Defining a Data Forwarder

Owned by Noga Aviram (Unlicensed)
Last updated: Feb 22, 2024 by XPLG Team
3 min read

XPLG provide an advanced and comprehensive forwarding data engine that is capable of sending data over variety of protocols (Syslog UDP/TCP, HTTP/S, Kafka, Files, etc.) with activation of filtering, transformers, and other manipulations of the data for different purposes.

The data can be sent to many consumers that require it in the organization, available targets: XpoLog Forwarder, Syslog Forwarder, HTTP Forwarder, Kafka Forwarder, File Forwarder, and Socket Forwarder.

image-20240222-091149.png

Next to each Forwarder you’ll see a summary of its targets that data is sent to, its status and last time it sent data. Possible statuses:

  • PENDING - waiting for initial execution and data forwarding

  • STREAMING - data is sent by the forwarder

  • SCHEDULED - the forwarder is scheduled to send data in specific time(s)

  • DISABLED - the forwarder is disabled. Enabling it will resume activity from the enablement moment based on its definitions.

  • SUSPENDED- the forwarder is suspended. Resuming it will resume activity from the suspension moment.

 

 

The Data Forwarding sections are as follows:

  • General Info – for setting the forwarder name and optionally providing a description for the usage of this customized/default forwarder.

  • Data Time Frame - On creation you may specify one of the following options:

    • From now - sending data from the saving time of the forward and going forward.

    • Set From/To - sending data from/to a specific time.

    • From the beginning of time - sending all available data from the defined source and going forward (note: if your sources are large in size, take into consideration that ALL their data will be sent to the specified targets).

(this option is not editable after saving a forwarder)

  • Input Sources – for selecting the logs from the XPLG repository that will be forwarded.

  • Forwarder Schedule – for defining the frequency of forwarding the data from XPLG.

  • Data Flows – for defining filtering, enrichment, masking, and transformation of the data before sending.

  • Output Targets – for defining the target hosts, the type of the forwarders and additional details such as port, protocol, etc.

To define a new Data Forwarder:

  1. Click on Manager pane, the Left Navigation Panel opens. select Data > Data Forwarding. 
    The Data Forwarding console opens. The available options are: Add Data Forwarder | Edit | Duplicate | Delete | Enable | Disabled | Suspend

  2. Click the New Data Forwarder button.

  3. In General Info, type the name and description of the Forwarder.

  4. In Data Time Frame, set the time from which to send data from and/or to in this forwarder

  5. In Input Sources, select the logs from the XPLG repository that will be forwarded.

  6. In Forwarder Schedule, define the frequency of forwarding:

    1. Send Data on Arrival - data will be sent as soon as it reaches XPLG.

    2. Set Frequency - define a scheduler to send the data on specific time/interval.

  7. In Data Flows, define the flow of the data before being sent. By default XPLG send unfiltered raw data, by defining data flows its possible to filter, enrich, customize and transform the data that will be sent. read more>

  8. In Output Targets, define the target hosts, the type of the forwarders and additional details such as port, protocol, etc. read more>

  9. Click Save. 
    The Data Forwarder is saved and data will be sent as defined.