File Forwarder

To send data from XPLG to Files, add a File Forwarder:

1. Go to Manager > Left Navigation Panel > Data > Data Forwarding > New Data Forwarder:

   Open

   

2. Enter a name for this forwarder.

3. Mark or define the sources which will be sent via this forwarder.

4. Set the Scheduler for this forwarder. (default and most recommended is ‘Send data on arrival’ - real-time)

5. Create Data Flows on the data (Optional). Read More>

6. As output target, choose File Forwarder:

   1. name - The name of the File Forwarder.

   2. Description -  The description of the File Forwarder.

   3. Enabled - The File forwarder is enabled by default, uncheck for disabling.

   4. File name - The name of the file that you want the data to be written to (full path), you can enter an existing file or a new one.

   5. Rolling pattern - the rolling pattern determines how the new files should be called.
      for time rolling add %d{some timestamp pattern} e.g. %d{HH-mm-ss}
      for max size rolling add %i - it is an increasing number in the file pattern.
      Usually this will be the same as the file name plus the format of the date or maximum size.

7. Append - if true the forwarder will adds data to the end of the file you provided. If false, it will overwrites the file. Default is true

8. Rolling behavior:

   1. Triggering policy:

      1. Rolling size trigger - the rolling will take place every time that the current file size is bigger the value of this field, value to should specified in log4j format e.g. “10MB”.

      2. Rolling time trigger - the rolling will take place every time that the time pattern in the file pattern changes depends on the given value at “rolling time trigger“ . for example - if “rolling time trigger“ is 3 and the rolling pattern is %d{HH-mm-ss} so every 3 seconds a new file will be created.

      3. No rolling - if you don't want to use rolling - leave ‘rolling time trigger’ to 0(default) and leave 'rolling size trigger' as null(default).

   2. Rollover policy:

      1. For DefaultRolloverStrategy:

         1. Rolling file index - if set to max the new files will be those with the higher numbers, if set to min the new files are those with lower number(every time that a new one is created every file’s number is growing by 1) . Default is max

         2. Max num of files - the max number of files that will be created, after that the oldest files will be deleted when new data will be sent. Default is 10
            enter “no max” to the field “Rolling file index“ if you want that there will be no deleting of files.

         3. Compression Level - Sets the compression level - can be used when the file pattern has zip/gz at the end. values are 0-9, 0 - none, 1 - best speed, 9 - best compression. Default is None.

      2. for DateDirectRolloverStrategy:
         leave the ‘file name’ empty - this strategy creates the online written file with the rolling pattern instead of writing online to a file with the name - ‘file name’ and copy it every time that roll occurred.

9. Create on-demand - if true, creates the file only when there is data. Default is false

10. Custom pattern - log4j layout for the data that being forwarded. Default is %m%n - Log4j – Log4j 2 Layouts

11. Immediate Flush - if true, each write will be followed by a flush. Default is true

12. Use random access - if true, uses the random access appender (there is a 20-200% performance improvement compared to regular FileAppender). Default is true.

13. Active Flows: Use one of the existing flows or send ‘All data’.

Examples:

1. Export all logs to files with a limit of 1GB per file:
   
   

   Open

   

2. Forward and keep last 5 files, using a rolling date name per 3 hours:
   
   

3. Forward and keep 20 files, using size based rolling - every file will be max 50MB and compression level of the files is 5:
   
   

4. Forward without rolling: