Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 11 Next »


When adding a log to XpoLog, you can configure the advanced settings that are available for the log type of the log that you are adding. The available advanced settings are:

  • Data Filter – enables filtering data during collection
  • Files Attributes – for local, Over SSH, and Hadoop HDFS logs
  • Files Filters – for local, Over SSH, and Hadoop HDFS logs;enables specifying which files in the logs directory are to be scanned
  • Regional Settings – for local, Over SSH, Hadoop HDFS, Google App Engine, Database, and Merge Logs logs
  • Other Types – for Windows Events logs

Note: Advanced settings are not available for Remote XpoLog logs.

Data Filter

You can filter out log events during collection so that some of the logs data will not be collected and available in XpoLog (alternatively it is also possible to mask data)

To set the data filter:

  1. Under Advanced Settings, click Data Filter. 
    The Data Filter definitions open.
  2. Enter a search query that filters only events that should be collected from the log (events that don't satisfy the query will be ignored).
    It is recommended to run the same query in the Search console prior to saving this filtering to ensure the result is the desired one.

Files Attributes

For local, Over SSH, or Hadoop HDFS log files, when using a generic name pattern that captures multiple files, it is important to define the following file attributes:

  • Sort physical files by – specify the option that will ensure that all the files in the log that are being captured are sorted correctly: last update time (default) or filename.
    Note: It is recommended to order according to the last update time, because XpoLog assumes that one file is written and closed in the source log, before another is written and closed.
  • File rotation policy – define the nature of rotation between the captured files of the log:
    • Cyclic – Files are overridden as part of the file rotation and log file names are modified during a rotation cycle.
    • New appending – File names are unique and are not changed as part of the rotation; however, the list of files changes as new files are added.
    • Static – The list of files never changes. Relevant for logs repository where the files are not changed.
  •  Add another log path – Clicking this link opens another Log path field, enabling you to add more than one source log into a single XpoLog log.   

To set the file attributes:

  1. Under Advanced Settings, click File Attributes. 
    The File Attributes definitions open.
  2. In Sort physical files by, select Last update time or File name.
  3. In File rotation policy, select Cyclic, New appending, or Static.
  4. If you want to import an additional log into the same XpoLog log, click Add another log path.
    An additional Log path field opens for selecting another log file to import into the same log.
  5. Repeat step 4 for all the source logs that you want to capture under the same XpoLog log.

Files Filters

If you used in the log path a name pattern that captured multiple log files, you can filter the files list to add only some of these captured files to XpoLog by filtering the list according to any or all of the following criteria:

  • Include or exclude the last specified number of files  
  • Include the files from the specified timeframe
  • Include files of or within a specified size range
  • Exclude specified name patterns

For example, you can filter the log files to include only those files from the last 7 days, or the last 50 files.

Important:  XpoLog does not allow two files of a single captured log to contain data from the same timeframe. In such a scenario, it is not possible to execute time-based operations. When different files contain an identical timeframe, they should be defined separately and can be merged by the merge-logs wizard.

To define the file filters:

  1. Under Advanced Settings, click Files Filters. 
    The Files Filters filtering criteria open.
  2. Fill in any of the criteria that you want to use to filter the captured files:
    • In the textbox within Include the last files, type the number of last files to include in the log
      OR
      In the textbox within Exclude the last files, type the number of last files to exclude from the log.
    • In the From the filter, select to filter the files from the last (default) or previous specified number of units of time – in the textbox, type the number of units of time and in the following dropdown list,  select minutes, hours (default), days, weeks, or months.
    • In With size, select equals, smaller than, smaller than or equals, greater than, greater than or equals, between, not equals, and in the following textbox, type the number of bytes.
      Note: If you select between, type a range of bytes.
    • In Exclude name patterns, type the name patterns (separated by commas) of the files to exclude from the log. See XpoLog Patterns Language for syntax of name patterns. 

End of Line Representation

Some logs contain special characters or methodology to end line of log records. When standard end of line characters are not used, it is possible to define to XpoLog the "logic" of different files separation:

  • Automatic (default) - XpoLog uses standard end of line characters to identify end of lines. This option is the most common one, only special logs require one of the below options in order to be supported.
  • Dynamic Line Length - Specify the number of characters at the beginning of each line which defines the line's length. XpoLog will use the first specified chars to calculate the line's length and then will read it on the next line to calculate its length and so on.
  • Fixed Line Length - Specify the fixed number of characters which represents a line.
  • Line Fixed Suffix - Specify the term which appears at the end of each line as is.
  • Line Fixed Prefix - Specify the term which appears at the beginning of each line as is.

Regional Settings 

In Regional Settings, you can set:

  • The time zone of the added log. This is especialy useful if you are managing more than one data center, and you want to view all logs of all data centers in the same time zone.
  • The character set used in the added log. The default is the language of your machine. 

To define the regional settings:

  1. Under Advanced Settings, click Regional Settings. 
    The Regional Settings section opens.
  2. In Log Time Zone, select the time zone of the added log (Default is Default UTC).
    Note: If the log doesn't have a date field, the time zone setting is ignored.

  3. In Character Set, select the charset of the added log (Default is Default (charset of your machine)).
    Note: Use the charset that applies to the file encoding and machine charset support.

Other Types

Windows Events logs that are added to XpoLog are usually classified as system, application, or security logs. In the Advanced Settings section, under Other Types, XpoLog enables you to alternately classify the incoming log as a Custom log or *.evt file.

To define other types:

  1. Under Advanced Settings, click Other Types. 
    The Other Types options open.
  2. Select one of the following options:
    Custom – Type the name of the custom log. 
     Note: If the custom registered log name cannot be found, the event logging service opens the Application log.
  3. File – Browse for or type the name of the file, and select its type:  system, application, or security.
  • No labels