Applying a pattern on a log enables viewing the log in organized tabular format, setting an alarm on a specific field, or aggregating on a certain field.
By default, when adding a log to XpoLog and clicking the Save button on the first page of the Add Log wizard, XpoLog applies an automated pattern to parse the logs.
For certain log types (local, Windows Network, Over SSH, and Hadoop HDFS), XpoLog enables you to tune the log and parse it more deeply to normalize the log records into tabular format, by applying patterns on the incoming log data.
This can be performed from the Patterns Administration page, accessed by clicking the Next button on the first page of the Add Log wizard or Edit Log wizard.
It may be necessary to configure more than one pattern for logs that have different types of records that cannot be represented by a single pattern. You can do so by clicking the New button in the central pane.
The Patterns Administration page is divided into three sections, as follows:
- Upper pane – Text sample from the selected log. This pane presents the first 20 records from the incoming log (original data). You can copy paste other records from the incoming log data into this section, and then view the results of applying a pattern on those records (see Verifying Patterns on Manually Selected Data).
- Central pane – There is a tab for each pattern that has been configured for the log, named Pattern1, Pattern2, and so on. There is also a New button, which can be clicked to configure a new pattern to apply on the log. On the right side, provides you with three different Pattern Editor methods for configuring the patterns to apply on the log data:
- Wizard – Opens a wizard for creating or modifying a pattern. Using the wizard, you can set different indications on each column such as type, length, optional, column name and more (more information on each type is presented in the wizard itself).
- Manual – For advanced users who are familiar with the Pattern language.
- Automatic – XpoLog matches patterns automatically and suggests possible patterns for deeper parsing. This is only available when adding a log; not when editing a log.
- Bottom pane – Log records analysis results. Shows the results of each parsing, i.e. applying the pattern to the log data.
To apply a pattern on the log:
- In the central pane of the Patterns Administration page, click the tab of a pattern to modify, or click the New button to configure a new pattern.
- Click one of the available Pattern Editor options, and configure the pattern:
Auto – see Selecting an Automatic Pattern.
Wizard – see Creating a Pattern Using the Builder.
Manual – see Configuring a Manual Pattern.
Note: You can also create a pattern in the wizard based on one of the automatic pattern suggestions (see Creating a Pattern Based On an Automatic Pattern Suggestion). - Repeat steps 1 and 2 for each pattern that you want to configure or modify.
- After you have modified and configured all patterns, click the Save button.
XpoLog applies the pattern on the incoming log, and the Log Viewer opens displaying the parsed records of the new log. The log name is displayed in the left pane in its selected location under Folders and Logs. If you put in the log path a {string} pattern, the various files of the log appear in the left pane. Otherwise, only one file appears. You can perform regular actions on this log.
Removing a Pattern
You can remove a pattern that you no longer want to apply on the log.
To remove a pattern:
- In the central pane of the Patterns Administration page, click the down arrow on the tab of the pattern that you want to remove, and in the menu that apppears, click Remove.
The pattern is removed. The renaming patterns are renumbered. For example, if there is a Pattern1 and Pattern2 and you remove Pattern1, Pattern2 now becomes Pattern1.
Moving a Pattern to the Left or Right
You can move a pattern one tab to the left or right.
To move a pattern one tab to the left:
- In the central pane of the Patterns Administration page, click the down arrow on the tab of the pattern that you want to move to the left, and in the menu that apppears, click Move left.
The pattern moves one tab to the left and is highlighted.
Note: Nothing happens when you select to move left the leftmost tab.
To move a pattern one tab to the right:
- In the central pane of the Patterns Administration page, click the down arrow on the tab of the pattern that you want to move to the right, and in the menu that apppears, click Move right.
The pattern moves one tab to the right and is highlighted.
Note: Nothing happens when you select to move right the rightmost tab.
Duplicating a Pattern
You can duplicate a pattern that you want to use as a basis for another pattern.
To duplicate a pattern:
- In the central pane of the Patterns Administration page, click the down arrow on the tab of the pattern that you want to duplicate, and in the menu that apppears, click Duplicate.
A new pattern tab is created with the contents of the duplicated pattern.