Add Logs Directory Use Case

Owned by Omry Koschitzky
Last updated: Dec 11, 2013 by Haim Koschitzky
2 min read

The Add Logs Directory wizard enables users to add to XpoLog multiple logs that are located in a local or remote directories. The user can scan their local or remote directory for the log directory from which they want to capture logs into XpoLog. Using this wizard may result with adding huge amount of logs and data to XpoLog, therefore it is critical to review the configuration properly prior to completing the execution of collection and indexing of data.

This procedure is important due to two main reasons: if XpoLog process data prior to major configuration changes such as data patterns, data will be reconstructed using the new configuration after it was already processed. which may take time based on the environment sizing. In addition, in order to get optimized search and analysis results it is mandatory to to get data parsed properly (it is not mandatory to parse all log fields however at least ensure the date/time stamp is parsed properly and the rest of the log record can be  under a single message field). 

Recommended steps:

  1. Prepare and save templates to log samples - use as many log types as possible that will be added during the environment processing and ensure data is parsed properly.

  2. Define a new collection policy (temp) with schedule set to 'Never'. This way no data will be processed before review.

  3. Scan the required directory/directories using templates (by setting Scan Method to use existing configuration) so the predefined configuration will be applied on the detected logs. Set the temporary collection policy on the scan result so no data will be collected and indexed before reviewing the scan results.

  4. Review the results on the scanning by entering selected log types that were detected by the scanner and ensure that all data is parsed and presented properly. 
    In case you identify log(s) which are not parsed well, make the required changes and replace / save a new template. Use the apply template on logs function to update all required logs based on the templates configuration. At the end of this process all logs should be parsed and presented properly.

  5. Apply the required collection policy on the logs that were added by the scan process, so that data will be collected and indexed using the accurate configuration.

 

The above steps will ensure that data is thoroughly reviewed prior to being processed by XpoLog resulting with a single data execution processing and optimized performance.