You can create an XML file to build an environment for scanning many servers, and per server, scanning many directories. The path to this XML file is placed in the Add Logs Directories Task, for adding multiple directories to XpoLog, and automating addition of directories.
DirectoryScanner XML General Structure
The following is the XML code of DirectoryScanner.
<DirectoryScanner>
<ScannerNode name="NAME_OF_FOLDER_IN_XPOLOG">
<Account name="ACCOUNT NAME" ... />
<ScanDirectories>
<ScanDirectory scanPath="DIRECTORY_PATH_ON_SOURCE">
<ScanConfiguration>
<!-- OPTIONAL CONF PARAMETERS -->
</ScanConfiguration>
</ScanDirectory>
</ScanDirectories>
</ScannerNode>
</DirectoryScanner>
The following table describes the general structure of DirectoryScanner.
| Tag Path | Mandatory/Optional | Description |
|---|---|---|
| DirectoryScanner | Mandatory | |
| DirectoryScanner/ScannerNode | Mandatory | The root folder that will be placed above its scanned directories. |
| DirectoryScanner/Account | Optional | Mandatory if XpoLog should connect to a remote server - Windows / UNIX (Creating an Account) |
| DirectoryScanner/ScanDirectories | Mandatory | |
| DirectoryScanner/ScanDirectories/ScanDirectory | Mandatory | Contains the scanPath |
| DirectoryScanner/ScanDirectories/ScanDirectory/ScanConfiguration | Optional | |
| DirectoryScanner/ScanDirectories/ScanDirectory/ScanConfiguration/ScanFileFilter | Optional | |
| DirectoryScanner/ScanDirectories/ScanDirectory/ScanConfiguration/ScanConfApplications | Optional |
XML Reference
ScannerNode Parameter
| Parameter | Mandatory/Optional | Description | Values |
|---|---|---|---|
| name | Mandatory | The name of the root folder that will be placed above its scanned directories. | String |
Example
<ScannerNode name="">
Account Parameters
| Parameter | Mandatory/Optional | Description | Values |
|---|---|---|---|
| name | Mandatory | The account name | String |
| useEncrypt | Mandatory | Indicates whether or not account password will be encrypted | Boolean |
| isPublicKey | Mandatory (SSH Only) | If authentication is done by private key, should be FALSE. If user/password are used, it should be TRUE, and a path should be specified under privateKeyPath (see the following parameter). | Boolean |
| privateKeyPath | Optional (SSH Only) | The path to the key, if authentication is done by private key | |
| isSystemAccount | Indicates whether or not account is a system account | Boolean | |
| isScriptAPI | Indicates whether or not account is Script API | Boolean | |
| isSSH | Indicates whether or not account is SSH | Boolean | |
| isEditable | Indicates whether or not account can be edited in the XpoLog Address Book | Boolean | |
| isCertificate | Indicates whether or not the account uses a certificate | Boolean | |
| description | Optional | Description of the account | |
| classKey | Mandatory | Windows: xpolog.eye.media.auth.win.WinAuthenticationAccount | |
| certificateID | Optional | The ID of the certificate, if account uses a certificate (see isCertificate). | String |
| UserName | Mandatory | The username that the account uses to connect | String |
| isDefault | Boolean | ||
| Port | Mandatory (SSH Only) | The port that will be used to establish the connection to the remote data source | Numeric |
| TYPE_SCP_SFTP | Optional (SSH Only) | Indicates if the SSH account will use SCP or SFTP (default) protocol | String |
| Password | Optional | The password that the account uses to connect. Optional only if SSH account uses Public/Private key | String |
| NetAddress | Mandatory | The IP/hostname of the remote data source used in the account | String |
Note 1: If a remote data source is scanned then an account to that source should be specified (it can be verified after execution under XpoLog > Tools > Address Book). In case an account for a specified machine already exists, XpoLog will automatically use it.
Note 2: In case XpoLog is running on a Windows machine, it is recommended to configure a service account on the Windows services panel and then all Windows network logs can be scanned as local without specifying an account in the ScannerNode (path may be \\<server-name>\<drive-name>$\...).
Windows Account Example
<Account name="ACCOUNT NAME" useEncrypt="false" isSystemAccount="false" isEditable="true" description="" classKey="xpolog.eye.media.auth.win.WinAuthenticationAccount" Password="PASSWORD" user="USERNAME" isDefault="false"/>
SSH Account Example
<Account name="ACCOUNT NAME" useEncrypt="false" privateKeyPath="" isSystemAccount="false" isScriptApi="false" isSSH="true" isPublicKey="false" isEditable="true" isCertificate="false" description="" classKey="xpolog.eye.media.telsh.TelnetAccount" certificateId="" UserName="USERNAME" TYPE_SCP_SFTP="SFTP" Port="22" Password="PASSWORD" NetAddress="IP"/>
ScanDirectory Parameter
| Parameter | Mandatory/Optional | Description | Values |
|---|---|---|---|
| scanPath | Mandatory | The full path to the directory that is to be scanned | Path |
Examples
<ScanDirectory scanPath="C:\logs\"> (Windows Local)
<ScanDirectory scanPath="\\qaserver\C$\logs\"> (Windows Network)
<ScanDirectory scanPath="/opt/application/logs/"> (UNIX Local / Over SSH)
ScanConfiguration Parameters
| Parameter | Mandatory/Optional | Description | Values |
|---|---|---|---|
| condenseLogsTree | Optional | A "true" value indicates that folders containing only one subfolder and without logs, will be omitted from the Folders and Logs tree. | Boolean |
| directoriesToHide | Optional | A comma separated list of name expressions of folders that will not be added to the Folders and Logs tree;their subfolders/logs will be added. | |
| fileSuffixesToIgnore | Optional | Unite logs with different suffixes into one log type (advanced) | |
| numberOfThreads | Optional | The number of threads to be used as part of the scanning operation | Integer |
| removeEmptyNodes | Optional | In case there are no matching files under one of the Folders and Logs members, remove it from the Folders and Logs tree. | Boolean |
| subdirsScanLevel | Optional | The number of subdirectories to scan from the given directory. Default is unlimited; any number can be specified. | Integer or "Unlimited" |
| scanMethod | Optional | 0 = Use existing configuration (file names and content) and automatic matching. 1 = Use existing configuration (file names and content). 2 = Use existing configuration (file names only). | 0, 1, or 2 |
| namePatternLogic | Optional | 0 = Capture each file separately (without name pattern). 1 = Unite files with a similar names (apply name pattern automatically). 2 = Unite files with a similar suffix (apply name pattern only at the end of the file name). | 0, 1, or 2 |
| filesToInclude | Optional | Comma separated list of the files to capture during a scan; for example: *.txt, *.log | |
| filesToExclude | Optional | Comma separated list of the files that will be skipped during a scan; for example: *.txt, *.log | |
| directoriesToExclude | Optional | Comma separated list of the directories that will be skipped during a scan; for example: DIR_NAME, *NAME* | |
| fileSuffixesToIgnore | Optional | Regular expression used to ignore part of the files names to define a name pattern | String |
| enableLogsIndex | Optional | True – Online logs that are added under Folders and Logs will be indexed. False – Disable indexing. | Boolean |
| enableLogsAnalytics | Optional | True – Online logs that are added under Folders and Logs will be analyzed by Analytics. False – Disable Analytics. | |
| addCollectors | Optional | True – All the logs that are added by the scanner task will be collected; default policy will be applied, unless a specific policy is specified. False – Disables logs collection. | Boolean |
| enableCollectedLogsAnalytics | Optional | True – enables Analytics analysis on collected logs. Online logs state will be taken from the collection policy; relevant only when addCollectors="true". False – Disables Analytics analysis on collected log. | Boolean |
| enableCollectedLogsIndex | Optional | True – enables indexing of collected logs. The online logs state will be taken from the collection policy; relevant only when addCollectors="true". False – Disables indexing of collected logs. | Boolean |
| onlineLogsApplication | Comma separated list of application name(s) that the online logs will be tagged to once created | ||
| collectedLogsApplication | Optional | Comma separated list of application name(s) that the collected logs will be tagged to once created; relevant only when addCollectors="true". | |
| assignedCollectionPolicy | Optional | The policy name of the collection policy that will be applied on the collected logs once created; relevant only when addCollectors="true". |
Example
<ScanConfiguration condenseLogsTree="false" directoriesToHide="log,home" fileSuffixesToIgnore="_lara_\d+" numberOfThreads="3" removeEmptyNodes="true">
ScanFileFilter Parameters
| Parameter | Mandatory/Optional | Description | Values |
|---|---|---|---|
| timeInterval/timeIntervalUnit | Optional | The scan will add only log files with last updated time that is within the specified time interval per log type. | Years, Months, Weeks, Days, Hours, Mins |
| maxNumberOfFiles | Optional | The maximum number of log files that are added per log type. | Integer |
Examples
<ScanFileFilter timeInteval="3" timeIntervalUnit="months"/>
<ScanFileFilter maxNumberOfFiles="2"/>
ScanConfApplications Parameters
| Parameter | Mandatory/Optional | Description | Values |
|---|---|---|---|
| applicationNamePattern | Optional | The pattern that is used to extract the application name. An application will be created as part of the scan process. | |
| applicationGroupNamePattern | Optional | The pattern that is used to extract the application group name. An application will be created as part of the scan process that all its sub-application are tagged to. |
Example
<ScanConfApplications applicationNamePattern="/\w+-(\w+).*" applicationGroupNamePattern="/(\w+)-\w+.*"/>
Templates:
Please use the following examples as templates and modify accordingly (multiple directories per host can be defined by adding more <scanDirecotry> entries / multiple hosts can be defined by adding multiple <scannerNode> entries:
Example 1 (scanner_example_Windows_logs_account_on_xpolog_service): scanner_example_Windows_logs_account_on_xpolog_service.xml
Example 2 (scanner_example_Windows_logs_using_windows_network_account): scanner_example_Windows_logs_using_windows_network_account.xml
Example 3 (scanner_example_Linux_local_logs): scanner_example_Linux_local_logs.xml
Example 4 (scanner_example_Linux_remote_logs): scanner_example_Linux_remote_logs.xml