Adding a LogSync Task
The LogSync task enables synchronizing a single or multiple logs directories from remote XpoLog to another XpoLog, and automating their execution using a scheduler. This feature should be used when you want to create an image of a directory or multiple directories content (logs and/or binaries) on another XpoLog.
Configuration of a LogSync task requires specifying a connectivity account to the remote XpoLog machine created , see Creating an Account - Remote XpoLog.
Configuration of a LogSync task for synchronizing multiple single log directory or multiple logs directories to XpoLog requires specifying the path to an XML-based LogSyc Configuration file, which contains the configuration for the synchronization from multiple servers and/or directories to XpoLog. For a detailed explanation on creating the XML-Based LogSyncConfiguration file, see below.
To add a LogSync task:
- Open the Tasks console (see Tasks), and click the New Task button
OR
In the XpoLog Manager homepage, under More Actions, click Add Task. - In the page that opens, select the LogSync option, and then click Continue.
The LogSync Task page opens. - In Name, type the name of the new task.
- In Description, type a description of the new task; optional.
- Under Details tab:
- Use a synchronization configuration file , and in Configuration file path, type the full path name to the LogSync configuration file or browse to it.
- Create Configuration, while synchronization takes place, XpoLog can also create configuration on all the logs which were synchronized under the Folders and Logs panel so all logs will be available for view and search in the XpoLog console. Check the 'create configuration' check-box if you wish to automatically add logs to XpoLog, and select/create a parent folder under which all synchronized logs from this task will be placed.
- To automate task execution, open the Schedule tab, and configure the scheduler as described in Scheduling a Task.
- Click Save.
The Add LogSync task is saved.
Note:
When right clicking the LogSync task the following options are available:
- Force Log Sync - Force the LogSync task to re-sync files that were deleted from the repository and still exist on the remote sources. The re-sync will take place during the next task execution.
- Reconstruct Configuration - Force the LogSync task to re-create missing Folders and Logs configuration, based on the structure of the repository. The reconstruction will take place during the next task execution.
- In case you are running XpoLog cluster with more than one processor, an option will be presented to determine which of the processors is assigned on the execution of the LogSync task.
LogSync XML Based Configuration
<SyncLogsConfiguration>
<SyncLogsRepository repositoryDirectory="C:\xplgshare\logSyncTest" timeToKeep="7" />
<SyncLogsNode repositoryDirectory="profiler">
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="XPOLOG Profiler"/>
</Remote>
<SyncLogsDirectory syncDirectory="C:\logSyncTest" directoriesToInclude="" directoriesToExclude="" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
<SyncLogsDirectory syncDirectory="C:\logSyncTest\zip3">
</SyncLogsDirectory>
</SyncLogsNode>
<SyncLogsNode repositoryDirectory="2" key="MY_KEY">
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="XPOLOG Profiler"/>
</Remote>
<SyncLogsDirectory syncDirectory="C:\logSyncTest\logs\Zip1" directoriesToInclude="" directoriesToExclude="" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
</SyncLogsNode>
</SyncLogsConfiguration>
The following table describes the general structure of SyncLogsConfiguration.
Tag Path | Mandatory/Optional | Description |
---|---|---|
SyncLogsConfiguration | Mandatory | |
SyncLogsConfiguration/SyncLogsRepository | Mandatory | The repository tag |
SyncLogsConfiguration/SyncLogsNode | Mandatory | A single log synchronizatoin node configuration |
SyncLogsConfiguration/SyncLogsNode/Remote | Mandatory | Contains the Remote XpoLog Account details to which with the directories will be synched. |
SyncLogsConfiguration/SyncLogsNode/Remote/Account | Mandatory | Mandatory as XpoLog should connect to a Remote XpoLog server (See: Creating an Account) |
SyncLogsConfiguration/SyncLogsNode/SyncLogsDirectory | Mandatory | Configure the remote path to synch with the local XpoLog |
XML Reference
SyncLogsRepository Parameters
Parameter | Mandatory/Optional | Description | Values |
---|---|---|---|
repositoryDirectory | Mandatory | The name of the root folder that will be synced with he content of the remote sync log directories. | String |
timeToKeep | Optional | Time to keep the synched data on the repository | String |
Example
<SyncLogsRepository repositoryDirectory="C:\xplgshare\logSyncTest" timeToKeep="7" />
CheckSum Parameters
Parameter | Mandatory/Optional | Description | Values |
---|---|---|---|
algorithm | Mandatory | The algorithm to use to calculate the checksum | MD5/SHA-1 |
enabled | Optional | Determines whether to activate checksum | true/false |
interval | Mandatory | The frequency that checksum will be calculated | m=minutes |
retryAttempts | Mandatory | The number of times that XpoLog will try to re-synchronize a file in case of a checksum validation failure before alerting | number |
mailRecipients | Optional | The list of recipients that will be alerted in case of a checksum validation failure | semicolon separated list of email addresses |
* The Checksum definition can be placed inside a specific SyncLogNode tag to be applied on that specific node, or inside the SyncLogsConfiguration tag to be applied globally on all nodes
Example:
<Checksum interval="2m" algorithm="MD5" mailRecipients="ops@xpolog.com" enabled="true" retryAttempts="1" />
SyncLogNode Parameters
Parameter | Mandatory/Optional | Description | Values |
---|---|---|---|
repositoryDirectory | Mandatory | The name of the folder within the repository to which remote directory from this node will be synced into | String - path |
key | Optional | The SyncLogNode unique key |
Example
<SyncLogsNode repositoryDirectory="profiler">
Account Parameters
Parameter | Mandatory/Optional | Description | Values |
---|---|---|---|
classKey | Mandatory | The account class to use | String |
name | Mandatory | The Remote XpoLog account name/ID to use | String (case sensitive) |
Remote Account Example
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="XPOLOG Profiler"/>
</Remote>
SyncLogsDirectory Parameters
Parameter | Mandatory/Optional | Description | Values |
---|---|---|---|
syncDirectory | Mandatory | The directory on the remote XpoLog Node that will be synchronized into the SyncLogNode repository directory | String - path |
repositoryDirectory | Optional | The name of the folder within the repository to which remote directory from this node will be synced into | String |
directoriesToInclude | Optional | Define which directories to include in the log sync scan: Using wildcard: Apps* - include all directories whose name starts with Apps Using a regular expression: regexp:\d\d\d\d-\d\d\-\d\d - include all directories whose name is a date, for example 2013-11-26 | String |
directoriesToExclude | Optional | Define which directories to exclude from the log sync scan: Using wildcard: Apps* - exclude all directories whose name starts with Apps Using a regular expression: regexp:\d\d\d\d-\d\d\-\d\d - exclude all directories whose name is a date, for example 2013-11-26 | String |
filesToInclude | Optional | Define which files to include in the log sync scan: Using wildcard: *.log,*.txt - include all files whose name ends with .log or .txt Using a regular expression: regexp:\w+\.log - include all files whose name is constructed of word characters only and ends with .log, for example helloWorld.log | String |
filesToExclude | Optional | Define which files to exclude in the log sync scan: Using wildcard: *.zip,*.gz - exclude all files whose name ends with .zip or .gz Using a regular expression: regexp:\w+\.tar\.gz - exclude all files whose name is constructed of word characters only and ends with .tar.gz, for example helloWorld.tar.gz | String |
subdirsScanLevel | Optional | Number of sub-folder depth to scan | Number |
timeInterval/timeIntervalUnit | Optional | The task will synchronize only files with last updated time that is within the specified time interval | years, months, weeks, days, hours, mins |
Examples
<SyncLogsConfiguration>
<SyncLogsRepository repositoryDirectory="/home/LOGS_REPOSITORY/" timeToKeep="7" />
<Checksum interval="2m" algorithm="MD5" mailRecipients="ops@xpolog.com" enabled="true" retryAttempts="1" />
<SyncLogsNode repositoryDirectory="Tent">
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Tent"/>
</Remote>
<SyncLogsDirectory syncDirectory="C:\configurations\4.5_Build\log\" directoriesToInclude="regexp:\d\d\d\d-\d\d-\d\d" directoriesToExclude="*temp*" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
</SyncLogsNode>
<SyncLogsNode repositoryDirectory="RedHat">
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="RedHat"/>
</Remote>
<Checksum interval="2m" algorithm="SHA-1" mailRecipients="ops@xpolog.com" enabled="true" retryAttempts="1" />
<SyncLogsDirectory repositoryDirectory="xpolog-logs" syncDirectory="/root/xpolog45/log" directoriesToInclude="" directoriesToExclude="" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
<SyncLogsDirectory repositoryDirectory="OS-logs" syncDirectory="/var/log/" directoriesToInclude="" directoriesToExclude="" filesToInclude="" filesToExclude="" subdirsScanLevel="" timeInterval="1" timeIntervalUnit="days" >
</SyncLogsDirectory>
</SyncLogsNode>
</SyncLogsConfiguration>
Configuration Parameters on synchronized Logs
The configuration parameters are relevant only if the Create Configuration option is selected. XpoLog will scan the repository to add all relevant logs.
It is possible to add <scanConfiguration> tag either globally (above all <SyncLogsNode> tag) or individually by placing it internally in a <SyncLogsNode>.
Using Proxy XpoLog for Log Synchronization
It is possible to use a remote XpoLog as a proxy to synchronize logs from remote agents which are managed by a remote XpoLog and not directly by the current XpoLog that executes the task.
- To do so add the account(s) of the remote XpoLog before the 'SyncLogsNode' tags.
The following example will synchronize logs from the directory 'C:\logs\' on the 'Agent 1' machine, where 'Agent 1' is not a direct agent of the current XpoLog, but an agent connected to 'Remote XpoLog Master1' and 'Remote XpoLog Master2':
<SyncLogsConfiguration>
<SyncLogsRepository repositoryDirectory="/home/LOGS_REPOSITORY/" timeToKeep="7" />
<Checksum interval="2m" algorithm="MD5" mailRecipients="ops@xpolog.com" enabled="true" retryAttempts="1" />
<ProxyRemote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Remote XpoLog Master1"/>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Remote XpoLog Master2"/>
</ProxyRemote>
<SyncLogsNode repositoryDirectory="Agent 1">
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Agent 1"/>
</Remote>
<SyncLogsDirectory syncDirectory="C:\logs\" directoriesToInclude="" directoriesToExclude="dump" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
</SyncLogsNode>
</SyncLogsConfiguration>
- It is also possible to define the <ProxyRemote> inside an individual <SynchLogNode>:
The following example will synchronize logs from the directory 'C:\logs\' on the 'Agent 1' machine, where 'Agent 1' is not a direct agent of the current XpoLog, but an agent connected to 'Remote XpoLog Master1' and 'Remote XpoLog Master2'. In addition, it will synchronize logs from the directory '/root/xpolog45/log' on the 'Agent 2' machine, where 'Agent 2' is a direct agent of the current XpoLog (no proxy has been defined for it):
<SyncLogsConfiguration>
<SyncLogsRepository repositoryDirectory="/home/LOGS_REPOSITORY/" timeToKeep="7" />
<Checksum interval="2m" algorithm="MD5" mailRecipients="ops@xpolog.com" enabled="true" retryAttempts="1" />
<SyncLogsNode repositoryDirectory="Agent 1">
<ProxyRemote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Remote XpoLog Master1"/>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Remote XpoLog Master2"/>
</ProxyRemote>
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Agent 1"/>
</Remote>
<SyncLogsDirectory syncDirectory="C:\logs\" directoriesToInclude="" directoriesToExclude="dump" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
</SyncLogsNode>
<SyncLogsNode repositoryDirectory="Agent 2">
<Remote>
<Account classKey="xpolog.eye.media.remote.RemoteXplgAccount" name="Agent 2"/>
</Remote>
<SyncLogsDirectory repositoryDirectory="xpolog-logs" syncDirectory="/root/xpolog45/log" directoriesToInclude="" directoriesToExclude="" filesToInclude="" filesToExclude="" subdirsScanLevel="">
</SyncLogsDirectory>
</SyncLogsNode>
</SyncLogsConfiguration>