Adding a Log to XpoLog

You can add a log to XpoLog Center from any of the following sources:

• Local – The log is on the same machine as XpoLog Center or can be directly accessed from a remote server (in the case of Windows) or a mounted directory (in the case of UNIX).
• Windows Network – The log is on a remote Windows machine.
• Over SSH – The log is on a remote UNIX machine (with SSH connecting protocol). 
• Hadoop HDFS – The log is in a Hadoop environment.
• AWS S3 Bucket - The log is in Amazon Web Services S3 storage.
• Google App Engine – The log is on a Google cloud.
• Database – Connection is made to a database to import a table from the database as a log.
• Windows Events  – The log is a Windows Events log.
• Remote XpoLog – Connection is made with http (protocol of navigating) or https (secure protocol of navigating, as in a bank) between XpoLog instances.
• Merge Logs – Multiple logs in the system are combined into a single unified view log and sorted chronologically.

Note: Windows Network and Windows Events logs are only available when Xpolog Center is installed on a Windows machine.

A single log can include many files of the same type or rotated files. For example, a messages log can include the files messages.1, messages.2, and more. It is recommended to capture multiple files of the same log type as one log, using a generic path. This can be done by appending a name pattern to the log path. For example, if a log type contains multiple files that follow a similar name pattern such as mylog.log.20110101_1, mylog.log.20110101_2, mylog.log.20110101_3, …, mylog.log.20110101_n, you can enter the generic log path: mylog.log.{date,yyyy-MM-dd}_{string}.

Examples:

• /var/log/messages{string}
• log.1.log, log.2.log, and log.3.log can be represented by the name pattern log.{string}.log.
• myapp.25-8-2009-22:30:00, myapp.26-8-2009-22:30:00, and myapp.27-8-2009-22:30:00 can be represented by the name pattern myapp.{date,dd-MM-yyyy-HH:mm:ss}.

As logs are written in free format, XpoLog uses its built-in mechanism to guess the structure of the incoming log, also called a pattern. For example, Xpolog guesses which field is the Date, the Time, and more, parses the data, and then adds the log records in bulk to XpoLog. Users can further normalize or tune the results of parsing. For example, if the results of parsing data is a log with columns Date, Text2, and Text3, the user can open the log under Folders and Logs, and click the log to open it and tune the parsing results. For example, Text2 column heading can be changed to Host, and Text3 column heading to Server Name. Next time you add a log of a similar type to the one that you edited, it uses the last structure that you applied to the new log.  

Once a log is brought into XpoLog it is indexed and undergoes Analytics. You can also run searches on it, and perform all log actions on it.

To add a log to XpoLog:

1. Open the XpoLog tab, and in XpoLog Manager, click Add Log
   OR
   Open the XpoLog tab, and in the Administration menu, select Add Log
   OR
   In the XpoLog homepage, in the left pane under More Actions, click Add log
   OR
   In the XpoLog homepage, in the left pane under More Actions, click Quick start wizard, and in the Quick Start Wizard that opens, click Add Log
   OR
   Open the XpoLog tab, and in the Administration menu, select Folders and Logs, and in the Folders and Logs console that opens, click the New Log button.
   The Add Log Wizard opens.
2. Set the basic information of the log being added to XpoLog, including log name, location, and tagging to applications  (see Setting Log General Information).
3. In Log Source, select the log source of the log to add to XpoLog
4. Fill in the connectivity information for the selected log type, as follows:
   1. For a Local log, complete the information as described in Adding a Local Log.
   2. For a Windows Network log, complete the information as described in Adding a Windows Network Log.
   3. For an Over SSH log, complete the information as described in Adding an Over SSH Log.
   4. For a Hadoop HDFS log, complete the information as described in Adding a Hadoop HDFS Log.
   5. For a AWS S3 Bucket log, complete the information as described in Adding an AWS S3 Bucket Log.
   6. For a Google App Engine log, complete the information as described in Adding a Google App Engine. 
   7. For a Database log, complete the information as described in Adding a Database Log.
   8. For a Windows Events log, complete the information as described in Adding a Windows Events Log.
   9. For a Remote XpoLog log, complete the information as described in Adding a Remote XpoLog Log.
   10. For a Merge Logs log, complete the information as described in Adding a Merged Log.   
5. At this point it is possible to save the log, or going 'Next' if applicable to review the Data Pattern Configuration.
6. Once the log is added to XpoLog, validate the log configuration (see Verifying Added Log Configuration).