HTTP/S

Owned by Omry Koschitzky
Last updated: Sept 03, 2020
3 min read

To get data over HTTP/S, configure an HTTP/S Listener account:

  1. Go to Manager > left panel Data > Listen to Data. The Listeners management console opens.
  2. Add HTTP account, for each account the following should be configured:
    1. Name: the name of the Listener account
    2. Description: the description of the Listener account
    3. Listening Node: the node in the cluster which will listen to the HTTP messages (appears only if a XpoLog cluster is deployed)
    4. Token: a unique token that is identified with this listener and must be included in the used URL when forwarding data to this account - the token ensures only permitted devices send data and also tags the sent data to the desired HTTP/S listener in XpoLog
    5. URL: the URL that devices that forward data to XpoLog should use - including the IP address/Hostname, port, path and token (in case of a cluster - the URL gets populated when selecting a Listening Node)
       
    • Advanced Settings:

      General Information:
      Enabled: determine whether this account is enabled or not

      Dynamic Log Creation Configuration:
      Note: it is optional to configure the dynamic log creation configuration in the listener XpoLog. Leave as default to recevie the data as is, or use the agent's configuration to send the information.
      Parent Folder: the parent folder which all logs from this listener will be place under in XpoLog Folders and Logs tree
      Collection Policy: the collection policy which will be associated to all logs from this listener (used mainly for storage location and data retention)
      AppTags: the AppTags  which will be associated to all the logs from this listener (used mainly for data enrichment)
      Log Name Prefix: a prefix which will be added to any of the the logs from this listener (used to easily distinguish between multiple listener accounts logs. Leave empty for no prefix)
      Split by Source Device: check to create a log for each unique source device value in the received message (a log will be created per device)
      1. Do not split - by default, XpoLog will not split the incoming data. All data will be stored under a single log in XpoLog.
      2. Create log by unique IP / host name - XpoLog will split the incoming data based on the source that sends it to different logs, the log name structure will be "Log_Name_Prefix Source_IP/Name"
      3. Create log by IP mask - XpoLog will split the incoming data based on matched source to the configured IP mask that sends it to different logs, the log name structure will be "Log_Name_Prefix IP_Mask"
      Message JSON Key: the JSON key of the message field. Used when sending a type to XpoLog Listener to match a template of that type - the key determine which one of the fields of the JSON contains the log message itself to be matched
      Split by JSON columns: A list of JSON keys, separated by '->', denoting the log's path and name. Append '*' to the key denoting the host.

      Advanced Dynamic Log Creation Configuration:
      It is possible to push information from the agents/devices sending the logs to XpoLog such as name, data type and their target Folders and Logs location in XpoLog. The Listener looks for the following JSON keys and, if any of them exists in the messages, uses them:
      - xpologPath: a path of folders, separated by '->', that the logs should be placed in, in XpoLog Folders and Logs
      - xpologName: the name of the log in XpoLog (if another split type is selected then it will be added to the name)
      - xpologType: the type of the log, if there's a template in XpoLog with the exact same type then XpoLog will automatically apply the template's pattern on the received message (note: in this case it is important to specify the 'Message JSON Key' detailed above).

      Listener Data:
      Listener Data Location: the location which data will be stored to, by default XpoLog stores it in its data directory
      Indexing Node: the node in the cluster which will index the received Syslog messages (appears only if a XpoLog cluster is deployed)
      Indexing Interval: the frequency in which received Syslog  messages are indexed

      JSON Parsing:
      JSON Parsing Level: set the maximum depth in the JSON object for which data will be extracted into columns
  3. Save the account.
  4. Data received to the HTTP/S listener account will be created under the configured parent folder and will be available for searching, reporting and alerting