Configuring Filebeat to send data to Logstash

Owned by Omry Koschitzky
Last updated: Dec 06, 2018
1 min read


Filebeat XpoLog Integration

Logstash can receive in its input raw data send by multiple Filebeat forwarders. In order to do this,  each Fliebeat should be configured to send its data to the Logstash along with proper tagging for each log.



Technical Details

The Filebeat configuration file should look like the following:

 

filebeat.inputs:

- type: log

enabled: true

# Paths that should be crawled and fetched. Glob based paths.

paths:

- FULL_PATH_TO_LOGS_DIRECTORY/FILE
# There should be one path per log - bellow there are examples for log paths - Linux or Windows
# - /var/log/syslog.log
# - c:\programdata\elasticsearch\logs\*

fields: {apptags: "<APPTAG_1,APPTAG_2,APPTAG_N>",app: "<FOLDER_NAME>", logname: "<LOG_NAME>", filename: <HOST_NAME>}

# apptags - the apptags that this log will be tagged to in XpoLog - there can be more than one tag comma separated.

# app - the folder in the Folders and Logs that this log(s) will be placed

# logname - the logname in XpoLog

# logname - the host name were the log of is originated from

 

# To add more than one log create another set of properties for the additional log

#- type: log

#enabled: true

#paths:

# - FULL_PATH_TO_LOGS_DIRECTORY/FILE

#fields: {apptags: "<APPTAG_NAME>",app: "<FOLDER_NAME>", logname: "<LOG_NAME>", filename: <HOST_NAME>}

 

filebeat.config.modules:

path: ${path.config}/modules.d/*.yml

reload.enabled: false

setup.template.settings:

index.number_of_shards: 3

output.logstash:

hosts: ["<LOGSTASH_HOSTNAME>:<PORT>"]

#Example:

# hosts: ["localhost:5044"]
# Note that 5044 is the default port set for Filebeat output

# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"

# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"