Table of Contents

, , , ,

logstash pipelines

If you want to add multiple config files, use the pipelines.

Edit pipelines.yml and add the path to configs there e.g.

- pipeline.id: web_log_pipelined
  queue.type: persisted
  path.config: "/etc/logstash.d/web.conf"
- pipeline.id: log2
  queue.type: persisted
  path.config: "/etc/logstash.d/otherlog.conf"
- pipeline.id: log3
  queue.type: persisted
  path.config: "/etc/logstash.d/errors.conf"
...

Now you need to start logstash without any arguments, simply execute logstash, just make sure you are pointing to the right config. You can use LS_SETTINGS_DIR env variable to point to the config directory.

LS_SETTINGS_DIR=/etc/logstash.d/ logstash

Example systemd service file:

# /etc/systemd/system/logstash.service
[Unit]
Description=Massage various logs and forward them to opensearch/elasticsearch
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/logstash
Environment="HOSTNAME={{ hostname }}"
Environment="LS_SETTINGS_DIR=/etc/logstash.d/"
Restart=on-failure

[Install]
WantedBy=multi-user.target

Tested on

See also

References