antisaWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{tag>logstash elasticsearch elk opensearch logging}}

====== logstash pipelines ======
If you want to add multiple config files, use the pipelines.

Edit ''pipelines.yml'' and add the path to configs there e.g.

<code>
- pipeline.id: web_log_pipelined
  queue.type: persisted
  path.config: "/etc/logstash.d/web.conf"
- pipeline.id: log2
  queue.type: persisted
  path.config: "/etc/logstash.d/otherlog.conf"
- pipeline.id: log3
  queue.type: persisted
  path.config: "/etc/logstash.d/errors.conf"
...
</code> 
Now you need to start logstash without any arguments, simply execute ''logstash'', just make sure you are pointing to the right config. You can use **LS_SETTINGS_DIR** env variable to point to the config directory.

  LS_SETTINGS_DIR=/etc/logstash.d/ logstash

Example systemd service file:

<code>
# /etc/systemd/system/logstash.service
[Unit]
Description=Massage various logs and forward them to opensearch/elasticsearch
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/logstash
Environment="HOSTNAME={{ hostname }}"
Environment="LS_SETTINGS_DIR=/etc/logstash.d/"
Restart=on-failure

[Install]
WantedBy=multi-user.target
</code>
====== Tested on ======
  * logstash-8.14.3 
  * OpenJDK 64-Bit Server VM 17.0.11+9

====== See also ======
  * [[wiki:logstash_setup|Logstash setup]]

====== References ======
  * https://discuss.elastic.co/t/pipelines-yml-ignored-on-logstash/314807
  * https://discuss.elastic.co/t/logstash-multiple-file-inputs/36397