antisaWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{tag>backup elasticsearch}}

====== Backup Elasticsearch indices ======
Below should work on bare metal and inside ES docker installation.
The snapshots are done using the ES API. <wrap em>Raw system file backups won't work.</wrap>
Test below was done via docker on Debian 11 provisioned by Vagrant.

===== Add path repo where the snapshots will be restored =====
Create the repo path:

  [root@97baaa7e0134 elasticsearch]# cat config/elasticsearch.yml 
  cluster.name: "docker-cluster"
  network.host: 0.0.0.0
  path.repo: /usr/share/elasticsearch/data/ # add this path

Restart ES container.

===== Register repository =====
<code>
curl -X PUT "localhost:9200/_snapshot/my_repository?pretty" -H 'Content-Type: application/json' -d'
{
  "type": "fs",
  "settings": {
    "location": "/usr/share/elasticsearch/data/backups"
  }
}
'
</code>

Above paths need to be created and owned by 'elasticsearch' user.

===== Manually create snapshot =====

  curl -X PUT "localhost:9200/_snapshot/my_repository/my_snapshot?wait_for_completion=true&pretty"


===== List all snapshots =====

  curl -X GET "localhost:9200/_snapshot/my_repository/_all?pretty"

===== Copy snapshot outside of container =====

  docker cp root-elasticsearch-1:/usr/share/elasticsearch/data/backups /vagrant/

You should also copy the configuration folder above (where //path.repo// is defined among other things)
  docker cp root-elasticsearch-1:/usr/share/elasticsearch/config /vagrant/

===== Restoring snapshots =====

Copy the files from snapshot repository back into docker container
  docker cp /vagrant/backups root-elasticsearch-1:/usr/share/elasticsearch/data/

Register the repository again as above

Before restoring you might need to delete the indexes if they already exist. This is especially the case when moving to new cluster with default docker installation and default indexes. 

To list the indexes do 
  curl -X GET "localhost:9200/_all?pretty"
  
===== If the Graylog is using ES =====

Stop the graylog docker then you can delete the indices with

  curl -X DELETE "localhost:9200/graylog_0,gl-system-events_0,gl-events_0?pretty"

This deletes the 3 default indices.

===== Final restore =====

List the snapshot and chose the one you want to restore. E.g. restoring "my_snapshot2"

  curl -X POST "localhost:9200/_snapshot/my_repository/my_snapshot2/_restore?wait_for_completion=true&pretty"

====== Tested on ======
  * Debian 11 Bullseye (vagrant)
  * docker_compose_version: v2.2.3
  * graylog_version: 4.2-jre11
  * mongodb_version: 4.2
  * elastic_search_version: 7.10.2

====== See also ======
  * [[Graylog backup and restore]]
====== References ======
  * https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-register-repository.html
  * https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-take-snapshot.html#manually-create-snapshot