Table of Contents
Rollover opensearch/elasticsearch indexes
Datastream indexes
In index management create a new policy. Most important is to match the index pattern with the name of your datastreams e.g. below we see a regex that will match datastreams as defined in Index Management > Data streams
In the policy you can define what will be done with the matching indexes, in this case the backing indexes of those datastreams. Here's an example JSON policy that changes state and rolls over to a new backing index after 20GB:
{
"id": "rollover_20GB",
"seqNo": 36234,
"primaryTerm": 1,
"policy": {
"policy_id": "rollover_20GB",
"description": "Rollover to a new index in datastream after the backing index reaches 20GB.",
"last_updated_time": 1725353927700,
"schema_version": 21,
"error_notification": null,
"default_state": "check index size",
"states": [
{
"name": "check index size",
"actions": [],
"transitions": [
{
"state_name": "rollover",
"conditions": {
"min_size": "20gb"
}
}
]
},
{
"name": "rollover",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"rollover": {
"min_size": "20gb",
"copy_alias": false
}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"logstash-nginx-access-logs-live*"
],
"priority": 10,
"last_updated_time": 1725021614800
}
]
}
}
The datastreams rely on a template (see nginx_logs_using_datastreams) which should have the advanced setting set to the name of the backing index e.g.
"index.plugins.index_state_management.rollover_alias": ".ds-logstash-nginx-*"
If you now go to Policy managed indexes you should see the indices from the data streams:
1 and 2 shows the state the index is in defined by whatever the names are defined in above policy
Here we see a successful rollover to a new index
Also if you go to Data streams and into one of the indexes you should the policy attached there
Any subsequent index should also be managed automatically now.
The docs mention that “The ISM policy is applied to the backing indexes at the time of their creation. When you associate a policy to a data stream, it only affects the future backing indexes of that data stream.” meaning you will need to manually rollever the indexes the first time to apply the policy → Select the datastreams then Actions → Roll over
Tested on
- Opensearch 2.15