Add a rollover policy to the setup plugin #269
Draft
+1,927
−5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 tasks
Policy DocumentThe setup plugin has been modified to index a document to the {
"policy": {
"policy_id": "wazuh_rollover_policy",
"description": "Example rollover policy.",
"last_updated_time": 1738947466825,
"schema_version": 21,
"error_notification": null,
"default_state": "rollover",
"states": [
{
"name": "rollover",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"rollover": {
"min_doc_count": 1,
"copy_alias": false
}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"test-index-*"
],
"priority": 100,
"last_updated_time": 1738947466825
}
],
"user": {
"name": "admin",
"backend_roles": [
"admin"
],
"roles": [
"own_index",
"all_access"
],
"custom_attribute_names": [],
"user_requested_tenant": null
}
}
} |
Index Management index templateDuring testing we found out that our plugin was quicker to load than the Index Management one, so a template must be set up for the We took the mappings from it from here: |
Trigger the jobAn index needs to be written, which matches the index pattern the policy expects, while also being set up as the write index for the rollover alias: curl -XPUT http://localhost:9200/test-index-0000 -H 'Content-Type: application/json' -d '{"aliases":{"test-alias":{"is_write_index":true}}}' |
Check that indices are being rotatedIn order to speed up execution of the rollover policy, the following command can be issued: curl -XPUT http://localhost:9200/_cluster/settings?pretty=true -H'Content-Type: application/json' -d '{"persistent": {"plugins.index_state_management.job_interval":1}}'Now we can index new commands pointing towards our index alias: curl -XPOST http://localhost:9200/test-alias/_doc -H 'Content-Type: application/json' -d '{"field":"value"}' |
Check the policy is reckonedWe can now check whether the ISM plugin is recognizing our policy and will apply it to the right index $ curl 'http://localhost:9200/_plugins/_ism/explain?pretty'
{
"test-index-0000" : {
"index.plugins.index_state_management.policy_id" : "wazuh_rollover_policy",
"index.opendistro.index_state_management.policy_id" : "wazuh_rollover_policy",
"index" : "test-index-0000",
"index_uuid" : "v6dYvuxlS9mJ3DE_78-IyA",
"policy_id" : "wazuh_rollover_policy",
"enabled" : true
},
"total_managed_indices" : 1
}
|
6 tasks
…plugins into 591-rollover-policy
…usly at the creation of the policy. Create the policy previously of the creation of wazuh-commands and wazuh-alerts
…hen rollover them.
|
When changing the OpenSearch version from 2.18.0 to 2.19.1 the solution stops working. At first the template was not configured in .opendistro-ism-config and generated problems with the mapping. Configuring the mapping directly in the .opendistro-ism-config index creation solves that problem. Although when the mapping started working the wazuh-command index and the wazuh-alerts index stopped being managed by the previously created policy. I think probably the problem is associated with the cluster reload process and this logs: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is meant as a proof of concept that a rollover policy can be set up from a plugin, outside the Index Management plugin itself.
It does so by writing a policy
jsonto the.opendistro-ism-configmuch like the Index Management plugin itself would do.Issues Resolved
wazuh/wazuh-indexer#591