Update unique keys for latest issues transform to catch all updates

[clement-fouque]

Proposed commit message

This pull request updates the configuration for the GitHub issues Elasticsearch transform to improve how the latest state of issues is tracked and stored. The main focus is on making the transform more robust by using more reliable unique keys and bumping the transform version.

The previous transform wasn't taking into account all updates (e.g. adding a label to an issue). Since the field github.issues.updated_at changes each time the issue is updated, this field covers more use cases that the initial github.issues.created_at field. I've also changed the unique issue identifier from url to github.issues.id.

Key changes:

Transform logic improvements:

• Changed the unique_key fields in latest_issues/transform.yml to use github.issues.id and github.issues.updated_at instead of repository and URL fields. This ensures that the transform identifies the latest issue state more reliably and efficiently.

Versioning:

• Updated the _meta.fleet_transform_version from 1.0.0 to 2.0.0 to indicate a breaking change and trigger reinstallation and restart of the transform.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[clement-fouque]

I confirmed locally that the transform is working as expected.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[efd6]

This is not a refactor; it changes behaviour. Can we have a clearer title to explain what is being done?

[clement-fouque]

I’ve updated the title and PR description, and changed the version from ⁠2.18.0 to ⁠2.17.1. Initially, I thought this was a breaking change. However, since the transform is deleted and recreated, it’s not a breaking change after all.

@efd6, could you please review? Thanks.

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 4e23935

History

• 💚 Build #32923 succeeded cb5309b
• 💔 Build #32919 failed b096197