[amazon_bedrock_agentcore] add runtime_application_logs data stream

[gpop63]

Overview

This PR adds the runtime_application_logs data stream. It also updates the agent runtime dashboard with a few visualizations for logs.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Closes: elastic/obs-integration-team/issues/685

Screenshots

[agithomas]

Suggested mapping for an input such as

{
    "resource_arn": "arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/example-agent-XXXXXXXXXX",
    "event_timestamp": 1763356735263,
    "account_id": "123456789012",
    "request_id": "d904a611-812d-43bd-9154-29dffe16090c",
    "session_id": "142e241a-567a-4ea1-9c8a-e1a0c1e5bcbd",
    "span_id": "c582eb7f19b33c86",
    "trace_id": "a3962c1ef0081a3724b65af6e44632d0",
    "service_name": "AgentCoreCodeRuntime",
    "operation": "InvokeAgentRuntime",
    "request_payload": {
        "prompt": "[REDACTED_USER_PROMPT]",
        "actor_id": "DEFAULT"
    }
}

as

{
  "@timestamp": "2025-11-25T15:25:35.263Z",
  "gen_ai": {
    "system": "aws_bedrock_agentcore",
    "operation": {
      "name": "InvokeAgentRuntime"
    },
    "provider": {
      "name": "aws"
    },
    "conversation": {
      "id": "142e241a-567a-4ea1-9c8a-e1a0c1e5bcbd"
    },
    "prompt": "[REDACTED_USER_PROMPT]"
  },
  "aws": {
    "bedrock_agentcore": {
      "request_id": "d904a611-812d-43bd-9154-29dffe16090c",
      "resource_arn": "arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/example-agent-XXXXXXXXXX",
      "account_id": "123456789012",
      "actor_id": "DEFAULT"
    }
  },
  "trace": {
    "id": "a3962c1ef0081a3724b65af6e44632d0"
  },
  "span": {
    "id": "c582eb7f19b33c86"
  },
  "service": {
    "name": "bedrock-agentcore"
  },
  "event": {
    "action": "InvokeAgentRuntime",
    "category": ["api"],
    "type": ["access"]
  },
  "cloud": {
    "provider": "aws",
    "service": {
      "name": "bedrock"
    },
    "account": {
      "id": "123456789012"
    }
  }
}

[agithomas]

1. Under the logs overview section, the most useful information is gen_ai.prompt value. Could you include it? The ordering of the columns could be

• timestamp
• Resource ARN
• gen_ai.prompt
• request_id
• session_id
• trace_id

2. The request_id, session_id, trace_id are values having high cardinality. I am doubtful about the practical benefit in having the following panels

• Reqeuest by Session ID
• Requests for RequestID
• Requests by Session ID

3, Request by ResourceARN may not be needed too, as the metrics section of the dashboard is already covering this part.

[agithomas]

Please rename Runtime logs as Application logs. Agentcore runtime has two logs - Application logs and runtime logs. This dataset covers Application logs, not runtime logs.
Please remove the panel title - Logs overview.

[agithomas]

Do you think, it will be good to mention the steps needed to fetch the log_group_arn, needed for configuring this dataset?

As it is the same steps for all the log-based datasets, maybe a common section (may be under What do I need to use this integration?) should suffice.

[gpop63]

I've added a small section to explain how to get the log group arn 👍

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[agithomas]

Please add support for S3 as the log data source.

[agithomas]

Please fix the PR link

[agithomas]

Kindly update the dashboard screenshots as well.

[gpop63]

Kindly update the dashboard screenshots as well.

@agithomas when I try to get metrics there's no data, only the logs lenses get data so the dashboard will look empty.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: cb1d145

History

• 💚 Build #34382 succeeded a404df2
• 💚 Build #34377 succeeded 391937f
• 💚 Build #34364 succeeded 67dce02
• 💔 Build #34322 failed 5c2712d
• 💔 Build #34301 failed 05b69ff
• 💔 Build #34288 failed e7bb9a9

cc @gpop63