Skip to content

Commit 6f34350

Browse files
jariy17jariy17
andauthored
fix(docs): document lambda:InvokeFunction for code-based evaluators (#1891)
`run eval` against a code-based evaluator fails with AccessDeniedException for callers using the documented user policy: Access denied when invoking Lambda function: ... User: ... is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:...:function:<Agent>-eval-<evaluator> `bedrock-agentcore:Evaluate` invokes the evaluator's Lambda under the *caller's* identity, so the caller needs `lambda:InvokeFunction`. A resource-based policy on the function alone does not cover it. Neither iam-policy-user.json nor the PERMISSIONS.md Evaluation table granted or mentioned it, because until the DeepEval/Autoevals tests added in #1828 every eval path exercised was either `Builtin.*` or `llm-as-a-judge` -- neither of which touches Lambda. Scoped to `arn:*:lambda:*:*:function:*-eval-*` (partition wildcard per the multi-partition rules in AGENTS.md) rather than `*`. The same statement was added to the e2e-github-actions CI role, which had no lambda:InvokeFunction on any of its 9 policies -- that is what broke e2e shard 5 on main: https://github.com/aws/agentcore-cli/actions/runs/30663047263/job/91263525976 Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>
1 parent 8fba816 commit 6f34350

2 files changed

Lines changed: 13 additions & 0 deletions

File tree

docs/PERMISSIONS.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -381,8 +381,15 @@ These EC2 and EFS `Describe*` actions do not support resource-level scoping, so
381381
| Action | CLI Commands | Purpose |
382382
| ------------------------------------------------ | ----------------------------------------- | --------------------------------------------- |
383383
| `bedrock-agentcore:Evaluate` | `run evals` | Run on-demand evaluation against agent traces |
384+
| `lambda:InvokeFunction` | `run evals` | Invoke code-based evaluator Lambda functions |
384385
| `bedrock-agentcore:UpdateOnlineEvaluationConfig` | `pause online-eval`, `resume online-eval` | Pause or resume online evaluation |
385386

387+
`bedrock-agentcore:Evaluate` invokes a code-based evaluator's Lambda function using the **caller's** identity, so
388+
`lambda:InvokeFunction` is required on the caller when running `run eval` against a `code-based` evaluator (including
389+
DeepEval and Autoevals third-party evaluators). Evaluator functions are named `<AgentName>-eval-<evaluatorName>`, so the
390+
permission can be scoped to `arn:*:lambda:*:*:function:*-eval-*`. Builtin and `llm-as-a-judge` evaluators do not need
391+
this permission.
392+
386393
### Batch evaluation and recommendations
387394

388395
| Action | CLI Commands | Purpose |

docs/policies/iam-policy-user.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,12 @@
5656
],
5757
"Resource": "*"
5858
},
59+
{
60+
"Sid": "CodeBasedEvaluatorInvocation",
61+
"Effect": "Allow",
62+
"Action": "lambda:InvokeFunction",
63+
"Resource": "arn:*:lambda:*:*:function:*-eval-*"
64+
},
5965
{
6066
"Sid": "AgentCoreResourceStatus",
6167
"Effect": "Allow",

0 commit comments

Comments
 (0)