Skip to content

test(toolpolicy): cover deny audit records - #84

Open
XnLemon wants to merge 1 commit into
mainfrom
codex/phase2-tool-deny-audit
Open

test(toolpolicy): cover deny audit records#84
XnLemon wants to merge 1 commit into
mainfrom
codex/phase2-tool-deny-audit

Conversation

@XnLemon

@XnLemon XnLemon commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Objective

Tighten Phase2 acceptance coverage for unauthorized tool denial audit logs.

Changes

  • Added a direct regression test for whitelist-based tool denial through CheckToolPermission.
  • Verified the deny decision writes one audit record with tenant/app/channel/binding/session/internal user/request/agent/tool/decision context.
  • Verified RedactedDetailRef contains only safe digest/bytes/decision details and does not leak raw tool args, Authorization headers, tokens, or paths.

Validation

  • go test ./platform/toolpolicy ./platform
  • go vet ./platform/toolpolicy ./platform
  • git diff --check
  • open-code-review subagent/manual review: 0 High / 0 Medium / 0 Low

Risks / Follow-up

  • This increment is test-only and focuses on the direct whitelist deny path; remaining Phase2 closure items will continue in follow-up fork-only PRs.

@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7064205f-8795-4b14-9689-12a9500186ea

📥 Commits

Reviewing files that changed from the base of the PR and between ffe5fe3 and a5f16ec.

📒 Files selected for processing (1)
  • platform/toolpolicy/policy_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • platform/toolpolicy/policy_test.go

📝 Walkthrough

English

  • Overview: Adds regression coverage for whitelist-based tool denial through CheckToolPermission. The test verifies one audit record with the expected tenant, application, channel, binding, session, user, request, agent, tool, and decision fields, while confirming RedactedDetailRef exposes only safe decision, digest, and byte-count data.
  • Compatibility and risks: Test-only change with no production behavior or API compatibility impact. It guards against future audit omissions or accidental leakage of tool arguments, authorization headers, tokens, or paths in denial records.
  • Recommended validation: Run the package tests, go vet, diff checks, and manual review. Confirm the test covers the direct whitelist-denial path and that redacted details remain free of sensitive input values.
中文
  • 变更概述:CheckToolPermission 增加基于白名单拒绝工具调用的回归测试。测试验证审计记录仅写入一条,并包含正确的租户、应用、频道、绑定、会话、用户、请求、代理、工具和决策信息;同时确认 RedactedDetailRef 只包含安全的决策、摘要和字节数数据。
  • 兼容性与行为风险: 这是仅测试代码变更,不影响生产行为或 API 兼容性。该测试可防止拒绝审计记录缺失,或意外泄露工具参数、授权请求头、令牌和路径。
  • 建议验证: 运行相关包测试、go vet 和 diff 检查,并进行人工审查。确认测试覆盖直接的白名单拒绝路径,且脱敏详情不包含任何敏感输入值。

Walkthrough

Adds a test covering denial of a non-whitelisted tool, audit record identity fields, redaction versioning, and exclusion of sensitive argument content from the audit detail reference.

Changes

Tool policy audit behavior

Layer / File(s) Summary
Whitelist denial and redacted audit assertions
platform/toolpolicy/policy_test.go
Adds an in-memory audit test that verifies the denial reason, emitted audit fields, redaction version, safe markers, and absence of raw sensitive arguments.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the test-only change: adding coverage for deny audit records in tool policy.
Description check ✅ Passed The description is directly aligned with the changeset and explains the new regression test and validation steps.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/phase2-tool-deny-audit

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@platform/toolpolicy/policy_test.go`:
- Around line 79-114: Strengthen the audit record assertions in the
CheckToolPermission test by validating record.UserIDHash against the expected
platform.UserIDHash("tenant", "wecom", "external-user") value. Keep the existing
context and other audit field checks unchanged, ensuring the hashed user
identity is verified without accepting an empty or raw external user ID.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3ee1b1bf-71e8-4604-8725-09640cbad031

📥 Commits

Reviewing files that changed from the base of the PR and between 170433e and ffe5fe3.

📒 Files selected for processing (1)
  • platform/toolpolicy/policy_test.go

Comment on lines +79 to +114
decision, err := p.CheckToolPermission(
ContextWithAuditContext(context.Background(), AuditContext{
Channel: "wecom",
BindingID: "binding",
SessionID: "session",
InternalUserID: "internal-user",
UserIDHash: platform.UserIDHash("tenant", "wecom", "external-user"),
RequestID: "request-1",
AgentName: "agent",
}),
request("shell", tool.ToolMetadata{}, []byte(`{"command":"rm -rf /tmp/demo","Authorization":"Bearer raw-token"}`)),
)
if err != nil {
t.Fatalf("CheckToolPermission: %v", err)
}
if decision.Action != tool.PermissionActionDeny {
t.Fatalf("expected deny, got %+v", decision)
}

records := audit.Records()
if len(records) != 1 {
t.Fatalf("expected one audit record, got %+v", records)
}
record := records[0]
if record.TenantID != "tenant" ||
record.AppID != "app" ||
record.Channel != "wecom" ||
record.BindingID != "binding" ||
record.SessionID != "session" ||
record.InternalUserID != "internal-user" ||
record.RequestID != "request-1" ||
record.AgentName != "agent" ||
record.ToolName != "shell" ||
record.Decision != string(tool.PermissionActionDeny) {
t.Fatalf("unexpected audit record: %+v", record)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assertions don't verify UserIDHash, despite it being set in the test's AuditContext.

The test populates UserIDHash via platform.UserIDHash(...) (Line 85), and applyAuditContext is known to copy it onto the audit record, but the validation block (Lines 103-114) never checks record.UserIDHash. This leaves a real regression path (e.g. hash function change, empty-string leak, or accidental exposure of the raw external user ID) uncaught. The PR objective explicitly calls out "user" as part of the expected audit context.

As per path instructions, "Whether assertions are strong enough to ensure the intended behavior is actually verified" should be checked for **/*_test.go files.

🐛 Proposed fix
 	if record.TenantID != "tenant" ||
 		record.AppID != "app" ||
 		record.Channel != "wecom" ||
 		record.BindingID != "binding" ||
 		record.SessionID != "session" ||
 		record.InternalUserID != "internal-user" ||
+		record.UserIDHash != platform.UserIDHash("tenant", "wecom", "external-user") ||
 		record.RequestID != "request-1" ||
 		record.AgentName != "agent" ||
 		record.ToolName != "shell" ||
 		record.Decision != string(tool.PermissionActionDeny) {
 		t.Fatalf("unexpected audit record: %+v", record)
 	}
中文

断言未验证 UserIDHash,尽管测试的 AuditContext 中已设置该值。

测试在第 85 行通过 platform.UserIDHash(...) 设置了 UserIDHash,且已知 applyAuditContext 会将其复制到审计记录,但第 103-114 行的校验块从未检查 record.UserIDHash。这会导致真实的回归路径(例如哈希函数变更、空字符串泄漏,或意外暴露原始外部用户 ID)无法被捕获。PR 目标中明确提到"user"是预期审计上下文的一部分。

根据路径指令,**/*_test.go 文件应检查"断言是否足够严格以确保预期行为得到实际验证"。

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
decision, err := p.CheckToolPermission(
ContextWithAuditContext(context.Background(), AuditContext{
Channel: "wecom",
BindingID: "binding",
SessionID: "session",
InternalUserID: "internal-user",
UserIDHash: platform.UserIDHash("tenant", "wecom", "external-user"),
RequestID: "request-1",
AgentName: "agent",
}),
request("shell", tool.ToolMetadata{}, []byte(`{"command":"rm -rf /tmp/demo","Authorization":"Bearer raw-token"}`)),
)
if err != nil {
t.Fatalf("CheckToolPermission: %v", err)
}
if decision.Action != tool.PermissionActionDeny {
t.Fatalf("expected deny, got %+v", decision)
}
records := audit.Records()
if len(records) != 1 {
t.Fatalf("expected one audit record, got %+v", records)
}
record := records[0]
if record.TenantID != "tenant" ||
record.AppID != "app" ||
record.Channel != "wecom" ||
record.BindingID != "binding" ||
record.SessionID != "session" ||
record.InternalUserID != "internal-user" ||
record.RequestID != "request-1" ||
record.AgentName != "agent" ||
record.ToolName != "shell" ||
record.Decision != string(tool.PermissionActionDeny) {
t.Fatalf("unexpected audit record: %+v", record)
}
decision, err := p.CheckToolPermission(
ContextWithAuditContext(context.Background(), AuditContext{
Channel: "wecom",
BindingID: "binding",
SessionID: "session",
InternalUserID: "internal-user",
UserIDHash: platform.UserIDHash("tenant", "wecom", "external-user"),
RequestID: "request-1",
AgentName: "agent",
}),
request("shell", tool.ToolMetadata{}, []byte(`{"command":"rm -rf /tmp/demo","Authorization":"Bearer raw-token"}`)),
)
if err != nil {
t.Fatalf("CheckToolPermission: %v", err)
}
if decision.Action != tool.PermissionActionDeny {
t.Fatalf("expected deny, got %+v", decision)
}
records := audit.Records()
if len(records) != 1 {
t.Fatalf("expected one audit record, got %+v", records)
}
record := records[0]
if record.TenantID != "tenant" ||
record.AppID != "app" ||
record.Channel != "wecom" ||
record.BindingID != "binding" ||
record.SessionID != "session" ||
record.InternalUserID != "internal-user" ||
record.UserIDHash != platform.UserIDHash("tenant", "wecom", "external-user") ||
record.RequestID != "request-1" ||
record.AgentName != "agent" ||
record.ToolName != "shell" ||
record.Decision != string(tool.PermissionActionDeny) {
t.Fatalf("unexpected audit record: %+v", record)
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@platform/toolpolicy/policy_test.go` around lines 79 - 114, Strengthen the
audit record assertions in the CheckToolPermission test by validating
record.UserIDHash against the expected platform.UserIDHash("tenant", "wecom",
"external-user") value. Keep the existing context and other audit field checks
unchanged, ensuring the hashed user identity is verified without accepting an
empty or raw external user ID.

Source: Path instructions

@XnLemon
XnLemon force-pushed the codex/phase2-redactor-spaced-secrets branch from 170433e to fb85850 Compare July 11, 2026 13:38
@XnLemon
XnLemon force-pushed the codex/phase2-tool-deny-audit branch from ffe5fe3 to 75002b5 Compare July 11, 2026 13:38
@XnLemon
XnLemon changed the base branch from codex/phase2-redactor-spaced-secrets to main July 13, 2026 02:23
@XnLemon
XnLemon force-pushed the codex/phase2-tool-deny-audit branch from 75002b5 to a5f16ec Compare July 13, 2026 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants