Skip to content

chore(ci): fix sandbox agent smoke image handoff and retries - #106811

Merged
trunk-io[bot] merged 5 commits into
masterfrom
chore/tasks-sandbox-smoke-image
Sep 25, 2026
Merged

trunk-io[bot] merged 5 commits into
masterfrom
chore/tasks-sandbox-smoke-image

Conversation

@tatoalo

@tatoalo tatoalo commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Automated sandbox agent releases stop before gateway validation, as in #106779.

The failed run requests the PR head tag, but the image build publishes the merge-commit tag. Retrying an existing PR skips candidate validation.

Changes

  • Both smoke stages use the image digest from the successful build attempt, including partial reruns.
  • Manual dispatch resumes an existing automated, pin-only PR and checks the expected agent version before gateway calls.
  • Scheduled runs keep checking the deployed agent. Approval still requires successful smoke tests and an unchanged PR head.

Before

flowchart LR
    B[Build and smoke merge-SHA image] --> C[Successful PR-head check]
    C --> P[Pull head-SHA image] --> F[Missing image]
    R[Retry existing PR] --> S[Skip candidate smoke]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phRed fill:#f54e00,stroke:#f54e00,color:#fff;
    class B,C,P,R phBlue;
    class F,S phRed;
Loading

After

flowchart LR
    R[New bump or manual retry] --> V[Validate pin-only PR]
    V --> B[Successful build attempt] --> D[Image digest]
    D --> G[Gateway smoke and version check] --> A[Approval guard]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phGray fill:#e5e7eb,stroke:#c7ccd1,color:#000;
    classDef phYellow fill:#f9bd2b,stroke:#f9bd2b,color:#000;
    class R,V,B,G phBlue;
    class D phGray;
    class A phYellow;
Loading

@tatoalo tatoalo added the skip-inkeep-docs Use this label to skip an Inkeep docs PR in posthog.com label Sep 25, 2026
@tatoalo tatoalo self-assigned this Sep 25, 2026
@trunk-io

trunk-io Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@tatoalo tatoalo added the security-review Request a security review label Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

🚨 Trunk lane — universal lane

This PR is assigned to the universal lane. It cannot merge in parallel with other PRs, so it can take longer to merge. Ask dev-ex if you think this is wrong.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

@posthog-security-review-bot

posthog-security-review-bot Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 8a7adb2203fd

2 findings posted as a review, in 19 min.

Add the security-review label to run again on the latest commit.

@tatoalo
tatoalo marked this pull request as ready for review September 25, 2026 15:13
@tatoalo
tatoalo requested a review from a team September 25, 2026 15:13
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team September 25, 2026 15:14
@greptile-apps

greptile-apps Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Retrigger

[High risk] Changes CI workflows and build infrastructure for sandbox agent.

The PR should not merge until digest resolution handles builds carried forward during partial reruns.

Reviews (1) · Last reviewed commit: "chore(ci): fix sandbox agent smoke image..."

Comment thread .github/workflows/update-sandbox-agent-version.yml Outdated
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 4640e3dd-e983-4d44-b50f-10c783a3ee8a

📥 Commits

Reviewing files that changed from the base of the PR and between 9a88836 and e2bad29.

📒 Files selected for processing (2)
  • .github/scripts/test_sandbox_agent_release.py
  • .github/workflows/update-sandbox-agent-version.yml

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.


📝 Walkthrough

Walkthrough

The wait-for-check action now outputs the details URL for the selected completed check. The base-image workflow records pull-request build metadata and checks platform-specific image digests. The agent-version workflow validates the related build artifact and passes its digest-pinned image to the gateway smoke check. New tests cover check outputs, artifact validation, platform digests, and workflow planning.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to e2bad

The image handoff is consistent in the covered rerun scenarios. No actionable merge-blocking issue is established beyond normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to e2bad

The new digest handoff adds a release-control dependency, but the inspected path checks the build’s identity and stops before approval when validation or smoke testing fails. An existing PR still does not appear to resume candidate validation on manual dispatch, despite that being a stated goal; this change does not establish a new approval bypass.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The security-relevant exposure is the automated sandbox release path: the selected image reaches gateway smoke, while the existing release approval token is acquired only on the bump path. No new production ingress was identified in the changed test harness.

Trust Boundaries and Controls

  • observed — Before an artifact-derived image is used, the workflow checks repository, branch, workflow, PR, and head identity. The gateway action verifies the installed agent version before requesting a scoped smoke token; approval rechecks the live PR head.

Resilience and Maintainability Implications

  • observed — Tests exercise mismatched run metadata, invalid digests, download failures, and partial reruns. Approval-time head changes and check-job identity fields are guarded in workflow source but are not independently varied by the focused tests.

Hardening Proposals

  • proposed — If manual recovery of an existing pin-only PR is required, route it through the same identity, image-smoke, gateway-smoke, and unchanged-head approval guards rather than treating dispatch alone as validation.
🚥 Pre-merge checks | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description clearly explains the problem, user-visible changes, and CI flow before and after the change. It omits the required testing details, release-status selection, automatic-notification and… Add the missing template sections. Document the automated tests that were run and their regression coverage, select exactly one release-status option, state the changelog and docs-update decisions, and complete the Agent context section or …
Full details: Description check

Explanation

The description clearly explains the problem, user-visible changes, and CI flow before and after the change. It omits the required testing details, release-status selection, automatic-notification and docs-update entries, and agent context, including agent-authorship and validation information.

Resolution

Add the missing template sections. Document the automated tests that were run and their regression coverage, select exactly one release-status option, state the changelog and docs-update decisions, and complete the Agent context section or remove it if no agent contributed.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - findings inline.

Comment thread .github/workflows/update-sandbox-agent-version.yml Outdated
Comment thread .github/workflows/update-sandbox-agent-version.yml
@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Sep 25, 2026
@trunk-io
trunk-io Bot merged commit 771bd3d into master Sep 25, 2026
238 checks passed
@trunk-io
trunk-io Bot deleted the chore/tasks-sandbox-smoke-image branch September 25, 2026 17:21
@deployment-status-posthog

deployment-status-posthog Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-09-25 18:06 UTC Run
prod-us ✅ Deployed 2026-09-25 18:19 UTC Run
prod-eu ✅ Deployed 2026-09-25 18:19 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-inkeep-docs Use this label to skip an Inkeep docs PR in posthog.com

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants