Skip to content

Persist exact workflow revisions additively - #6960

Draft
loganj wants to merge 5 commits into
mainfrom
larry/workflow-revision-foundation-v4
Draft

Persist exact workflow revisions additively#6960
loganj wants to merge 5 commits into
mainfrom
larry/workflow-revision-foundation-v4

Conversation

@loganj

@loganj loganj commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

🤖 I’m Larry. This is the additive foundation for recording the exact signed workflow definition revision selected when a run begins.

Summary

  • Preserve the signed definition event ID when a workflow is newly created or updated.
  • Copy the selected revision ID into new runs across event, schedule, webhook, and manual/bridge paths.
  • Keep unknown legacy provenance nullable. Existing workflows and historical runs are not backfilled by matching present-day content.

Details

Workflow definitions are materialized into database rows today, but those rows do not retain the exact signed event that produced them. This change adds nullable, 32-byte revision IDs to definitions and runs so subsequent manual-trigger and agent-wake features can verify the owner-signed instructions selected for a run.

Signed-event persistence and workflow materialization commit in one transaction. A database trigger clears the workflow binding on every materialization update—including equal-value writes from older pods. New ingest rebinds within the same transaction while retaining the row lock, so readers cannot observe an intermediate binding. Operational status/enabled changes preserve it; existing runs retain their selected revision even when a later legacy write clears the workflow's binding.

Compatibility and scope

  • Migration 0041 adds the nullable columns, constraints, and invalidation trigger. Desired-state schema contains the same invariants.
  • Unknown legacy workflow and run provenance stays NULL. Automatic historical inference was retired in #6963; it is not an anticipated follow-up.
  • No new trigger or delivery feature is introduced here. #6964 adds owner-authorized manual triggering; #6965 adds verified agent mention wakes.
  • Existing execution remains compatible with NULL revision IDs.

The permanent PostgreSQL CI gate covers legacy-write invalidation, atomic rebinding/visibility, row-lock exclusion, rollback, and retention of already-created run revisions.

Originating Buzz workstream: buzz://message?channel=11ae68c1-38e7-4591-958e-e4da94d758f0&id=ee768d6e6f981d9450e2afc727a632d99d9480bbbe4399995332c5dd19e12c4c

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is c432a111ca9ddd31a85e1312d5995f8b92191b82...bb7de1e37872746cec63a1c7b1feda4a242b32b9.
A new review must complete for this exact range. When manual authorization
is required, a Block organization member must comment exactly
@buzz-security-review bb7de1e37872746cec63a1c7b1feda4a242b32b9 to authorize a new review.
Any previous review applies only to its recorded range.

@loganj
loganj marked this pull request as ready for review August 27, 2026 21:15
@loganj
loganj requested a review from a team as a code owner August 27, 2026 21:15
@loganj
loganj marked this pull request as draft August 28, 2026 16:54
@loganj
loganj force-pushed the larry/workflow-revision-foundation-v4 branch from ca2c16c to e191591 Compare August 28, 2026 16:56
@loganj

loganj commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 I’m Larry. The mixed-version rollout blocker is fixed at e191591780ea75499a5790053bec0384a16ce169.

A column-targeted database trigger now clears workflows.definition_event_id on every materialization update, including equal-value predecessor upserts and the sibling update_workflow path. New ingest rebinds in the existing signed-event transaction while holding the row lock. Operational status/enabled updates preserve the pointer; already-created runs retain their selected revision. A later legacy write clears the workflow pointer, so subsequent runs use NULL compatibility rather than false provenance.

Rebased onto main 58cc4b7e9be71223f7898ea5f930a4248830ccfb; the additive migration is now 0040, after main’s operator migrations 35–39. This changes no protected PR branches.

Exact-head Blox evidence: workstation larry-buzz-foundation-rollout / 2169747; fmt and all-target clippy for DB/workflow/relay passed. Full package suites: DB 113 + 1, workflow 169, relay 994 + 13, all passing with their existing ignored tests reported. All 11 workflow Postgres tests also passed on both databases built from migrations and from actual pgschema apply. These include changed/equal-value legacy materialization, new-run NULL compatibility, retained historical run bindings, sibling updates, operational updates, transaction visibility, row-lock exclusion, and rollback.

Downloaded evidence: WORK_LOGS/FOUNDATION_ROLLOUT_20260828/FINAL.log, SHA-256 9cefe1b826331fb11a48f208e9efdd59ccef3d98a724e19679f3469310a8db47. Local publication skipped Rust/Tauri test hooks using that Blox matrix; branch-skew, file-size, push-head, and DCO checks remained enabled.

Not merge-cleared: restored draft status; fresh Royals review and GitHub CI are pending. The previous approval/green state does not establish this candidate’s clearance. The PR body’s anticipated automatic historical migration is not a delivery commitment: historical matching cannot establish provenance across the predecessor’s separate materialization/event commits. Legacy rows remain NULL until new signed ingest.

@loganj

loganj commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 I’m Larry. Fixed the Royals CI-gate finding at 53b92fc6d0e5ad0ae28617aebe75f491cb73c276: Backend Integration runs all 11 workflow DB tests from its existing nextest archive and PostgreSQL fixture. No Rust or SQL changed after the rollout review. The exact archive selector and full DB/workflow/relay matrix passed on Blox, all three Royals reviewers cleared this head with no findings, and all applicable GitHub CI checks are now green—including the new PostgreSQL step. No unresolved review threads remain. The PR stays draft: GitHub still requires approval, and the separate Codex security review is stale/awaiting exact-head authorization; the ordinary Security job passing does not satisfy that separate review.

@loganj

loganj commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

@buzz-security-review 53b92fc

@loganj
loganj force-pushed the larry/workflow-revision-foundation-v4 branch from 53b92fc to 8368014 Compare August 28, 2026 21:01
@loganj

loganj commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 I’m Larry. Rebased onto main a3730784f and published exact head 8368014b324e98dab2b513d7f421f6df1cd978e3.

The conflict came from #6987's runtime/store extraction. I kept its crate-root facade unchanged, applied the revision/transaction arguments to the relocated Db workflow wrappers, moved the regression module beside its store owner, and updated the permanent nextest selector to store::workflow::. SQL, trigger, relay ingest, and run-capture behavior are unchanged from the reviewed candidate. I inspected the integration delta; this does not restart the completed Royals feature review.

Exact-head Blox evidence on larry-buzz-foundation-rollout / 2169747: fmt and all-target DB/workflow/relay clippy passed; full package tests DB 113+2, workflow 169, relay 994+13 passed. All 11 workflow PostgreSQL cases passed against both the existing migrated and pgschema-created databases, and the corrected archive selector ran 11/11. Local log SHA-256: 3738c252b741dcd3b127627701c860f8836cc5a29edafe26e586bbfe2e7cbd28 (WORK_LOGS/FOUNDATION_ROLLOUT_20260828/REBASE_VALIDATION.log). Publication skipped only the Blox-validated Rust/Tauri test lanes; time-dependent and safety guards passed. Final-head GitHub CI is running.

Correction to my previous status comment: Codex's stale-review notice is advisory, not one of main's required checks. Both manually authorized attempts timed out without a verdict; that is neither a security finding nor a mandatory merge blocker. The actual required Security check is distinct. I am carrying this PR through final-head required checks for human review, not waiting on advisory automation or human approval to call it review-ready.

loganj and others added 5 commits August 28, 2026 17:26
Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Signed-off-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Clear bindings at the database materialization boundary, even on equal-value legacy updates. Rebind inside atomic signed ingest while holding the row lock.

Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Signed-off-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Signed-off-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Signed-off-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Signed-off-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj
loganj force-pushed the larry/workflow-revision-foundation-v4 branch from 8368014 to bb7de1e Compare August 28, 2026 21:30
@loganj

loganj commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 I’m Larry. Main advanced again while the previous candidate completed CI. Replacement head bb7de1e37872746cec63a1c7b1feda4a242b32b9 is now rebased onto c432a111ca9ddd31a85e1312d5995f8b92191b82.

Main's push-message migration owns 0040; the unchanged workflow revision SQL is now 0041. Migration inventory assertions preserve both. No workflow behavior changed, and main's CI/Cargo.lock/schema changes remain intact.

Exact-head Blox validation on workstation 2169747 passed: fresh migrations 0001–0041 in order; fresh pgschema bootstrap plus reconciliation; fmt; all-target DB/workflow/relay clippy; full package tests DB 114+2, workflow 169, relay 994+13; all 11 workflow PostgreSQL tests on both fresh databases; archive-selector 11/11. Evidence: MAIN2_VALIDATION.log, SHA-256 4f06452bf13cf0503719dd335b0ae67047d2677dfcdc03df0303fd2da0ad4c6d. The publication skipped only validated Rust/Tauri test hooks; skew and safety guards passed. Remote SHA verified. New-head CI is running; descendant owners have the replacement foundation and are restacking in parallel. Prior feature review remains applicable; this integration delta was inspected without restarting the whole review cycle.

@loganj

loganj commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 I’m Larry. Ready for human review at bb7de1e37872746cec63a1c7b1feda4a242b32b9.

Final-head GitHub checks are complete: 30 successful, 10 skipped, none pending or failing. CI run 33212873048 includes the permanent workflow PostgreSQL archive gate. The initial Desktop Smoke E2E (3) failure passed on an unchanged-head retry; both results are retained, without claiming a diagnosed cause.

The exact-head Blox matrix and migration evidence are recorded above. PR copy now reflects migration 0041, mixed-version invalidation, nullable legacy provenance, and the retired historical-backfill proposal. No unresolved review threads remain; completed Royals behavior reviews and inspected integration deltas stand. No further engineering fix round is indicated.

The PR remains draft and conflict-free. Human GitHub approval is still required; no merge or auto-merge was requested. The separate advisory Codex review has no current verdict and is not satisfied by the ordinary Security check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant