-
-
Notifications
You must be signed in to change notification settings - Fork 134
fix(security): block r2-direct manifest jsonb poison path #3206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
d21b751
fix(security): block r2-direct manifest jsonb poison path
cursoragent 5c4845f
test: fix manifest poison and swap cleanup regressions
cursoragent b1fb82f
fix(security): block r2-direct manifest jsonb smuggle on finalize
cursoragent b5ecdd4
test: align manifest s3_path in audit skip regression
cursoragent 6d6a5ae
test: use canonical r2_path in audit fat-fields regression
cursoragent 26f2ad4
test: fix audit fat-fields regression for r2-direct manifest lock
cursoragent 58da280
test: assert set_manifest inserts rows after jsonb block
cursoragent 640dd0d
ci: retrigger test workflow
cursoragent cfbc5c5
fix(security): block r2-direct manifest jsonb on INSERT too
cursoragent a0b5fee
test: fix audit fat-fields setup for r2 content lock
cursoragent 57d6ddd
ci: retrigger test workflow
cursoragent 64d5c96
test: guard set_manifest cleanup when version insert fails
cursoragent 0700671
ci: retrigger tests after flaky organization-api shard
cursoragent 01adafa
ci: retrigger full test suite
cursoragent 2eb2f07
fix(db): re-stamp r2-direct manifest guard migration after main advance
cursoragent ec08b82
ci: retrigger tests (no concurrent pushes)
cursoragent 6cbcf99
ci: retry after flaky cloudflare workers shard
cursoragent 57fd9d5
ci: final test run
cursoragent f00a2af
fix(db): document manifest helper profile and satisfy SQLFluff LT05
cursoragent 5fd7081
fix(db): re-stamp r2-direct manifest guard after sso migration on main
cursoragent 9cf1b2c
ci: sync CodeRabbit bot trigger workflow from main
cursoragent 2a3b8ee
ci: reduce CodeRabbit trigger workflow pull-request permissions to read
cursoragent faf2d4a
ci: document why CodeRabbit trigger workflow needs pull-requests write
cursoragent e8b66bd
merge: sync main into cursor/fix-manifest-poison-r2-direct-6e3f
cursoragent b8b72b9
ci: retry CodeRabbit trigger when prior attempt had no review on HEAD
cursoragent 77a039e
ci: use push SHA for CodeRabbit trigger marker
cursoragent 6b7803d
merge: sync main into cursor/fix-manifest-poison-r2-direct-6e3f
cursoragent a751277
ci: allow CodeRabbit trigger retry when review is CHANGES_REQUESTED
cursoragent fecfb60
test: retry Cloudflare worker-restart 503 in fetchTestRequest
cursoragent a827a89
test: restrict fetchTestRequest retries to safe methods by default
cursoragent a7263a9
ci: retry CodeRabbit trigger until review is APPROVED
cursoragent 67dad7c
ci: retrigger CodeRabbit review after rate limit window
cursoragent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
267 changes: 267 additions & 0 deletions
267
supabase/migrations/20260826101500_block_r2_direct_manifest_jsonb_writes.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,267 @@ | ||
| -- Block PostgREST writes to app_versions.manifest while a bundle is still | ||
| -- in-progress (storage_provider = r2-direct). Legitimate delta uploads use | ||
| -- POST /private/set_manifest, which inserts into public.manifest directly. | ||
| -- | ||
| -- Execution profile for app_version_manifest_jsonb_unmigrated: | ||
| -- - Called from check_encrypted_bundle_on_insert on public.app_versions | ||
| -- INSERT/UPDATE when manifest jsonb is cleared or compared (at most once | ||
| -- per affected row). | ||
| -- - Roles: service_role only; not exposed to anon/authenticated PostgREST. | ||
| -- - Frequency: console/API app_versions writes; not plugin /updates hot path. | ||
| -- - Cardinality: p_manifest is bounded by bundle file count (thousands of | ||
| -- entries at most); each entry probes public.manifest via app_version_id. | ||
| -- - Indexes: idx_manifest_app_version_id on (app_version_id); per-entry | ||
| -- s3_path/file_hash filter on the index-scanned row set. | ||
| -- - Worst case: Nested Loop from unnest(p_manifest) to Index Scan on | ||
| -- idx_manifest_app_version_id with s3_path/file_hash filters. Bounded by | ||
| -- manifest entry count, not table cardinality. | ||
|
|
||
| CREATE OR REPLACE FUNCTION public.app_version_manifest_jsonb_unmigrated( | ||
| p_version_id bigint, | ||
| p_manifest public.manifest_entry[] | ||
| ) | ||
| RETURNS boolean | ||
| LANGUAGE sql | ||
| STABLE | ||
| SET search_path = '' | ||
| AS $$ | ||
| SELECT EXISTS ( | ||
| SELECT 1 | ||
| FROM pg_catalog.unnest(p_manifest) AS entry(file_name, s3_path, file_hash) | ||
| WHERE NOT EXISTS ( | ||
| SELECT 1 | ||
| FROM public.manifest AS m | ||
| WHERE m.app_version_id = p_version_id | ||
| AND m.s3_path = entry.s3_path | ||
| AND m.file_hash = entry.file_hash | ||
| ) | ||
| ); | ||
| $$; | ||
|
|
||
| ALTER FUNCTION public.app_version_manifest_jsonb_unmigrated( | ||
| bigint, public.manifest_entry[] | ||
| ) OWNER TO postgres; | ||
| REVOKE ALL ON FUNCTION public.app_version_manifest_jsonb_unmigrated( | ||
| bigint, public.manifest_entry[] | ||
| ) FROM PUBLIC; | ||
| GRANT ALL ON FUNCTION public.app_version_manifest_jsonb_unmigrated( | ||
| bigint, public.manifest_entry[] | ||
| ) TO service_role; | ||
|
|
||
| CREATE OR REPLACE FUNCTION "public"."check_encrypted_bundle_on_insert"() RETURNS "trigger" | ||
| LANGUAGE "plpgsql" SECURITY DEFINER | ||
| SET "search_path" TO '' | ||
| AS $$ | ||
| DECLARE | ||
| org_id uuid; | ||
| org_enforcing boolean; | ||
| org_required_key varchar(21); | ||
| bundle_is_encrypted boolean; | ||
| bundle_key_id varchar(20); | ||
| bundle_was_ready boolean; | ||
| r2_direct_manifest_err constant text := | ||
| 'r2_direct_manifest_jsonb: Use POST /private/set_manifest for in-progress ' | ||
| || 'r2-direct uploads instead of app_versions.manifest jsonb.'; | ||
| BEGIN | ||
| IF TG_OP = 'INSERT' | ||
| AND NEW.storage_provider = 'r2-direct' | ||
| AND NEW.manifest IS NOT NULL | ||
| THEN | ||
| PERFORM public.pg_log('deny: BUNDLE_CONTENT_LOCKED_TRIGGER', | ||
| pg_catalog.jsonb_build_object( | ||
| 'org_id', NEW.owner_org, | ||
| 'app_id', NEW.app_id, | ||
| 'version_name', NEW.name, | ||
| 'user_id', NEW.user_id, | ||
| 'old_storage_provider', NULL, | ||
| 'new_storage_provider', NEW.storage_provider, | ||
| 'reason', 'r2_direct_manifest_jsonb' | ||
| )); | ||
| RAISE EXCEPTION '%', r2_direct_manifest_err; | ||
| END IF; | ||
|
|
||
| IF TG_OP = 'UPDATE' THEN | ||
| IF pg_catalog.current_setting('capgo.reclaim_manifest_null', true) = 'on' | ||
| AND NEW.manifest IS NULL | ||
| AND OLD.manifest IS NOT NULL | ||
| AND NEW.native_packages IS NOT DISTINCT FROM OLD.native_packages | ||
| AND NEW.name IS NOT DISTINCT FROM OLD.name | ||
| AND NEW.app_id IS NOT DISTINCT FROM OLD.app_id | ||
| AND NEW.session_key IS NOT DISTINCT FROM OLD.session_key | ||
| AND NEW.key_id IS NOT DISTINCT FROM OLD.key_id | ||
| AND NEW.storage_provider IS NOT DISTINCT FROM OLD.storage_provider | ||
| AND NEW.r2_path IS NOT DISTINCT FROM OLD.r2_path | ||
| AND NEW.external_url IS NOT DISTINCT FROM OLD.external_url | ||
| AND NEW.checksum IS NOT DISTINCT FROM OLD.checksum | ||
| THEN | ||
| RETURN NEW; | ||
| END IF; | ||
|
|
||
| IF NEW.manifest IS NULL | ||
| AND OLD.manifest IS NOT NULL | ||
| AND public.app_version_manifest_jsonb_unmigrated(OLD.id, OLD.manifest) | ||
| THEN | ||
| RAISE EXCEPTION '%', | ||
| 'bundle_manifest_not_migrated: Cannot clear app_versions.manifest ' | ||
| || 'until every entry exists in public.manifest.'; | ||
| END IF; | ||
|
|
||
| bundle_was_ready := OLD.storage_provider IS DISTINCT FROM 'r2-direct'; | ||
|
|
||
| IF bundle_was_ready | ||
| AND ( | ||
| NEW.name IS DISTINCT FROM OLD.name | ||
| OR NEW.app_id IS DISTINCT FROM OLD.app_id | ||
| OR NEW.session_key IS DISTINCT FROM OLD.session_key | ||
| OR NEW.key_id IS DISTINCT FROM OLD.key_id | ||
| OR NEW.storage_provider IS DISTINCT FROM OLD.storage_provider | ||
| OR NEW.r2_path IS DISTINCT FROM OLD.r2_path | ||
| OR NEW.external_url IS DISTINCT FROM OLD.external_url | ||
| OR NEW.checksum IS DISTINCT FROM OLD.checksum | ||
| OR (NEW.manifest IS DISTINCT FROM OLD.manifest AND NEW.manifest IS NOT NULL) | ||
| OR ( | ||
| NEW.manifest IS NULL | ||
| AND OLD.manifest IS NOT NULL | ||
| AND public.app_version_manifest_jsonb_unmigrated(OLD.id, OLD.manifest) | ||
| ) | ||
| OR NEW.native_packages IS DISTINCT FROM OLD.native_packages | ||
| ) | ||
| THEN | ||
| PERFORM public.pg_log('deny: BUNDLE_CONTENT_LOCKED_TRIGGER', | ||
| pg_catalog.jsonb_build_object( | ||
| 'org_id', OLD.owner_org, | ||
| 'app_id', OLD.app_id, | ||
| 'version_name', OLD.name, | ||
| 'user_id', OLD.user_id, | ||
| 'old_storage_provider', OLD.storage_provider, | ||
| 'new_storage_provider', NEW.storage_provider, | ||
| 'reason', 'bundle_ready' | ||
| )); | ||
| RAISE EXCEPTION '%', | ||
| 'bundle_already_ready: Bundle content cannot be changed ' | ||
| || 'after upload is complete. Upload a new bundle instead.'; | ||
| END IF; | ||
|
|
||
| -- In-progress r2-direct uploads must use POST /private/set_manifest. | ||
| -- Block any non-null manifest jsonb write, including r2-direct -> r2 finalize | ||
| -- requests that try to smuggle manifest rows through on_version_update. | ||
| IF OLD.storage_provider = 'r2-direct' | ||
| AND NEW.manifest IS DISTINCT FROM OLD.manifest | ||
| AND NEW.manifest IS NOT NULL | ||
| THEN | ||
| PERFORM public.pg_log('deny: BUNDLE_CONTENT_LOCKED_TRIGGER', | ||
| pg_catalog.jsonb_build_object( | ||
| 'org_id', OLD.owner_org, | ||
| 'app_id', OLD.app_id, | ||
| 'version_name', OLD.name, | ||
| 'user_id', OLD.user_id, | ||
| 'old_storage_provider', OLD.storage_provider, | ||
| 'new_storage_provider', NEW.storage_provider, | ||
| 'reason', 'r2_direct_manifest_jsonb' | ||
| )); | ||
| RAISE EXCEPTION '%', r2_direct_manifest_err; | ||
| END IF; | ||
| END IF; | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| IF TG_OP = 'UPDATE' | ||
| AND NEW.session_key IS NOT DISTINCT FROM OLD.session_key | ||
| AND NEW.key_id IS NOT DISTINCT FROM OLD.key_id | ||
| AND NEW.name IS NOT DISTINCT FROM OLD.name | ||
| AND NEW.app_id IS NOT DISTINCT FROM OLD.app_id | ||
| AND NEW.storage_provider IS NOT DISTINCT FROM OLD.storage_provider | ||
| AND NEW.r2_path IS NOT DISTINCT FROM OLD.r2_path | ||
| AND NEW.external_url IS NOT DISTINCT FROM OLD.external_url | ||
| AND NEW.checksum IS NOT DISTINCT FROM OLD.checksum | ||
| AND NEW.native_packages IS NOT DISTINCT FROM OLD.native_packages | ||
| AND ( | ||
| NEW.manifest IS NOT DISTINCT FROM OLD.manifest | ||
| OR ( | ||
| NEW.manifest IS NULL | ||
| AND OLD.manifest IS NOT NULL | ||
| AND NOT public.app_version_manifest_jsonb_unmigrated(OLD.id, OLD.manifest) | ||
| ) | ||
| ) | ||
| THEN | ||
| RETURN NEW; | ||
| END IF; | ||
|
|
||
| SELECT apps.owner_org INTO org_id | ||
| FROM public.apps | ||
| WHERE apps.app_id = NEW.app_id; | ||
|
|
||
| IF org_id IS NULL THEN | ||
| org_id := NEW.owner_org; | ||
| END IF; | ||
|
|
||
| IF org_id IS NULL THEN | ||
| RETURN NEW; | ||
| END IF; | ||
|
|
||
| SELECT enforce_encrypted_bundles, required_encryption_key | ||
| INTO org_enforcing, org_required_key | ||
| FROM public.orgs | ||
| WHERE id = org_id; | ||
|
|
||
| IF org_enforcing IS NULL OR org_enforcing = false THEN | ||
| RETURN NEW; | ||
| END IF; | ||
|
|
||
| bundle_is_encrypted := public.is_bundle_encrypted(NEW.session_key); | ||
| bundle_key_id := NULLIF(pg_catalog.btrim(NEW.key_id), '')::varchar(20); | ||
|
|
||
| IF NOT bundle_is_encrypted THEN | ||
| PERFORM public.pg_log('deny: ORG_REQUIRES_ENCRYPTED_BUNDLES_TRIGGER', | ||
| pg_catalog.jsonb_build_object( | ||
| 'org_id', org_id, | ||
| 'app_id', NEW.app_id, | ||
| 'version_name', NEW.name, | ||
| 'user_id', NEW.user_id, | ||
| 'reason', 'not_encrypted' | ||
| )); | ||
| RAISE EXCEPTION '%', | ||
| 'encryption_required: This organization requires all bundles to be ' | ||
| || 'encrypted. Please upload an encrypted bundle with a session_key.'; | ||
| END IF; | ||
|
|
||
| IF org_required_key IS NOT NULL AND org_required_key <> '' THEN | ||
| IF bundle_key_id IS NULL THEN | ||
| PERFORM public.pg_log('deny: ORG_REQUIRES_SPECIFIC_ENCRYPTION_KEY_TRIGGER', | ||
| pg_catalog.jsonb_build_object( | ||
| 'org_id', org_id, | ||
| 'app_id', NEW.app_id, | ||
| 'version_name', NEW.name, | ||
| 'user_id', NEW.user_id, | ||
| 'required_key', org_required_key, | ||
| 'bundle_key_id', bundle_key_id, | ||
| 'reason', 'missing_key_id' | ||
| )); | ||
| RAISE EXCEPTION '%', | ||
| 'encryption_key_required: This organization requires bundles to be ' | ||
| || 'encrypted with a specific key. The uploaded bundle does not have ' | ||
| || 'a key_id.'; | ||
| END IF; | ||
|
|
||
| IF NOT ( | ||
| bundle_key_id = pg_catalog.left(org_required_key, 20) | ||
| OR pg_catalog.left(bundle_key_id, pg_catalog.length(org_required_key)) = org_required_key | ||
| ) THEN | ||
| PERFORM public.pg_log('deny: ORG_REQUIRES_SPECIFIC_ENCRYPTION_KEY_TRIGGER', | ||
| pg_catalog.jsonb_build_object( | ||
| 'org_id', org_id, | ||
| 'app_id', NEW.app_id, | ||
| 'version_name', NEW.name, | ||
| 'user_id', NEW.user_id, | ||
| 'required_key', org_required_key, | ||
| 'bundle_key_id', bundle_key_id, | ||
| 'reason', 'key_mismatch' | ||
| )); | ||
| RAISE EXCEPTION '%', | ||
| 'encryption_key_mismatch: This organization requires bundles to be ' | ||
| || 'encrypted with a specific key. The uploaded bundle was encrypted ' | ||
| || 'with a different key.'; | ||
| END IF; | ||
| END IF; | ||
|
|
||
| RETURN NEW; | ||
| END; | ||
| $$; | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.