ci(sdk): stage placeholder credentials when secrets are unavailable - #2129
ci(sdk): stage placeholder credentials when secrets are unavailable#2129Utkal059 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adjusts the CI workflow so SDK tests can run on fork pull requests where secrets.CONNECTOR_SPECIFIC_AUTH is unavailable, by ensuring a creds.json file is always staged for the SDK harnesses.
Changes:
- In
.github/workflows/ci.yml, whenCONNECTOR_SPECIFIC_AUTHis missing, copycreds_dummy.jsontocreds.jsoninstead of logging “dry-run mode”. - Add workflow comments explaining why
creds.jsonmust exist and why the dummy values must remain unsanitized.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
A pull request from a fork receives no CONNECTOR_SPECIFIC_AUTH, so the "Create connector credentials" step wrote no creds.json and logged "running in dry-run mode" — but nothing downstream implements that mode. Every SDK harness resolves creds.json, and the four disagree on what a missing file means: the Rust gRPC harness exits with "Configuration error", Kotlin throws IllegalArgumentException, JavaScript and Python fall back to placeholders. The job died in its first test step, and the FFI and mock steps behind it never ran — both need the file too. Copy creds_dummy.json, the file the mock tests already use, so all four harnesses take one path. The values stay <REPLACE_WITH_YOUR_VALUE> rather than being scrubbed to "placeholder": has_valid_credentials() reads a recognised placeholder as "nothing configured", every harness then skips every connector, and passed == 0 && skipped > 0 exits 1. With the dummy values the flows run, the connector rejects the key, and each harness records that as "skipped (connector error)" with the connector still passing — so packaging, native-library loading, transport and request building stay covered without secrets. Runs that hold the secret are unaffected; that branch is untouched.
12fd55f to
5e16685
Compare
|
Worth recording why the two obvious alternatives don't hold, since staging a credentials file isn't the first thing that comes to mind for this failure.
Skipping the step when credentials are absent. That drops SDK gRPC coverage for every fork PR, including the run #2029 leans on to show its resolver change works end to end (stage native → pack → install → load). Fork PRs are where most SDK contributions come from; they'd be the ones losing the check. "The harnesses already support dry-run and mock." Only the FFI ones. Even in the FFI harness the modes don't help: The distinction that matters: this keeps the tests running and failing honestly. The alternatives get the check green by making it stop looking. |
Fork pull requests cannot pass
SDK Tests, and the reason is one line in this workflow rather than anything insdk/.A PR from a fork receives no
secrets.CONNECTOR_SPECIFIC_AUTH, so "Create connector credentials" writes nothing and logs "running in dry-run mode". Nothing downstream implements that mode. Every SDK harness resolvescreds.json, and the four disagree on what a missing file means:Run SDK gRPC Testsis the first test step, so the job dies there — and the two steps behind it need the file just as much:Run SDK FFI Tests: all four harnesses abort.run_testspanics withCredentials file not found(sdk/rust/smoke-test/src/main.rs),loadCredentialsthrows (sdk/java/smoke-test/src/main/kotlin/SmokeTest.kt), and the JavaScript and Python FFI harnesses raise as well — only their gRPC harnesses degrade.Run SDK Mock Tests:scripts/run_smoke_tests_parallel.py:499copiesrepo_root/creds.jsonunconditionally.So three of the job's steps are unreachable without secrets, and the message a contributor sees points at their own diff.
Change
Copy
creds_dummy.json— the file the mock tests already use — when the secret is absent, so all four harnesses take one path.cp creds_dummy.json creds.jsonThe values stay
<REPLACE_WITH_YOUR_VALUE>rather than being scrubbed to"placeholder", and that detail is the whole fix.has_valid_credentials()reads a recognised placeholder as nothing configured, so every harness would skip every connector, and all four then agree on this:A scrubbed file turns four crashes into four green-looking skips that still exit 1. The unscrubbed dummy makes the flows actually run: the connector rejects the key, each harness classifies that as
skipped (connector error)with the connector itself passing, exactly as the JavaScript and Python gRPC harnesses do on fork PRs today. Packaging, native-library loading, transport and request building stay covered.Runs that hold the secret are untouched — the live branch is unchanged, and the placeholder branch is reachable only when
CONNECTOR_SPECIFIC_AUTHis empty.Verified
Loading the real predicates out of
sdk/python/smoke-test/test_smoke.py(AST-extracted, so it is the shipped code, not a re-implementation):The Rust, Kotlin and JavaScript harnesses carry the same two predicates and the same
passed == 0 && skipped > 0rule; the Rust and Kotlin gRPC harnesses classify a non-transport error asskipped (connector error)and leave the connectorpassed, so they land where JavaScript and Python already do.bash -non the changed step, and the workflow parses.This PR is its own test: it changes
.github/**, which setschanges.ci, which is one of the triggers forsdk-test— and being a fork PR, it runs the placeholder branch it adds.Not included
The harness divergence is real on its own —
make -C sdk test-grpcwithout acreds.jsongives a contributor ✓✓✗✗ locally too. Converging that is four code paths across two languages I cannot build here, and it is a separate concern from unblocking CI; happy to follow up if it is wanted.One cosmetic leftover:
Run SDK FFI Testsstill branches on[[ -f "creds.json" ]]to choose its log line, which is now always true, so a fork run prints "with connector credentials". Say the word and I will make it read the mode instead.Refs #2029