Repository navigation
test: cover callback handlers, fetch, metrics-only, payload capture in local integration harness - #823
Merged
Conversation
This comment has been minimized.
This comment has been minimized.
joeyzhao2018
marked this pull request as ready for review
September 1, 2026 13:17
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
rithikanarayan
approved these changes
Sep 2, 2026
…, payload capture
joeyzhao2018
force-pushed
the
joey/l2-coverage-pr2b
branch
from
September 2, 2026 16:42
a96a6c6 to
26a364d
Compare
joeyzhao2018
added a commit
that referenced
this pull request
Sep 17, 2026
… serverless-e2e-tests) The integration_tests/ AWS suite is retired. Its coverage now lives in two places, each stronger at what it owns: - Behavior: integration_tests_local/ (docker/RIE, GitHub Actions, per PR) is now the behavior oracle — hermetic, contributor-runnable, and strictly stronger than the AWS suite at behavior (its goldens preserve span meta/metrics keys the AWS normalization stripped wholesale). After #821 and #823 it covers every variant the AWS suite pinned, including callback-style handlers, fetch, metrics-only mode, payload capture, and the manual-wrap no-init path. - Real-AWS-only signals: the four variants whose value IS the platform — esm (published layer zip artifact + ESM loader), sync-metrics (direct-API sink to the real intake), http-requests (_X_AMZN_TRACE_ID pass-through + real egress), status-code-500s (real API Gateway) — move to a dedicated integration-tests-residual suite in serverless-e2e-tests, which deploys against published/candidate layer ARNs and keeps the same snapshots and byte-identical normalization pipeline. Deleted here: serverless.yml, the six top-level handlers, all 450 snapshots, and scripts/run_integration_tests.sh (459 files). Kept: input_events/, parse-json.js, and container/ — the local suite references the first two and uses container/ as fixture; a new integration_tests/README.md documents this. CI: the GitLab `integration test` job and its sign/publish `needs` entries are removed (build-layer keeps CI_ENABLE_CONTAINER_IMAGE_BUILDS, which it uses to docker-build the layer). CONTRIBUTING.md now points contributors at the local suite. Release-gate note: the per-pipeline AWS gate on sign/publish goes away. The candidate-layer gate moves to the residual suite in serverless-e2e-tests — manual until the e2e release-gate automation lands (migration roadmap), so release runbooks should trigger it until then.
joeyzhao2018
added a commit
that referenced
this pull request
Sep 17, 2026
… serverless-e2e-tests) The integration_tests/ AWS suite is retired. Its coverage now lives in two places, each stronger at what it owns: - Behavior: integration_tests_local/ (docker/RIE, GitHub Actions, per PR) is now the behavior oracle — hermetic, contributor-runnable, and strictly stronger than the AWS suite at behavior (its goldens preserve span meta/metrics keys the AWS normalization stripped wholesale). After #821 and #823 it covers every variant the AWS suite pinned, including callback-style handlers, fetch, metrics-only mode, payload capture, and the manual-wrap no-init path. - Real-AWS-only signals: the four variants whose value IS the platform — esm (published layer zip artifact + ESM loader), sync-metrics (direct-API sink to the real intake), http-requests (_X_AMZN_TRACE_ID pass-through + real egress), status-code-500s (real API Gateway) — move to a dedicated integration-tests-residual suite in serverless-e2e-tests, which deploys against published/candidate layer ARNs and keeps the same snapshots and byte-identical normalization pipeline. Deleted here: serverless.yml, the six top-level handlers, all 450 snapshots, and scripts/run_integration_tests.sh (459 files). Kept: input_events/, parse-json.js, and container/ — the local suite references the first two and uses container/ as fixture; a new integration_tests/README.md documents this. CI: the GitLab `integration test` job and its sign/publish `needs` entries are removed (build-layer keeps CI_ENABLE_CONTAINER_IMAGE_BUILDS, which it uses to docker-build the layer). CONTRIBUTING.md now points contributors at the local suite. Release-gate note: the per-pipeline AWS gate on sign/publish goes away. The candidate-layer gate moves to the residual suite in serverless-e2e-tests — manual until the e2e release-gate automation lands (migration roadmap), so release runbooks should trigger it until then.
joeyzhao2018
added a commit
that referenced
this pull request
Sep 17, 2026
… serverless-e2e-tests) The integration_tests/ AWS suite is retired. Its coverage now lives in two places, each stronger at what it owns: - Behavior: integration_tests_local/ (docker/RIE, GitHub Actions, per PR) is now the behavior oracle — hermetic, contributor-runnable, and strictly stronger than the AWS suite at behavior (its goldens preserve span meta/metrics keys the AWS normalization stripped wholesale). After #821 and #823 it covers every variant the AWS suite pinned, including callback-style handlers, fetch, metrics-only mode, payload capture, and the manual-wrap no-init path. - Real-AWS-only signals: the four variants whose value IS the platform — esm (published layer zip artifact + ESM loader), sync-metrics (direct-API sink to the real intake), http-requests (_X_AMZN_TRACE_ID pass-through + real egress), status-code-500s (real API Gateway) — move to a dedicated integration-tests-residual suite in serverless-e2e-tests, which deploys against published/candidate layer ARNs and keeps the same snapshots and byte-identical normalization pipeline. Deleted here: serverless.yml, the six top-level handlers, all 450 snapshots, and scripts/run_integration_tests.sh (459 files). Kept: input_events/, parse-json.js, and container/ — the local suite references the first two and uses container/ as fixture; a new integration_tests/README.md documents this. CI: the GitLab `integration test` job and its sign/publish `needs` entries are removed (build-layer keeps CI_ENABLE_CONTAINER_IMAGE_BUILDS, which it uses to docker-build the layer). CONTRIBUTING.md now points contributors at the local suite. Release-gate note: the per-pipeline AWS gate on sign/publish goes away. The candidate-layer gate moves to the residual suite in serverless-e2e-tests — manual until the e2e release-gate automation lands (migration roadmap), so release runbooks should trigger it until then.
joeyzhao2018
added a commit
that referenced
this pull request
Sep 18, 2026
…e e2e and the RIE based tests (#826) * test: retire the AWS-based integration suite (moved to local docker + serverless-e2e-tests) The integration_tests/ AWS suite is retired. Its coverage now lives in two places, each stronger at what it owns: - Behavior: integration_tests_local/ (docker/RIE, GitHub Actions, per PR) is now the behavior oracle — hermetic, contributor-runnable, and strictly stronger than the AWS suite at behavior (its goldens preserve span meta/metrics keys the AWS normalization stripped wholesale). After #821 and #823 it covers every variant the AWS suite pinned, including callback-style handlers, fetch, metrics-only mode, payload capture, and the manual-wrap no-init path. - Real-AWS-only signals: the four variants whose value IS the platform — esm (published layer zip artifact + ESM loader), sync-metrics (direct-API sink to the real intake), http-requests (_X_AMZN_TRACE_ID pass-through + real egress), status-code-500s (real API Gateway) — move to a dedicated integration-tests-residual suite in serverless-e2e-tests, which deploys against published/candidate layer ARNs and keeps the same snapshots and byte-identical normalization pipeline. Deleted here: serverless.yml, the six top-level handlers, all 450 snapshots, and scripts/run_integration_tests.sh (459 files). Kept: input_events/, parse-json.js, and container/ — the local suite references the first two and uses container/ as fixture; a new integration_tests/README.md documents this. CI: the GitLab `integration test` job and its sign/publish `needs` entries are removed (build-layer keeps CI_ENABLE_CONTAINER_IMAGE_BUILDS, which it uses to docker-build the layer). CONTRIBUTING.md now points contributors at the local suite. Release-gate note: the per-pipeline AWS gate on sign/publish goes away. The candidate-layer gate moves to the residual suite in serverless-e2e-tests — manual until the e2e release-gate automation lands (migration roadmap), so release runbooks should trigger it until then. * merge main * docs: clarify AWS integration test deprecation * ci: retry pipeline * test: pin RIE tracer per runtime * ci: retrigger pipeline
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
PR 2b of the dd-trace migration roadmap (Phase 0). PR 2a brought the local RIE harness to
coverage parity with the old AWS suite; this PR goes beyond the old suite, adding the
first four customer usage modes that have zero end-to-end coverage on any surface — L1
specs cover the units, but nothing pins these paths through a real invocation.
It moves no library code. Fixtures, case registrations, docs, and goldens only — all
goldens captured from the current pre-migration implementation.
Base: this branches off
joey/l2-coverage-parity(PR 2a) and should land after it.The four cases
manual-callback(event, context, callback)handler under manualdatadog()wrap, through the RIE invoke (async completion viasetTimeout, so the wrapper can't mistake it for a sync return)tracePromisewrapper replacedpromisifiedHandler's call site, and callback handlers returnednullto API Gateway.handler.spec.tspins the units; until now nothing pinned it end to endcjs-fetch-requestsfetchoutbound in redirect mode; mock echo shows the injectedx-datadog-*headers (18/18 requests carry all four)fetchis undici, instrumented by dd-trace's undici plugin — a different injection path than the http/https plugin thatcjs-http-requestspins via axios. Main-path pattern for Node 18+ customersmanual-metrics-onlyDD_TRACE_ENABLED=false: enhanced + custom metrics still flush (viaDD_FLUSH_TO_LOG), and noaws.lambdaspan, no trace JSON, nodd.trace_idlog correlationmanual-send-metrics(verified), so the toggle is pinned to change only the tracing surfacecjs-capture-payloadDD_CAPTURE_LAMBDA_PAYLOAD=truein redirect mode:function.request/function.responsespan tags holding the captured payloadsDocs and ledger
covered locally" list in the emulation-gaps section — response streaming /
time_to_first_byte, direct-API and KMS/Secrets metric paths, aws-sdk client spans,durable execution — each with its owning suite, so the scoping discussion doesn't recur.
migration_parity.md: the callback and payload-capture rows now point at the newgoldens; new dd-core ownership rows for fetch/undici injection and Lambda profiling
(
DD_PROFILING_ENABLEDis a documented, shipped feature that had no ledger row — thecapability lives in dd-trace's profiler, covered by its suites plus the L3 Profiling
row).
Verification
All four cases green in strict compare mode on the node22 and node18 legs against shared
goldens — no per-runtime overrides needed.
manual-callbackadditionally verified on thenode26 preview leg (newest RIC + callback style was the riskiest combination). The CI
workflow runs every case per runtime leg by default, so no workflow change is needed;
expect 16 cases × 5 runtimes = 80 legs after this lands on top of PR 2a.
Remaining PR 2b items (not in this PR): kinesis/eventbridge/sns-sqs event goldens,
negative-toggle variants (
DD_ENHANCED_METRICS=false/DD_LOGS_INJECTION=false), and anauthorizer encode/decode fixture.
Trying it locally