feat(destinations): Meta Conversions API destination (#1054) - #1073
Merged
Conversation
drt's first ad-platform destination beyond Google Ads. Sends
warehouse-computed conversion events to POST /{pixel_id}/events,
batched up to 1000 per request per Meta's documented limit.
Request shape and hashing verified directly against Meta's own API
docs: email/phone are SHA-256-hashed after normalization (lowercase +
trim for email, digits-only for phone) per Meta's customer-information
parameters spec; client_ip_address/client_user_agent/fbc/fbp stay
plain text, a documented easy-to-get-wrong distinction. Hash output
independently verified against precomputed SHA-256 test vectors.
Meta's synchronous response reports an aggregate events_received
count with no per-event error array, so a partial-ack batch is
conservatively treated as fully failed rather than guessing which
records succeeded.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…destination
- Suppress httpx's own INFO-level request logging in --log-format json
(logging.getLogger("httpx").setLevel(WARNING)) — verified empirically
that httpx logs full request URLs including query strings at INFO,
and this connector's access_token is a query param, so every request
was leaking the long-lived token into JSON log output. The default
(non-JSON) text logging path doesn't raise root to INFO, so it was
never affected. No other destination in this repo passes credentials
via URL, confirming this was new to this connector, not a pre-existing
repo-wide gap.
- Require event_id_field — without a stable id, with_retry's default
3-attempt retry can resubmit a batch Meta already accepted after an
ambiguous timeout, with no way for Meta to deduplicate it.
- Fail closed on a malformed acknowledgement: a missing, null, wrong-type,
or boolean events_received previously fell through to recording success
for up to 1000 unconfirmed events; now treated as a batch failure like
an explicit count mismatch already was.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nt-age window - Reject blank per-row event_id instead of silently sending an unkeyed event (same class of gap as the Klaviyo companion PR #1052). - Meta's API requires at least one customer-information identifier in user_data (verified against Meta's own docs); require at least one identifier field configured, and reject a row where none resolve to a value. - Reject event_time values older than Meta's documented 7-day acceptance window before sending, rather than letting the API reject it with less specific attribution. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ite-event fields - Move the httpx INFO-logging suppression to module import time in meta_conversions.py itself, so it applies regardless of entry point (CLI, dagster-drt, Airflow, MCP, direct library use) rather than only when drt's own --log-format json CLI path runs. Verified no Authorization-header alternative is documented for Meta's Conversions or Graph API before ruling that fix out. - Require event_source_url_field and client_user_agent_field (config- and row-level) whenever action_source is "website" (the default) — Meta's own docs state website events without these "may be discarded." Scoped specifically to action_source == "website"; other action_source values aren't required to set these, since their own parameter requirements weren't verified here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Round-4 adversarial review fixes for #1054: - custom_data.value is validated with math.isfinite() per row before batching, so one non-finite value fails only that row instead of the whole batch (httpx's JSON encoder previously rejected the whole request, failing up to 999 otherwise-valid events). - Meta 400 responses with error.is_transient: true now retry via with_retry's retry_on predicate; other 400s still fail fast. Made retry_on actually apply on the httpx.HTTPStatusError path in the shared with_retry helper (previously only consulted there indirectly via retryable_status_codes) — backward compatible, verified against every other with_retry call site (161 tests). - Documented the accepted residual risk of the import-time httpx logging suppression (defeatable by a caller reconfiguring logging after import) rather than adding a logging.Filter with the same defeat mode. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… safety Round-5 adversarial review fixes for #1054: - access_token now sent via Authorization: Bearer header instead of a URL query param. Verified directly against the real Meta Graph API (with an invalid test token) that both a POST body field and a Bearer header are accepted identically to the query param (all three return the same "cannot parse token" OAuthException, vs. a distinctly different error with no token at all). - Removed the connector-level httpx logging suppression and its residual-risk documentation — no longer needed since the token never appears in a logged URL. - Extended the per-row JSON-safety check from custom_data.value only to the full assembled event dict (json.dumps(..., allow_nan=False)), so a non-serializable value in any field (event_id, URLs, user agent, IP, fbc, fbp) fails only that row instead of poisoning the whole batch. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Round-6 adversarial review fixes for #1054: - event_id is now coerced to str for scalar (str/int/float, not bool) values, matching Meta's schema requirement that it be a string; non-scalar values are rejected as a row error. A raw numeric event_id previously passed the JSON-safety check (an int is valid JSON) but could cause Meta to reject the whole HTTP batch. - Email/phone hashing now validates input type before hashing instead of blindly stringifying: email requires str; phone requires str or int, rejecting float outright (a float phone number's formatting isn't reliably recoverable). Previously e.g. float('nan') silently hashed to the SHA-256 of the literal string "nan" — a well-formed, Meta-accepted, but meaningless identifier with no error anywhere. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
tests/contracts/test_destination_provider_uri_empty_batch.py and test_destination_api_empty_batch.py construct MetaConversionsDestinationConfig directly and were never updated when an earlier round of this PR added a validator requiring event_source_url_field/client_user_agent_field for the default action_source: "website". Both files failed at collection time (pydantic ValidationError), breaking real GitHub Actions CI (test 3.10-3.13) on every push since — masked locally because this sandbox's own test runs report all failures under a generic "sandbox denies localhost socket binding" bucket without distinguishing a collection error from an actual socket-bound test. Three sibling fixture files (test_describe.py, test_credential_resolution_call_sites.py, test_docs_safe_labels.py) already had the correct fields — only these two contract files were missed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Round-7 adversarial review fix for #1054: the previous round's event_id coercion accepted any scalar float, including NaN/inf/-inf, which stringify to "nan"/"inf"/"-inf". NaN is a common missing-value sentinel in tabular data, so distinct rows with a missing numeric id would silently collapse onto the same event_id — Meta deduplicates matching event name + event_id pairs, discarding all but one, with no error surfaced anywhere. event_time's separate "silently defaults to sync time" finding from this same review round is a design/default-behavior question, not a narrow bug — filed as #1077 rather than fixed here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Author
|
Final adversarial review round flagged the same watermark-on-partial-failure pattern already tracked as engine-level in #1074 (found during #1072's review, applies identically to this connector and to every per-record destination using |
…sions # Conflicts: # CHANGELOG.md # integrations/vscode-drt/CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1054. drt's first ad-platform destination beyond Google Ads — sends warehouse-computed conversion events to Meta's Conversions API (
POST /{pixel_id}/events), batched up to Meta's documented 1000-events-per-request limit.What changed
MetaConversionsDestination(drt/destinations/meta_conversions.py) +MetaConversionsDestinationConfig.?access_token=...), not OAuth2 — simpler than Google Ads.user_data.em/phare SHA-256 hashes of normalized values (email: lowercase + trim; phone: digits-only). Hash output independently cross-checked against precomputed SHA-256 test vectors — confirmed byte-for-byte correct.client_ip_address/client_user_agent/fbc/fbpdeliberately stay plain text — Meta's docs call out hashing these as a common mistake.api_versiondefaults tov25.0(the field-tested prior release, not the just-shippedv26.0), fully configurable since Meta versions sunset on a ~2-year cycle.events_received != len(batch)) is conservatively treated as a full-batch failure, since Meta's synchronous response has no per-event error array to attribute individual failures against.docs/connectors/meta-conversions.md), README/comparison.md connector counts (34→35), VS Code bundled schema (regenerated from a clean venv, verified byte-identical against an independent second clean-venv regen — avoiding the chore(ci): vscode-schema-drift check depends on an unpinned pydantic version, can fail PRs that never touched schemas #1070 stale-pydantic trap).Verification
ruff check drt tests/mypy drt— cleanpytest tests/unit/test_meta_conversions_destination.py tests/unit/test_config.py tests/unit/test_destination_contract.py— 239 passedpytest tests/unit— full suite green except the known-unrelated scratchpad-path CLI-version test artifactsha256("test@example.com")andsha256("14155551234")reproduced exactly, including from unnormalized inputs (mixed-case/whitespace email, formatted phone)🤖 Generated with Claude Code