Shopify Product Update Webhook Service — Technical & Functional Specification

Feature: FastAPI microservice that receives Shopify product update webhooks, logs payloads as structured JSON, and returns 200. Includes a production-grade multi-stage Dockerfile. Date: 2025-06-05 Version: 1.0 Status: Planned Branches: N/A (greenfield)

Feature Overview
Architecture Diagram
Component Map
API Reference
Data Schema
Protocol / Event Reference
State Machine
Message / Data Flow Diagrams
Visual Mockups
Implementation Status
Gap Analysis
Testing Requirements
Test Coverage Map
Known Edge Cases
Acceptance Criteria
Out of Scope
Open Questions
Persistence / Session Behaviour

1. Feature Overview

What the feature does

A lightweight FastAPI service that exposes a single webhook endpoint for Shopify product/update events. When Shopify (or any HTTP client) sends a POST request with a JSON payload to POST /webhooks/shopify/product-update, the service:

Validates the payload against a Pydantic model (top-level Shopify product fields only).
Logs the entire payload as a single structured JSON line via structlog.
Returns 200 OK with a minimal JSON body.

A companion GET /health endpoint returns 200 OK for container health checks and load-balancer probes.

The service runs inside a production-grade Docker container built with a multi-stage Dockerfile, runs as a non-root user, pins all dependency versions, and includes a HEALTHCHECK instruction.

End-to-end user journey

Shopify sends a product/update webhook POST to the service's /webhooks/shopify/product-update endpoint.
Uvicorn receives the request and routes it to the FastAPI app.
The Pydantic model validates the top-level fields of the JSON payload.
If validation passes, structlog emits a single structured JSON log line containing the full payload plus metadata (timestamp, event type, product ID).
The service returns 200 OK with body {"status": "received"}.
If validation fails, the service returns 422 Unprocessable Entity with FastAPI's standard validation error detail.
A Docker HEALTHCHECK periodically hits GET /health; the endpoint returns 200 OK with body {"status": "ok"}.

Key design decisions

Decision	Rationale
Skip HMAC verification	Per user clarification — simplifies initial deployment; can be added later as middleware
Pydantic model validates top-level fields only	Shopify's product schema is large and versioned; strict full validation is fragile and breaks on schema changes
`structlog` for JSON logging	Single JSON line per event is easy for log shippers (Fluentd, Datadog, CloudWatch) to ingest
Multi-stage Dockerfile	Smaller final image; build tools not included in runtime image
Non-root user in container	Security best practice; required by most Kubernetes pod security policies
No database / no retry queue	Per user clarification — this is a fire-and-forget logging sink
`GET /health` endpoint	Required for Docker `HEALTHCHECK` and load-balancer probes
Python 3.12 + pip	Per user clarification — keeps Dockerfile simpler than Poetry/pdm

2. Architecture Diagram

                    ┌─────────────────┐
                    │     Shopify     │
                    │  (Webhook Sender)│
                    └────────┬────────┘
                             │
                    HTTPS POST (JSON)
                             │
                             ▼
                ┌────────────────────────┐
                │   Docker Container     │
                │  ┌──────────────────┐  │
                │  │   Uvicorn        │  │
                │  │   (ASGI server)  │  │
                │  └────────┬─────────┘  │
                │           │            │
                │  ┌────────▼─────────┐  │
                │  │   FastAPI App    │  │
                │  │                  │  │
                │  │  POST /webhooks/ │  │
                │  │  shopify/        │  │
                │  │  product-update  │  │
                │  │                  │  │
                │  │  GET /health     │  │
                │  └────────┬─────────┘  │
                │           │            │
                │  ┌────────▼─────────┐  │
                │  │   structlog      │  │
                │  │   (JSON stdout)  │  │
                │  └────────┬─────────┘  │
                │           │            │
                │  ┌────────▼─────────┐  │
                │  │   stdout         │  │
                │  │   (container log)│  │
                │  └──────────────────┘  │
                └────────────────────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │  Log Shipper    │
                    │  (Fluentd,      │
                    │   CloudWatch,   │
                    │   Datadog, etc.)│
                    └─────────────────┘

3. Component Map

Backend

File path	Role
`app/__init__.py`	Package marker; empty
`app/main.py`	App factory — creates FastAPI instance, configures structlog, includes routers
`app/routers/__init__.py`	Package marker; empty
`app/routers/webhooks.py`	`POST /webhooks/shopify/product-update` endpoint and Pydantic model
`app/routers/health.py`	`GET /health` endpoint
`app/models.py`	Pydantic models for Shopify product webhook payload
`app/logging_config.py`	structlog configuration — JSON renderer, processor chain
`Dockerfile`	Multi-stage production build
`requirements.txt`	Pinned Python dependencies
`.dockerignore`	Exclude unnecessary files from Docker build context
`tests/__init__.py`	Package marker; empty
`tests/test_webhooks.py`	Unit + integration tests for webhook endpoint
`tests/test_health.py`	Unit tests for health endpoint
`tests/conftest.py`	Shared fixtures (TestClient, etc.)

4. API Reference

`POST /webhooks/shopify/product-update`

Receives a Shopify product update webhook payload, validates top-level fields, logs the payload as structured JSON, and returns 200.

Request

{
  "id": 123456789,
  "title": "Product Title",
  "body_html": "<p>Description</p>",
  "vendor": "Vendor Name",
  "product_type": "Type",
  "handle": "product-handle",
  "published_at": "2025-01-01T00:00:00-00:00",
  "created_at": "2025-01-01T00:00:00-00:00",
  "updated_at": "2025-06-05T12:00:00-00:00",
  "status": "active",
  "variants": [],
  "options": [],
  "images": [],
  "metafields": []
}

Response — 200 OK

{
  "status": "received"
}

Response — 422 Unprocessable Entity (validation failure)

{
  "detail": [
    {
      "loc": ["body", "id"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

Behaviour

Condition	Result
Valid JSON with required top-level fields	200, payload logged as structured JSON
Valid JSON but missing required field `id`	422, FastAPI validation error
Valid JSON but missing optional fields	200, defaults applied per model
Invalid JSON (malformed body)	422, FastAPI request parsing error
Empty body	422, validation error
Request with extra nested fields	200, extra fields passed through to log (model uses `model_config = ConfigDict(extra="allow")`)

`GET /health`

Health check endpoint for Docker HEALTHCHECK and load-balancer probes.

Request: No parameters.

Response — 200 OK

{
  "status": "ok"
}

Condition	Result
Service running	200
Service starting up	200 (Uvicorn serves immediately on import)

5. Data Schema

Pydantic Model — `ShopifyProductUpdatePayload`

class ShopifyProductUpdatePayload(BaseModel):
    model_config = ConfigDict(extra="allow")

    id: int                           # Shopify product ID (required)
    title: str                        # Product title (required)
    body_html: str | None = None      # HTML description
    vendor: str | None = None         # Vendor name
    product_type: str | None = None   # Product type
    handle: str | None = None         # URL handle
    published_at: str | None = None   # ISO 8601 datetime
    created_at: str | None = None     # ISO 8601 datetime
    updated_at: str | None = None     # ISO 8601 datetime
    status: str | None = None         # active, draft, archived
    tags: str | None = None           # Comma-separated tag string

Only id and title are required — all other fields are optional with None defaults.
extra="allow" — Shopify webhooks frequently include additional fields (variants, options, images, metafields, etc.) that are passed through to the log without validation.
Nested objects (variants, options, images, metafields) are NOT modelled — they are accepted as part of the extra fields and logged as-is.

Redis / Database

N/A — no persistence layer.

Log Output Schema

Each log line is a JSON object:

{
  "event": "shopify.product_update",
  "timestamp": "2025-06-05T12:00:00.000000Z",
  "level": "info",
  "product_id": 123456789,
  "title": "Product Title",
  "payload": { ... }
}

Field	Type	Description
`event`	`str`	Constant `"shopify.product_update"`
`timestamp`	`str`	ISO 8601 UTC timestamp
`level`	`str`	Log level (`info`, `warning`, `error`)
`product_id`	`int`	Extracted from `payload.id` for quick filtering
`title`	`str`	Extracted from `payload.title` for quick filtering
`payload`	`object`	Full webhook payload as received

6. Protocol / Event Reference

N/A — this service is a pure HTTP sink. No WebSocket, no event bus, no message queue.

7. State Machine

This service is stateless. There is no persistent state machine. The per-request lifecycle is:

[Request Received] ──→ [Parse JSON Body] ──→ [Validate Pydantic Model]
                                                    │
                                     ┌──────────────┴──────────────┐
                                     │                             │
                                  Valid                       Invalid
                                     │                             │
                                     ▼                             ▼
                              [Log via structlog]          [422 Validation Error]
                                     │
                                     ▼
                              [200 OK Response]

8. Message / Data Flow Diagrams

Normal happy path — product update webhook

Shopify                Uvicorn             FastAPI            structlog           stdout
  │                      │                   │                   │                 │
  │  POST /webhooks/     │                   │                   │                 │
  │  shopify/            │                   │                   │                 │
  │  product-update      │                   │                   │                 │
  │─────────────────────→│                   │                   │                 │
  │                      │  route to         │                   │                 │
  │                      │  webhook handler  │                   │                 │
  │                      │──────────────────→│                   │                 │
  │                      │                   │  validate         │                 │
  │                      │                   │  payload          │                 │
  │                      │                   │──────┐            │                 │
  │                      │                   │      │ valid      │                 │
  │                      │                   │←─────┘            │                 │
  │                      │                   │  log payload      │                 │
  │                      │                   │──────────────────→│                 │
  │                      │                   │                   │  JSON line      │
  │                      │                   │                   │────────────────→│
  │                      │                   │  return 200       │                 │
  │                      │←──────────────────│                   │                 │
  │  200 OK              │                   │                   │                 │
  │←─────────────────────│                   │                   │                 │

Error path — invalid payload

Client                 Uvicorn             FastAPI
  │                      │                   │
  │  POST (invalid JSON) │                   │
  │─────────────────────→│                   │
  │                      │  route to         │
  │                      │  webhook handler  │
  │                      │──────────────────→│
  │                      │                   │  validate
  │                      │                   │──────┐
  │                      │                   │      │ invalid
  │                      │                   │←─────┘
  │                      │  422 error        │
  │                      │←──────────────────│
  │  422 Unprocessable   │                   │
  │←─────────────────────│                   │

Health check flow

Docker / LB            Uvicorn             FastAPI
  │                      │                   │
  │  GET /health         │                   │
  │─────────────────────→│                   │
  │                      │  route to         │
  │                      │  health handler   │
  │                      │──────────────────→│
  │                      │  200 OK           │
  │                      │←──────────────────│
  │  200 {"status":"ok"} │                   │
  │←─────────────────────│                   │

9. Visual Mockups

As a backend-only service with no user interface, the mockups are rendered as flow diagrams showing the system states and data flows.

A — Default / Idle State

The service is running, listening on port 8080, no active requests.

B — First Interaction (Webhook Received)

A POST request arrives at the webhook endpoint; the service begins parsing and validation.

C — Loading / Validation State

The Pydantic model is validating the incoming payload against the schema.

D — Active / In-Use State (Logging)

Validation passed; structlog is writing the structured JSON line to stdout.

E — Success / Completion State

200 OK response returned to Shopify; log line emitted to stdout.

F — Alternate Layout (Health Check)

Load balancer or Docker HEALTHCHECK hits GET /health; service responds 200 OK.

G — Error State

Invalid payload fails Pydantic validation; 422 Unprocessable Entity returned.

10. Implementation Status

Story	Description	Status	Notes
WH-1	FastAPI app factory with structlog config	❌ Not built	Greenfield
WH-2	`POST /webhooks/shopify/product-update` endpoint	❌ Not built	Greenfield
WH-3	Pydantic model for Shopify product payload	❌ Not built	Greenfield
WH-4	`GET /health` endpoint	❌ Not built	Greenfield
WH-5	Production Dockerfile (multi-stage, non-root, healthcheck)	❌ Not built	Greenfield
WH-6	Unit + integration tests	❌ Not built	Greenfield
WH-7	requirements.txt with pinned versions	❌ Not built	Greenfield

11. Gap Analysis

🔴 GAP-1: No HMAC verification

Shopify webhooks include an X-Shopify-Hmac-Sha256 header for request authenticity. Skipping it means any client can send payloads to the endpoint.
Recommended fix: Add optional HMAC verification middleware in a future iteration. Configurable via environment variable VERIFY_HMAC=true|false so it can be enabled without code changes. The HMAC secret should be read from SHOPIFY_HMAC_SECRET env var.

🟡 GAP-2: No rate limiting

Without rate limiting, the endpoint is vulnerable to abuse or accidental flood (e.g., a Shopify misconfiguration sending thousands of updates).
Recommended fix: Add an in-memory rate limiter (e.g., slowapi) configurable via RATE_LIMIT env var. Default: 100 requests/minute.

🟡 GAP-3: No request ID / correlation ID

Log lines have no request-level correlation ID, making it hard to trace a single webhook through downstream systems (if added later).
Recommended fix: Add a request_id field to every log entry. Generate UUID4 per request, or read from X-Request-ID header if provided.

🟡 GAP-4: No graceful shutdown handling

Uvicorn's default shutdown may drop in-flight requests.
Recommended fix: Configure Uvicorn with --graceful-timeout 5 and handle SIGTERM to drain connections before exiting.

12. Testing Requirements

Unit Tests

Test	File	Assertions
`test_webhook_valid_payload`	`tests/test_webhooks.py`	POST with valid top-level fields returns 200; body contains `{"status": "received"}`
`test_webhook_minimal_payload`	`tests/test_webhooks.py`	POST with only `id` and `title` returns 200
`test_webhook_missing_id`	`tests/test_webhooks.py`	POST without `id` field returns 422; error detail mentions `id`
`test_webhook_missing_title`	`tests/test_webhooks.py`	POST without `title` field returns 422; error detail mentions `title`
`test_webhook_extra_fields_allowed`	`tests/test_webhooks.py`	POST with extra nested fields (`variants`, `images`) returns 200; extra fields not rejected
`test_webhook_invalid_json`	`tests/test_webhooks.py`	POST with malformed JSON body returns 422
`test_webhook_empty_body`	`tests/test_webhooks.py`	POST with empty body returns 422
`test_webhook_logs_payload`	`tests/test_webhooks.py`	POST with valid payload causes structlog to emit a log line with `event=shopify.product_update`, correct `product_id`, and full `payload`
`test_health_endpoint`	`tests/test_health.py`	GET /health returns 200 with `{"status": "ok"}`
`test_health_method_not_allowed`	`tests/test_health.py`	POST /health returns 405 Method Not Allowed

E2E Browser Tests

N/A — this is a pure backend API service with no browser UI. All testing is done via HTTP requests using httpx/TestClient.

Test Infrastructure

Framework: pytest with httpx (via FastAPI TestClient)
Location: tests/ directory at project root
Run command: docker run --rm <image> pytest tests/ -v or pytest tests/ -v locally
CI-runnable: Yes — all tests are unit/integration via TestClient, no external dependencies
structlog capture: Use capfd or a custom structlog processor to capture log output in tests

13. Test Coverage Map

Test File	What it covers	Status
`tests/test_webhooks.py`	Webhook endpoint validation, response codes, logging	❌ Missing
`tests/test_health.py`	Health endpoint response	❌ Missing
`tests/conftest.py`	Shared fixtures (TestClient)	❌ Missing

Area	Coverage level
Webhook endpoint — valid payloads	❌ Not tested
Webhook endpoint — invalid payloads	❌ Not tested
Webhook endpoint — logging output	❌ Not tested
Health endpoint	❌ Not tested
Docker build	❌ Not tested (manual)

14. Known Edge Cases

Case	Current Handling
Very large payload (>1 MB)	Uvicorn default limits apply; no explicit cap configured. Could cause memory issues under sustained high volume.
Payload with null `id`	Pydantic rejects — `id` is `int` (required), `null` fails validation → 422
Payload with string `id` instead of int	Pydantic rejects — type mismatch → 422
Payload with nested variants/images containing thousands of items	Accepted (extra="allow"), logged as-is. Could produce very large log lines.
Concurrent requests	Uvicorn handles async; no locking needed (stateless)
Missing `Content-Type: application/json`	FastAPI still parses body as JSON if valid; otherwise 422
Shopify retries webhook (if we returned non-200)	Not applicable — service always returns 200 for valid payloads; no downstream failures possible
Container OOM under high load	No memory limit in Dockerfile; rely on orchestrator/resource limits

15. Acceptance Criteria

16. Out of Scope

HMAC signature verification — explicitly skipped per user clarification
Any form of persistence (database, file, cache) — fire-and-forget logging only
Retry queue or background task processing
Rate limiting or throttling
Authentication or authorization on the endpoint
Webhook topic verification (verifying the X-Shopify-Topic header)
Webhook deduplication (idempotency keys)
Docker Compose or multi-container orchestration
CI/CD pipeline configuration
Kubernetes manifests or Helm charts
Monitoring / metrics endpoints (Prometheus, etc.)
API documentation UI (Swagger/Redoc) — FastAPI provides this by default; not a deliverable
Internationalization (i18n) — English-only error messages

17. Open Questions

#	Question	Owner
—	All clarifications resolved. No open questions.	—

18. Persistence / Session Behaviour

N/A — the service is entirely stateless. No data is persisted between requests. Each webhook request is processed independently: validate → log → respond. There is no session state, no cookies, no tokens, and no in-memory caches that survive across requests.

Uh oh!

FilesExpand file tree

spec.md

Latest commit

History