Skip to content

api-e2e.md

Latest commit

 

History

History
135 lines (103 loc) · 5 KB

api-e2e.md

File metadata and controls

135 lines (103 loc) · 5 KB
Raw

API E2E Design

1. Definition of complete coverage

Complete V1 coverage means all 16 operations in contracts/gateway-v1-operations.yaml are discovered in the runtime OpenAPI document and executed at least once through a successful user journey.

It also means representative protocol and failure paths are verified. It does not mean forcing every backend-only status such as every possible 500 or 504 for every operation; those require fault injection and belong to a later suite.

2. Contract sources

The runner uses two independent inputs:

  1. The checked-in operation ledger specifies the expected method, path, success statuses, and documented response statuses.
  2. GET /v1/openapi.json supplies the runtime OpenAPI 3.1 schemas.

The run fails before executing scenarios when an expected operation is absent, duplicated, has a different method/path, or removes a declared status. Every captured successful or error response is then validated against the matching OpenAPI response schema. The runtime document is saved as evidence.

Once Gateway 1.0 is declared stable, the testbed should additionally pin a reviewed normalized OpenAPI snapshot. Changes to that snapshot require an explicit testbed review rather than silently accepting a server-side change.

3. Main journey

The runner generates names from the run ID and performs this ordered journey:

  1. getHealth, getReady, getOpenApi, and listClusters.
  2. createDatabase, followed by listDatabases.
  3. createTable(validate_only=true) and prove no table was created.
  4. Create a partitioned catalog table, a Log table, and a KV table.
  5. listTables and describeTable for exact schema round-trip checks.
  6. alterTable for add-column, set-config, and reset-config behavior.
  7. createPartition, listPartitions, and dropPartition.
  8. writeRecords with Log append entries.
  9. writeRecords with KV upsert, delete, and partial update entries.
  10. Stop using Gateway responses as evidence and read data directly through fluss-rs: scan the Log table and lookup expected KV keys.
  11. dropTable for every table, then dropDatabase.
  12. List resources again to prove cleanup completed.

Cleanup is also registered as a best-effort guard. If a scenario fails halfway, the guard still removes known leaf resources in reverse order.

4. V1 table definitions

Log table:

event_time  BIGINT NOT NULL
sequence    BIGINT NOT NULL
payload     STRING

KV table:

id          BIGINT NOT NULL PRIMARY KEY
version     BIGINT NOT NULL
payload     STRING
note        STRING NULL

The E2E dataset stays small and deterministic. It proves exact durable state, including input ordering, upsert replacement, deletion, nullable values, and partial-update preservation.

5. Protocol assertions

Every response must carry x-request-id. JSON responses must use application/json. Errors must have the shared envelope:

{
  "error": {
    "code": "invalid_argument",
    "message": "...",
    "request_id": "..."
  }
}

The body request ID must equal the response header. Responses with an empty body must not accidentally return JSON text.

Representative negative cases include:

  • Unknown route and wrong method.
  • Unexpected query parameters.
  • Unknown cluster, database, table, and partition.
  • Missing/wrong Content-Type for JSON body operations.
  • Malformed JSON and unknown fields.
  • Unsupported Accept on writeRecords.
  • Empty write entries and mixed append/upsert batches.
  • Duplicate entry IDs and rows that do not match the table schema.
  • Write row count and body size above configured limits.
  • Drop of a non-empty database returning conflict.
  • Invalid table config and invalid alter operation.
  • Rate/concurrency rejection returning 429 and Retry-After in a separately configured admission-control case.

Validation failure for a write batch must be all-or-nothing. The oracle verifies that the nominally valid row beside an invalid row was not persisted.

6. Oracle rules

V1 uses the released fluss-rs client because the project is Rust-first and the crate exposes both Log scanning and KV lookup. The client version is pinned to the target Fluss release rather than imported from a sibling source tree.

The oracle never writes test data. It only:

  • Scans the bounded E2E Log table and compares every expected row.
  • Looks up every expected KV key and checks the full value.
  • Confirms deleted keys are absent.
  • Confirms a rejected batch left no rows behind.

Using a Java oracle remains a release-hardening option because it would avoid sharing the same Rust client implementation as the Gateway.

7. Initial acceptance criteria

  • All 16 expected operations are present in the runtime OpenAPI document.
  • Every operation has a passing success scenario.
  • Every observed response matches its OpenAPI schema.
  • All protocol assertions pass.
  • Durable Log and KV state exactly matches the expected model.
  • Cleanup completes or is clearly reported as a separate failure.
  • JUnit, JSON, Markdown, runtime OpenAPI, and manifest artifacts are complete.