Skip to content

Endpoint normalization: modern identifier/format coverage gaps (modern IDs, case-collapse, matrix params, gRPC-Web, tRPC) #97

Description

@Serhatcck

Summary

The deterministic normalization layer (Tier-0 protocol detection + Tier-1 segment classification) recognizes a classic-REST vocabulary but misses a range of modern identifier and protocol formats. Each gap causes the same symptom: distinct requests fragment into separate endpoint rows (or, in one case, distinct resources wrongly merge), and the endpoint table bloats / access-control aggregation is defeated. All items below were CONFIRMED by a deterministic harness run against the real normalize functions (no LLM).

What already works (control — verified)

Numeric / UUID / hex-hash(>=12) / email / JWT path slotting; MongoDB ObjectId -> {hash}; Snowflake -> {id}; GraphQL (single + batch) and JSON-RPC operation modeling; form-urlencoded and multipart body-shape keying. These are solid.

Coverage gaps (all confirmed by harness)

1. Modern ID formats not slotted — every id fragments

Tier-1 DYNAMIC_PATTERNS (src/session/normalize/tier1.ts) has no pattern for: TypeID (<type>_<ulid>), bare ULID, KSUID, nanoid, cuid/cuid2, base62 ids, Stripe-style prefixed ids (cus_...). Each is classified ambiguous, so distinct ids stay literal and each yields a distinct normalized_path/key_hash — e.g. ~30 distinct ids for one logical collection became 30 separate endpoints.

Fix — preserve the type prefix, slot only the id. Do NOT collapse <type>_<ulid> to {id}: the prefix is a type discriminator, and collapsing it would over-merge polymorphic endpoints (/node/order_X and /node/user_Y must not both become /node/{id}). Template as order_{ulid} / user_{ulid}. Bare (prefix-less) ids can use a full-segment {ulid}/{id}.
Suggested patterns: ULID ^[0-9A-HJKMNP-TV-Z]{26}$; TypeID ^([a-z][a-z0-9_]*)_([0-9a-hjkmnp-tv-z]{26})$ -> ${type}_{ulid}. (Implementation note: DYNAMIC_PATTERNS maps regex -> one fixed placeholder; TypeID needs a partial-slot classification that emits ${prefix}_{ulid}.)

2. Case-mis-collapse — distinct ids wrongly merged (opposite failure)

Path segments are lowercased before hashing, so two case-sensitive ids that differ only in case (base62 / nanoid, which are case-sensitive) merge to one key_hash — distinct resources conflated into one endpoint. Fix: do not lowercase segments that classify as opaque/case-sensitive ids.

3. Matrix / ;-params never stripped (also a mild security concern)

;-delimited matrix params (/cart;jsessionid=..., /products;color=red;size=xl/detail) are kept literal in the segment. A ;jsessionid= therefore fragments the endpoint per session AND leaks a session token into the recorded endpoint identity. Fix: strip matrix params from the segment before classification/slotting (and treat a session-id-like matrix param as redactable, not identity).

4. gRPC-Web bodies unparsed — every call fragments

For application/grpc-web+proto, the binary body is not parsed (no bodyKeyHash), so key_hash falls back to the value-bearing bodyHash and every call fragments into its own endpoint.

5. tRPC not modeled as operations

/trpc/proc.a,proc.b?batch=1 is treated as a plain path; the procedures survive only because they sit in the URL, and distinct batches fragment. Like GraphQL/JSON-RPC, tRPC is an operation-over-a-URL protocol and should be modeled at Tier-0.

Why the deterministic layer matters here

The only thing that could rescue an ambiguous segment at runtime is the Tier-3 LLM classifier, and that fallback is provider-dependent and separately unreliable, so correctness should not rest on it — these formats belong in the deterministic layer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions