Skip to content

http: a client — response parser, url type, https routing point - #33

Merged
alii merged 3 commits into
masterfrom
http-client
Aug 13, 2026
Merged

http: a client — response parser, url type, https routing point#33
alii merged 3 commits into
masterfrom
http-client

Conversation

@alii

@alii alii commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

scarlet/http was server-only. This adds the response parser, a Url type, and the routing point that picks cleartext or TLS — unblocked by #26 landing TLS.

Transport shape: parameterise, not a sum type

Socket and TlsSocket are deliberately different types, so the three shapes were: duplicate per transport, a Transport sum the client matches on, or parameterise over the transport's operations.

Duplication loses because the response reader is where framing bugs live, and two copies means two places to fix one smuggling reject.

The sum type is not absurd — it keeps the types across three small matches. It loses because it is closed, and what it forfeits is a transport backed by a Binary, which is how the whole request/response path is tested with no network. Adding a Fake arm to a production sum to recover that is a test hook on the wire.

Io was made opaque and then reverted: opaque leaves plain/secure as the only doors, which shuts out the in-memory transport that motivated the shape in the first place.

What parameterising gives up, stated rather than hidden: nothing proves all three closures address one connection. That needs an existential — Io(h) would carry it, but then Io(Socket) and Io(TlsSocket) differ and connect has no return type. Filed T-217; the cost is one unenforced invariant on a public type.

One grammar, not two

The parser is a VM op, so the header block below a status line is literally parse_header_block — the same code as a request head. That is the load-bearing claim, and plant 2 proves it: dropping the whitespace-before-colon reject turns both goldens red from one edit, http_parse and http_response together.

Plants — all watched red, all restored

plant observed
status code defaults to 200 header bytes returned as a body: code=200 body="Content-Length: 0\r\n\r\n"
drop whitespace-before-colon reject both goldens red from one edit
body shorter than Content-Length body="he" instead of Err Transport(UnexpectedEof)
response framing falls back to the request rule close-framed reply truncated to empty
drop request-header validation X-Injected: yes on the wire as its own header
rename client.get export check fires

One plant was inert and was replaced — disabling the obs-fold reject left the golden green, because obs-fold is caught twice. That is the only reason the check that shipped is known to be real. Two further plants tripped checks the author did not write: a no-progress read_more made read_head spin rather than fail, and adding 7 ABI slots without binding them tripped fixture_binds_every_slot.

Gates

fmt 0 · clippy 0 · gen-editor-syntax --check 0 · test --workspace 0 — 37 binaries, 1265 passed (baseline 1264; +1 is the new golden).

core_ir goldens: pure +512 type-id shift, 28 line pairs, 2 ids. The shift checker was itself planted twice — a changed variant index gives NOT A SHIFT: - ctor 7936.0 / + ctor 8448.1, an inconsistent id gives type id 7936 maps to both 8448 and 9999.

Deliberately not done

No connection pool, so T-118's own DONE-WHEN ("over a pooled connection") is not met — every request sends Connection: close and an https request is a handshake. Filed T-220, and T-165 should not be closed on this alone.

No read deadline on either scheme. net/tls has no read_within (T-128), and giving only the cleartext adapter a deadline would make liveness depend on the URL scheme — invisible until the https request is the one that hangs.

Also filed T-219: dis.rs's constant-pool ceiling was < 600, measured at 520 without these files and 604 with. Dedup was working; the ceiling was snug against stdlib size with 15% headroom. Raised to 1000, but the assertion tests the wrong thing — the failure it guards is multiplicative.

Goes stale on merge

The website's TLS page says "There is no HTTP client in the standard library. An HTTPS request is request bytes written by hand over a TlsSocket." Both sentences are false as of this branch. Separate repo, needs its own ticket.

@alii
alii force-pushed the http-client branch 2 times, most recently from e15a486 to caad60f Compare August 13, 2026 04:47
alii added 3 commits August 12, 2026 21:52
T-118. scarlet/http was server-only: h1 had parse_request and no response
parser, and no code anywhere turned an https:// string into a transport.

h1 gains parse_response/response_framing, and a new module scarlet/http/url
gives Scheme as a value so http-vs-https is decided once. scarlet/http/client
is the request/response path and the routing point.

TRANSPORT SHAPE. Socket and TlsSocket are different types on purpose, so the
client takes an `Io` of three closures over one connection; plain(Socket) and
secure(TlsSocket) build them. Rejected: duplicating send per transport (the
response reader is where framing bugs live, two copies is two places to fix a
smuggling reject in one of), and a Transport sum matched per read/write (keeps
the types, three small matches, but closed — an Io backed by a Binary is how
the whole path is tested with no network, and a Fake arm on a production sum
type is a test hook on the wire). What shape 3 gives up is that all three
closures address one connection, which needs an existential: T-217.

ONE HEADER GRAMMAR. parse_response is a VM op, not Scarlet, so the field block
below the status line is literally parse_header_block — the same code as a
request head. Watched: deleting the whitespace-before-colon reject turns
http_parse AND http_response red from one edit. parse_header_block's Bad now
carries FieldReject (Malformed/TooLarge) instead of a request status code, so
neither caller needs a `_` arm for a condition it cannot produce.

RESPONSE FRAMING IS NOT REQUEST FRAMING. Absent framing fields mean no body on
a request and until-close on a response, so ResponseFraming is its own type;
reusing Framing truncates every close-framed reply to empty (watched).

NOT DONE, deliberately: no connection pool (every request is Connection:
close, so an https request is a handshake — the ticket's DONE-WHEN asked for
pooling and this is scoped short of it), no redirects, cookies, retry, or
streaming response bodies. No read deadline on either scheme: net/tls has no
read_within (T-128), and a deadline only the cleartext side honours makes
liveness depend on the URL scheme.

Constant-pool ceiling in dis.rs raised 600 -> 1000: measured 520 without the
two new modules, 604 with. Dedup was working; the ceiling was snug. T-219.
#36 and #33 each merge clean and the merged tree does not build: 14 calls
expect a bare Binary. Fourteen unwraps would restore exactly the silence #36
exists to remove, with a Result in the signature to make it look handled, so
every site was classified instead.

FIVE were the tail, slice_bytes(b, at, byte_size(b) - at). drop_bytes already
names that operation and is honestly total, so they carry no Err arm at all:
url.split_bracketed's rest, url.split_at_colon's port_text, url.port_of's
digits, client.read_head's rest, http_response's wire_loop.

ONE takes master's own idiom, result.then(slice_bytes(..), to_string) or '?',
byte-identical to what #36 did to http_parse.scrl's twin line.

ONE is a genuine error and is now visible. http_response's "consumed threads
to the body" asserts a 5-byte window against a Content-Length of 5, so a
consumed that does not thread there has to print as an error rather than as an
empty window.

SEVEN carry an Err arm argued unreachable, each naming the bound it rests on:
an index_of hit, first_of's clamp to size, the leading-[ guard, the
byte_size >= n guard, int.min. That is an argument in a comment rather than a
type, which is T-286. Six of the seven fall through to a loud error anyway
(UnsupportedScheme, EmptyHost, MalformedResponse); target_of's does not, and
says so.

url.scrl's malformed-URL cases turn out to be rejected before any window is
computed — scheme_sep, split_bracketed and port_of all fail first — so none of
its 8 became a new UrlError variant. A variant nothing can construct is worse
than the comment it replaces.

Not done here: those 7 are all one operation, "cut b at a position something
else already proved is inside b". drop_bytes names the tail half and the head
half has no name. T-309 carries the site list and the argument that
split_at_bytes is total for drop_bytes's own reason, and is not the take_bytes
#36 refused — window_bytes(b, from, to) is, which is why it is not that.

dis.rs's pool ceiling re-measured on the merged tree: 714 entries, not the 712
the comment claimed. The 1000 ceiling is unchanged and not close.

Plants, all watched red and restored. #33's load-bearing one first: dropping
the whitespace-before-colon disjunct in vm/http.rs reds http_parse AND
http_response from that one edit, rc=101 — the response head parser is still
literally the request head parser after a 14-site edit. Then two of mine.
Truncating the framed fixture prints "OUT OF RANGE" in place of a short
window, rc=101, so the genuine error reaches the caller. Weakening
read_exactly's guard to >= n - 1 prints Err MalformedResponse(BadFraming) in
place of a truncated body, rc=101, so the argued-unreachable arm is live and
loud when its bound is wrong.

fmt 0, clippy -D warnings 0, test --workspace 0, gen-editor-syntax --check 0,
SCARLET_GC_STRESS=1 0, dylint --all 0 with the cache cleared and sources
touched (0 findings), hawk check -D warnings 0 (0 findings).

Filed alongside: T-309 (split_at_bytes), T-310 (h1.scrl's HeadFlags still
carries two connection bools where the Rust side is one ConnTokens).
@alii
alii merged commit 80b5135 into master Aug 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant