diff --git a/README.md b/README.md index 2d9e040..5acd566 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,38 @@ # WASI HTTP -(This is a placeholder so there can be a PR to fill in the contents.) +A proposed [WebAssembly System Interface](https://github.com/WebAssembly/WASI) API. + +This proposal currently only contains the proposed Wit interfaces with light +explanation in comments; more work is necessary to fully document the proposal. +The Wit comments annotate where the proposed interface is expected to change in +the short term (for Preview2) once resources and handles are re-added to Wit, +and then after that (for Preview3) once native stream support is added to the +Component Model and Wit. + +The `wit` directory currently validates and can generate bindings with: +``` +wit-bindgen c wit/ --world proxy +``` +or can be manipulated in other ways with: +``` +wasm-tools component wit wit/ ... +``` + +The HTTP proposal depends on the WASI IO and Logging proposals. For simplicity, +the Wit files for these proposals are currently copied into the `wit/deps` +directory and will be updated periodically to match their respective proposals. +As the Wit tooling develops, we should be able to avoid this form of manual +vendoring. + +### Current Phase + +wasi-http is currently in [Phase 1](https://github.com/WebAssembly/WASI/blob/main/Proposals.md). + +### Champions + +Piotr Sikora, Jiaxiao Zhou, Dan Chiarlone, David Justice + +### TODO + +This readme needs to be expanded to cover a number of additional fields suggested in the +[WASI Proposal template](https://github.com/WebAssembly/wasi-proposal-template). diff --git a/wit/deps/io/streams.wit b/wit/deps/io/streams.wit new file mode 100644 index 0000000..9fa533e --- /dev/null +++ b/wit/deps/io/streams.wit @@ -0,0 +1,161 @@ +/// WASI I/O is an I/O abstraction API which is currently focused on providing +/// stream types. +/// +/// In the future, the component model is expected to add built-in stream types; +/// when it does, they are expected to subsume this API. +default interface streams { + use poll.poll.{pollable} + + /// An error type returned from a stream operation. Currently this + /// doesn't provide any additional information. + record stream-error {} + + /// An input bytestream. In the future, this will be replaced by handle + /// types. + /// + /// This conceptually represents a `stream`. It's temporary + /// scaffolding until component-model's async features are ready. + /// + /// `input-stream`s are *non-blocking* to the extent practical on underlying + /// platforms. I/O operations always return promptly; if fewer bytes are + /// promptly available than requested, they return the number of bytes promptly + /// available, which could even be zero. To wait for data to be available, + /// use the `subscribe-to-input-stream` function to obtain a `pollable` which + /// can be polled for using `wasi_poll`. + /// + /// And at present, it is a `u32` instead of being an actual handle, until + /// the wit-bindgen implementation of handles and resources is ready. + /// + /// This [represents a resource](https://github.com/WebAssembly/WASI/blob/main/docs/WitInWasi.md#Resources). + type input-stream = u32 + + /// Read bytes from a stream. + /// + /// This function returns a list of bytes containing the data that was + /// read, along with a bool indicating whether the end of the stream + /// was reached. The returned list will contain up to `len` bytes; it + /// may return fewer than requested, but not more. + /// + /// Once a stream has reached the end, subsequent calls to read or + /// `skip` will always report end-of-stream rather than producing more + /// data. + /// + /// If `len` is 0, it represents a request to read 0 bytes, which should + /// always succeed, assuming the stream hasn't reached its end yet, and + /// return an empty list. + /// + /// The len here is a `u64`, but some callees may not be able to allocate + /// a buffer as large as that would imply. + /// FIXME: describe what happens if allocation fails. + read: func( + this: input-stream, + /// The maximum number of bytes to read + len: u64 + ) -> result, bool>, stream-error> + + /// Skip bytes from a stream. + /// + /// This is similar to the `read` function, but avoids copying the + /// bytes into the instance. + /// + /// Once a stream has reached the end, subsequent calls to read or + /// `skip` will always report end-of-stream rather than producing more + /// data. + /// + /// This function returns the number of bytes skipped, along with a bool + /// indicating whether the end of the stream was reached. The returned + /// value will be at most `len`; it may be less. + skip: func( + this: input-stream, + /// The maximum number of bytes to skip. + len: u64, + ) -> result, stream-error> + + /// Create a `pollable` which will resolve once either the specified stream + /// has bytes available to read or the other end of the stream has been + /// closed. + subscribe-to-input-stream: func(this: input-stream) -> pollable + + /// Dispose of the specified `input-stream`, after which it may no longer + /// be used. + drop-input-stream: func(this: input-stream) + + /// An output bytestream. In the future, this will be replaced by handle + /// types. + /// + /// This conceptually represents a `stream`. It's temporary + /// scaffolding until component-model's async features are ready. + /// + /// `output-stream`s are *non-blocking* to the extent practical on + /// underlying platforms. Except where specified otherwise, I/O operations also + /// always return promptly, after the number of bytes that can be written + /// promptly, which could even be zero. To wait for the stream to be ready to + /// accept data, the `subscribe-to-output-stream` function to obtain a + /// `pollable` which can be polled for using `wasi_poll`. + /// + /// And at present, it is a `u32` instead of being an actual handle, until + /// the wit-bindgen implementation of handles and resources is ready. + /// + /// This [represents a resource](https://github.com/WebAssembly/WASI/blob/main/docs/WitInWasi.md#Resources). + type output-stream = u32 + + /// Write bytes to a stream. + /// + /// This function returns a `u64` indicating the number of bytes from + /// `buf` that were written; it may be less than the full list. + write: func( + this: output-stream, + /// Data to write + buf: list + ) -> result + + /// Write multiple zero bytes to a stream. + /// + /// This function returns a `u64` indicating the number of zero bytes + /// that were written; it may be less than `len`. + write-zeroes: func( + this: output-stream, + /// The number of zero bytes to write + len: u64 + ) -> result + + /// Read from one stream and write to another. + /// + /// This function returns the number of bytes transferred; it may be less + /// than `len`. + /// + /// Unlike other I/O functions, this function blocks until all the data + /// read from the input stream has been written to the output stream. + splice: func( + this: output-stream, + /// The stream to read from + src: input-stream, + /// The number of bytes to splice + len: u64, + ) -> result, stream-error> + + /// Forward the entire contents of an input stream to an output stream. + /// + /// This function repeatedly reads from the input stream and writes + /// the data to the output stream, until the end of the input stream + /// is reached, or an error is encountered. + /// + /// Unlike other I/O functions, this function blocks until the end + /// of the input stream is seen and all the data has been written to + /// the output stream. + /// + /// This function returns the number of bytes transferred. + forward: func( + this: output-stream, + /// The stream to read from + src: input-stream + ) -> result + + /// Create a `pollable` which will resolve once either the specified stream + /// is ready to accept bytes or the other end of the stream has been closed. + subscribe-to-output-stream: func(this: output-stream) -> pollable + + /// Dispose of the specified `output-stream`, after which it may no longer + /// be used. + drop-output-stream: func(this: output-stream) +} diff --git a/wit/deps/logging/handler.wit b/wit/deps/logging/handler.wit new file mode 100644 index 0000000..c9632b9 --- /dev/null +++ b/wit/deps/logging/handler.wit @@ -0,0 +1,32 @@ +/// WASI Logging is a logging API intended to let users emit log messages with +/// simple priority levels and context values. +default interface handler { + /// A log level, describing a kind of message. + enum level { + /// Describes messages about the values of variables and the flow of + /// control within a program. + trace, + + /// Describes messages likely to be of interest to someone debugging a + /// program. + debug, + + /// Describes messages likely to be of interest to someone monitoring a + /// program. + info, + + /// Describes messages indicating hazardous situations. + warn, + + /// Describes messages indicating serious errors. + error, + } + + /// Emit a log message. + /// + /// A log message has a `level` describing what kind of message is being + /// sent, a context, which is an uninterpreted string meant to help + /// consumers group similar messages, and a string containing the message + /// text. + log: func(level: level, context: string, message: string) +} diff --git a/wit/deps/poll/poll.wit b/wit/deps/poll/poll.wit new file mode 100644 index 0000000..28f08e1 --- /dev/null +++ b/wit/deps/poll/poll.wit @@ -0,0 +1,39 @@ +/// A poll API intended to let users wait for I/O events on multiple handles +/// at once. +default interface poll { + /// A "pollable" handle. + /// + /// This is conceptually represents a `stream<_, _>`, or in other words, + /// a stream that one can wait on, repeatedly, but which does not itself + /// produce any data. It's temporary scaffolding until component-model's + /// async features are ready. + /// + /// And at present, it is a `u32` instead of being an actual handle, until + /// the wit-bindgen implementation of handles and resources is ready. + /// + /// `pollable` lifetimes are not automatically managed. Users must ensure + /// that they do not outlive the resource they reference. + /// + /// This [represents a resource](https://github.com/WebAssembly/WASI/blob/main/docs/WitInWasi.md#Resources). + type pollable = u32 + + /// Dispose of the specified `pollable`, after which it may no longer + /// be used. + drop-pollable: func(this: pollable) + + /// Poll for completion on a set of pollables. + /// + /// The "oneoff" in the name refers to the fact that this function must do a + /// linear scan through the entire list of subscriptions, which may be + /// inefficient if the number is large and the same subscriptions are used + /// many times. In the future, this is expected to be obsoleted by the + /// component model async proposal, which will include a scalable waiting + /// facility. + /// + /// Note that the return type would ideally be `list`, but that would + /// be more difficult to polyfill given the current state of `wit-bindgen`. + /// See + /// for details. For now, we use zero to mean "not ready" and non-zero to + /// mean "ready". + poll-oneoff: func(in: list) -> list +} diff --git a/wit/deps/random/random.wit b/wit/deps/random/random.wit new file mode 100644 index 0000000..2080ddf --- /dev/null +++ b/wit/deps/random/random.wit @@ -0,0 +1,42 @@ +/// WASI Random is a random data API. +/// +/// It is intended to be portable at least between Unix-family platforms and +/// Windows. +default interface random { + /// Return `len` cryptographically-secure pseudo-random bytes. + /// + /// This function must produce data from an adequately seeded + /// cryptographically-secure pseudo-random number generator (CSPRNG), so it + /// must not block, from the perspective of the calling program, and the + /// returned data is always unpredictable. + /// + /// This function must always return fresh pseudo-random data. Deterministic + /// environments must omit this function, rather than implementing it with + /// deterministic data. + get-random-bytes: func(len: u64) -> list + + /// Return a cryptographically-secure pseudo-random `u64` value. + /// + /// This function returns the same type of pseudo-random data as + /// `get-random-bytes`, represented as a `u64`. + get-random-u64: func() -> u64 + + /// Return a 128-bit value that may contain a pseudo-random value. + /// + /// The returned value is not required to be computed from a CSPRNG, and may + /// even be entirely deterministic. Host implementations are encouraged to + /// provide pseudo-random values to any program exposed to + /// attacker-controlled content, to enable DoS protection built into many + /// languages' hash-map implementations. + /// + /// This function is intended to only be called once, by a source language + /// to initialize Denial Of Service (DoS) protection in its hash-map + /// implementation. + /// + /// # Expected future evolution + /// + /// This will likely be changed to a value import, to prevent it from being + /// called multiple times and potentially used for purposes other than DoS + /// protection. + insecure-random: func() -> tuple +} diff --git a/wit/incoming-handler.wit b/wit/incoming-handler.wit new file mode 100644 index 0000000..1ecff0a --- /dev/null +++ b/wit/incoming-handler.wit @@ -0,0 +1,24 @@ +// The `wasi:http/incoming-handler` interface is meant to be exported by +// components and called by the host in response to a new incoming HTTP +// response. +// +// NOTE: in Preview3, this interface will be merged with +// `wasi:http/outgoing-handler` into a single `wasi:http/handler` interface +// that takes a `request` parameter and returns a `response` result. +// +default interface incoming-handler { + use pkg.types.{incoming-request, response-outparam} + + // The `handle` function takes an outparam instead of returning its response + // so that the component may stream its response while streaming any other + // request or response bodies. The callee MUST write a response to the + // `response-out` and then finish the response before returning. The `handle` + // function is allowed to continue execution after finishing the response's + // output stream. While this post-response execution is taken off the + // critical path, since there is no return value, there is no way to report + // its success or failure. + handle: func( + request: incoming-request, + response-out: response-outparam + ) +} diff --git a/wit/outgoing-handler.wit b/wit/outgoing-handler.wit new file mode 100644 index 0000000..abe812f --- /dev/null +++ b/wit/outgoing-handler.wit @@ -0,0 +1,18 @@ +// The `wasi:http/outgoing-handler` interface is meant to be imported by +// components and implemented by the host. +// +// NOTE: in Preview3, this interface will be merged with +// `wasi:http/outgoing-handler` into a single `wasi:http/handler` interface +// that takes a `request` parameter and returns a `response` result. +// +default interface outgoing-handler { + use pkg.types.{outgoing-request, request-options, future-incoming-response} + + // The parameter and result types of the `handle` function allow the caller + // to concurrently stream the bodies of the outgoing request and the incoming + // response. + handle: func( + request: outgoing-request, + options: option + ) -> future-incoming-response +} diff --git a/wit/proxy.wit b/wit/proxy.wit new file mode 100644 index 0000000..ff8b53a --- /dev/null +++ b/wit/proxy.wit @@ -0,0 +1,45 @@ +// The `wasi:http/proxy` world captures a widely-implementable intersection of +// hosts that includes HTTP forward and reverse proxies. Components targeting +// this world may concurrently stream in and out any number of incoming and +// outgoing HTTP requests. +default world proxy { + // HTTP proxies have access to time and randomness. + import random: random.random + // TODO: add `import wall-clock: clocks.wall-clock` + // TODO: add `import monotonic-clock: clocks.monotonic-clock` + + // This is the default logging handler to use when user code simply wants to + // log to a developer-facing console (e.g., via `console.log()`). + import console: logging.handler + + // TODO: Once the underlying Wit template machinery is implemented, add: + // + // import loggers: interface { + // *: logging.handler + // } + // + // which will allow a component to import any number of non-default logging + // backends that different categories of log messages can be sent to. + + // TODO: add `import metrics: metrics.counters` + + // This is the default handler to use when user code simply wants to make an + // HTTP request (e.g., via `fetch()`) but doesn't otherwise specify a + // particular handler. + import default-outgoing-HTTP: pkg.outgoing-handler + + // TODO: Once the underlying Wit template machinery is implemented, add: + // + // import upstreams: interface { + // *: pkg.outgoing-handler + // } + // + // which will allow a component to import any number of non-default backends + // that HTTP requests can be dispatched to. + + // The host delivers incoming HTTP requests to a component by calling the + // `handle` function of this exported interface. A host may arbitrarily reuse + // or not reuse component instance when delivering incoming HTTP requests and + // thus a component must be able to handle 0..N calls to `handle`. + export HTTP: pkg.incoming-handler +} diff --git a/wit/types.wit b/wit/types.wit new file mode 100644 index 0000000..6a8b0dc --- /dev/null +++ b/wit/types.wit @@ -0,0 +1,155 @@ +// The `wasi:http/types` interface is meant to be imported by components to +// define the HTTP resource types and operations used by the component's +// imported and exported interfaces. +default interface types { + use io.streams.{input-stream, output-stream} + use poll.poll.{pollable} + + // This type corresponds to HTTP standard Methods. + variant method { + get, + head, + post, + put, + delete, + connect, + options, + trace, + patch, + other(string) + } + + // This type corresponds to HTTP standard Related Schemes. + variant scheme { + HTTP, + HTTPS, + other(string) + } + + // TODO: perhaps better align with HTTP semantics? + // This type enumerates the different kinds of errors that may occur when + // initially returning a response. + variant error { + invalid-url(string), + timeout-error(string), + protocol-error(string), + unexpected-error(string) + } + + // This following block defines the `fields` resource which corresponds to + // HTTP standard Fields. Soon, when resource types are added, the `type + // fields = u32` type alias can be replaced by a proper `resource fields` + // definition containing all the functions using the method syntactic sugar. + type fields = u32 + drop-fields: func(fields: fields) + new-fields: func(entries: list>) -> fields + fields-get: func(fields: fields, name: string) -> list + fields-set: func(fields: fields, name: string, value: list) + fields-delete: func(fields: fields, name: string) + fields-append: func(fields: fields, name: string, value: string) + fields-entries: func(fields: fields) -> list> + fields-clone: func(fields: fields) -> fields + + type headers = fields + type trailers = fields + + // The following block defines stream types which corresponds to the HTTP + // standard Contents and Trailers. With Preview3, all of these fields can be + // replaced by a stream>. In the interim, we need to + // build on separate resource types defined by `wasi:io/streams`. The + // `finish-` functions emulate the stream's result value and MUST be called + // exactly once after the final read/write from/to the stream before dropping + // the stream. + type incoming-stream = input-stream + type outgoing-stream = output-stream + finish-incoming-stream: func(s: incoming-stream) -> option + finish-outgoing-stream: func(s: outgoing-stream, trailers: option) + + // The following block defines the `incoming-request` and `outgoing-request` + // resource types that correspond to HTTP standard Requests. Soon, when + // resource types are added, the `u32` type aliases can be replaced by + // proper `resource` type definitions containing all the functions as + // methods. Later, Preview2 will allow both types to be merged together into + // a single `request` type (that uses the single `stream` type mentioned + // above). The `consume` and `write` methods may only be called once (and + // return failure thereafter). + type incoming-request = u32 + type outgoing-request = u32 + drop-incoming-request: func(request: incoming-request) + drop-outgoing-request: func(request: outgoing-request) + incoming-request-method: func(request: incoming-request) -> method + incoming-request-path: func(request: incoming-request) -> string + incoming-request-scheme: func(request: incoming-request) -> option + incoming-request-authority: func(request: incoming-request) -> string + incoming-request-headers: func(request: incoming-request) -> headers + incoming-request-consume: func(request: incoming-request) -> result + new-outgoing-request: func( + method: method, + path: string, + scheme: option, + authority: string, + headers: headers + ) -> outgoing-request + outgoing-request-write: func(request: outgoing-request) -> result + + // Additional optional parameters that can be set when making a request. + record request-options { + // The following timeouts are specific to the HTTP protocol and work + // independently of the overall timeouts passed to `io.poll.poll-oneoff`. + + // The timeout for the initial connect. + connect-timeout-ms: option, + + // The timeout for receiving the first byte of the response body. + first-byte-timeout-ms: option, + + // The timeout for receiving the next chunk of bytes in the response body + // stream. + between-bytes-timeout-ms: option + } + + // The following block defines a special resource type used by the + // `wasi:http/incoming-handler` interface. When resource types are added, this + // block can be replaced by a proper `resource response-outparam { ... }` + // definition. Later, with Preview3, the need for an outparam goes away entirely + // (the `wasi:http/handler` interface used for both incoming and outgoing can + // simply return a `stream`). + type response-outparam = u32 + drop-response-outparam: func(response: response-outparam) + set-response-outparam: func(response: result) -> result + + // This type corresponds to the HTTP standard Status Code. + type status-code = u16 + + // The following block defines the `incoming-response` and `outgoing-response` + // resource types that correspond to HTTP standard Responses. Soon, when + // resource types are added, the `u32` type aliases can be replaced by proper + // `resource` type definitions containing all the functions as methods. Later, + // Preview2 will allow both types to be merged together into a single `response` + // type (that uses the single `stream` type mentioned above). The `consume` and + // `write` methods may only be called once (and return failure thereafter). + type incoming-response = u32 + type outgoing-response = u32 + drop-incoming-response: func(response: incoming-response) + drop-outgoing-response: func(response: outgoing-response) + incoming-response-status: func(response: incoming-response) -> status-code + incoming-response-headers: func(response: incoming-response) -> headers + incoming-response-consume: func(response: incoming-response) -> result + new-outgoing-response: func( + status-code: status-code, + headers: headers + ) -> outgoing-response + outgoing-response-write: func(response: outgoing-response) -> result + + // The following block defines a special resource type used by the + // `wasi:http/outgoing-handler` interface to emulate + // `future>` in advance of Preview3. Given a + // `future-incoming-response`, the client can call the non-blocking `get` + // method to get the result if it is available. If the result is not available, + // the client can call `listen` to get a `pollable` that can be passed to + // `io.poll.poll-oneoff`. + type future-incoming-response = u32 + drop-future-incoming-response: func(f: future-incoming-response) + future-incoming-response-get: func(f: future-incoming-response) -> option> + listen-to-future-incoming-response: func(f: future-incoming-response) -> pollable +}