Use utoipa crates to generate and serve basic OpenAPI description

[Turbo87]

This PR implements the basic infrastructure to have an experimental, best-effort, automatically generated OpenAPI description for the crates.io API. The OpenAPI description will be available at https://crates.io/api/openaip.json, with a viewer available at https://crates.io/api/openaip.

Due to the viewer being on the same domain as the Ember.js frontend, it will automatically share the cookie authentication, which may or may not be a problem. We could alternatively consider serving the viewer from a different subdomain if this turns out to be problematic.

Note that this PR only migrates a single route for now to prove the viability of this approach. The remaining routes can be migrated incrementally.

Related:

• Add an HTTP API reference, if stable #741

[Turbo87]

the OpenAPI spec does not specify the format of the version field other than that it's a "string". we could potentially use the git SHA1 or commit timestamp as the version, if that seems useful 🤔

[LawnGnome]

I thinking a combination of both would probably be most informative: something like 2024-12-11-21-45-00-4b33fdda250e7a8ebb4c40eff4d6205ce09c7e2a.

[LawnGnome]

As I said earlier on a call, weirdly, I've also been playing around with utoipa in the last week.

In general, I like the idea of specifying our API (which is a significant superset of the basic Cargo Registry API) more formally, and providing it in a form that can be used downstream by users to do things like generating API clients.¹ OpenAPI seems to be as good a choice as any, so 👍 for that.

That said, I'm wondering if this is too bare bones right now, even for an initial exploratory PR. To my mind, the benefit of this is in showing the actual shape of the data going in and out — so, in the case of this particular controller, the query parameters that are accepted, and then the structure that comes back in the response. Unfortunately, this annotation doesn't really do that.

I threw a rough, incomplete commit on top of this. This implements more or less what I think is the minimum viable OpenAPI description of this endpoint, just without some of the descriptive text we'd eventually need.

Issues that I found along the way include:

1. This doesn't really work with our current usage of ErasedJson for small JSON blobs in responses — the only way to really get useful types generated is to write proper struct definitions that derive ToSchema, and at that point, you might as well just derive Serialize as well and use them. I don't know if this is actually a problem — if anything, I probably prefer this code — but it does increase our code size a bit.
2. I'm pretty dissatisfied with how query parameters actually appear in the Swagger UI, but the schema itself looks OK enough.
3. We'd have to develop some patterns around when types get inlined into other types, naming, etc.
4. The responses(...) tuples are going to be repeated and mixed and matched in different ways in different controllers², so we'll probably need to develop some macros to handle the common cases.

Also, a bonus issue is that I had never fully grappled with the idiosyncrasies of the search API's parameters and how they fit together, but that's a separate issue.

Realistically, many (all?) of these probably growing pains that we'd have to deal with regardless if we do move towards a more explicitly defined API schema, but it's worth calling them out, I think.

So I guess my fundamental question here isn't so much around this PR (it's fine), but more whether it's complete enough for us to merge as-is, and what we consider to be our baseline for including a controller in the OpenAPI schema if we decide that we're going to go in that direction.

What do you think, @Turbo87?

Footnotes

1. I've been playing around with wiring up auto-generation of a TypeScript client from a Rust utoipa server, and then using that client from a frontend webapp (in this case, using SvelteKit, but the exact details aren't important). It's very slick when it's all wired up. ↩

2. Almost all controllers are going to be able to return 200, 500, and 503. From there, it gets hairier. ↩

[LawnGnome]

I thinking a combination of both would probably be most informative: something like 2024-12-11-21-45-00-4b33fdda250e7a8ebb4c40eff4d6205ce09c7e2a.

[Turbo87]

That said, I'm wondering if this is too bare bones right now, even for an initial exploratory PR.

yeah, I intentionally kept it small for now to reduce the size of the diff and focus mostly on the plumbing first. what you did in that extra commit is roughly what I had in mind for a followup.

I guess my fundamental question here isn't so much around this PR (it's fine), but more whether it's complete enough for us to merge as-is, and what we consider to be our baseline for including a controller in the OpenAPI schema if we decide that we're going to go in that direction.

my idea was to migrate all existing routes into very basic utoipa::path annotated routes to migrate the router.rs in one go and then incrementally improve the individual endpoints in follow-up PRs. IMHO the generated docs don't need to be perfect on inception as long as we clearly indicate that this is experimental/WIP.

[eth3lbert]

The OpenAPI description will be available at https://crates.io/api/openaip.json, with a viewer available at https://crates.io/api/openaip.

I guess we could also just expose the OpenAPI JSON in production and the viewer in the development environment initially? Since I expect it would be used more for development purposes and most viewers provide a way to import from a given JSON URL or file.

[Turbo87]

@eth3lbert I mostly included it for local development for now, but I just found https://zudoku.dev/demo?api-url=http://127.0.0.1:8888/api/openapi.json as an alternative, so I've remove the Swagger UI commit, including the CSP header change.

[LawnGnome]

yeah, I intentionally kept it small for now to reduce the size of the diff and focus mostly on the plumbing first. what you did in that extra commit is roughly what I had in mind for a followup.

OK, then I'm fine merging this and figuring the rest out as we go. I just wanted to get a sense of how you were thinking about it first.

I do think we'll actually want the UI more for production in the longer term, but I'm totally fine dropping it until we have some minimal level of coverage across the API, and not just one or two routes.