Not .well-known

[darrelmiller]

Not .well-known

Unfortunately .well-known has become too well known and we need to stop abusing it. RF8615 states:

Some applications on the Web require the discovery of information
about an origin [RFC6454] (sometimes called "site-wide metadata")
before making a request.

The use of .well-known for discovering applications like MCP Servers and A2A agents that are hosted on some origin is a misuse of .well-known, because applications are not "site-wide metadata".

Interestingly, even the specfication notes that .well-known is really not a great idea in the first place.

Aren't well-known locations bad for the Web?
They are, but for various reasons -- both technical and social --
they are sometimes necessary. This memo defines a "sandbox" for
them, to reduce the risks of collision and to minimise the impact
upon preexisting URIs on sites.

https://www.rfc-editor.org/rfc/rfc8615.html#appendix-A

It goes on to clarify why .well-known MUST be defined immediately after the authority component of the URI and must not contain any path segments:

Why aren't per-directory well-known locations defined?
Allowing every URI path segment to have a well-known location
(e.g., "/images/.well-known/") would increase the risks of
colliding with a preexisting URI on a site, and generally these
solutions are found not to scale well because they're too
"chatty".

The obscure character sequence of ".well-known" was chosen to minimize the risk of colliding with preexisting URIs on sites. Prefixing the .well-known with some other path segment negates that benefit.

For more details on the risks of colliding with preexisting URIs on sites, refer to BCP 56 Building Protocols with HTTP

But what about OpenID Connect Discovery?

You may have noticed that OpenID Connect Discovery uses .well-known that is prefixed by an issuer component that allows a path.

GET /issuer1/.well-known/openid-configuration HTTP/1.1
Host: example.com

Using path components enables supporting multiple issuers per host. This is required in
some multi-tenant hosting configurations. This use of .well-known is for supporting
multiple issuers per host; unlike its use in RFC 5785 [RFC5785], it does not provide
general information about the host.

RFC5785 is the predecessor to RF8615 and is now obsoleted by it. The use of .well-known in OpenID Connect Discovery is an intentional violation of the specification that defines .well-known. Whether you agree or disagree with this decision, it is clear that it causes some confusion.

But MCP uses Protected Resource Metadata with path segments!

Protected Resource Metadata uses a .well-known URI and it supports path segments, but they go after the .well-known, not before.

GET /.well-known/oauth-protected-resource/resource1 HTTP/1.1
Host: resource.example.com

My guess as to why this is the case, is that PRM is an IETF specification, and the IETF review process would likely have not allowed a direct contradiction of the .well-known specification. Having resource segments after the .well-known segment does not risk URL collisons.

https://www.rfc-editor.org/rfc/rfc9728.html#section-3.1-5

And we wonder why people are confused about .well-known. Using specifications for standardized behavior is usually a great practice. But when we start creating variants usages that contradict each other, it really doesn't help.

How does .well-known help discovery?

The idea is that given a URL, a developer can extract the host component, concatenate a "well-known" path segment, and then make a request to that URL to discover information about the origin. For example, given the URL https://example.com/some/path, a developer can extract the host component example.com, concatenate the "well-known" path segment to get https://example.com/.well-known/, and then make a request to that URL to discover information about the origin.

If you assume that a host only has one MCP Server, or one A2A agent, then yes, you can use .well-known to discover that server or agent. But how often is that the case? Why would we take a dependency on a discovery mechanism with such a significant constraint?

We could choose to follow in the path of others and pick somewhere to put additional path segments. However, let's consider what it is that we are discovering when it comes to A2A Agents and MCP Servers. We are discovering an agent card or server card that points to the Agent or MCP Server along with metadata about it.

If we need Host + Some Identifying Path + .well-known to find the agent/server card, then couldn't someone just share a URL directly to the agent/server card instead? Why do we need .well-known at all?

Can we make this easier?

I have been advocating to the A2A TSC that we can make this much simpler by enabling agents to return their Agent Card at their agent URL. In A2A, a GET request on the agent URL is currently undefined, but we could define it to return the Agent Card/AI Card. This would eliminate the need for .well-known and make discovery much easier.

MCP currently does have use for GET on the MCP Server URL, however, it is expected that clients will send Accept: text/event-stream to that URL to establish an SSE connection. We could define that if a client sends a GET request without that Accept header, then the MCP Server could return a Server Card. This would eliminate the need for .well-known in MCP as well.

This does not preclude the inclusion of agent cards/server cards/ai cards in other locations, like registries, but it does mean that we would not need to rely on .well-known for discovery. An Agent or MCP Server has a card to describe itself and a HTTP GET on that entry point URL would return that card. Simple!

[Fannon]

@darrelmiller thanks for the extensive write up, this is good reading!

I like to add one more argument for your case:

• Not every agent / MCP developer will have access to the root /.well-known/ of the server, in this case the approach will not work and will need a failover solution (like directly providing the full URL to the discovery entry-point URL)

I'm not sure every assumption you make here has to be done like this:

• I agree, assuming there is juts one Agent or MCP server on a host is wrong, in my companies case this would never work. But this is not a fault of .well-known, it's how its used.
• Yes, we stretch the meaning of the RFC a bit, but I guess we do that with a lot of older web standards and they mostly hold up well enough. It's better than re-inventing something completely new I think.
• How would you know the Agent URL in the first place? I think this is the real purpose of a .well-known URI entry point. It is basically a list of all relevant links and necessary information to understand what you get and how to get it. It's not the resources itself you should put there.

I created a SEP here, having this 2-stage design mentioned above. It doesn't really contain any shared model between MCP and A2A, from what I've ready somewhere else - there might be good opportunities to do this, too.

• SEP 0014 - AI Card Discovery #14

[darrelmiller]

@Fannon I'm glad you took it in the spirit of trying to be constructive, which is what I had hoped for, but I tend to get a bit ranty about this topic.

Yes, we stretch the meaning of the RFC a bit, but I guess we do that with a lot of older web standards and they mostly hold up well enough. It's better than re-inventing something completely new I think.

Putting the resource path at the end of the URL is "stretching". Putting it before is breaking the standard. But putting it after is also kind of weird from a design perspective.

How would you know the Agent URL in the first place?

How do you know the host? You either need to start with a URL on that host, or you are a web-crawler and you are trawling through domain registries.