[EDU-2101] - Update llmstxt.ts to structure llms.txt a bit cleaner

[GregHolmes]

Description

Restructured llms.txt generation with product categorisation This update reorganises the generated llms.txt file from a flat list into a structured hierarchy with categories (Platform, Pub/Sub, Chat, Spaces, LiveObjects, LiveSync, Asset Tracking) and subcategories based on URL patterns. Added a more informative preamble describing Ably's core features, support for Kotlin and React languages, and logic to skip redundant language-specific URLs when the language already matches the page slug (e.g., prevents /getting-started/go?lang=go). The structured format improves LLM navigation and discovery of relevant documentation.

Attached an example of the llms.txt generated:

llms.txt

Build output:

info onPostBuild: Found 205 pages to place into llms.txt (26 textile, 179 MDX)
info onPostBuild: Successfully wrote llms.txt with 591 pages

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch EDU-2101-Update-llms.txt-to-be-better-structured

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[kennethkalmer]

@GregHolmes should we even have asset tracking in here since it is being deprecated?

[m-hulbert]

This is a really great idea, Greg.

There's a few tweaks I think we need to make to Platform and Pub/Sub and I also wonder if we should look at breaking down other products too (at least maybe Chat and LiveObjects) into further categories. WDYT?

[m-hulbert]

This only has 1 at the moment and is for Chat... Let's move it into platform for now maybe.

[m-hulbert]

I think there should be some additional sub-categories here:

• architecture
• pricing
• integrations

[m-hulbert]

Let's remove this. I think it's time we removed the page too.

[m-hulbert]

This seems to be used for the Pub/Sub basics and advanced pages which seems odd. I wonder if we need an 'unclassified' product level one. So it's just a category of 'Pub/Sub' without a sub-category. WDYT?

[m-hulbert]

I think APIs should sit within the relevant product.

[m-hulbert]

This just pulls in the product landing page. See my comment about an empty sub-category suggestion.

[m-hulbert]

Lost my comment these;

• Let's remove 'integrations' from a heading about products.
• Should we list all languages in the SDK row to make it complete?