Search: Research Preview 1 🧪

[squidfunk]

Important

This PR presents an early preview of our new search feature for gathering user feedback. It is not yet ready for production and is subject to modifications. Currently, it lacks support for search suggestions and does not cater to Japanese, due to the absence of segmentation functionality.

Overview of this PR

This PR offers a sneak peek at our revamped search functionality. While still in development and devoid of UI changes (for now), it marks a departure from the current implementation. Unlike the existing system that forms a union of all query terms, this new approach leverages an intersection. We are eager to know whether you find this change enhances or diminishes the search results. To illustrate, here's a comparative example:

Query	master Branch	This PR
custom	54 documents	54 documents
custom code	62 documents	13 documents
custom code annotations	62 documents	6 documents

While the ranking of search results remains unchanged, the introduction of more terms now leads to fewer results, contrary to the prior behavior that yielded more results. Our tests indicate this approach significantly improves the typeahead experience. This overhaul was a substantial effort, as rewriting a codebase with thousands of lines (Lunr.js + extensions) from scratch is no small feat.

Your feedback is essential 🙋‍♀️

We want to co-create the search feature with our community. Our goal is to deliver an unparalleled search experience to all Material for MkDocs users, not just our sponsors. This journey begins with a revamped interface and, more critically, heightened accuracy to help users find what they need quickly and effortlessly. After establishing a solid foundation, we will explore many exciting extensions, which, as always, will first be released exclusively to our sponsors. But first, we aim to elevate the experience for everyone.

To test this version featured in the PR, you can install it using pip:

pip install git+https://github.com/squidfunk/mkdocs-material.git@spike/search-preview-1

We warmly invite you to share your honest feedback on this PR! Please note that this PR is specifically designed for feedback collection and will not be merged. Your insights are invaluable to us! ☺️

[alexvoss]

Wonderful to see the fruits of your hard work see the light of day! Just one question: there is no need to configure anything, just use the preview version to build a site?

[squidfunk]

You can just install over any existing installation, no need for reconfiguration ☺️

[vedranmiletic]

Gave it a quick try locally on our site (mostly teaching materials, 450 pages, 550k words). Much better user experience, without the unnecessary clutter of irrelevant results. I haven't benchmarked, but it also feels faster, which is expected since less content gets displayed.

[squidfunk]

Thanks for trying @vedranmiletic! It is indeed expected to be faster. Funnily, in the end, it will be even faster than it currently is. In order to integrate it into the current architecture, I needed to jump through some hoops. Once everything is finished, 500 or even 1,000 pages shouldn't be a problem anymore. Additionally, we could make it possible to shard among multiple workers to support 2,000 or even 4,000 pages, if users need that.

[guerda]

Great news and I can already see huge improvements. I tested it out on a documentation with > 600 pages and > 36k lines of markdown.
The results come faster and narrowing down makes the search results way more relevant. I tested out German umlauts and other special characters, which work smoothly.

What does not work is quotes: If you use two single or double quotes, the results are empty, even if the exact search clause occurs in a page. It looks like the search does not interpret the quotes, but takes them as a literal.

[squidfunk]

What does not work is quotes: If you use two single or double quotes, the results are empty, even if the exact search clause occurs in a page. It looks like the search does not interpret the quotes, but takes them as a literal.

Good catch! Yes, we currently don't have support for special syntax, so +foo -bar "baz" does not work. We deliberately left it out to see whether people would miss it ☺️ Of course, we can add it back as part of a plugin.

[PaulSonOfLars]

Just tried this - minus the comments mentioned above, it works a charm! Really impressive stuff 😄
Looking forward to seeing the full capabilities of the final result!

[squidfunk]

Thanks for the feedback! The next research preview is ready to be tested in #6372! It has a new ranking function, and have better accuracy, as can be seen in the example screenshots in the PR.