Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

super-high cost when searching for documents #1097

Open
Roiocam opened this issue Feb 3, 2024 · 10 comments · May be fixed by sbt/sbt-paradox-material-theme#75
Open

super-high cost when searching for documents #1097

Roiocam opened this issue Feb 3, 2024 · 10 comments · May be fixed by sbt/sbt-paradox-material-theme#75

Comments

@Roiocam
Copy link
Member

Roiocam commented Feb 3, 2024

Reproduce

Just search from https://pekko.apache.org/docs/pekko/current//index.html

Happened on:

  • m1 macOS Safari
  • m1 macOS Chrome
  • iPhone Safari

I think this maybe a issue from paradox.

Self-diagnosis

Just a quick investigation from myself, the profiler shows that most of the time costs from a reduce function.

I am not a frontend developer, can not deep dive to it.

截屏2024-02-04 00 43 51 截屏2024-02-04 00 49 34 截屏2024-02-04 00 50 09

Performance Profile

This is my profiler result.

Trace-20240204T004355.json

@Roiocam
Copy link
Member Author

Roiocam commented Feb 3, 2024

After some digging, I just found the issue caused by a very large search_index.json, it will load when the user clicks the search input box.

截屏2024-02-04 02 34 49

This javascript file comes from the upstream repo: https://github.com/squidfunk/mkdocs-material, in the latest documentation website of them, it will pre-loaded search index, and the index file size is only 200kb.

Then I checked the content of pekko search index file, which is very large json array.

截屏2024-02-04 02 48 11

Finally, I decided to look up the answer in the upstream repo, and I found this: squidfunk/mkdocs-material#904

@pjfanning
Copy link
Contributor

pjfanning commented Feb 3, 2024

It is Paradox that builds this json file. I'm not sure what we can do in the short term. Maybe, we should look at trying to offload the search to Google instead of using our own search with its own JSON file.

See https://poi.apache.org/ - another Apache project site - its search is Google based.

@pjfanning
Copy link
Contributor

An example Google search

actor site:pekko.apache.org

https://www.google.com/search?q=actor+site%3Apekko.apache.org

@He-Pin
Copy link
Member

He-Pin commented Feb 3, 2024

IIRC, it was using the algolia for indexing and searching

@Roiocam
Copy link
Member Author

Roiocam commented Feb 4, 2024

IIRC, it was using the algolia for indexing and searching

I don't think so, only akka uses algolia, we use paradox-material-theme which depends on mkdocs-material

Maybe, we should look at trying to offload the search to Google instead of using our own search with its own JSON file.

I am using Docusaurus at work, they use the same way to implement offline search but won't stuck UI thread.

@Roiocam
Copy link
Member Author

Roiocam commented Feb 4, 2024

I will try to handle this issue via an update paradox-material-theme.

@mdedetrich
Copy link
Contributor

I will try to handle this issue via an update paradox-material-theme.

Note that sbt-paradox-material-theme was just transferred to the sbt org/community and we are currently in the process of making the necessary changes so it may take a bit of time before we can get to deploying the change

@Roiocam
Copy link
Member Author

Roiocam commented Feb 4, 2024

I will try to handle this issue via an update paradox-material-theme.

I tried to upgrade to the latest mkdocs-material and found that the way it was built changed after the 5.x version, and only up-to 5.x version can avoid search blocking the main thread, and it seems that the implementation is through the need to precompile a search index file.

I think this upgrade is no less difficult than rewriting upstream https://github.com/sbt/sbt-paradox-material-theme. We should consider replacing the search implementation to solve this ISSUE.

@squidfunk
Copy link

Author of Material for MkDocs here. v5.x is from 2020, so pretty old. We made significant improvements on search in 9.x, which should be twice as fast and significantly cut down on index size, and will be replacing it with an entirely new implementation that will be much faster and more powerful in the near future. Related:

@mdedetrich
Copy link
Contributor

@Roiocam sbt-paradox-theme has been ported over to the sbt org/package and 0.7.0 has just been published so you are now free to make changes against sbt-paradox-theme.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants