Welcome to staticSearch Discussions!

[martindholmes]

👋 Welcome!

We’re using Discussions as a place to connect with other users and potential users of staticSearch. We hope that you:

• Ask questions you’re wondering about.
• Share ideas and pitch suggestions more informally than you might do when raising an issue.
• Engage with other community members.
• Welcome others and are open-minded.

To get started, comment below with an introduction of yourself and tell us about what you use staticSearch for, or what you're thinking about using it for.

[mprell]

First of all, many thanks for the staticSearch and its continuous maintenance! In my opinion, there is currently nothing comparable that fits so well with X technologies and continues to be maintained. And staticSearch seems to me to be a great contribution not only to longevity but also to the greening of DH projects.
I'm Martin and I have implemented staticSearch in a website for the presentation of diaries of sea voyages of the Moravian Church in the 18th century.
Search site: https://www.moravians-at-sea.de/suche.html
Data and configuration file etc.: https://github.com/mprell/moravians-at-sea-Prod
I would like to ask two questions:

1. Is there already experience, what amount of data (e.g. the number of json files in the stems folder or the size of the indexed static xhtml files) noticeably slow down the performance of StaticSearch? I am asking in order to be able to estimate whether staticSearch is also suitable for projects with 100,000 and more letters and diary entries or something similar. It is difficult to find out what quantity of data the reference projects mentioned here use.
2. Does anyone know of a project that integrates a German stemmer or is the implementation of a German stemmer possibly even in progress? So far I have been using wildcard searches.

[joeytakeda]

Hi @mprell ! Thanks for the kind words and support :-) Great to see your project too -- we'd be glad to list it on the "Projects Using StaticSearch" page you link, if that's OK with you?

To answer your questions:

Is there already experience, what amount of data (e.g. the number of json files in the stems folder or the size of the indexed static xhtml files) noticeably slow down the performance of StaticSearch? I am asking in order to be able to estimate whether staticSearch is also suitable for projects with 100,000 and more letters and diary entries or something similar. It is difficult to find out what quantity of data the reference projects mentioned here use.

We have some better stats and figures here for document collections: https://www.balisage.net/Proceedings/vol27/html/Takeda01/BalisageVol27-Takeda01.html#statistics

In terms of front-end: With lots of documents, there can be a slight delay in the initial start-up of staticSearch since it loads in all of the titles on initialization. Otherwise, though, there's no performance hit per search when the size of site increases. staticSearch only searches for the string that was searched, so, in other words, the front-end JS doesn't know or care about the rest of the stems, just the ones it is looking for.

In terms of back-end: lots of documents can slow down the indexing processing. You may need to give ant additional memory, depending on the size of the document collection (see #323 and #157 for more details).

Does anyone know of a project that integrates a German stemmer or is the implementation of a German stemmer possibly even in progress? So far I have been using wildcard searches

I'm not aware of a German implementation in progress, but since there is a Porter implementation, this is something that could be plugged in:

http://snowball.tartarus.org/algorithms/german/stemmer.html

It looks like there's already one in JS (https://github.com/klaemo/snowball-german/blob/master/index.js), so this would just need to be implemented in XSLT as well.

We do have some "blue sky" plans for ways of adopting all of the Porter stemmers in one go (#244 ), but that's been put a bit on the back-burner while we're working toward a new major release.

[mprell]

Hi @joeytakeda
It would be a great pleasure to see my project in the staticSearch projects list. Thank you!
Many thanks for the hint to your Balisage article, as well; I didn't know it yet. This allows me to better assess the performance effects of large amounts of data on the front-end.
I'll take a closer look at the Porter Stemmer for German, many thanks!