I wonder if we can improve the Gardener's ability to suggest great topics by showing it more documents at a time. Currently, I have some Gardener runs that only have three or four files, even when I am using a 700k context window. Simply: I have some big notes. But that size is probably overkill for topic resolution.
Could we use a small, fast, cheap model like gemini-2.5-flash (or possibly gemini-flash-lite-latest) to summarise large documents and then inject the summary, rather than the full document, for the Gardener? The idea would be to ensure we always have, say, 200+ documents for the Gardener to consider (when using models with 1M+ input tokens window), increasing the chance that it finds interesting, non-trivial subjects that are widely relevant for my vault.
For example, for documents over 12,000 characters (about 3,000-4,000 tokens) we ask gemini-flash-lite-latest to summarise in 12k characters or less. That uses O(3,500) tokens per document, allowing us to fit 200 documents in my 700k limit. (Since some documents, most documents, even, will be smaller, we should usually have more than 200 in the context for the Gardener.)
User settings: We would make both the 12,000 character limits (trigger for summary and the limit for the summariser prompt) and the summariser model selection (e.g., gemini-2.5.-flash or gemini-flash-lite-latest) into user settings (which, ideally, could even be a different provider, e.g., use a local Ollama model for summary and better cost). We should allow users to opt-out of summarising (effectively setting the trigger limit to infinite). We would increase the max documents default for the Gardener, which I think is currently 100, to something like 500 or even higher.
Caching: We would likely want some way to cache the summaries. These are specialised summaries for the purposes of topic identification so I am not sure they should go in the front matter (except, perhaps, as an option) but think they may be better in our database, though that requires careful purging of stale cache entries.
Prompt engineering: We would need to think carefully about the prompt to the summariser model: some experimentation may be needed and it should certainly be a user setting. (Summarise, but keep all important keywords as they are, I think would be part of the prompt.)
(I vaguely wonder if there is a possible positive side effect that the summariser model can provide the summary in the main language, should the user wish so, helping to normalise the ontology of multi-lingual vaults. But let's not worry about this initially.)
I wonder if we can improve the Gardener's ability to suggest great topics by showing it more documents at a time. Currently, I have some Gardener runs that only have three or four files, even when I am using a 700k context window. Simply: I have some big notes. But that size is probably overkill for topic resolution.
Could we use a small, fast, cheap model like
gemini-2.5-flash(or possiblygemini-flash-lite-latest) to summarise large documents and then inject the summary, rather than the full document, for the Gardener? The idea would be to ensure we always have, say, 200+ documents for the Gardener to consider (when using models with 1M+ input tokens window), increasing the chance that it finds interesting, non-trivial subjects that are widely relevant for my vault.For example, for documents over 12,000 characters (about 3,000-4,000 tokens) we ask
gemini-flash-lite-latestto summarise in 12k characters or less. That uses O(3,500) tokens per document, allowing us to fit 200 documents in my 700k limit. (Since some documents, most documents, even, will be smaller, we should usually have more than 200 in the context for the Gardener.)User settings: We would make both the 12,000 character limits (trigger for summary and the limit for the summariser prompt) and the summariser model selection (e.g.,
gemini-2.5.-flashorgemini-flash-lite-latest) into user settings (which, ideally, could even be a different provider, e.g., use a local Ollama model for summary and better cost). We should allow users to opt-out of summarising (effectively setting the trigger limit to infinite). We would increase the max documents default for the Gardener, which I think is currently 100, to something like 500 or even higher.Caching: We would likely want some way to cache the summaries. These are specialised summaries for the purposes of topic identification so I am not sure they should go in the front matter (except, perhaps, as an option) but think they may be better in our database, though that requires careful purging of stale cache entries.
Prompt engineering: We would need to think carefully about the prompt to the summariser model: some experimentation may be needed and it should certainly be a user setting. (Summarise, but keep all important keywords as they are, I think would be part of the prompt.)
(I vaguely wonder if there is a possible positive side effect that the summariser model can provide the summary in the main language, should the user wish so, helping to normalise the ontology of multi-lingual vaults. But let's not worry about this initially.)