[BUG] Citation Highlight feature rarely works under small LLMs

[Yanjieij]

Description

The LLM I use is Llama3.1-8B running on Ollama.

All functions running smoothly but the citation highlight. For most of the time, there is no green high light in my Information Panel, but rarely it works properly (less than once in ten times).

Then I check the log, found that even when the CitationPipeline returns "evidences" in json, the subsequent output could still be "Got 0 cited docs".

I have no idea about this, maybe the text matching rules are too strict? Or the prompts of CitationPipeline is too hard to understand, so the small-scale LLMs cannot give the correct output in evidences json?

BTW, under my conditions, the new "inline" feature could work, but always with only one citation and a very short answer.

Thanks for your helping.

Reproduction steps

1. Set the options correctly: Citation style->highlight on.
2. Conduct the QA.

Screenshots

Logs

CitationPipeline: invoking LLM
CitationPipeline: finish invoking LLM
CitationPipeline: {"evidences":["...(Omit Here)"]}
Got 0 cited docs

Browsers

Chrome

OS

Linux