[SimBe195]

This PR adds a new LabelScorer which can be used for Speech LLMs. Structurally it is roughly similar to the StateManagedOnnxLabelScorer from #206. The main difference is how the initial states are constructed. While the StateMangedOnnxLabelScorer only gets one or more start labels as initial history while the PrefixSpeechLmOnnxLabelScorer has a separate initializer ONNX model which gets an initial-prompt, encoder-states and suffix-prompt to produce the initial scores and states. Also, the StateManagedOnnxLabelScorer feeds in the encoder-states at every scoring step while the PrefixSpeechLmOnnxLabelScorer only feeds the encoder-states for the initialization.

Depends on #287.

Prompts can be given as token ids (initial-prompt-labels, suffix-prompt-labels) or as text
(initial-prompt, suffix-prompt), tokenized with the tokenizer configured under tokenizer.*.

[larissakl]

I have one suggestion for this label scorer (I can also implement this myself in a separate PR). Speech LLMs usually have a very large vocab, but only a very small percentage of label extensions survive in the beam search. However, all scores are copied from GPU to CPU in every step, therefore I think it would make sense to make RASR copy only the top-max-beam-size scores per hypothesis.
Concretely, I thought we could introduce a SparseScoreSpan (parallel to DenseScoreSpan) and a new SparseScoreAccessors, in which the getScore() function returns the actual score for labels in the list and the max score for all others. In this label scorer here, we would then need optional step model outputs (e.g. topk-scores and topk-indices) in the io-map. If they are set, only those are read, sorted, de-duplicated and cached as sparse scores. The actual top-k "pruning" then has to be implemented by the ONNX model, which should however not be a large change.

[SimBe195]

I have one suggestion for this label scorer (I can also implement this myself in a separate PR). Speech LLMs usually have a very large vocab, but only a very small percentage of label extensions survive in the beam search. However, all scores are copied from GPU to CPU in every step, therefore I think it would make sense to make RASR copy only the top-max-beam-size scores per hypothesis. Concretely, I thought we could introduce a SparseScoreSpan (parallel to DenseScoreSpan) and a new SparseScoreAccessors, in which the getScore() function returns the actual score for labels in the list and the max score for all others. In this label scorer here, we would then need optional step model outputs (e.g. topk-scores and topk-indices) in the io-map. If they are set, only those are read, sorted, de-duplicated and cached as sparse scores. The actual top-k "pruning" then has to be implemented by the ONNX model, which should however not be a large change.

Yes I think this is a great idea but in a separate PR. It would also ideally come with some modifications to the search so that it only actually iterates over the non-max labels instead of still looping over all possible successors.