Doubt from new readme.md

[nnnarvaez]

Hi, first of all thanks for your work!

I had tested the TQ+ build with Gemma4 and are seeing a lot of Reasoning Loops I came here to see if others were experiencing/seeing the same, and saw this update to the readme. (see image)

So i wanted to ask: Do i need to TQ quantize the models to use TurboQuant with this build?

I apologize if this is a noob question, i am trying to teach my self how all this works, and really appreciate whatever feedback i get.

[vboyk]

Tried quantizing the Gemma4-26b model to TQ4_1S. When I ran the model and typed "hello", I got a infinite stream of hallucinations on the reasoning step.

[mgeldi]

I tried the very same thing with many of the models.
The issue is the "GGML_TQ_NATIVE=1".
This parameter is the thing that is supposed to use the cuda path to boost speeds by a lot, according to the readme.
However, I tried MANY models, including the Gemma 4 family. I'm unable to get them to work with the "native" path. Meaning: I get the regular speeds.

Currently, fastest models for me are TQ4 with MTP. However, that path is also completely broken on any MoE model.

[mgeldi]

I belive I've missunderstood the GGML_TQ_NATIVE=1 parameter.
It turns out, this disables the default TQ4_1S -> Q8 load-time conversion. So it would actually make it slower. The "faster path" is with this variable unset. I believe the "bug" comes from the fact that you'd need to set -b 1 and -ub 1 to make it work. Didn't test this yet, since it's basically not what I want anyways.

I tested this and I can verify it. Models that run TQ3_1S with MTP are significantly slower than TQ4_1S models without MTP.