Live - "client-content" without end of turn blocks voice response.

[tiagoefreitas]

Description of the bug:

I was trying to add hidden context to gemini live audio to give it instructions during audio conversations that are hidden from the users (not only at the beggining). The gemini docs say we can add previous context with clientcontent but the model always responds, even if I add the model response as a turn with turncomplete=true like this:

{
  "clientContent": {
    "turns": [
      {
        "role": "user",
        "parts": [
          {
            "text": "Context xxx"
          }
        ]
      },
      {
        "role": "model",
        "parts": [
          {
            "text": "ok."
          }
        ]
      }
    ],
    "turnComplete": true
  }
}

The model will still reply in audio with "ok" again

And if turncomplete is false, the model will not reply to audio content until I send a text message to end the turn.

If I instruct in the prompt not to reply, it seems the model was trained to always reply so 90% of the times it says “ok” or something.

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[MarkDaoust]

Is this something you can't do with the system instructions at the start of the conversation?

And if turncomplete is false, the model will not reply to audio content until I send a text message to end the turn.

I haven't tried mixing in client content without an end of turn. But this is not the behavior I would expect.

[tiagoefreitas]

@MarkDaoust it doesn't follow the instructions 90% of the time, it still replies.
This is kind of expected as the model was likely trained with pairs of request/reply.

but with turncomplete=true and a model role turn, it should not reply again, but it does.

[MarkDaoust]

Thanks for the feedback, I've raised this with the internal API team.

[MarkDaoust]

We're going to make turn_complete the default so this doesn't happen by accident.