Not able to set seed parameter in generation_config

[zya]

Description of the bug:

Both of the following results in an error.

model = genai.GenerativeModel(
        model_name="models/gemini-1.5-pro-001",
        system_instruction=(...),
        generation_config={
           ...,
           "seed": 1
        }
    )

model = genai.GenerativeModel(
        model_name="models/gemini-1.5-pro-001",
        system_instruction=(...),
        generation_config={
           ...,
        }
    )
model.generate_content(prompt, generation_config={"seed": 1})

Error:

ValueError: Unknown field for GenerationConfig: seed

Seed is mentioned in the docs here:

• https://cloud.google.com/vertex-ai/docs/reference/rest/v1/GenerationConfig
• https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference

Actual vs expected behavior:

Actual: Error is thrown when seed is used in generation_config
Expected: seed to be supported and passed to the underlying generate content API.

Any other information you'd like to share?

No response

[Gunand3043]

Hi @zya,

You can set the seed parameter in generation_config only when using the Vertex AI SDK. For reference, you can check this link. If you are using Python SDK, here is the documentation link.

[zya]

@Gunand3043 Thanks. Do you happen to know why seed isn't supported? And more generally, what is the purpose of having these two API's separate? Are we supposed to use Gemini API for experimentation and Vertex for production?

[Gunand3043]

At the moment, I'm not exactly sure why 'seed' isn't supported in the Google AI SDK.

I kind of agree with your idea of using the Google AI SDK for experimentation and the Vertex AI SDK for production. Since Vertex AI SDK offers more features than the Google AI SDK, such as deploying generative AI applications and building solutions end-to-end, Vertex AI is the better option. For a more detailed comparison, please check out the below link.

• https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/migrate-google-ai#google-ai

Hope it helps.

[zya]

In that case perhaps we want to change the type of this ticket to be a feature request to add support for seed in the Gemini API?

[t1u1]

What does "seed" not supported in Gemini API mean? That it is constant or random?

In other words, I would like to know if the responses for the same input and configuration are going to be the same, or they will be different for each invocation?

[MarkDaoust]

Right now it's different each time. They're working on it.

[sr228822]

any update on this? Unclear from above discussion whether theres plan to support seed in genai.GenerativeModel interface or whether I should switch to VertexAI?