RuntimeError: Bad Response to AzureGuidance Request (404 Not Found)

[shandilyabh]

I’m encountering a 404 Not Found error when trying to use AzureGuidance. I’m following the typical structure (as mentioned here: https://github.com/microsoft/Phi-3CookBook/blob/main/code/01.Introduce/guidance.ipynb) for setting up, but I’m not sure if I’ve misconfigured something or if there’s an issue with how Guidance interacts with an Azure deployment endpoint.

phi3_url = "https://model.region.models.ai.azure.com"
phi3_api_key = "api_key"
phi3_lm = AzureGuidance(f"{phi3_url}/guidance#auth={phi3_api_key}")

lm = phi3_lm

with system():
    lm += "You are a helpful assistant."
with user():
    lm += "what's your name. tell me something about yourself"
with assistant():
    lm += gen(temperature=0.5, max_tokens=128)

The error message:

RuntimeError: Bad response to Guidance request
Request: https://model.region.models.ai.azure.com/v1/chat/completions/guidance (authorization: Bearer ie...ab)
Response: 404 Not Found
NOT FOUND

[ahmed33033]

I wasn't facing this issue when you posted the question, but now, after running it again, I'm getting the same error. 😢

[shandilyabh]

I wasn't facing this issue when you posted the question, but now, after running it again, I'm getting the same error. 😢

actually, I forgot to update the discussion. guidance actually doesn't support phi3. more specifically, it is an experimental tool that currently only supports phi 3.5

cheers

[ahmed33033]

Oh I didn't event realize you were referring to PHI-3. I was running the PHI-3.5 and I was getting the 404 issue 😂

[shandilyabh]

now that's new 😂

[shandilyabh]

Guidance does not support Phi 3. therefore, regardless of the endpoint and deployment, it won't work with it.

[serdarsemen]

Hi
I have used phi3.5; same 404 error though I have used the below setup steps. Any help please.

RuntimeError: Bad response to Guidance request
Request: https://phi-3-5-mini-instruct-rwxob.eastus2.models.ai.azure.com/guidance (authorization: Bearer qj...a7)
Response: 404 Not Found
NOT FOUND
Output is truncated. View as a scrollable element or open in a text editor.

Setup

1. Install Guidance with pip install guidance --pre
2. Deploy a Phi 3.5 mini endpoint in Azure by going to https://ai.azure.com/explore/models/Phi-3.5-mini-instruct/version/2/registry/azureml and clicking the "Deploy" button
3. Store your endpoint's API key in an environment variable called AZURE_PHI3_KEY and the URL in an environment variable called AZURE_PHI3_URL

[ahmed33033]

Hey!
It seems that the integration with Phi models on Azure is not supported at the moment (was taken down recently). Harsha informs us about it here:
#1074