FoundryLocal-Cloud

FastAPI server exposing OpenAI-compatible endpoints for Phi-4 ONNX models, deployable to Azure App Service with Azure File Share–mounted models.

1) What you need

• Azure CLI logged in: az login
• Bash shell (WSL / Git Bash / Linux)
• ONNX model files (recommended: Phi-4-Instruct/Chat variant in int4 ONNX format)
• API key you will pass as Authorization: Bearer <API_KEY>

2) Quick Azure deploy (App Service)

export RG=phi4-rg
export LOCATION=eastus
export ACR_NAME=phi4acr123
export APP_NAME=phi4app123
export API_KEY="replace-with-strong-key"

./deploy.sh $RG $LOCATION $ACR_NAME $APP_NAME $API_KEY

What this does:

• Creates resource group, storage account, file share
• Mounts the share to /app/model in App Service
• Builds/pushes image to ACR (if missing)
• Configures app settings (API_KEY, WEBSITES_PORT, MODELS_ROOT)
• Restarts the web app

To tail logs after deploy:

az webapp log tail -g $RG -n $APP_NAME

3) Model storage

• Default: Azure File Share mounted to /app/model (no image rebuild needed)
• To update a model: upload new files to the share, then restart the web app

  az storage file upload-batch --account-name <storage> --destination <share> --source <local_model_dir>
  az webapp restart -g $RG -n $APP_NAME

• Embedding models in the image is not default. If you want that, extend Dockerfile with COPY model/ /app/model and rebuild.

4) Use the API

Set BASE_URL=https://<app_name>.azurewebsites.net (or http://localhost:8000 when local).

• List models

curl -H "Authorization: Bearer $API_KEY" $BASE_URL/v1/models

• Chat completion

curl -X POST $BASE_URL/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"phi4","messages":[{"role":"user","content":"What is 2+2?"}],"stream":false}'

• Shortcut route

curl -X POST $BASE_URL/model/phi4/chat \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}]}'

5) Run locally

Python dev:

python -m venv .venv
source .venv/bin/activate          # Windows: .\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
export API_KEY=dev-key
export MODELS_ROOT=/app/model       # or your local path
uvicorn app:app --host 0.0.0.0 --port 8000 --timeout-keep-alive 120 --reload

Docker:

docker build -t phi4:local .
docker run -it --rm \
  -p 8000:8000 \
  -e API_KEY=dev-key \
  -e MODELS_ROOT=/app/model \
  -v /absolute/path/to/FoundryModels:/app/model \
  phi4:local

6) Known issues (important)

• Using a base Phi-4 model (e.g., Phi-4-generic) will generate only ! characters.
  • Fix: use an instruct/chat variant (e.g., microsoft/Phi-4-instruct) and place it in the mounted share.
• ONNX Runtime GenAI 0.6.0 Python lacks tokenizer.apply_chat_template(); app.py already uses manual ChatML formatting with <|im_sep|>.

7) Minimal troubleshooting

• Tail logs: az webapp log tail -g $RG -n $APP_NAME
• Check app settings: az webapp config appsettings list -g $RG -n $APP_NAME
• If models missing, upload to the file share and restart the app.

8) Cleanup

az group delete -n <resource_group> --yes --no-wait

9) Project files

• app.py – FastAPI app with OpenAI-compatible endpoints
• requirements.txt – Python dependencies
• Dockerfile – Minimal image (does not copy models by default)
• deploy.sh – Azure automation (ACR, App Service, storage, mount, app settings)