tutorials/get-started-notebooks/deploy-model.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "# Deploy a model as an online endpoint\n",
    "\n",
    "Learn to deploy a model to an online endpoint, using Azure Machine Learning Python SDK v2.\n",
    "\n",
    "In this tutorial, you deploy and use a model that predicts the likelihood of a customer defaulting on a credit card payment.\n",
    "\n",
    "The steps you take are:\n",
    "\n",
    "> * Register your model\n",
    "> * Create an endpoint and a first deployment\n",
    "> * Deploy a trial run\n",
    "> * Manually send test data to the deployment\n",
    "> * Get details of the deployment\n",
    "> * Create a second deployment\n",
    "> * Manually scale the second deployment\n",
    "> * Update allocation of production traffic between both deployments\n",
    "> * Get details of the second deployment\n",
    "> * Roll out the new deployment and delete the first one"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## Prerequisites\n",
    "\n",
    "1. Open in the studio and select a compute instance.\n",
    "    * If you opened this notebook from Azure Machine Learning studio, you need a compute instance to run the code. If you don't have a compute instance, select **Create compute** on the toolbar to first create one.  You can use all the default settings.  \n",
    "    \n",
    "    ![Create compute](./media/create-compute.png)\n",
    "    \n",
    "    * If you're seeing this notebook elsewhere, complete [Create resources you need to get started](https://docs.microsoft.com/azure/machine-learning/quickstart-create-resources) to create an Azure Machine Learning workspace and a compute instance.\n",
    "    \n",
    "1. View your VM quota and ensure you have enough quota available to create online deployments. In this tutorial, you will need at least 8 cores of `STANDARD_DS3_v2` and 12 cores of `STANDARD_F4s_v2`. To view your VM quota usage and request quota increases, see [Manage resource quotas](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-quotas#view-your-usage-and-quotas-in-the-azure-portal).\n",
    "\n",
    "## Set your kernel\n",
    "\n",
    "* If your compute instance is stopped, start it now.  \n",
    "        \n",
    "    ![Start compute](./media/start-compute.png)\n",
    "\n",
    "* Once your compute instance is running, make sure the that the kernel, found on the top right, is `Python 3.10 - SDK v2`.  If not, use the dropdown to select this kernel.\n",
    "\n",
    "    ![Set the kernel](./media/set-kernel.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## Create handle to workspace\n",
    "\n",
    "Before you dive in the code, you need a way to reference your workspace. Create `ml_client` for a handle to the workspace and use the `ml_client` to manage resources and jobs.\n",
    "\n",
    "In the next cell, enter your Subscription ID, Resource Group name, and Workspace name. To find these values:\n",
    "\n",
    "1. In the upper right Azure Machine Learning studio toolbar, select your workspace name.\n",
    "1. Copy the value for workspace, resource group, and subscription ID into the code.\n",
    "1. You'll need to copy one value, close the area and paste, then come back for the next one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1671511884101
    }
   },
   "outputs": [],
   "source": [
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "# authenticate\n",
    "credential = DefaultAzureCredential()\n",
    "\n",
    "# Get a handle to the workspace\n",
    "ml_client = MLClient(\n",
    "    credential=credential,\n",
    "    subscription_id=\"<SUBSCRIPTION_ID>\",\n",
    "    resource_group_name=\"<RESOURCE_GROUP>\",\n",
    "    workspace_name=\"<AML_WORKSPACE_NAME>\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> [!NOTE]\n",
    "> Creating `MLClient` won't connect to the workspace. The client initialization is lazy and waits for the first time it needs to make a call (this happens in the next code cell).\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register the model\n",
    "\n",
    "If you already completed the earlier training tutorial, [Train a model](https://learn.microsoft.com/en-us/azure/machine-learning/tutorial-train-model), you registered an MLflow model as part of the training script and can skip to the next section.\n",
    "\n",
    "If you didn't complete the training tutorial, you need to register the model. Registering your model before deployment is a recommended best practice.\n",
    "\n",
    "The following code specifies the `path` (where to upload files from) inline. If you [cloned the tutorials folder](https://learn.microsoft.com/en-us/azure/machine-learning/quickstart-create-resources##learn-from-sample-notebooks), then run the following code as-is. Otherwise, download the files and metadata for the model from the [credit_defaults_model folder](https://github.com/Azure/azureml-examples/tree/main/tutorials/get-started-notebooks/deploy/credit_defaults_model). Save the files you downloaded into a local version of the *credit_defaults_model* folder on your computer and update the path in the following code to the location of the downloaded files.\n",
    "\n",
    "The SDK automatically uploads the files and registers the model. \n",
    "\n",
    "For more information on registering your model as an asset, see [Register your model as an asset in Machine Learning by using the SDK](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-models?tabs=use-local%2Ccli#register-your-model-as-an-asset-in-machine-learning-by-using-the-sdk)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "gather": {
     "logged": 1671511925700
    }
   },
   "outputs": [],
   "source": [
    "# Import the necessary libraries\n",
    "from azure.ai.ml.entities import Model\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "\n",
    "# Provide the model details, including the\n",
    "# path to the model files, if you've stored them locally.\n",
    "mlflow_model = Model(\n",
    "    path=\"./deploy/credit_defaults_model/\",\n",
    "    type=AssetTypes.MLFLOW_MODEL,\n",
    "    name=\"credit_defaults_model\",\n",
    "    description=\"MLflow Model created from local files.\",\n",
    ")\n",
    "\n",
    "# Register the model\n",
    "ml_client.models.create_or_update(mlflow_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Confirm that the model is registered\n",
    "\n",
    "You can check the **Models** page in [Azure Machine Learning studio](https://ml.azure.com/) to identify the latest version of the registered model.\n",
    "\n",
    "![View model](./media/registered-model-in-studio.png)\n",
    "\n",
    "Alternatively, the following code retrieves the latest version number for you to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "registered_model_name = \"credit_defaults_model\"\n",
    "\n",
    "# Let's pick the latest version of the model\n",
    "latest_model_version = max(\n",
    "    [int(m.version) for m in ml_client.models.list(name=registered_model_name)]\n",
    ")\n",
    "\n",
    "print(latest_model_version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have a registered model, you can create an endpoint and deployment. The next section briefly covers some key details about these topics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Endpoints and deployments\n",
    "\n",
    "After you train a machine learning model, you need to deploy it so that others can use it for inferencing. For this purpose, Azure Machine Learning allows you to create **endpoints** and add **deployments** to them.\n",
    "\n",
    "An **endpoint**, in this context, is an HTTPS path that provides an interface for clients to send requests (input data) to a trained model and receive the inferencing (scoring) results from the model. An endpoint provides:\n",
    "\n",
    "- Authentication using \"key or token\" based auth \n",
    "- [TLS(SSL)](https://simple.wikipedia.org/wiki/Transport_Layer_Security) termination\n",
    "- A stable scoring URI (endpoint-name.region.inference.ml.azure.com)\n",
    "\n",
    "\n",
    "A **deployment** is a set of resources required for hosting the model that does the actual inferencing. \n",
    "\n",
    "A single endpoint can contain multiple deployments. Endpoints and deployments are independent Azure Resource Manager resources that appear in the Azure portal.\n",
    "\n",
    "Azure Machine Learning allows you to implement [online endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints#what-are-online-endpoints) for real-time inferencing on client data, and [batch endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints#what-are-batch-endpoints) for inferencing on large volumes of data over a period of time. \n",
    "\n",
    "In this tutorial, you go through the steps of implementing a _managed online endpoint_. Managed online endpoints work with powerful CPU and GPU machines in Azure in a scalable, fully managed way that frees you from the overhead of setting up and managing the underlying deployment infrastructure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## Create an online endpoint\n",
    "\n",
    "Now that you have a registered model, it's time to create your online endpoint. The endpoint name needs to be unique in the entire Azure region. For this tutorial, you create a unique name using a universally unique identifier [`UUID`](https://en.wikipedia.org/wiki/Universally_unique_identifier). For more information on the endpoint naming rules, see [managed online endpoint limits](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-quotas#azure-machine-learning-managed-online-endpoints)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import uuid\n",
    "\n",
    "# Create a unique name for the endpoint\n",
    "online_endpoint_name = \"credit-endpoint-\" + str(uuid.uuid4())[:8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, define the endpoint, using the `ManagedOnlineEndpoint` class.\n",
    "\n",
    "\n",
    "\n",
    "> [!TIP]\n",
    "> * `auth_mode` : Use `key` for key-based authentication. Use `aml_token` for Azure Machine Learning token-based authentication. A `key` doesn't expire, but `aml_token` does expire. For more information on authenticating, see [Authenticate clients for online endpoints](https://learn.microsoft.com/azure/machine-learning/how-to-authenticate-online-endpoint).\n",
    "> * Optionally, you can add a description and tags to your endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import ManagedOnlineEndpoint\n",
    "\n",
    "# define an online endpoint\n",
    "endpoint = ManagedOnlineEndpoint(\n",
    "    name=online_endpoint_name,\n",
    "    description=\"this is an online endpoint\",\n",
    "    auth_mode=\"key\",\n",
    "    tags={\n",
    "        \"training_dataset\": \"credit_defaults\",\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `MLClient` created earlier, we'll now create the endpoint in the workspace. This command starts the endpoint creation and returns a confirmation response while the endpoint creation continues.\n",
    "\n",
    "> [!NOTE]\n",
    "> Expect the endpoint creation to take approximately 2 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# create the online endpoint\n",
    "# expect the endpoint to take approximately 2 minutes.\n",
    "\n",
    "endpoint = ml_client.online_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "Once you create the endpoint, you can retrieve it as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "endpoint = ml_client.online_endpoints.get(name=online_endpoint_name)\n",
    "\n",
    "print(\n",
    "    f'Endpoint \"{endpoint.name}\" with provisioning state \"{endpoint.provisioning_state}\" is retrieved'\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding online deployments\n",
    "\n",
    "The key aspects of a deployment include:\n",
    "\n",
    "- `name` - Name of the deployment.\n",
    "- `endpoint_name` - Name of the endpoint that will contain the deployment.\n",
    "- `model` - The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification.\n",
    "- `environment` - The environment to use for the deployment (or to run the model). This value can be either a reference to an existing versioned environment in the workspace or an inline environment specification. The environment can be a Docker image with Conda dependencies or a Dockerfile.\n",
    "- `code_configuration` - the configuration for the source code and scoring script.\n",
    "    - `path`- Path to the source code directory for scoring the model.\n",
    "    - `scoring_script` - Relative path to the scoring file in the source code directory. This script executes the model on a given input request. For an example of a scoring script, see [Understand the scoring script](https://learn.microsoft.com/azure/machine-learning/how-to-deploy-online-endpoints#understand-the-scoring-script) in the \"Deploy an ML model with an online endpoint\" article.\n",
    "- `instance_type` - The VM size to use for the deployment. For the list of supported sizes, see [Managed online endpoints SKU list](https://learn.microsoft.com/azure/machine-learning/reference-managed-online-endpoints-vm-sku-list).\n",
    "- `instance_count` - The number of instances to use for the deployment.\n",
    "    \n",
    "### Deployment using an MLflow model\n",
    "\n",
    "Azure Machine Learning supports no-code deployment of a model created and logged with MLflow. This means that you don't have to provide a scoring script or an environment during model deployment, as the scoring script and environment are automatically generated when training an MLflow model. If you were using a custom model, though, you'd have to specify the environment and scoring script during deployment.\n",
    "\n",
    "> [!IMPORTANT]\n",
    "> If you typically deploy models using scoring scripts and custom environments and want to achieve the same functionality using MLflow models, we recommend reading [Guidelines for deploying MLflow models](https://learn.microsoft.com/azure/machine-learning/how-to-deploy-mlflow-models)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## Deploy the model to the endpoint\n",
    "\n",
    "Begin by creating a single deployment that handles 100% of the incoming traffic. Choose an arbitrary color name (*blue*) for the deployment. To create the deployment for the endpoint, use the `ManagedOnlineDeployment` class.\n",
    "\n",
    "> [!NOTE]\n",
    "> No need to specify an environment or scoring script as the model to deploy is an MLflow model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import ManagedOnlineDeployment\n",
    "\n",
    "# Choose the latest version of the registered model for deployment\n",
    "model = ml_client.models.get(name=registered_model_name, version=latest_model_version)\n",
    "\n",
    "# define an online deployment\n",
    "# if you run into an out of quota error, change the instance_type to a comparable VM that is available.\n",
    "# Learn more on https://azure.microsoft.com/en-us/pricing/details/machine-learning/.\n",
    "blue_deployment = ManagedOnlineDeployment(\n",
    "    name=\"blue\",\n",
    "    endpoint_name=online_endpoint_name,\n",
    "    model=model,\n",
    "    instance_type=\"Standard_DS3_v2\",\n",
    "    instance_count=1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `MLClient` created earlier, now create the deployment in the workspace. This command starts the deployment creation and returns a confirmation response while the deployment creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# create the online deployment\n",
    "blue_deployment = ml_client.online_deployments.begin_create_or_update(\n",
    "    blue_deployment\n",
    ").result()\n",
    "\n",
    "# blue deployment takes 100% traffic\n",
    "# expect the deployment to take approximately 8 to 10 minutes.\n",
    "endpoint.traffic = {\"blue\": 100}\n",
    "ml_client.online_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check the status of the endpoint\n",
    "You can check the status of the endpoint to see whether the model was deployed without error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# return an object that contains metadata for the endpoint\n",
    "endpoint = ml_client.online_endpoints.get(name=online_endpoint_name)\n",
    "\n",
    "# print a selection of the endpoint's metadata\n",
    "print(\n",
    "    f\"Name: {endpoint.name}\\nStatus: {endpoint.provisioning_state}\\nDescription: {endpoint.description}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# existing traffic details\n",
    "print(endpoint.traffic)\n",
    "\n",
    "# Get the scoring URI\n",
    "print(endpoint.scoring_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test the endpoint with sample data\n",
    "\n",
    "Now that the model is deployed to the endpoint, you can run inference with it. Begin by creating a sample request file that follows the design expected in the run method found in the scoring script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create a directory to store the sample request file.\n",
    "deploy_dir = \"./deploy\"\n",
    "os.makedirs(deploy_dir, exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, create the file in the deploy directory. The following code cell uses IPython magic to write the file into the directory you just created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile {deploy_dir}/sample-request.json\n",
    "{\n",
    "  \"input_data\": {\n",
    "    \"columns\": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22],\n",
    "    \"index\": [0, 1],\n",
    "    \"data\": [\n",
    "            [20000,2,2,1,24,2,2,-1,-1,-2,-2,3913,3102,689,0,0,0,0,689,0,0,0,0],\n",
    "            [10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 10, 9, 8]\n",
    "            ]\n",
    "                }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `MLClient` created earlier, we'll get a handle to the endpoint. You can invoke the endpoint by using the `invoke` command with the following parameters:\n",
    "\n",
    "* `endpoint_name` - Name of the endpoint\n",
    "* `request_file` - File with request data\n",
    "* `deployment_name` - Name of the specific deployment to test in an endpoint\n",
    "\n",
    "Test the blue deployment with the sample data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test the blue deployment with the sample data\n",
    "ml_client.online_endpoints.invoke(\n",
    "    endpoint_name=online_endpoint_name,\n",
    "    deployment_name=\"blue\",\n",
    "    request_file=\"./deploy/sample-request.json\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get logs of the deployment\n",
    "Check the logs to see whether the endpoint/deployment were invoked successfully.\n",
    "If you face errors, see [Troubleshooting online endpoints deployment](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-online-endpoints?tabs=cli)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "logs = ml_client.online_deployments.get_logs(\n",
    "    name=\"blue\", endpoint_name=online_endpoint_name, lines=50\n",
    ")\n",
    "print(logs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a second deployment \n",
    "Deploy the model as a second deployment called `green`. In practice, you can create several deployments and compare their performance. These deployments could use a different version of the same model, a completely different model, or a more powerful compute instance.\n",
    "\n",
    "In this example, you deploy the same model version, using a more powerful compute instance that could potentially improve performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pick the model to deploy. Here you use the latest version of the registered model\n",
    "model = ml_client.models.get(name=registered_model_name, version=latest_model_version)\n",
    "\n",
    "# define an online deployment using a more powerful instance type\n",
    "# if you run into an out of quota error, change the instance_type to a comparable VM that is available.\n",
    "# Learn more on https://azure.microsoft.com/en-us/pricing/details/machine-learning/.\n",
    "green_deployment = ManagedOnlineDeployment(\n",
    "    name=\"green\",\n",
    "    endpoint_name=online_endpoint_name,\n",
    "    model=model,\n",
    "    instance_type=\"Standard_F4s_v2\",\n",
    "    instance_count=1,\n",
    ")\n",
    "\n",
    "# create the online deployment\n",
    "# expect the deployment to take approximately 8 to 10 minutes\n",
    "green_deployment = ml_client.online_deployments.begin_create_or_update(\n",
    "    green_deployment\n",
    ").result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scale deployment to handle more traffic\n",
    "\n",
    "Using the `MLClient` created earlier, you can get a handle to the `green` deployment. You can then scale it by increasing or decreasing the `instance_count`.\n",
    "\n",
    "In the following code, you increase the VM instance manually. However, it's also possible to autoscale online endpoints. Autoscale automatically runs the right amount of resources to handle the load on your application. Managed online endpoints support autoscaling through integration with the Azure monitor autoscale feature. To configure autoscaling, see [Autoscale online endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-autoscale-endpoints?tabs=python)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# update definition of the deployment\n",
    "green_deployment.instance_count = 2\n",
    "\n",
    "# update the deployment\n",
    "# expect the deployment to take approximately 8 to 10 minutes\n",
    "ml_client.online_deployments.begin_create_or_update(green_deployment).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Update traffic allocation for deployments\n",
    "You can split production traffic between deployments. You may first want to test the `green` deployment with sample data, just like you did for the `blue` deployment. Once you've tested your green deployment, allocate a small percentage of traffic to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint.traffic = {\"blue\": 80, \"green\": 20}\n",
    "ml_client.online_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test traffic allocation by invoking the endpoint several times:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can invoke the endpoint several times\n",
    "for i in range(30):\n",
    "    ml_client.online_endpoints.invoke(\n",
    "        endpoint_name=online_endpoint_name,\n",
    "        request_file=\"./deploy/sample-request.json\",\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show logs from the `green` deployment to check that there were incoming requests and the model was scored successfully. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "logs = ml_client.online_deployments.get_logs(\n",
    "    name=\"green\", endpoint_name=online_endpoint_name, lines=50\n",
    ")\n",
    "print(logs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View metrics using Azure Monitor\n",
    "You can view various metrics (request numbers, request latency, network bytes, CPU/GPU/Disk/Memory utilization, and more) for an online endpoint and its deployments by following links from the endpoint's **Details** page in the studio. Following any of these links takes you to the exact metrics page in the Azure portal for the endpoint or deployment.\n",
    "\n",
    "![metrics page 1](./media/deployment-metrics-from-endpoint-details-page.png)\n",
    "\n",
    "\n",
    "If you open the metrics for the online endpoint, you can set up the page to see metrics such as the average request latency as shown in the following figure.\n",
    "\n",
    "![metrics page 2](./media/view-endpoint-metrics-in-azure-portal.png)\n",
    "\n",
    "For more information on how to view online endpoint metrics, see [Monitor online endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-online-endpoints#metrics)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Send all traffic to the new deployment\n",
    "Once you're fully satisfied with your `green` deployment, switch all traffic to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint.traffic = {\"blue\": 0, \"green\": 100}\n",
    "ml_client.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delete the old deployment\n",
    "Remove the old (blue) deployment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.online_deployments.begin_delete(\n",
    "    name=\"blue\", endpoint_name=online_endpoint_name\n",
    ").result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean up resources\n",
    "\n",
    "If you aren't going use the endpoint and deployment after completing this tutorial, you should delete them.\n",
    "\n",
    "> [!NOTE]\n",
    "> Expect the complete deletion to take approximately 20 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.online_endpoints.begin_delete(name=online_endpoint_name).result()"
   ]
  }
 ],
 "metadata": {
  "description": {
   "description": "Learn to deploy a model to an online endpoint, using Azure Machine Learning Python SDK v2."
  },
  "kernel_info": {
   "name": "python310-sdkv2"
  },
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}