Predictive maintenance NASA sample.ipynb

{
  "cells": [
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "# Microsoft Azure ML Automated Machine Learning\n![alt text](https://www.nasa.gov/sites/all/themes/custom/nasatwo/images/nasa-logo.svg \"NASA Ames\")\n## _NASA Predictive Maintenance Sample_ "
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Purpose and Challenge"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "The purpose of this notebook is for the user to build and deploy a Machine Learning (ML) application using Azure Machine Learning (AML) Service.\n\nThe challenge we will tackle is predictive maintenance: when will a certain piece of machinery will fail, so that we are prepared to fix or replace it in advance _before_ it fails.\n\nThis notebook has the complete code to load, prep, train and deploy the model. We chose a small public data set for this demo so as to run the entire process in only few minutes.\n\nFollowing are the high level steps:\n\n1. Acquire and Prepare Data\n2. Train using automated machine learning to get the best possible model\n3. Deploy the model\n"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Prepare the environment for training"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import logging\nimport os\nimport random\nimport time\n\nfrom matplotlib import pyplot as plt\nfrom matplotlib.pyplot import imshow\nimport numpy as np\nimport pandas as pd\n\nimport azureml.core\nfrom azureml.core.experiment import Experiment\nfrom azureml.core.workspace import Workspace\nfrom azureml.train.automl import AutoMLConfig\nfrom azureml.train.automl.run import AutoMLRun\nfrom azureml.widgets import RunDetails\nfrom azureml.core.model import Model",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Define the experiment name and read the config file"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Retrieve workspace\nws = Workspace.from_config()\n\n# Choose a name for the experiment and specify the project folder.\nexperiment_name = 'automl-predictive-rul'\nproject_folder = './sample_projects/automl-demo-predmain'\n\nexperiment = Experiment(ws, experiment_name)\n\noutput = {}\noutput['SDK version'] = azureml.core.VERSION\noutput['Subscription ID'] = ws.subscription_id\noutput['Workspace Name'] = ws.name\noutput['Resource Group'] = ws.resource_group\noutput['Location'] = ws.location\noutput['Project Directory'] = project_folder\noutput['Experiment Name'] = experiment.name\npd.set_option('display.max_colwidth', -1)\npd.DataFrame(data = output, index = ['']).T",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 1. Acquire and Prepare Data\nFor this notebook, we will use the NASA Prognostics Center's Turbo-Fan Failure dataset.  It is located here: https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/#turbofan"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We have it as .txt file in the same folder. We read it into a Pandas DataFrame.\nNote the headers were not in the space seperated txt file, so we assign them from the ReadMe in the zip file. In pandas we use read_csv with the delimiter option, even with a space delimited file."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import pandas as pd\ntrain = pd.read_csv(\"train_FD001.txt\", delimiter=\"\\s|\\s\\s\", index_col=False, engine='python', names=['unit','cycle','os1','os2','os3','sm1','sm2','sm3','sm4','sm5','sm6','sm7','sm8','sm9','sm10','sm11','sm12','sm13','sm14','sm15','sm16','sm17','sm18','sm19','sm20','sm21'])",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Take a quick look at the data"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "train.head(5)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Our dataset has a number of units in it, with each engine flight listed as a cycle. The cycles count up until the engine fails. What we would like to predict is the no. of cycles until failure. \nSo we need to calculate a new column called \"Remaining Useful Life\", or RUL, for short. It will be the last cycle value minus each cycle value per unit."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "def assignrul(df):\n    maxi = df['cycle'].max()\n    df['rul'] = maxi - df['cycle']\n    return df\n    \ntrain_new = train.groupby('unit').apply(assignrul)\n\ntrain_new.columns",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now our dataframe has the 'RUL' column.  Predicting this value will be the objective of this exercise."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "train_new.head(192)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "First note that some of the sensor measurements do seem to be changing as we near 0 RUL (sm3, sm4, sm14, sm17). This implies that we should be able to make a model that will be useful enough for business value.\n\nWe are now ready to train a model on this data using Automated ML."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 2. Train using automated machine learning"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Here we utilize Azure's AutoML package to automate the scaling of the sensors, selection of sensors, and automatically train and evaluate many different types of ML models."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Create training data"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# remove the unit ID and cycle number\nX_train = train_new.iloc[:,2:26].values\n# extract the RUL column to be the target column\ny_train = train_new.iloc[:,26:27].values.astype(int).flatten()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Split data to train and test"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split(X_train,\n                                                    y_train,\n                                                    test_size=0.3,\n                                                    random_state=100)\nX_train = pd.DataFrame(X_train)\nX_test = pd.DataFrame(X_test)\nprint(X_train.shape)\nprint(X_test.shape)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "X_test[0:1]",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now we are ready to configure automated ML.  We provide necessary information on: what we want to predict, what accuracy metric we want to use, how many models we want to try, and many other parameters.  AutoML will also automatically scale the data for us."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Configure Automated ML\n\nSet the automated ML run. Full list of parameters is available [here](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-auto-train).\n\n|Property|Description|\n|-|-|\n|**task**|classification or regression|\n|**primary_metric**|This is the metric that you want to optimize. Regression supports the following primary metrics: <br><i>spearman_correlation</i><br><i>normalized_root_mean_squared_error</i><br><i>r2_score</i><br><i>normalized_mean_absolute_error</i>|\n|**iteration_timeout_minutes**|Time limit in minutes for each iteration.|\n|**iterations**|Number of iterations. In each iteration AutoML trains a specific pipeline with the data.|\n|**n_cross_validations**|Number of cross validation splits.|\n|**X**|(sparse) array-like, shape = [n_samples, n_features]|\n|**y**|(sparse) array-like, shape = [n_samples, ], [n_samples, n_classes]<br>Multi-class targets. An indicator matrix turns on multilabel classification. This should be an array of integers.|\n|**path**|Relative path to the project folder. AutoML stores configuration files for the experiment under this folder. You can specify a new empty folder.|\n|**preprocess**|set this to True to enable pre-processing of data eg. string to numeric using one-hot encoding|\n|**exit_score**|Target score for experiment. It is associated with the metric. eg. exit_score=0.995 will exit experiment after that|"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Create Auto ML configuration\nAutoml_config = AutoMLConfig(task = 'regression',\n                             primary_metric = 'r2_score',\n                             iteration_timeout_minutes = 5,\n                             iterations = 5,                             \n                             blacklist_models = ['KNN','RandomForest'],\n                             X = X_train,\n                             y = y_train,\n                             n_cross_validations = 3,\n                             preprocess = False,\n                             experiment_exit_score = 0.985,\n                             path=project_folder)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Finally we are ready to launch AutoML.  This step can take many minutes, but AutoML will give you updates as models are trained and evaluated by the metric we specified above.  AutoML also let us know which scaling method was used.  The information from each ML model training will be stored in the Experiment section of the ML Workspace, where we can review it through Azure Portal."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Submit the training job. The output will show the iterations as they finish one by one\nexperiment=Experiment(ws, experiment_name)\nlocal_run = experiment.submit(Automl_config, show_output=True)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### View the training run in a graphic widget"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "RunDetails(local_run).show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Retrieve the best model (according to the primary metric)"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# find the run with the highest accuracy value.\nbest_run, fitted_model = local_run.get_output()\nprint(best_run)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 3. Deploy Model"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# register best model in workspace. The output of this cell is important\ndescription = 'AutoML NASA RUL Regression'\ntags = None\nmodel = local_run.register_model(description=description, tags=tags)\nlocal_run.model_id # Use the model id that is printed out in the cell below",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "After we register the model in our ML Workspace, it should be visible in Azure Portal.\n\nNow we want to deploy the model as a REST API that we can feed a row or rows of \"X\" data to, and return the predicted 'RUL' value.  To accomplish this, we will build a container image in our AML Workspace and deploy that image as a Container instance in Azure's ACI service.  We will then obtain an IP address where we can submit data and receive back the predicted 'RUL' value.\n\nThere are 3 things we need: \n1. A score.py file that contains the init() and run() functions with instructions on how to load and socre with the model\n2. A myenv.yml file that contains information on the python environment in which the model needs to run\n3. Configurations for our images and our services, using functions provided by AzureML service.\n\nThe cells below help you set these up.   You will need to use the registered model name provided by the cell above."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Create scoring script"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "%%writefile score.py\n# Scoring Script\nimport json\nimport numpy as np\nimport os\nimport pickle\nfrom sklearn.externals import joblib\nfrom sklearn.linear_model import LogisticRegression\n\nfrom azureml.core.model import Model\n\nimport azureml.train.automl\n\ndef init():\n    global model\n    model_path = Model.get_model_path('<<modelid>>')\n    print(model_path)\n    model = joblib.load(model_path)\n    \n\ndef run(raw_data):\n    # grab and prepare the data\n    data = (np.array(json.loads(raw_data)['data'])).reshape(1,-1)\n    # make prediction\n    y_hat = model.predict(data)\n    return json.dumps(y_hat.tolist())",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Replace the 'modelid' tag with the actual model ID"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Substitute the actual model id in the script file.\n\nscript_file_name = 'score.py'\n\nwith open(script_file_name, 'r') as cefr:\n    content = cefr.read()\n\nwith open(script_file_name, 'w') as cefw:\n    cefw.write(content.replace('<<modelid>>', local_run.model_id))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Create the conda environment file"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.conda_dependencies import CondaDependencies\n\nmyenv = CondaDependencies.create(conda_packages=['numpy','scikit-learn','lightgbm'], pip_packages=['azureml-sdk[automl]'])\n\nconda_env_file_name = 'myenv.yml'\nmyenv.save_to_file('.', conda_env_file_name)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "with open(\"myenv.yml\",\"r\") as f:\n    print(f.read())",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Create the webservice configuration"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.webservice import AciWebservice\n\naciconfig = AciWebservice.deploy_configuration(cpu_cores=2, \n                                               memory_gb=2, \n                                               tags={\"data\": \"RUL\",  \"method\" : \"sklearn\"}, \n                                               description='Predict RUL with Azure AutoML')",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Create the container image and deploy as a webservice"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Finally, configure the container image and deploy the service. Make sure the filenames match, your Workspace is in variable ws, and your model name is correct. It will create your containter image and deploy it as a webservice.\n\nThis process can take up to 10 minutes, so please be patient. You can check the progress bar periodically ..."
    },
    {
      "metadata": {
        "scrolled": true,
        "trusted": true
      },
      "cell_type": "code",
      "source": "%%time\nfrom azureml.core.webservice import Webservice\nfrom azureml.core.image import ContainerImage\n\n# configure the image\nimage_config = ContainerImage.image_configuration(execution_script=\"score.py\", \n                                                  runtime=\"python\", \n                                                  conda_file=\"myenv.yml\",\n                                                  tags = {'ml': \"Regression\", 'type': \"automl\"},\n                                                  description = \"Image for automated ML NASA predictive maintenance\")\n\n# deploy the image to a webservice\nservice = Webservice.deploy_from_model(workspace=ws,\n                                       name='automl-rul-regression',\n                                       deployment_config=aciconfig,\n                                       models=[model],\n                                       image_config=image_config)\n\nservice.wait_for_deployment(show_output=True)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Just as a check, we can retrieve the URI for the scoring function."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "print(service.scoring_uri)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Test the service\n\nLet's check to see if the service is working.  Here we submit a single row of data from X_train to see if it returns a reasonable prediction."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import requests\nimport json\n\n# send a random row from the test set to score\n#random_index = np.random.randint(0, len(X_train)-1)\ninput_data = \"{\\\"data\\\": \" + str(X_test[1:2].values.tolist()) + \"}\" #str(list(X_train[0].reshape(1,-1)[0])) + \"}\"\n\nheaders = {'Content-Type':'application/json'}\n\n# for AKS deployment you'd need to the service key in the header as well\n# api_key = service.get_key()\n# headers = {'Content-Type':'application/json',  'Authorization':('Bearer '+ api_key)} \n\nresp = requests.post(service.scoring_uri, input_data, headers=headers)\n\nprint(\"POST to url\", service.scoring_uri)\nprint(\"input data:\", input_data)\nprint(\"label:\", y_test[1:2])\nprint(\"prediction:\", resp.text)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Here we see one engine evolving through many flights, or cycles.  As we approach failure, the rul declines to zero, as does the prediction.  This is a good example of how the predictive model can assist in estimate the future failure of the engine.\n\nNote that the model does not perform well at high rul.  This is an acceptable outcome as the engine is far from failure."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Delete the web service resource\n\nTo avoid any run-away Azure costs, we always delete un-necessary services when we are done."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "service.delete()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "If the workspace will not be in use, it is advisable to delete it also"
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}