deepracer_rl.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Distributed DeepRacer RL training with SageMaker and RoboMaker\n",
    "\n",
    "---\n",
    "## Introduction\n",
    "\n",
    "\n",
    "In this notebook, we will train a fully autonomous 1/18th scale race car using reinforcement learning using Amazon SageMaker RL and AWS RoboMaker's 3D driving simulator. [AWS RoboMaker](https://console.aws.amazon.com/robomaker/home#welcome) is a service that makes it easy for developers to develop, test, and deploy robotics applications.  \n",
    "\n",
    "This notebook provides a jailbreak experience of [AWS DeepRacer](https://console.aws.amazon.com/deepracer/home#welcome), giving us more control over the training/simulation process and RL algorithm tuning.\n",
    "\n",
    "![Training in Action](./deepracer-reinvent-track.jpg)\n",
    "\n",
    "\n",
    "---\n",
    "## How it works?  \n",
    "\n",
    "![How training works](./training.png)\n",
    "\n",
    "The reinforcement learning agent (i.e. our autonomous car) learns to drive by interacting with its environment, e.g., the track, by taking an action in a given state to maximize the expected reward. The agent learns the optimal plan of actions in training by trial-and-error through repeated episodes.  \n",
    "  \n",
    "The figure above shows an example of distributed RL training across SageMaker and two RoboMaker simulation envrionments that perform the **rollouts** - execute a fixed number of episodes using the current model or policy. The rollouts collect agent experiences (state-transition tuples) and share this data with SageMaker for training. SageMaker updates the model policy which is then used to execute the next sequence of rollouts. This training loop continues until the model converges, i.e. the car learns to drive and stops going off-track. More formally, we can define the problem in terms of the following:  \n",
    "\n",
    "1. **Objective**: Learn to drive autonomously by staying close to the center of the track.\n",
    "2. **Environment**: A 3D driving simulator hosted on AWS RoboMaker.\n",
    "3. **State**: The driving POV image captured by the car's head camera, as shown in the illustration above.\n",
    "4. **Action**: Six discrete steering wheel positions at different angles (configurable)\n",
    "5. **Reward**: Positive reward for staying close to the center line; High penalty for going off-track. This is configurable and can be made more complex (for e.g. steering penalty can be added)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use below commands only if you want to make changes to the simulation application (Robomaker code changes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #\n",
    "# # Run these commands if you want to modify the simapp\n",
    "# #\n",
    "# # Clean the build directory if present\n",
    "# !python3 sim_app_bundler.py --clean\n",
    "\n",
    "# # Download Robomaker simApp from the deepracer public s3 bucket\n",
    "# simulation_application_bundle_location = \"s3://deepracer-managed-resources-us-east-1/deepracer-simapp.tar.gz\"\n",
    "# !aws s3 cp {simulation_application_bundle_location} ./\n",
    "\n",
    "# # Untar the simapp bundle\n",
    "# !python3 sim_app_bundler.py --untar ./deepracer-simapp.tar.gz\n",
    "\n",
    "# # Now modify the simapp from build directory and run this command.\n",
    "\n",
    "# # Most of the simapp files can be found here (Robomaker changes)\n",
    "# # bundle/opt/install/sagemaker_rl_agent/lib/python3.5/site-packages/\n",
    "# # bundle/opt/install/deepracer_simulation_environment/share/deepracer_simulation_environment/\n",
    "# # bundle/opt/install/deepracer_simulation_environment/lib/deepracer_simulation_environment/\n",
    "\n",
    "# # # Copying the notebook src/markov changes to the simapp (For sagemaker container)\n",
    "# !rsync -av ./src/markov/ ./build/simapp/bundle/opt/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov\n",
    "\n",
    "# !python3 sim_app_bundler.py --tar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get started, we'll import the Python libraries we need, set up the environment with a few prerequisites for permissions and configurations.\n",
    "\n",
    "You can run this notebook from your local machine or from a SageMaker notebook instance. In both of these scenarios, you can run the following to launch a training job on SageMaker and a simulation job on RoboMaker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import sagemaker\n",
    "import sys\n",
    "import os\n",
    "import re\n",
    "import numpy as np\n",
    "import subprocess\n",
    "sys.path.append(\"common\")\n",
    "from misc import get_execution_role, wait_for_s3_object\n",
    "from docker_utils import build_and_push_docker_image\n",
    "from sagemaker.rl import RLEstimator, RLToolkit, RLFramework\n",
    "from time import gmtime, strftime\n",
    "import time\n",
    "from IPython.display import Markdown\n",
    "from markdown_helper import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initializing basic parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select the instance type\n",
    "instance_type = \"ml.c4.2xlarge\"\n",
    "#instance_type = \"ml.p2.xlarge\"\n",
    "#instance_type = \"ml.c5.4xlarge\"\n",
    "\n",
    "# Starting SageMaker session\n",
    "sage_session = sagemaker.session.Session()\n",
    "\n",
    "# Create unique job name.\n",
    "job_name_prefix = 'deepracer-notebook'\n",
    "\n",
    "# Duration of job in seconds (1 hours)\n",
    "job_duration_in_seconds = 3600\n",
    "\n",
    "# AWS Region\n",
    "aws_region = sage_session.boto_region_name\n",
    "if aws_region not in [\"us-west-2\", \"us-east-1\", \"eu-west-1\"]:\n",
    "    raise Exception(\"This notebook uses RoboMaker which is available only in US East (N. Virginia),\"\n",
    "                    \"US West (Oregon) and EU (Ireland). Please switch to one of these regions.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup S3 bucket\n",
    "Set up the linkage and authentication to the S3 bucket that we want to use for checkpoint and metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# S3 bucket\n",
    "s3_bucket = sage_session.default_bucket()\n",
    "\n",
    "# SDK appends the job name and output folder\n",
    "s3_output_path = 's3://{}/'.format(s3_bucket)\n",
    "\n",
    "#Ensure that the S3 prefix contains the keyword 'sagemaker'\n",
    "s3_prefix = job_name_prefix + \"-sagemaker-\" + strftime(\"%y%m%d-%H%M%S\", gmtime())\n",
    "\n",
    "# Get the AWS account id of this account\n",
    "sts = boto3.client(\"sts\")\n",
    "account_id = sts.get_caller_identity()['Account']\n",
    "\n",
    "print(\"Using s3 bucket {}\".format(s3_bucket))\n",
    "print(\"Model checkpoints and other metadata will be stored at: \\ns3://{}/{}\".format(s3_bucket, s3_prefix))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create an IAM role\n",
    "Either get the execution role when running from a SageMaker notebook `role = sagemaker.get_execution_role()` or, when running from local machine, use utils method `role = get_execution_role('role_name')` to create an execution role."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    sagemaker_role = sagemaker.get_execution_role()\n",
    "except:\n",
    "    sagemaker_role = get_execution_role('sagemaker')\n",
    "\n",
    "print(\"Using Sagemaker IAM role arn: \\n{}\".format(sagemaker_role))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Please note that this notebook cannot be run in `SageMaker local mode` as the simulator is based on AWS RoboMaker service."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Permission setup for invoking AWS RoboMaker from this notebook\n",
    "In order to enable this notebook to be able to execute AWS RoboMaker jobs, we need to add one trust relationship to the default execution role of this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Markdown(generate_help_for_robomaker_trust_relationship(sagemaker_role)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Permission setup for Sagemaker to S3 bucket\n",
    "\n",
    "The sagemaker writes the Redis IP address, models to the S3 bucket. This requires PutObject permission on the bucket. Make sure the sagemaker role you are using as this permissions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Markdown(generate_s3_write_permission_for_sagemaker_role(sagemaker_role)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Permission setup for Sagemaker to create KinesisVideoStreams\n",
    "\n",
    "The sagemaker notebook has to create a kinesis video streamer. You can observer the car making epsiodes in the kinesis video streamer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Markdown(generate_kinesis_create_permission_for_sagemaker_role(sagemaker_role)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and push docker image\n",
    "\n",
    "The file ./Dockerfile contains all the packages that are installed into the docker. Instead of using the default sagemaker container. We will be using this docker container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "from copy_to_sagemaker_container import get_sagemaker_docker, copy_to_sagemaker_container, get_custom_image_name\n",
    "cpu_or_gpu = 'gpu' if instance_type.startswith('ml.p') else 'cpu'\n",
    "repository_short_name = \"sagemaker-docker-%s\" % cpu_or_gpu\n",
    "custom_image_name = get_custom_image_name(repository_short_name)\n",
    "try:\n",
    "    print(\"Copying files from your notebook to existing sagemaker container\")\n",
    "    sagemaker_docker_id = get_sagemaker_docker(repository_short_name)\n",
    "    copy_to_sagemaker_container(sagemaker_docker_id, repository_short_name)\n",
    "except Exception as e:\n",
    "    print(\"Creating sagemaker container\")\n",
    "    docker_build_args = {\n",
    "        'CPU_OR_GPU': cpu_or_gpu, \n",
    "        'AWS_REGION': boto3.Session().region_name,\n",
    "    }\n",
    "    custom_image_name = build_and_push_docker_image(repository_short_name, build_args=docker_build_args)\n",
    "    print(\"Using ECR image %s\" % custom_image_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean the docker images\n",
    "Remove this only when you want to completely remove the docker or clean up the space of the sagemaker instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !docker rm -f $(docker ps -a -q);\n",
    "# !docker rmi -f $(docker images -q);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure VPC\n",
    "\n",
    "Since SageMaker and RoboMaker have to communicate with each other over the network, both of these services need to run in VPC mode. This can be done by supplying subnets and security groups to the job launching scripts.  \n",
    "We will check if the deepracer-vpc stack is created and use it if present (This is present if the AWS Deepracer console is used atleast once to create a model). Else we will use the default VPC stack."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ec2 = boto3.client('ec2')\n",
    "\n",
    "#\n",
    "# Check if the user has Deepracer-VPC and use that if its present. This will have all permission.\n",
    "# This VPC will be created when you have used the Deepracer console and created one model atleast\n",
    "# If this is not present. Use the default VPC connnection\n",
    "#\n",
    "deepracer_security_groups = [group[\"GroupId\"] for group in ec2.describe_security_groups()['SecurityGroups']\\\n",
    "                             if group['GroupName'].startswith(\"aws-deepracer-\")]\n",
    "\n",
    "# deepracer_security_groups = False\n",
    "if(deepracer_security_groups):\n",
    "    print(\"Using the DeepRacer VPC stacks. This will be created if you run one training job from console.\")\n",
    "    deepracer_vpc = [vpc['VpcId'] for vpc in ec2.describe_vpcs()['Vpcs'] \\\n",
    "                     if \"Tags\" in vpc for val in vpc['Tags'] \\\n",
    "                     if val['Value'] == 'deepracer-vpc'][0]\n",
    "    deepracer_subnets = [subnet[\"SubnetId\"] for subnet in ec2.describe_subnets()[\"Subnets\"] \\\n",
    "                         if subnet[\"VpcId\"] == deepracer_vpc]\n",
    "else:\n",
    "    print(\"Using the default VPC stacks\")\n",
    "    deepracer_vpc = [vpc['VpcId'] for vpc in ec2.describe_vpcs()['Vpcs'] if vpc[\"IsDefault\"] == True][0]\n",
    "\n",
    "    deepracer_security_groups = [group[\"GroupId\"] for group in ec2.describe_security_groups()['SecurityGroups'] \\\n",
    "                                 if 'VpcId' in group and group[\"GroupName\"] == \"default\" and group[\"VpcId\"] == deepracer_vpc]\n",
    "\n",
    "    deepracer_subnets = [subnet[\"SubnetId\"] for subnet in ec2.describe_subnets()[\"Subnets\"] \\\n",
    "                         if subnet[\"VpcId\"] == deepracer_vpc and subnet['DefaultForAz']==True]\n",
    "\n",
    "print(\"Using VPC:\", deepracer_vpc)\n",
    "print(\"Using security group:\", deepracer_security_groups)\n",
    "print(\"Using subnets:\", deepracer_subnets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Route Table\n",
    "A SageMaker job running in VPC mode cannot access S3 resourcs. So, we need to create a VPC S3 endpoint to allow S3 access from SageMaker container. To learn more about the VPC mode, please visit [this link.](https://docs.aws.amazon.com/sagemaker/latest/dg/train-vpc.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Explain to customer what CREATE_ROUTE_TABLE is doing\n",
    "CREATE_ROUTE_TABLE = True\n",
    "\n",
    "def create_vpc_endpoint_table():\n",
    "    print(\"Creating \")\n",
    "    try:\n",
    "        route_tables = [route_table[\"RouteTableId\"] for route_table in ec2.describe_route_tables()['RouteTables']\\\n",
    "                        if route_table['VpcId'] == deepracer_vpc]\n",
    "    except Exception as e:\n",
    "        if \"UnauthorizedOperation\" in str(e):\n",
    "            display(Markdown(generate_help_for_s3_endpoint_permissions(sagemaker_role)))\n",
    "        else:\n",
    "            display(Markdown(create_s3_endpoint_manually(aws_region, deepracer_vpc)))\n",
    "        raise e\n",
    "\n",
    "    print(\"Trying to attach S3 endpoints to the following route tables:\", route_tables)\n",
    "    \n",
    "    if not route_tables:\n",
    "        raise Exception((\"No route tables were found. Please follow the VPC S3 endpoint creation \"\n",
    "                         \"guide by clicking the above link.\"))\n",
    "    try:\n",
    "        ec2.create_vpc_endpoint(DryRun=False,\n",
    "                                VpcEndpointType=\"Gateway\",\n",
    "                                VpcId=deepracer_vpc,\n",
    "                                ServiceName=\"com.amazonaws.{}.s3\".format(aws_region),\n",
    "                                RouteTableIds=route_tables)\n",
    "        print(\"S3 endpoint created successfully!\")\n",
    "    except Exception as e:\n",
    "        if \"RouteAlreadyExists\" in str(e):\n",
    "            print(\"S3 endpoint already exists.\")\n",
    "        elif \"UnauthorizedOperation\" in str(e):\n",
    "            display(Markdown(generate_help_for_s3_endpoint_permissions(role)))\n",
    "            raise e\n",
    "        else:\n",
    "            display(Markdown(create_s3_endpoint_manually(aws_region, deepracer_vpc)))\n",
    "            raise e\n",
    "\n",
    "if CREATE_ROUTE_TABLE:\n",
    "    create_vpc_endpoint_table()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup the environment\n",
    "\n",
    "The environment is defined in a Python file called “deepracer_racetrack_env.py” and the file can be found at `src/markov/environments/`. This file implements the gym interface for our Gazebo based RoboMakersimulator. This is a common environment file used by both SageMaker and RoboMaker. The environment variable - `NODE_TYPE` defines which node the code is running on. So, the expressions that have `rospy` dependencies are executed on RoboMaker only.  \n",
    "\n",
    "We can experiment with different reward functions by modifying `reward_function` in `src/markov/rewards/`. Action space and steering angles can be changed by modifying `src/markov/actions/`.json file\n",
    "\n",
    "### Configure the preset for RL algorithm\n",
    "\n",
    "The parameters that configure the RL training job are defined in `src/markov/presets/`. Using the preset file, you can define agent parameters to select the specific agent algorithm. We suggest using Clipped PPO for this example.  \n",
    "You can edit this file to modify algorithm parameters like learning_rate, neural network structure, batch_size, discount factor etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the pygmentize code lines to see the code\n",
    "\n",
    "# Reward function\n",
    "#!pygmentize src/markov/rewards/default.py\n",
    "\n",
    "# Action space\n",
    "#!pygmentize src/markov/actions/single_speed_stereo_shallow.json\n",
    "\n",
    "# Preset File\n",
    "#!pygmentize src/markov/presets/default.py\n",
    "#!pygmentize src/markov/presets/preset_attention_layer.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Copy custom files to S3 bucket so that sagemaker & robomaker can pick it up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_location = \"s3://%s/%s\" % (s3_bucket, s3_prefix)\n",
    "print(s3_location)\n",
    "\n",
    "# Clean up the previously uploaded files\n",
    "!aws s3 rm --recursive {s3_location}\n",
    "\n",
    "!aws s3 cp src/markov/rewards/default.py {s3_location}/customer_reward_function.py\n",
    "\n",
    "!aws s3 cp src/markov/actions/default.json {s3_location}/model/model_metadata.json\n",
    "\n",
    "#!aws s3 cp src/markov/presets/default.py {s3_location}/presets/preset.py\n",
    "#!aws s3 cp src/markov/presets/preset_attention_layer.py {s3_location}/presets/preset.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the RL model using the Python SDK Script mode\n",
    "\n",
    "Next, we define the following algorithm metrics that we want to capture from cloudwatch logs to monitor the training progress. These are algorithm specific parameters and might change for different algorithm. We use [Clipped PPO](https://coach.nervanasys.com/algorithms/policy_optimization/cppo/index.html) for this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metric_definitions = [\n",
    "    # Training> Name=main_level/agent, Worker=0, Episode=19, Total reward=-102.88, Steps=19019, Training iteration=1\n",
    "    {'Name': 'reward-training',\n",
    "     'Regex': '^Training>.*Total reward=(.*?),'},\n",
    "    \n",
    "    # Policy training> Surrogate loss=-0.32664725184440613, KL divergence=7.255815035023261e-06, Entropy=2.83156156539917, training epoch=0, learning_rate=0.00025\n",
    "    {'Name': 'ppo-surrogate-loss',\n",
    "     'Regex': '^Policy training>.*Surrogate loss=(.*?),'},\n",
    "     {'Name': 'ppo-entropy',\n",
    "     'Regex': '^Policy training>.*Entropy=(.*?),'},\n",
    "   \n",
    "    # Testing> Name=main_level/agent, Worker=0, Episode=19, Total reward=1359.12, Steps=20015, Training iteration=2\n",
    "    {'Name': 'reward-testing',\n",
    "     'Regex': '^Testing>.*Total reward=(.*?),'},\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the RLEstimator for training RL jobs.\n",
    "\n",
    "1. Specify the source directory which has the environment file, preset and training code.\n",
    "2. Specify the entry point as the training code\n",
    "3. Specify the choice of RL toolkit and framework. This automatically resolves to the ECR path for the RL Container.\n",
    "4. Define the training parameters such as the instance count, instance type, job name, s3_bucket and s3_prefix for storing model checkpoints and metadata. **Only 1 training instance is supported for now.**\n",
    "4. Set the RLCOACH_PRESET as \"deepracer\" for this example.\n",
    "5. Define the metrics definitions that you are interested in capturing in your logs. These can also be visualized in CloudWatch and SageMaker Notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator = RLEstimator(entry_point=\"training_worker.py\",\n",
    "                        source_dir='src',\n",
    "                        image_name=custom_image_name,\n",
    "                        dependencies=[\"common/\"],\n",
    "                        role=sagemaker_role,\n",
    "                        train_instance_type=instance_type,\n",
    "                        train_instance_count=1,\n",
    "                        output_path=s3_output_path,\n",
    "                        base_job_name=job_name_prefix,\n",
    "                        metric_definitions=metric_definitions,\n",
    "                        train_max_run=job_duration_in_seconds,\n",
    "                        hyperparameters={\n",
    "                            \"s3_bucket\": s3_bucket,\n",
    "                            \"s3_prefix\": s3_prefix,\n",
    "                            \"aws_region\": aws_region,\n",
    "                            \"model_metadata_s3_key\": \"%s/model/model_metadata.json\" % s3_prefix,\n",
    "                            \"reward_function_s3_source\": \"%s/customer_reward_function.py\" % s3_prefix,\n",
    "                            \"batch_size\": \"64\",\n",
    "                            \"num_epochs\": \"10\",\n",
    "                            \"stack_size\": \"1\",\n",
    "                            \"lr\": \"0.0003\",\n",
    "                            \"exploration_type\": \"Categorical\",\n",
    "                            \"e_greedy_value\": \"1\",\n",
    "                            \"epsilon_steps\": \"10000\",\n",
    "                            \"beta_entropy\": \"0.01\",\n",
    "                            \"discount_factor\": \"0.999\",\n",
    "                            \"loss_type\": \"Huber\",\n",
    "                            \"num_episodes_between_training\": \"20\"\n",
    "                        },\n",
    "                        subnets=deepracer_subnets,\n",
    "                        security_group_ids=deepracer_security_groups,\n",
    "                    )\n",
    "\n",
    "estimator.fit(wait=False)\n",
    "job_name = estimator.latest_training_job.job_name\n",
    "print(\"Training job: %s\" % job_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_job_arn = estimator.latest_training_job.describe()['TrainingJobArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create the Kinesis video stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kvs_stream_name = \"dr-kvs-{}\".format(job_name)\n",
    "\n",
    "!aws --region {aws_region} kinesisvideo create-stream --stream-name {kvs_stream_name} --media-type video/h264 --data-retention-in-hours 24\n",
    "print (\"Created kinesis video stream {}\".format(kvs_stream_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start the Robomaker job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "robomaker = boto3.client(\"robomaker\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Simulation Application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "robomaker_s3_key = 'robomaker/simulation_ws.tar.gz'\n",
    "robomaker_source = {'s3Bucket': s3_bucket,\n",
    "                    's3Key': robomaker_s3_key,\n",
    "                    'architecture': \"X86_64\"}\n",
    "simulation_software_suite={'name': 'Gazebo',\n",
    "                           'version': '7'}\n",
    "robot_software_suite={'name': 'ROS',\n",
    "                      'version': 'Kinetic'}\n",
    "rendering_engine={'name': 'OGRE',\n",
    "                  'version': '1.x'}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the DeepRacer bundle provided by RoboMaker service and upload it in our S3 bucket to create a RoboMaker Simulation Application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists('./build/output.tar.gz'):\n",
    "    print(\"Using the latest simapp from public s3 bucket\")\n",
    "    # Download Robomaker simApp for the deepracer public s3 bucket\n",
    "    simulation_application_bundle_location = \"s3://deepracer-managed-resources-us-east-1/deepracer-simapp.tar.gz\"\n",
    "    !aws s3 cp {simulation_application_bundle_location} ./\n",
    "\n",
    "    # Remove if the Robomaker sim-app is present in s3 bucket\n",
    "    !aws s3 rm s3://{s3_bucket}/{robomaker_s3_key}\n",
    "\n",
    "    # Uploading the Robomaker SimApp to your S3 bucket\n",
    "    !aws s3 cp ./deepracer-simapp.tar.gz s3://{s3_bucket}/{robomaker_s3_key}\n",
    "\n",
    "    # Cleanup the locally downloaded version of SimApp\n",
    "    !rm deepracer-simapp.tar.gz\n",
    "else:\n",
    "    print(\"Using the simapp from build directory\")\n",
    "    !aws s3 cp ./build/output.tar.gz s3://{s3_bucket}/{robomaker_s3_key}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "app_name = \"deepracer-notebook-application\" + strftime(\"%y%m%d-%H%M%S\", gmtime())\n",
    "\n",
    "print(app_name)\n",
    "try:\n",
    "    response = robomaker.create_simulation_application(name=app_name,\n",
    "                                                       sources=[robomaker_source],\n",
    "                                                       simulationSoftwareSuite=simulation_software_suite,\n",
    "                                                       robotSoftwareSuite=robot_software_suite,\n",
    "                                                       renderingEngine=rendering_engine)\n",
    "    simulation_app_arn = response[\"arn\"]\n",
    "    print(\"Created a new simulation app with ARN:\", simulation_app_arn)\n",
    "except Exception as e:\n",
    "    if \"AccessDeniedException\" in str(e):\n",
    "        display(Markdown(generate_help_for_robomaker_all_permissions(role)))\n",
    "        raise e\n",
    "    else:\n",
    "        raise e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Launch the Simulation job on RoboMaker\n",
    "\n",
    "We create [AWS RoboMaker](https://console.aws.amazon.com/robomaker/home#welcome) Simulation Jobs that simulates the environment and shares this data with SageMaker for training. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "s3_yaml_name=\"training_params.yaml\"\n",
    "world_name = \"reInvent2019_track\"\n",
    "\n",
    "!touch {s3_yaml_name}\n",
    "!echo \"WORLD_NAME:                           \\\"{world_name}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"SAGEMAKER_SHARED_S3_BUCKET:           \\\"{s3_bucket}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"SAGEMAKER_SHARED_S3_PREFIX:           \\\"{s3_prefix}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"TRAINING_JOB_ARN:                     \\\"{training_job_arn}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"METRICS_S3_BUCKET:                    \\\"{s3_bucket}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"METRICS_S3_OBJECT_KEY:                \\\"{s3_prefix}/training_metrics.json\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"AWS_REGION:                           \\\"{aws_region}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"TARGET_REWARD_SCORE:                  \\\"None\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"NUMBER_OF_EPISODES:                   \\\"0\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"ROBOMAKER_SIMULATION_JOB_ACCOUNT_ID:  \\\"{account_id}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"JOB_TYPE:                             \\\"TRAINING\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"CHANGE_START_POSITION:                \\\"true\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"ALTERNATE_DRIVING_DIRECTION:          \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"KINESIS_VIDEO_STREAM_NAME:            \\\"{kvs_stream_name}\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"REWARD_FILE_S3_KEY:                   \\\"{s3_prefix}/customer_reward_function.py\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"MODEL_METADATA_FILE_S3_KEY:           \\\"{s3_prefix}/model/model_metadata.json\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "!echo \"NUMBER_OF_OBSTACLES:                  \\\"0\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"MIN_DISTANCE_BETWEEN_OBSTACLES:       \\\"2.0\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"RANDOMIZE_OBSTACLE_LOCATIONS:         \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"PSEUDO_RANDOMIZE_OBSTACLE_LOCATIONS:  \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"NUMBER_OF_PSEUDO_RANDOM_PLACEMENTS:   \\\"2\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"IS_OBSTACLE_BOT_CAR:                  \\\"false\\\"\" | tee -a {s3_yaml_name}   \n",
    "\n",
    "!echo \"IS_LANE_CHANGE:                       \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"LOWER_LANE_CHANGE_TIME:               \\\"3.0\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"UPPER_LANE_CHANGE_TIME:               \\\"5.0\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"LANE_CHANGE_DISTANCE:                 \\\"1.0\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "!echo \"NUMBER_OF_BOT_CARS:                   \\\"6\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"MIN_DISTANCE_BETWEEN_BOT_CARS:        \\\"2.0\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"RANDOMIZE_BOT_CAR_LOCATIONS:          \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"BOT_CAR_SPEED:                        \\\"0.2\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "!echo \"CAR_COLOR:                            \\\"LightBlue\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"FIRST_PERSON_VIEW:                    \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"CHANGE_START_POSITION:                \\\"true\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "print(\"Upload yaml settings to S3\")\n",
    "!aws s3 cp ./training_params.yaml {s3_location}/training_params.yaml\n",
    "!rm training_params.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_simulation_workers = 1\n",
    "\n",
    "envriron_vars = {\n",
    "    \"S3_YAML_NAME\": s3_yaml_name,\n",
    "    \"SAGEMAKER_SHARED_S3_PREFIX\": s3_prefix,\n",
    "    \"SAGEMAKER_SHARED_S3_BUCKET\": s3_bucket,\n",
    "    \"WORLD_NAME\": world_name,\n",
    "    \"KINESIS_VIDEO_STREAM_NAME\": kvs_stream_name,\n",
    "    \"APP_REGION\": aws_region,\n",
    "    \"MODEL_METADATA_FILE_S3_KEY\": \"%s/model/model_metadata.json\" % s3_prefix\n",
    "}\n",
    "\n",
    "simulation_application = {\"application\":simulation_app_arn,\n",
    "                          \"launchConfig\": {\"packageName\": \"deepracer_simulation_environment\",\n",
    "                                           \"launchFile\": \"distributed_training.launch\",\n",
    "                                           \"environmentVariables\": envriron_vars}\n",
    "                         }\n",
    "\n",
    "\n",
    "vpcConfig = {\"subnets\": deepracer_subnets,\n",
    "             \"securityGroups\": deepracer_security_groups,\n",
    "             \"assignPublicIp\": True}\n",
    "\n",
    "responses = []\n",
    "for job_no in range(num_simulation_workers):\n",
    "    client_request_token = strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "    response =  robomaker.create_simulation_job(iamRole=sagemaker_role,\n",
    "                                            clientRequestToken=client_request_token,\n",
    "                                            maxJobDurationInSeconds=job_duration_in_seconds,\n",
    "                                            failureBehavior=\"Continue\",\n",
    "                                            simulationApplications=[simulation_application],\n",
    "                                            vpcConfig=vpcConfig\n",
    "                                            )\n",
    "    responses.append(response)\n",
    "\n",
    "print(\"Created the following jobs:\")\n",
    "job_arns = [response[\"arn\"] for response in responses]\n",
    "for response in responses:\n",
    "    print(\"Job ARN\", response[\"arn\"]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing the simulations in RoboMaker\n",
    "You can visit the RoboMaker console to visualize the simulations or run the following cell to generate the hyperlinks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Markdown(generate_robomaker_links(job_arns, aws_region)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating temporary folder top plot metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp_dir = \"/tmp/{}\".format(job_name)\n",
    "os.system(\"mkdir {}\".format(tmp_dir))\n",
    "print(\"Create local folder {}\".format(tmp_dir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot metrics for training job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import json\n",
    "\n",
    "training_metrics_file = \"training_metrics.json\"\n",
    "training_metrics_path = \"{}/{}\".format(s3_bucket, training_metrics_file)\n",
    "wait_for_s3_object(s3_bucket, training_metrics_path, tmp_dir)\n",
    "\n",
    "json_file = \"{}/{}\".format(tmp_dir, training_metrics_file)\n",
    "with open(json_file) as fp:  \n",
    "    data = json.load(fp)\n",
    "\n",
    "df = pd.DataFrame(data['metrics'])\n",
    "x_axis = 'episode'\n",
    "y_axis = 'reward_score'\n",
    "\n",
    "plt = df.plot(x=x_axis,y=y_axis, figsize=(12,5), legend=True, style='b-')\n",
    "plt.set_ylabel(y_axis);\n",
    "plt.set_xlabel(x_axis);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean up RoboMaker and SageMaker training job\n",
    "\n",
    "Execute the cells below if you want to kill RoboMaker and SageMaker job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Cancelling robomaker job\n",
    "# for job_arn in job_arns:\n",
    "#     robomaker.cancel_simulation_job(job=job_arn)\n",
    "\n",
    "# # Stopping sagemaker training job\n",
    "# sage_session.sagemaker_client.stop_training_job(TrainingJobName=estimator._current_job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation - ReInvent Track"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_yaml_name=\"evaluation_params.yaml\"\n",
    "world_name = \"reInvent2019_track\"\n",
    "\n",
    "!touch {s3_yaml_name}\n",
    "echo \"WORLD_NAME:                           \\\"{world_name}\\\"\" | tee {s3_yaml_name}\n",
    "echo \"MODEL_S3_BUCKET:                      \\\"{s3_bucket}\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"MODEL_S3_PREFIX:                      \\\"{s3_prefix}\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"AWS_REGION:                           \\\"{aws_region}\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"METRICS_S3_BUCKET:                    \\\"{s3_bucket}\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"METRICS_S3_OBJECT_KEY:                \\\"{s3_prefix}/evaluation_metrics.json\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"NUMBER_OF_TRIALS:                     \\\"10\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"ROBOMAKER_SIMULATION_JOB_ACCOUNT_ID:  \\\"{account_id}\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"JOB_TYPE:                             \\\"EVALUATION\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "echo \"NUMBER_OF_OBSTACLES:                  \\\"0\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"MIN_DISTANCE_BETWEEN_OBSTACLES:       \\\"2.0\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"RANDOMIZE_OBSTACLE_LOCATIONS:         \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"PSEUDO_RANDOMIZE_OBSTACLE_LOCATIONS:  \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"NUMBER_OF_PSEUDO_RANDOM_PLACEMENTS:   \\\"2\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"IS_OBSTACLE_BOT_CAR:                  \\\"false\\\"\" | tee -a {s3_yaml_name}  \n",
    "\n",
    "echo \"IS_LANE_CHANGE:                       \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"LOWER_LANE_CHANGE_TIME:               \\\"3.0\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"UPPER_LANE_CHANGE_TIME:               \\\"5.0\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"LANE_CHANGE_DISTANCE:                 \\\"1.0\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "echo \"NUMBER_OF_BOT_CARS:                   \\\"6\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"MIN_DISTANCE_BETWEEN_BOT_CARS:        \\\"2.0\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"RANDOMIZE_BOT_CAR_LOCATIONS:          \\\"true\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"BOT_CAR_SPEED:                        \\\"0.2\\\"\" | tee -a {s3_yaml_name}\n",
    "echo \"CAR_COLOR:                            \\\"LightBlue\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "!echo \"CAR_COLOR:                           \\\"LightBlue\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"FIRST_PERSON_VIEW:                   \\\"false\\\"\" | tee -a {s3_yaml_name}\n",
    "!echo \"CHANGE_START_POSITION:               \\\"true\\\"\" | tee -a {s3_yaml_name}\n",
    "\n",
    "print(\"Upload yaml settings to S3\")\n",
    "!aws s3 cp ./evaluation_params.yaml {s3_location}/evaluation_params.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sys.path.append(\"./src\")\n",
    "\n",
    "num_simulation_workers = 1\n",
    "\n",
    "envriron_vars = {\n",
    "    \"S3_YAML_NAME\": s3_yaml_name,\n",
    "    \"MODEL_S3_PREFIX\": s3_prefix,\n",
    "    \"MODEL_S3_BUCKET\": s3_bucket,\n",
    "    \"WORLD_NAME\": world_name,\n",
    "    \"KINESIS_VIDEO_STREAM_NAME\": kvs_stream_name,\n",
    "    \"APP_REGION\": aws_region,\n",
    "    \"MODEL_METADATA_FILE_S3_KEY\": \"%s/model/model_metadata.json\" % s3_prefix\n",
    "}\n",
    "\n",
    "simulation_application = {\n",
    "    \"application\":simulation_app_arn,\n",
    "    \"launchConfig\": {\n",
    "         \"packageName\": \"deepracer_simulation_environment\",\n",
    "         \"launchFile\": \"evaluation.launch\",\n",
    "         \"environmentVariables\": envriron_vars\n",
    "    }\n",
    "}\n",
    "                            \n",
    "vpcConfig = {\"subnets\": deepracer_subnets,\n",
    "             \"securityGroups\": deepracer_security_groups,\n",
    "             \"assignPublicIp\": True}\n",
    "\n",
    "responses = []\n",
    "for job_no in range(num_simulation_workers):\n",
    "    response =  robomaker.create_simulation_job(clientRequestToken=strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime()),\n",
    "                                                outputLocation={ \n",
    "                                                  \"s3Bucket\": s3_bucket,\n",
    "                                                  \"s3Prefix\": s3_prefix\n",
    "                                                },\n",
    "                                                maxJobDurationInSeconds=job_duration_in_seconds,\n",
    "                                                iamRole=sagemaker_role,\n",
    "                                                failureBehavior=\"Continue\",\n",
    "                                                simulationApplications=[simulation_application],\n",
    "                                                vpcConfig=vpcConfig)\n",
    "    responses.append(response)\n",
    "\n",
    "# print(\"Created the following jobs:\")\n",
    "for response in responses:\n",
    "    print(\"Job ARN\", response[\"arn\"]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating temporary folder top plot metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluation_metrics_file = \"evaluation_metrics.json\"\n",
    "evaluation_metrics_path = \"{}/{}\".format(s3_bucket, evaluation_metrics_file)\n",
    "wait_for_s3_object(s3_bucket, evaluation_metrics_path, tmp_dir)\n",
    "\n",
    "json_file = \"{}/{}\".format(tmp_dir, evaluation_metrics_file)\n",
    "with open(json_file) as fp:  \n",
    "    data = json.load(fp)\n",
    "\n",
    "df = pd.DataFrame(data['metrics'])\n",
    "# Converting milliseconds to seconds\n",
    "df['elapsed_time'] = df['elapsed_time_in_milliseconds']/1000\n",
    "df = df[['trial', 'completion_percentage', 'elapsed_time']]\n",
    "\n",
    "display(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean Up Simulation Application Resource"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# robomaker.delete_simulation_application(application=simulation_app_arn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean your S3 bucket (Uncomment the awscli commands if you want to do it)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Uncomment if you only want to clean the s3 bucket\n",
    "# sagemaker_s3_folder = \"s3://{}/{}\".format(s3_bucket, s3_prefix)\n",
    "# !aws s3 rm --recursive {sagemaker_s3_folder}\n",
    "\n",
    "# robomaker_s3_folder = \"s3://{}/{}\".format(s3_bucket, job_name)\n",
    "# !aws s3 rm --recursive {robomaker_s3_folder}\n",
    "\n",
    "# robomaker_sim_app = \"s3://{}/{}\".format(s3_bucket, 'robomaker')\n",
    "# !aws s3 rm --recursive {robomaker_sim_app}\n",
    "\n",
    "# model_output = \"s3://{}/{}\".format(s3_bucket, s3_bucket)\n",
    "# !aws s3 rm --recursive {model_output}"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "conda_amazonei_tensorflow_p36",
   "language": "python",
   "name": "conda_amazonei_tensorflow_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}