Tensorflow - Complete ML Tutorial with TF and Keras.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Tensorflow - Complete ML Tutorial with TF and Keras.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOGMZ0wWxXt3wMdshh8QhIT",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sahug/ds-tensorflow-colab/blob/master/Tensorflow%20-%20Complete%20ML%20Tutorial%20with%20TF%20and%20Keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Tensorflow - Complete ML Tutorial with TF and Keras**"
      ],
      "metadata": {
        "id": "twGNNeLl1N5C"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Tensorflow** has 2 main components that we will try to understand\n",
        "1. Graphs\n",
        "2. Sessions\n",
        "\n",
        "Tensorflow works by building a graph of a defined computations. Nothing is computed or stored in this graph. It is simply a way of defining the operations that have been written in code.\n",
        "\n",
        "A TF Sessions allows part of the graphs to be executed. It allocates memory and resources and handels the execution of the operations and computatuions we have defined.\n",
        "\n",
        "In some instances we will need to run a session to be able to execute parts of the graphs we've created earlier."
      ],
      "metadata": {
        "id": "J6a3aEQQ2FI7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Installing Tensorflow**"
      ],
      "metadata": {
        "id": "OwT4di6Z34v9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install tensorflow"
      ],
      "metadata": {
        "id": "BeyhHr3E31Jw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you have a CUDA enabled GPU you can install the GPU version of TensorFlow. "
      ],
      "metadata": {
        "id": "hdIpbpFx4F1A"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Eh_iJYAj1CeV"
      },
      "outputs": [],
      "source": [
        "!pip install tensorflow-gpu"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Importing TensorFlow**\n",
        "\n",
        "The first step here is going to be to select the correct version of TensorFlow from within collabratory!"
      ],
      "metadata": {
        "id": "gCObFhH54j5b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%tensorflow_version 2.x "
      ],
      "metadata": {
        "id": "asVsdLr_4qbX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf #now import the tensorflow module\n",
        "print(tf.version) #Make sure the version is 2.x"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "j7DVINnV719t",
        "outputId": "b4b646c3-2333-4d97-decf-d4b3350f2c07"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<module 'tensorflow._api.v2.version' from '/usr/local/lib/python3.7/dist-packages/tensorflow/_api/v2/version/__init__.py'>\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Tensors**\n",
        "\n",
        "\"A **Tensor** is a generalization of vectors and matrices to potentially higher dimensions. Internally, TensorFlow represents tensors as n-dimensional arrays of base datatypes.\" (https://www.tensorflow.org/guide/tensor)\n",
        "\n",
        "It should't surprise you that tensors are a fundemental apsect of TensorFlow. They are the main objects that are passed around and manipluated throughout the program. Each tensor represents a partialy defined computation that will eventually produce a value. TensorFlow programs work by building a graph of Tensor objects that details how tensors are related. Running different parts of the graph allow results to be generated.\n",
        "\n",
        "Each tensor has a data type and a shape.\n",
        "\n",
        "**Data Types Include**: float32, int32, string and others.\n",
        "\n",
        "**Shape**: Represents the dimension of data.\n",
        "\n",
        "Just like vectors and matrices tensors can have operations applied to them like addition, subtraction, dot product, cross product etc.\n",
        "\n",
        "In the next sections we will discuss some different properties of tensors. This is to make you more familiar with how tensorflow represnts data and how you can manipulate this data."
      ],
      "metadata": {
        "id": "NthS87Ja8ISy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Creating Tensors**\n",
        "\n",
        "Below is an example of how to create some different tensors.\n",
        "\n",
        "You simply define the value of the tensor and the datatype and you are good to go! It's worth mentioning that usually we deal with tensors of numeric data, it is quite rare to see string tensors.\n",
        "\n",
        "For a full list of datatypes please refer to the following guide.\n",
        "\n",
        "https://www.tensorflow.org/api_docs/python/tf/dtypes/DType?version=stable"
      ],
      "metadata": {
        "id": "7hjhFU6O8xOp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "string = tf.Variable(\"this is a string\", tf.string)\n",
        "number = tf.Variable(324, tf.int16)\n",
        "floating = tf.Variable(3.567, tf.float64)\n",
        "\n",
        "string, number, floating"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yDquVDCq8r05",
        "outputId": "9f92008a-3f89-4c2c-b08a-b2454b39acf6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=() dtype=string, numpy=b'this is a string'>,\n",
              " <tf.Variable 'Variable:0' shape=() dtype=int32, numpy=324>,\n",
              " <tf.Variable 'Variable:0' shape=() dtype=float32, numpy=3.567>)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Rank/Degree of Tensors**\n",
        "\n",
        "Another word for rank is degree, these terms simply mean the number of dimensions involved in the tensor. What we created above is a tensor of rank 0, also known as a scalar.\n",
        "\n",
        "Now we'll create some tensors of higher degrees/ranks."
      ],
      "metadata": {
        "id": "3vfDcp4xDGt_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "rank1_tensor = tf.Variable([\"Test\"], tf.string)\n",
        "rank2_tensor = tf.Variable([[\"Test\", \"Ok\"], [\"Test\", \"Ok\"]], tf.string) \n",
        "\n",
        "rank1_tensor, rank2_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w_ln2Y6h-EAP",
        "outputId": "5a712d00-ba4f-4a6e-ca21-8a48dfdfe48f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=(1,) dtype=string, numpy=array([b'Test'], dtype=object)>,\n",
              " <tf.Variable 'Variable:0' shape=(2, 2) dtype=string, numpy=\n",
              " array([[b'Test', b'Ok'],\n",
              "        [b'Test', b'Ok']], dtype=object)>)"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "To determine the rank of a tensor we can call the following method."
      ],
      "metadata": {
        "id": "azVCzEGPDnPN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tf.rank(rank1_tensor), tf.rank(rank2_tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N6EUF4xaDmEx",
        "outputId": "fa77d213-1e68-4c53-8130-7b550ac7ff24"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int32, numpy=1>,\n",
              " <tf.Tensor: shape=(), dtype=int32, numpy=2>)"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The rank of a tensor is direclty related to the deepest level of nested lists. You can see in the first example [\"Test\"] is a rank 1 tensor as the deepest level of nesting is 1. Where in the second example [[\"test\", \"ok\"], [\"test\", \"yes\"]] is a rank 2 tensor as the deepest level of nesting is 2."
      ],
      "metadata": {
        "id": "BBeibluYD128"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "To get the **shape** of a tensor we use the shape attribute."
      ],
      "metadata": {
        "id": "GyM3QpE5D5Sr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "rank1_tensor.shape, rank2_tensor.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9RClVflzD7Uh",
        "outputId": "0e41ea9d-3612-4d5b-d78e-e72db4184ab1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([1]), TensorShape([2, 2]))"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Changing Shape**\n",
        "\n",
        "The number of elements of a tensor is the product of the sizes of all its shapes. There are often many shapes that have the same number of elements, making it convient to be able to change the shape of a tensor.\n",
        "\n",
        "The example below shows how to change the shape of a tensor."
      ],
      "metadata": {
        "id": "o1ugAAnwEPVN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# tf.ones() creates a shape [1,2,3] tensor full of ones\n",
        "tensor1 = tf.ones([1,2,3])\n",
        "\n",
        "# reshape existing data to shape [2,3,1]\n",
        "tensor2 = tf.reshape(tensor1, [2,3,1])\n",
        "\n",
        "# -1 tells the tensor to calculate the size of the dimension in that place # this will reshape the tensor to [3,3]\n",
        "tensor3 = tf.reshape(tensor2, [3, -1])\n",
        "\n",
        "# The numer of elements in the reshaped tensor MUST match the number in the original"
      ],
      "metadata": {
        "id": "LBBRFA0mERmK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(tensor1)\n",
        "print(tensor2)\n",
        "print(tensor3)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Lpqkme_tEtn1",
        "outputId": "688c03b9-8660-4157-d8dd-b0cfd6e6a465"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(\n",
            "[[[1. 1. 1.]\n",
            "  [1. 1. 1.]]], shape=(1, 2, 3), dtype=float32)\n",
            "tf.Tensor(\n",
            "[[[1.]\n",
            "  [1.]\n",
            "  [1.]]\n",
            "\n",
            " [[1.]\n",
            "  [1.]\n",
            "  [1.]]], shape=(2, 3, 1), dtype=float32)\n",
            "tf.Tensor(\n",
            "[[1. 1.]\n",
            " [1. 1.]\n",
            " [1. 1.]], shape=(3, 2), dtype=float32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Slicing Tensors**\n",
        "\n",
        "You may be familiar with the term \"slice\" in python and its use on lists, tuples etc. Well the slice operator can be used on tensors to select specific axes or elements.\n",
        "\n",
        "When we slice or select elements from a tensor, we can use comma seperated values inside the set of square brackets. Each subsequent value refrences a different dimension of the tensor.\n",
        "\n",
        "`Ex: tensor[dim1, dim2, dim3]`\n",
        "\n",
        "I've included a few examples that will hopefully help illustrate how we can manipulate tensors with the slice operator."
      ],
      "metadata": {
        "id": "sYuBdxfdFH6y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Creating a 2D tensor\n",
        "matrix = [[1,2,3,4,5],\n",
        "          [6,7,8,9,10],\n",
        "          [11,12,13,14,15],\n",
        "          [16,17,18,19,20]]\n",
        "\n",
        "tensor = tf.Variable(matrix, dtype=tf.int32) \n",
        "print(tf.rank(tensor))\n",
        "print(tensor.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PeQgoAmrFHRb",
        "outputId": "8f1738e5-1cb5-403b-c067-472a84360ac2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(2, shape=(), dtype=int32)\n",
            "(4, 5)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Now lets select some different rows and columns from our tensor\n",
        "\n",
        "three = tensor[0,2]  # selects the 3rd element from the 1st row\n",
        "print(three)  # -> 3\n",
        "\n",
        "row1 = tensor[0]  # selects the first row\n",
        "print(row1)\n",
        "\n",
        "column1 = tensor[:, 0]  # selects the first column\n",
        "print(column1)\n",
        "\n",
        "row_2_and_4 = tensor[1::2]  # selects second and fourth row\n",
        "print(row_2_and_4)\n",
        "\n",
        "column_1_in_row_2_and_3 = tensor[1:3, 0]\n",
        "print(column_1_in_row_2_and_3)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s5WLHAKeF2BZ",
        "outputId": "0aa7a6e2-8433-40ba-a7f6-79f47274740e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(3, shape=(), dtype=int32)\n",
            "tf.Tensor([1 2 3 4 5], shape=(5,), dtype=int32)\n",
            "tf.Tensor([ 1  6 11 16], shape=(4,), dtype=int32)\n",
            "tf.Tensor(\n",
            "[[ 6  7  8  9 10]\n",
            " [16 17 18 19 20]], shape=(2, 5), dtype=int32)\n",
            "tf.Tensor([ 6 11], shape=(2,), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Types of Tensors**\n",
        "Before we go to far, I will mention that there are diffent types of tensors. These are the most used and we will talk more in depth about each as they are used.\n",
        "\n",
        "- Variable\n",
        "- Constant\n",
        "- Placeholder\n",
        "- SparseTensor\n",
        "\n",
        "With the execption of Variable all these tensors are immuttable, meaning their value may not change during execution.\n",
        "\n",
        "For now, it is enough to understand that we use the Variable tensor when we want to potentially change the value of our tensor."
      ],
      "metadata": {
        "id": "q4pNzZstF9P5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**TensorFlow Core Learning Algorithms**\n",
        "\n",
        "In this notebook we will walk through 4 fundemental machine learning algorithms. We will apply each of these algorithms to unique problems and datasets before highlighting the use cases of each.\n",
        "\n",
        "The algorithms we will focus on include:\n",
        "\n",
        "- Linear Regression\n",
        "- Classification\n",
        "- Clustering\n",
        "- Hidden Markov Models\n",
        "\n",
        "It is worth noting that there are many tools within TensorFlow that could be used to solve the problems we will see below. I have chosen the tools that I belive give the most variety and are easiest to use."
      ],
      "metadata": {
        "id": "LcHI4ca9KLB5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Linear Regression**\n",
        "\n",
        "Linear regression is one of the most basic forms of machine learning and is used to predict numeric values.\n",
        "\n",
        "In this tutorial we will use a linear model to predict the survival rate of passangers from the titanic dataset.\n",
        "\n",
        "This section is based on the following documentation: https://www.tensorflow.org/tutorials/estimator/linear"
      ],
      "metadata": {
        "id": "PaIxQjRsKT0O"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Setup and Imports**\n",
        "\n",
        "Before we get started we must install sklearn and import the following modules."
      ],
      "metadata": {
        "id": "aeZQ0qFPKWMV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython.display import clear_output\n",
        "from six.moves import urllib\n",
        "\n",
        "import tensorflow.compat.v2.feature_column as fc\n",
        "\n",
        "import tensorflow as tf"
      ],
      "metadata": {
        "id": "O7uFG8ZbF8jT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load dataset.\n",
        "dftrain = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/train.csv') # training data\n",
        "dfeval = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/eval.csv') # testing data\n",
        "y_train = dftrain.pop('survived')\n",
        "y_eval = dfeval.pop('survived')"
      ],
      "metadata": {
        "id": "TJP9Z3VXF8oG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Feature Columns**\n",
        "\n",
        "In our dataset we have two different kinds of information: **Categorical and Numeric**\n",
        "\n",
        "Our categorical data is anything that is not numeric! For example, the sex column does not use numbers, it uses the words \"male\" and \"female\".\n",
        "\n",
        "Before we continue and create/train a model we must convet our categorical data into numeric data. We can do this by encoding each category with an integer (ex. male = 1, female = 2).\n",
        "\n",
        "Fortunately for us TensorFlow has some tools to help!"
      ],
      "metadata": {
        "id": "U8yQm50cKqnk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "CATEGORICAL_COLUMNS = ['sex', 'n_siblings_spouses', 'parch', 'class', 'deck',\n",
        "                       'embark_town', 'alone']\n",
        "NUMERIC_COLUMNS = ['age', 'fare']\n",
        "\n",
        "feature_columns = []\n",
        "for feature_name in CATEGORICAL_COLUMNS:\n",
        "  vocabulary = dftrain[feature_name].unique()  # gets a list of all unique values from given feature column\n",
        "  feature_columns.append(tf.feature_column.categorical_column_with_vocabulary_list(feature_name, vocabulary))\n",
        "\n",
        "for feature_name in NUMERIC_COLUMNS:\n",
        "  feature_columns.append(tf.feature_column.numeric_column(feature_name, dtype=tf.float32))\n",
        "\n",
        "print(feature_columns)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fkOUeJVPKk1V",
        "outputId": "c4679673-3923-4fe4-d90b-f3fb1ec92563"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[VocabularyListCategoricalColumn(key='sex', vocabulary_list=('male', 'female'), dtype=tf.string, default_value=-1, num_oov_buckets=0), VocabularyListCategoricalColumn(key='n_siblings_spouses', vocabulary_list=(1, 0, 3, 4, 2, 5, 8), dtype=tf.int64, default_value=-1, num_oov_buckets=0), VocabularyListCategoricalColumn(key='parch', vocabulary_list=(0, 1, 2, 5, 3, 4), dtype=tf.int64, default_value=-1, num_oov_buckets=0), VocabularyListCategoricalColumn(key='class', vocabulary_list=('Third', 'First', 'Second'), dtype=tf.string, default_value=-1, num_oov_buckets=0), VocabularyListCategoricalColumn(key='deck', vocabulary_list=('unknown', 'C', 'G', 'A', 'B', 'D', 'F', 'E'), dtype=tf.string, default_value=-1, num_oov_buckets=0), VocabularyListCategoricalColumn(key='embark_town', vocabulary_list=('Southampton', 'Cherbourg', 'Queenstown', 'unknown'), dtype=tf.string, default_value=-1, num_oov_buckets=0), VocabularyListCategoricalColumn(key='alone', vocabulary_list=('n', 'y'), dtype=tf.string, default_value=-1, num_oov_buckets=0), NumericColumn(key='age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='fare', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**The Training Process**\n",
        "\n",
        "So, we are almost done preparing our dataset and I feel as though it's a good time to explain how our model is trained. Specifically, how input data is fed to our model.\n",
        "\n",
        "For this specific model data is going to be streamed into it in small batches of **32**. This means we will not feed the entire dataset to our model at once, but simply small batches of entries. We will feed these batches to our model multiple times according to the number of epochs.\n",
        "\n",
        "An epoch is simply one stream of our entire dataset. The number of epochs we define is the amount of times our model will see the entire dataset. We use multiple epochs in hope that after seeing the same data multiple times the model will better determine how to estimate it.\n",
        "\n",
        "Ex. if we have **10** ephocs, our model will see the same dataset 10 times.\n",
        "\n",
        "Since we need to feed our data in batches and multiple times, we need to create something called an input function. The input function simply defines how our dataset will be converted into batches at each epoch."
      ],
      "metadata": {
        "id": "c-yiZma_Les5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Input Function**\n",
        "\n",
        "The TensorFlow model we are going to use requires that the data we pass it comes in as a tf.data.Dataset object. This means we must create a input function that can convert our current pandas dataframe into that object.\n",
        "\n",
        "Below you'll see a seemingly complicated input function, this is straight from the TensorFlow documentation (https://www.tensorflow.org/tutorials/estimator/linear). I've commented as much as I can to make it understandble, but you may want to refer to the documentation for a detailed explination of each method.\n",
        "\n",
        "In our function, we pass in x, which is our **dictionary**, and y, which is our **label**. We can also pass in our **batch** size, number of **epochs**, and whether or not to **shuffle** the data. Please note that you **always want to shuffle your data**. The batch size is a hyper parameter that you should file empirically. The number of epochs is how many times you would like to go over your data. For training, set any number. For test, set this to 1."
      ],
      "metadata": {
        "id": "MrGBSoyPLpY2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def make_input_fn(data_df, label_df, num_epochs=10, shuffle=True, batch_size=32):\n",
        "  def input_function():  # inner function, this will be returned\n",
        "    ds = tf.data.Dataset.from_tensor_slices((dict(data_df), label_df))  # create tf.data.Dataset object with data and its label\n",
        "    if shuffle:\n",
        "      ds = ds.shuffle(1000)  # randomize order of data\n",
        "    ds = ds.batch(batch_size).repeat(num_epochs)  # split dataset into batches of 32 and repeat process for number of epochs\n",
        "    return ds  # return a batch of the dataset\n",
        "  return input_function  # return a function object for use\n",
        "\n",
        "train_input_fn = make_input_fn(dftrain, y_train)  # here we will call the input_function that was returned to us to get a dataset object we can feed to the model\n",
        "eval_input_fn = make_input_fn(dfeval, y_eval, num_epochs=1, shuffle=False)"
      ],
      "metadata": {
        "id": "wzZMIsuZLt1Z"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Creating the Model**\n",
        "\n",
        "In this tutorial we are going to use a **linear estimator** to utilize the linear regression algorithm.\n",
        "\n",
        "Creating one is pretty easy! Have a look below."
      ],
      "metadata": {
        "id": "d28sW9UHL1in"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "linear_est = tf.estimator.LinearClassifier(feature_columns=feature_columns)\n",
        "# We create a linear estimtor by passing the feature columns we created earlier"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LGA0DEOaL0rh",
        "outputId": "2684bda1-6aa3-4bd3-d025-778bb75ab53c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INFO:tensorflow:Using default config.\n",
            "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpju_5v05t\n",
            "INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpju_5v05t', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
            "graph_options {\n",
            "  rewrite_options {\n",
            "    meta_optimizer_iterations: ONE\n",
            "  }\n",
            "}\n",
            ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training the Model**\n",
        "\n",
        "Training the model is as easy as passing the input functions that we created earlier."
      ],
      "metadata": {
        "id": "TnIahLr4kjyr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "linear_est.train(train_input_fn)  # train\n",
        "result = linear_est.evaluate(eval_input_fn)  # get model metrics/stats by testing on tetsing data\n",
        "\n",
        "clear_output()  # clears console output\n",
        "print(result['accuracy'])  # the result variable is simply a dict of stats about our model"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qpd986v5km_l",
        "outputId": "596856d0-9f3a-49f1-85a5-baf44a55b85c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.75757575\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pred_dicts = list(linear_est.predict(eval_input_fn))\n",
        "probs = pd.Series([pred['probabilities'][1] for pred in pred_dicts])\n",
        "\n",
        "probs.plot(kind='hist', bins=20, title='predicted probabilities')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 457
        },
        "id": "KytmDx0mlFFm",
        "outputId": "99c81bf6-42c5-48e1-c4e8-2acb9bf2fcfb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INFO:tensorflow:Calling model_fn.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/canned/linear.py:1478: UserWarning: `layer.add_variable` is deprecated and will be removed in a future version. Please use `layer.add_weight` method instead.\n",
            "  getter=tf.compat.v1.get_variable)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "INFO:tensorflow:Restoring parameters from /tmp/tmpju_5v05t/model.ckpt-200\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f66bb8d7110>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXyElEQVR4nO3deZhddZ3n8feHTRbZU0YMhKgd0TSOiIXL0+0aUEEkdDemYZCONG20dVwGH9uAOtB248A4iri0dAQlbiyiSGzcMKKM3Q0aFhEIDBASSQgkhiCLCgY/88f5lX27ppaTSp17q+p8Xs9TT539fH83lfu553fOPUe2iYiI9tmm1wVERERvJAAiIloqARAR0VIJgIiIlkoARES0VAIgIqKlEgAxIUlaJenQMnyqpPO6sM9XSFrT9H7Kvt4k6cdjXHfEOiWdK+mDQy0r6RZJrxhh3W9LWjCWumLy2a7XBUSMxvaH6ywn6QJgje0PNFvRxGb7rSPM++OBYUmnA39k+40d8w9vtrqYSHIEEI2T1LoPGm1sc0w+CYAYk9JFc4qkWyVtkvR5STuWea+QtEbS+yTdB3xe0jaSFkm6S9JGSZdI2qtjeydIWl3mvX/Qvk6X9KWO8T+V9G+SHpR0T+lOWQgcD/ydpEckfbMs+zRJX5O0QdLdkt7ZsZ2dJF1Q6r8VOGSUNlvSOyWtlPRLSR+RtE2Z9yZJ/yrpbEkbgdMl7S7pC2XfqyV9YGD5/9ikPiXpV5JukzS3Y8aJklZIerjs7y1D1HNqqWOVpOM7pl8g6R9H+Hc7VNJrgVOBvyyv18/K/B9K+puO5f+61LFJ0ncl7T9QeGnrekkPSfq5pANHev1i4kkAxNY4HngN8EzgWUBn18tTgb2A/YGFwDuAo4GXA08DNgGfBpA0B/gMcEKZtzew71A7LG9A3wY+CfQBBwE32l4MfBn4X7afbPv15c32m8DPgBnAXODdkl5TNndaqf2ZpR11+r7/DOgHDgbmAX/dMe9FwEpgOnBGqXF34Bml3X8FnDho+buAaaWWr3eE4nrgSGC3ss7Zkg7uWPepZb0Zpe7Fkg6oUT8Atr8DfBi4uLxezxu8jKR5VCHx51Sv9f8BLiyzXw28jOrffXdgPrCx7v5jYkgAxNb4lO17bD9A9YZ3XMe83wOn2X7M9m+AtwLvt73G9mPA6cAxpavkGOBfbF9d5n2wrD+U/wp83/aFtn9ne6PtG4dZ9hCgz/aHbD9ueyXwWeDYMn8+cIbtB2zfA3yiRpvPKsv/Avj4oDbfa/uTtjcDj5f9nGL7YdurgI9ShdyA9cDHSzsuBm4HXgdg+wrbd7nyI+B7wEsH1fLB8vr+CLiitGc8vRX4n7ZXlDZ9GDiohPDvgF2BZwMqy6wb5/1HwxIAsTXu6RheTfXpfcAG27/tGN8fuKx02zwIrACeoPq0/LTObdl+lOE/Te5H9am5jv2Bpw3ss+z31LJPBu+3tGE0I7W5c940YPtB21xN9Yl9wFr/57sx/mF7kg6XdI2kB0rdR5RtDthUXqfhahkP+wPndLx2DwACZtj+AfApqqO49ZIWS9ptnPcfDUsAxNbYr2N4JnBvx/jg28zeAxxue4+Onx1trwXWdW5L0s5U3UBDuYeqy2YoQ+3z7kH73NX2EWX+f9pvacNo6rb5l1SfkvcftPzajvEZkjR4e5KeBHwN+N/AdNt7AN+ievMdsKekXUaopY7RbgV8D/CWQa/fTrb/DcD2J2y/AJhD1RX03i3cf/RYAiC2xtsl7Vv6rd8PXDzCsucCZ3ScROwrfcwAlwJHlpO7OwAfYvi/zS8Dh0qaL2k7SXtLOqjMu5+qv33AT4CHy8nonSRtK+lASQMney8BTpG0p6R9qc5TjOa9Zfn9gHcN12bbT5TtnyFp19Luk4EvdSz2FOCdkraX9AbgOVRv9DsATwI2AJslHU7V5z7Y30vaQdJLqc4XfLVG/Z3uB2YNOjHd6Vyq1+ePAcpJ7TeU4UMkvUjS9sCjwG8ZvtsuJqgEQGyNr1D1Ta+k6pYZ8sqT4hxgKfA9SQ8D11CdBMX2LcDby/bWUZ0gHvKLTqXv/QjgPVRdEjcCAycwzwfmlC6Lb5Q34SOpThTfTfWp/Dyqk5YAf0/VdXJ3accXa7T5cuC6st8ryj6H8w6qN8eVwI9L+z7XMf9aYHap6wzgmHJO42HgnVQBsonqvMfSQdu+r8y7lyoU32r7thr1dxoIjI2Srh880/ZlwFnARZIeAm4GBr4nsBvV+ZRNVK/hRuAjW7j/6DHlgTAxFpJWAX9j+/u9rqVbJBmYbfvOXtcSMR5yBBAR0VIJgIiIlkoXUERES+UIICKipSbFDaumTZvmWbNm9bqMiIhJ5brrrvul7b7h5k+KAJg1axbLly/vdRkREZOKpBG/3Z4uoIiIlkoARES0VAIgIqKlEgARES2VAIiIaKkEQERESyUAIiJaKgEQEdFSCYCIiJaaFN8E7pVZi64Y87qrznzdOFYSETH+cgQQEdFSjQWApAMk3djx85Ckd0vaS9KVku4ov/dsqoaIiBheYwFg+3bbB9k+CHgB8GvgMmARsMz2bGBZGY+IiC7rVhfQXOAu26uBecCSMn0JcHSXaoiIiA7dCoBjgQvL8HTb68rwfcD0oVaQtFDScknLN2zY0I0aIyJapfEAkLQDcBTw1cHzXD2PcshnUtpebLvfdn9f37DPM4iIiDHqxhHA4cD1tu8v4/dL2geg/F7fhRoiImKQbgTAcfxH9w/AUmBBGV4AXN6FGiIiYpBGA0DSLsBhwNc7Jp8JHCbpDuDQMh4REV3W6DeBbT8K7D1o2kaqq4IiIqKH8k3giIiWSgBERLRUAiAioqUSABERLZUAiIhoqQRARERLJQAiIloqARAR0VIJgIiIlkoARES0VAIgIqKlEgARES2VAIiIaKkEQERESyUAIiJaKgEQEdFSCYCIiJZKAEREtFQCICKipRIAEREt1WgASNpD0qWSbpO0QtJLJO0l6UpJd5TfezZZQ0REDK3pI4BzgO/YfjbwPGAFsAhYZns2sKyMR0RElzUWAJJ2B14GnA9g+3HbDwLzgCVlsSXA0U3VEBERw2vyCODpwAbg85JukHSepF2A6bbXlWXuA6YPtbKkhZKWS1q+YcOGBsuMiGinJgNgO+Bg4DO2nw88yqDuHtsGPNTKthfb7rfd39fX12CZERHt1GQArAHW2L62jF9KFQj3S9oHoPxe32ANERExjMYCwPZ9wD2SDiiT5gK3AkuBBWXaAuDypmqIiIjhbdfw9t8BfFnSDsBK4ESq0LlE0knAamB+wzVERMQQGg0A2zcC/UPMmtvkfiMiYnT5JnBEREslACIiWioBEBHRUgmAiIiWSgBERLRUAiAioqUSABERLZUAiIhoqQRARERLJQAiIloqARAR0VIJgIiIlkoARES0VAIgIqKlEgARES2VAIiIaKkEQERESyUAIiJaKgEQEdFSCYCIiJZq9KHwklYBDwNPAJtt90vaC7gYmAWsAubb3tRkHRER8f/rxhHAK20fZLu/jC8CltmeDSwr4xER0WW96AKaBywpw0uAo3tQQ0RE6zUdAAa+J+k6SQvLtOm215Xh+4DpQ60oaaGk5ZKWb9iwoeEyIyLap9FzAMCf2l4r6SnAlZJu65xp25I81Iq2FwOLAfr7+4dcJiIixq7RIwDba8vv9cBlwAuB+yXtA1B+r2+yhoiIGFpjASBpF0m7DgwDrwZuBpYCC8piC4DLm6ohIiKG12QX0HTgMkkD+/mK7e9I+ilwiaSTgNXA/AZriIiIYTQWALZXAs8bYvpGYG5T+42IiHryTeCIiJZKAEREtFQCICKipRIAEREtlQCIiGipBEBEREvVCgBJz226kIiI6K66RwD/JOknkt4mafdGK4qIiK6oFQC2XwocD+wHXCfpK5IOa7SyiIhoVO1zALbvAD4AvA94OfAJSbdJ+vOmiouIiObUPQfwXySdDawAXgW83vZzyvDZDdYXERENqXsvoE8C5wGn2v7NwETb90r6QCOVRUREo+oGwOuA39h+AkDSNsCOtn9t+4uNVRcREY2pew7g+8BOHeM7l2kRETFJ1Q2AHW0/MjBShndupqSIiOiGugHwqKSDB0YkvQD4zQjLR0TEBFf3HMC7ga9KuhcQ8FTgLxurKiIiGlcrAGz/VNKzgQPKpNtt/665siIiomlb8kjIQ4BZZZ2DJWH7C41UFRERjasVAJK+CDwTuBF4okw2kACIiJik6h4B9ANzbHtLdyBpW2A5sNb2kZKeDlwE7A1cB5xg+/Et3W5ERGydulcB3Ux14ncs3kV1C4kBZwFn2/4jYBNw0hi3GxERW6FuAEwDbpX0XUlLB35GW0nSvlTfIj6vjIvq/kGXlkWWAEdvedkREbG16nYBnT7G7X8c+Dtg1zK+N/Cg7c1lfA0wY6gVJS0EFgLMnDlzjLuPiIjh1H0ewI+AVcD2ZfinwPUjrSPpSGC97evGUpjtxbb7bff39fWNZRMRETGCulcBvZnq0/heVFcDzQDOBeaOsNqfAEdJOgLYEdgNOAfYQ9J25ShgX2Dt2MuPiIixqnsO4O1Ub+gPwR8eDvOUkVawfYrtfW3PAo4FfmD7eOAq4Jiy2ALg8jHUHRERW6luADzWeammpO2ovgcwFu8DTpZ0J9U5gfPHuJ2IiNgKdU8C/0jSqcBO5VnAbwO+WXcntn8I/LAMrwReuGVlRkTEeKt7BLAI2AD8HHgL8C2q5wNHRMQkVfdmcL8HPlt+IiJiCqh7FdDdDNHnb/sZ415RRER0xZbcC2jAjsAbqC4JjYiISaruF8E2dvystf1xqls8RETEJFW3C+jgjtFtqI4ItuRZAhERMcHUfRP/aMfwZqrbQswf92oiIqJr6l4F9MqmC4mIiO6q2wV08kjzbX9sfMqJiIhu2ZKrgA4BBp4B8HrgJ8AdTRQVERHNqxsA+wIH234YQNLpwBW239hUYRER0ay6t4KYDnQ+t/fxMi0iIiapukcAXwB+IumyMn401eMcIyJikqp7FdAZkr4NvLRMOtH2Dc2VFRERTavbBQSwM/CQ7XOANZKe3lBNERHRBbUCQNJpVA9yOaVM2h74UlNFRURE8+oeAfwZcBTwKIDte4FdmyoqIiKaVzcAHrdtyi2hJe3SXEkREdENdQPgEkn/DOwh6c3A98nDYSIiJrVRrwKSJOBi4NnAQ8ABwP+wfeUo6+0IXA08qeznUtunlZPHF1E9EP464ITOB85HRER3jBoAti3pW7afC4z4pj/IY8CrbD8iaXvgx+VS0pOBs21fJOlc4CTgM2MpPiIixq5uF9D1kg7Zkg278kgZ3b78GHgVcGmZvoTqS2UREdFldQPgRcA1ku6SdJOkn0u6abSVJG0r6UZgPdXRw13Ag7Y3l0XWADPGUnhERGydEbuAJM20/QvgNWPZuO0ngIMk7QFcRnUeoRZJC4GFADNnzhzL7iMiYgSjHQF8A8D2auBjtld3/tTdie0HgauAl1BdSTQQPPsCa4dZZ7Htftv9fX19dXcVERE1jRYA6hh+xpZsWFJf+eSPpJ2Aw4AVVEFwTFlsAXD5lmw3IiLGx2hXAXmY4Tr2AZZI2pYqaC6x/S+SbgUukvSPwA3A+Vu43YiIGAejBcDzJD1EdSSwUxmmjNv2bsOtaPsm4PlDTF8JvHCM9UZExDgZMQBsb9utQiIioru25HbQERExhSQAIiJaKgEQEdFSCYCIiJZKAEREtFQCICKipRIAEREtlQCIiGipBEBEREslACIiWioBEBHRUqM+EzjGZtaiK7Zq/VVnvm6cKomIGFqOACIiWioBEBHRUgmAiIiWSgBERLRUAiAioqUSABERLZUAiIhoqcYCQNJ+kq6SdKukWyS9q0zfS9KVku4ov/dsqoaIiBhek0cAm4H32J4DvBh4u6Q5wCJgme3ZwLIyHhERXdZYANheZ/v6MvwwsAKYAcwDlpTFlgBHN1VDREQMryvnACTNAp4PXAtMt72uzLoPmD7MOgslLZe0fMOGDd0oMyKiVRoPAElPBr4GvNv2Q53zbBvwUOvZXmy733Z/X19f02VGRLROowEgaXuqN/8v2/56mXy/pH3K/H2A9U3WEBERQ2vyKiAB5wMrbH+sY9ZSYEEZXgBc3lQNERExvCZvB/0nwAnAzyXdWKadCpwJXCLpJGA1ML/BGiIiYhiNBYDtHwMaZvbcpvYbERH15IEwE9TWPFAmD5OJiDpyK4iIiJZKAEREtFQCICKipRIAEREtlQCIiGipBEBEREslACIiWioBEBHRUgmAiIiWSgBERLRUAiAioqWm/L2AtuaeOhFTVe41FZAjgIiI1koARES0VAIgIqKlEgARES015U8CR0xVucAhtlaOACIiWioBEBHRUo11AUn6HHAksN72gWXaXsDFwCxgFTDf9qamaoiIqSXfXxhfTR4BXAC8dtC0RcAy27OBZWU8IiJ6oLEAsH018MCgyfOAJWV4CXB0U/uPiIiRdfsqoOm215Xh+4Dpwy0oaSGwEGDmzJldKC16LYf3MRVN5L/rnp0Etm3AI8xfbLvfdn9fX18XK4uIaIduB8D9kvYBKL/Xd3n/ERFRdLsLaCmwADiz/L68y/uPUUzkw9WpZrJ+kWuy/o1M1te7SY0dAUi6EPh34ABJaySdRPXGf5ikO4BDy3hERPRAY0cAto8bZtbcpvYZERH15ZvAEREtlQCIiGipBEBEREvldtBTUK526J681lsmr9fEkiOAiIiWSgBERLRUAiAioqUSABERLZWTwDFuenmCb7LeniCil3IEEBHRUgmAiIiWSgBERLRUAiAioqUSABERLZWrgKL1cnuCaKscAUREtFQCICKipRIAEREtlQCIiGipBEBEREv1JAAkvVbS7ZLulLSoFzVERLRd1wNA0rbAp4HDgTnAcZLmdLuOiIi268URwAuBO22vtP04cBEwrwd1RES0Wi++CDYDuKdjfA3wosELSVoILCyjj0i6fYRtTgN+OW4VTj5pf9qf9k9BOqvWYiO1f/+RVpyw3wS2vRhYXGdZSctt9zdc0oSV9qf9aX/aP5Z1e9EFtBbYr2N83zItIiK6qBcB8FNgtqSnS9oBOBZY2oM6IiJaretdQLY3S/pvwHeBbYHP2b5lKzdbq6toCkv72y3tb7cxt1+2x7OQiIiYJPJN4IiIlkoARES01KQJgNFuHyHpSZIuLvOvlTSr+1U2p0b7T5Z0q6SbJC2TNOL1v5NR3VuISPoLSZY0pS4NrNN+SfPL38Etkr7S7RqbVOP/wExJV0m6ofw/OKIXdTZB0uckrZd08zDzJekT5bW5SdLBtTZse8L/UJ0svgt4BrAD8DNgzqBl3gacW4aPBS7udd1dbv8rgZ3L8N9OpfbXfQ3KcrsCVwPXAP29rrvLfwOzgRuAPcv4U3pdd5fbvxj42zI8B1jV67rHsf0vAw4Gbh5m/hHAtwEBLwaurbPdyXIEUOf2EfOAJWX4UmCuJHWxxiaN2n7bV9n+dRm9hur7FVNJ3VuI/ANwFvDbbhbXBXXa/2bg07Y3Adhe3+Uam1Sn/QZ2K8O7A/d2sb5G2b4aeGCEReYBX3DlGmAPSfuMtt3JEgBD3T5ixnDL2N4M/ArYuyvVNa9O+zudRPVpYCoZ9TUoh7372Z6KD/mt8zfwLOBZkv5V0jWSXtu16ppXp/2nA2+UtAb4FvCO7pQ2IWzpewQwgW8FEWMj6Y1AP/DyXtfSTZK2AT4GvKnHpfTSdlTdQK+gOgK8WtJzbT/Y06q65zjgAtsflfQS4IuSDrT9+14XNlFNliOAOreP+MMykrajOgTc2JXqmlfr9hmSDgXeDxxl+7Eu1dYto70GuwIHAj+UtIqqH3TpFDoRXOdvYA2w1PbvbN8N/F+qQJgK6rT/JOASANv/DuxIdaO0NhjTLXYmSwDUuX3EUmBBGT4G+IHL2ZEpYNT2S3o+8M9Ub/5Tqe93wIivge1f2Z5me5btWVTnQY6yvbw35Y67Ov8HvkH16R9J06i6hFZ2s8gG1Wn/L4C5AJKeQxUAG7paZe8sBf6qXA30YuBXtteNttKk6ALyMLePkPQhYLntpcD5VId8d1KdLDm2dxWPr5rt/wjwZOCr5dz3L2wf1bOix1nN12DKqtn+7wKvlnQr8ATwXttT4ii4ZvvfA3xW0n+nOiH8pqnyIVDShVThPq2c4zgN2B7A9rlU5zyOAO4Efg2cWGu7U+T1iYiILTRZuoAiImKcJQAiIloqARAR0VIJgIiIlkoARES0VAIgIqKlEgARES31/wAK4TG21d5zvgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "That's it for linear regression! Now onto classification."
      ],
      "metadata": {
        "id": "r7RrxOQTn6e9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Classification**\n",
        "\n",
        "Now that we've covered linear regression it is time to talk about classification. Where regression was used to predict a numeric value, classification is used to seperate data points into classes of different labels. In this example we will use a TensorFlow estimator to classify flowers.\n",
        "\n",
        "Since we've touched on how estimators work earlier, I'll go a bit quicker through this example.\n",
        "\n",
        "This section is based on the following guide from the TensorFlow website. https://www.tensorflow.org/tutorials/estimator/premade"
      ],
      "metadata": {
        "id": "FVLqSaEHn7t6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Imports and Setup**"
      ],
      "metadata": {
        "id": "IUoACUUNn_6b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "import pandas as pd"
      ],
      "metadata": {
        "id": "rCVnU_uDoDpr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Dataset**"
      ],
      "metadata": {
        "id": "mqh14xRJoIyV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "CSV_COLUMN_NAMES = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth', 'Species']\n",
        "SPECIES = ['Setosa', 'Versicolor', 'Virginica']\n",
        "# Lets define some constants to help us later on"
      ],
      "metadata": {
        "id": "alNfJRCsoGPP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_path = tf.keras.utils.get_file(\n",
        "    \"iris_training.csv\", \"https://storage.googleapis.com/download.tensorflow.org/data/iris_training.csv\")\n",
        "test_path = tf.keras.utils.get_file(\n",
        "    \"iris_test.csv\", \"https://storage.googleapis.com/download.tensorflow.org/data/iris_test.csv\")\n",
        "\n",
        "train = pd.read_csv(train_path, names=CSV_COLUMN_NAMES, header=0)\n",
        "test = pd.read_csv(test_path, names=CSV_COLUMN_NAMES, header=0)\n",
        "# Here we use keras (a module inside of TensorFlow) to grab our datasets and read them into a pandas dataframe"
      ],
      "metadata": {
        "id": "FGs3GPbyoLJO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "hOoeAD9CoY0m",
        "outputId": "a6f54768-cf00-4d11-a15e-e6b859aef4fb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-1e3c73cf-ebbf-4b80-986a-8aa0b0e95a0d\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SepalLength</th>\n",
              "      <th>SepalWidth</th>\n",
              "      <th>PetalLength</th>\n",
              "      <th>PetalWidth</th>\n",
              "      <th>Species</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6.4</td>\n",
              "      <td>2.8</td>\n",
              "      <td>5.6</td>\n",
              "      <td>2.2</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>5.0</td>\n",
              "      <td>2.3</td>\n",
              "      <td>3.3</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4.9</td>\n",
              "      <td>2.5</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1.7</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4.9</td>\n",
              "      <td>3.1</td>\n",
              "      <td>1.5</td>\n",
              "      <td>0.1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5.7</td>\n",
              "      <td>3.8</td>\n",
              "      <td>1.7</td>\n",
              "      <td>0.3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1e3c73cf-ebbf-4b80-986a-8aa0b0e95a0d')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-1e3c73cf-ebbf-4b80-986a-8aa0b0e95a0d button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-1e3c73cf-ebbf-4b80-986a-8aa0b0e95a0d');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   SepalLength  SepalWidth  PetalLength  PetalWidth  Species\n",
              "0          6.4         2.8          5.6         2.2        2\n",
              "1          5.0         2.3          3.3         1.0        1\n",
              "2          4.9         2.5          4.5         1.7        2\n",
              "3          4.9         3.1          1.5         0.1        0\n",
              "4          5.7         3.8          1.7         0.3        0"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can pop the species column off and use that as our label."
      ],
      "metadata": {
        "id": "xxPXXgvHobT3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_y = train.pop('Species')\n",
        "test_y = test.pop('Species')\n",
        "train.head() # the species column is now gone"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "0fqBwcxzoaW6",
        "outputId": "917d1a7a-ad88-4390-810f-8715f3582679"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-228104b1-f866-4033-ab1a-74ff977877c1\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SepalLength</th>\n",
              "      <th>SepalWidth</th>\n",
              "      <th>PetalLength</th>\n",
              "      <th>PetalWidth</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6.4</td>\n",
              "      <td>2.8</td>\n",
              "      <td>5.6</td>\n",
              "      <td>2.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>5.0</td>\n",
              "      <td>2.3</td>\n",
              "      <td>3.3</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4.9</td>\n",
              "      <td>2.5</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4.9</td>\n",
              "      <td>3.1</td>\n",
              "      <td>1.5</td>\n",
              "      <td>0.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5.7</td>\n",
              "      <td>3.8</td>\n",
              "      <td>1.7</td>\n",
              "      <td>0.3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-228104b1-f866-4033-ab1a-74ff977877c1')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-228104b1-f866-4033-ab1a-74ff977877c1 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-228104b1-f866-4033-ab1a-74ff977877c1');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   SepalLength  SepalWidth  PetalLength  PetalWidth\n",
              "0          6.4         2.8          5.6         2.2\n",
              "1          5.0         2.3          3.3         1.0\n",
              "2          4.9         2.5          4.5         1.7\n",
              "3          4.9         3.1          1.5         0.1\n",
              "4          5.7         3.8          1.7         0.3"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train.shape  # we have 120 entires with 4 features"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yv1DjxEiohzl",
        "outputId": "e640cb55-2b89-4f71-e870-24b406796bff"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(120, 4)"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Input Function**\n",
        "\n",
        "Remember that nasty input function we created earlier. Well we need to make another one here! Fortunatly for us this one is a little easier to digest."
      ],
      "metadata": {
        "id": "BAm-8gx1okNt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def input_fn(features, labels, training=True, batch_size=256):\n",
        "    # Convert the inputs to a Dataset.\n",
        "    dataset = tf.data.Dataset.from_tensor_slices((dict(features), labels))\n",
        "\n",
        "    # Shuffle and repeat if you are in training mode.\n",
        "    if training:\n",
        "        dataset = dataset.shuffle(1000).repeat()\n",
        "    \n",
        "    return dataset.batch(batch_size)"
      ],
      "metadata": {
        "id": "Tll7i6B5omfD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Feature Columns**"
      ],
      "metadata": {
        "id": "427d6_otosch"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Feature columns describe how to use the input.\n",
        "my_feature_columns = []\n",
        "for key in train.keys():\n",
        "    my_feature_columns.append(tf.feature_column.numeric_column(key=key))\n",
        "print(my_feature_columns) "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_pdvv4udou6w",
        "outputId": "724dd406-f934-43fc-f7f8-a8812422e615"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[NumericColumn(key='SepalLength', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='SepalWidth', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='PetalLength', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='PetalWidth', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Building the Model**\n",
        "\n",
        "And now we are ready to choose a model. For classification tasks there are variety of different estimators/models that we can pick from. Some options are listed below.\n",
        "\n",
        "- DNNClassifier (Deep Neural Network)\n",
        "- LinearClassifier\n",
        "\n",
        "We can choose either model but the DNN seems to be the best choice. This is because we may not be able to find a linear coorespondence in our data.\n",
        "\n",
        "So let's build a model!"
      ],
      "metadata": {
        "id": "NXHT17uro0NC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Build a DNN with 2 hidden layers with 30 and 10 hidden nodes each.\n",
        "classifier = tf.estimator.DNNClassifier(\n",
        "    feature_columns=my_feature_columns,\n",
        "    # Two hidden layers of 30 and 10 nodes respectively.\n",
        "    hidden_units=[30, 10],\n",
        "    # The model must choose between 3 classes.\n",
        "    n_classes=3)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X9r1cgx6o8ql",
        "outputId": "68a71d01-d82d-4a5d-cd11-110442980813"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INFO:tensorflow:Using default config.\n",
            "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpalxs5wlx\n",
            "INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpalxs5wlx', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
            "graph_options {\n",
            "  rewrite_options {\n",
            "    meta_optimizer_iterations: ONE\n",
            "  }\n",
            "}\n",
            ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "What we've just done is created a deep neural network that has two hidden layers. These layers have 30 and 10 neurons respectively. This is the number of neurons the TensorFlow official tutorial uses so we'll stick with it. However, it is worth mentioning that the number of hidden neurons is an arbitrary number and many experiments and tests are usually done to determine the best choice for these values. Try playing around with the number of hidden neurons and see if your results change."
      ],
      "metadata": {
        "id": "8vJC9O05pH8v"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training**\n",
        "\n",
        "Now it's time to train the model!"
      ],
      "metadata": {
        "id": "mB8PmWrxpJhK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "classifier.train(\n",
        "    input_fn=lambda: input_fn(train, train_y, training=True),\n",
        "    steps=5000)  #This simply tells the classifier to run for 5000 steps. \n",
        "# We include a lambda to avoid creating an inner function previously"
      ],
      "metadata": {
        "id": "5ctyJQULpIkX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Evaluation**\n",
        "\n",
        "Now let's see how this trained model does!"
      ],
      "metadata": {
        "id": "ZnQf-Vnb-Stq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "eval_result = classifier.evaluate(\n",
        "    input_fn=lambda: input_fn(test, test_y, training=False))\n",
        "\n",
        "print('\\nTest set accuracy: {accuracy:0.3f}\\n'.format(**eval_result))"
      ],
      "metadata": {
        "id": "z1tjGRe6-Ub1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Notice this time we didn't specify the number of steps. This is because during evaluation the model will only look at the testing data one time."
      ],
      "metadata": {
        "id": "ylY79Edj-gra"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Predictions**\n",
        "\n",
        "Now that we have a trained model it's time to use it to make predictions. I've written a little script below that allows you to type the features of a flower and see a prediction for its class."
      ],
      "metadata": {
        "id": "ZMh4rnQ6-jJB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def input_fn(features, batch_size=256):\n",
        "  #Convert the inputs to a Dataset without labels\n",
        "  return tf.data.Dataset.from_tensor_slices(dict(features)).batch(batch_size)"
      ],
      "metadata": {
        "id": "O61YByPj-kzY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "features = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth']\n",
        "predict = {}\n",
        "\n",
        "print(\"Please type numeric values as prompted.\")\n",
        "for feature in features:\n",
        "  valid = True\n",
        "  while valid: \n",
        "    val = input(feature + \": \")\n",
        "    if not val.isdigit(): valid = False\n",
        "\n",
        "  predict[feature] = [float(val)]\n",
        "\n",
        "predictions = classifier.predict(input_fn=lambda: input_fn(predict))\n",
        "\n",
        "for pred_dict in predictions:\n",
        "  class_id = pred_dict[\"class_ids\"][0]\n",
        "  probability = pred_dict[\"probabilities\"][class_id]\n",
        "\n",
        "  \n",
        "  print('Prediction is \"{}\" ({:.1f}%)'.format(SPECIES[class_id], 100 * probability))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1PgwSs30_SNd",
        "outputId": "8c5f3a5e-1fa0-43bb-badc-56b69999ffdd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Please type numeric values as prompted.\n",
            "SepalLength: 23\n",
            "SepalLength: 12\n",
            "SepalLength: 12\n",
            "SepalLength: 4\n",
            "SepalLength: 3\n",
            "SepalLength: 2\n",
            "SepalLength: 0.5\n",
            "SepalWidth: 2\n",
            "SepalWidth: 0.4\n",
            "PetalLength: 0.5\n",
            "PetalWidth: 0.3\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "INFO:tensorflow:Restoring parameters from /tmp/tmpalxs5wlx/model.ckpt-5000\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "Prediction is \"Setosa\" (37.2%)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And that's pretty much it for classification!"
      ],
      "metadata": {
        "id": "kl5gb2sZAPH-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Clustering**\n",
        "\n",
        "Now that we've covered regression and classification it's time to talk about clustering data!\n",
        "\n",
        "Clustering is a Machine Learning technique that involves the grouping of data points. In theory, data points that are in the same group should have similar properties and/or features, while data points in different groups should have highly dissimilar properties and/or features. (https://towardsdatascience.com/the-5-clustering-algorithms-data-scientists-need-to-know-a36d136ef68)\n",
        "\n",
        "Unfortunalty there are issues with the current version of TensorFlow and the implementation for KMeans. This means we cannot use KMeans without writing the algorithm from scratch. We aren't quite at that level yet, so we'll just explain the basics of clustering for now.\n",
        "\n",
        "Basic Algorithm for K-Means.\n",
        "\n",
        "- Step 1: Randomly pick K points to place K centroids\n",
        "- Step 2: Assign all the data points to the centroids by distance. The closest centroid to a point is the one it is assigned to.\n",
        "- Step 3: Average all the points belonging to each centroid to find the middle of those clusters (center of mass). Place the corresponding centroids into that position.\n",
        "- Step 4: Reassign every point once again to the closest centroid.\n",
        "- Step 5: Repeat steps 3-4 until no point changes which centroid it belongs to.\n",
        "\n",
        "Please refer to the video for an explanation of KMeans clustering."
      ],
      "metadata": {
        "id": "ru29ksgRARaE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Data**\n",
        "\n",
        "Let's start by discussing the type of data we use when we work with a hidden markov model.\n",
        "\n",
        "In the previous sections we worked with large datasets of 100's of different entries. For a markov model we are only interested in probability distributions that have to do with states.\n",
        "\n",
        "We can find these probabilities from large datasets or may already have these values. We'll run through an example in a second that should clear some things up, but let's discuss the components of a markov model.\n",
        "\n",
        "States: In each markov model we have a finite set of states. These states could be something like \"warm\" and \"cold\" or \"high\" and \"low\" or even \"red\", \"green\" and \"blue\". These states are \"hidden\" within the model, which means we do not direcly observe them.\n",
        "\n",
        "Observations: Each state has a particular outcome or observation associated with it based on a probability distribution. An example of this is the following: On a hot day Tim has a 80% chance of being happy and a 20% chance of being sad.\n",
        "\n",
        "Transitions: Each state will have a probability defining the likelyhood of transitioning to a different state. An example is the following: a cold day has a 30% chance of being followed by a hot day and a 70% chance of being follwed by another cold day.\n",
        "\n",
        "To create a hidden markov model we need.\n",
        "\n",
        "- States\n",
        "- Observation Distribution\n",
        "- Transition Distribution\n",
        "\n",
        "For our purpose we will assume we already have this information available as we attempt to predict the weather on a given day."
      ],
      "metadata": {
        "id": "HOicUpFQAtVO"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Imports and Setup**"
      ],
      "metadata": {
        "id": "cprEG7B7A1dc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Due to a version mismatch with tensorflow v2 and tensorflow_probability we need to install the most recent version of tensorflow_probability"
      ],
      "metadata": {
        "id": "bV1KfNvEA7am"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install --upgrade tensorflow-probability"
      ],
      "metadata": {
        "id": "retUEIpaA3uX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow_probability as tfp\n",
        "import tensorflow as tf"
      ],
      "metadata": {
        "id": "QXXEu7r-Byi2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Weather Model**\n",
        "Taken direclty from the TensorFlow documentation (https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/HiddenMarkovModel).\n",
        "\n",
        "We will model a simple weather system and try to predict the temperature on each day given the following information.\n",
        "\n",
        "- Cold days are encoded by a 0 and hot days are encoded by a 1.\n",
        "- The first day in our sequence has an 80% chance of being cold.\n",
        "- A cold day has a 30% chance of being followed by a hot day.\n",
        "- A hot day has a 20% chance of being followed by a cold day.\n",
        "- On each day the temperature is normally distributed with mean and standard deviation 0 and 5 on a cold day and mean and standard deviation 15 and 10 on a hot day.\n",
        "\n",
        "If you're unfamiliar with standard deviation it can be put simply as the range of expected values.\n",
        "\n",
        "In this example, on a hot day the average temperature is 15 and ranges from 5 to 25.\n",
        "\n",
        "To model this in TensorFlow we will do the following."
      ],
      "metadata": {
        "id": "-hBjbFSDFYDc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tfd = tfp.distributions  # making a shortcut for later on\n",
        "initial_distribution = tfd.Categorical(probs=[0.2, 0.8])  # Refer to point 2 above\n",
        "transition_distribution = tfd.Categorical(probs=[[0.5, 0.5],\n",
        "                                                 [0.2, 0.8]])  # refer to points 3 and 4 above\n",
        "observation_distribution = tfd.Normal(loc=[0., 15.], scale=[5., 10.])  # refer to point 5 above\n",
        "\n",
        "# the loc argument represents the mean and the scale is the standard devitation"
      ],
      "metadata": {
        "id": "at1wZetYFXgZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = tfd.HiddenMarkovModel(\n",
        "    initial_distribution=initial_distribution,\n",
        "    transition_distribution=transition_distribution,\n",
        "    observation_distribution=observation_distribution,\n",
        "    num_steps=7)"
      ],
      "metadata": {
        "id": "iyNcQFlTFk1N"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The number of steps represents the number of days that we would like to predict information for. In this case we've chosen 7, an entire week.\n",
        "\n",
        "To get the expected temperatures on each day we can do the following."
      ],
      "metadata": {
        "id": "fQ4OgwXLFm7y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mean = model.mean()\n",
        "\n",
        "# due to the way TensorFlow works on a lower level we need to evaluate part of the graph\n",
        "# from within a session to see the value of this tensor\n",
        "\n",
        "# in the new version of tensorflow we need to use tf.compat.v1.Session() rather than just tf.Session()\n",
        "with tf.compat.v1.Session() as sess:  \n",
        "  print(mean.numpy())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xUQYgK-TFuZu",
        "outputId": "06c749df-8a1b-44e1-87d5-f66394a12eef"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[11.999999 11.1      10.830001 10.748999 10.724699 10.71741  10.715222]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Neurtal Networks**"
      ],
      "metadata": {
        "id": "ra-AQ1uOHm25"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Keras**\n",
        "\n",
        "Before we dive in and start discussing neural networks, I'd like to give a breif introduction to keras.\n",
        "\n",
        "From the keras official documentation (https://keras.io/) keras is described as follows.\n",
        "\n",
        "\"Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation.\n",
        "\n",
        "Use Keras if you need a deep learning library that:\n",
        "\n",
        "- Allows for easy and fast prototyping (through user friendliness, modularity, and extensibility).\n",
        "- Supports both convolutional networks and recurrent networks, as well as combinations of the two.\n",
        "- Runs seamlessly on CPU and GPU.\"\n",
        "\n",
        "Keras is a very powerful module that allows us to avoid having to build neural networks from scratch. It also hides a lot of mathematical complexity (that otherwise we would have to implement) inside of helpful packages, modules and methods.\n",
        "\n",
        "In this guide we will use keras to quickly develop neural networks."
      ],
      "metadata": {
        "id": "NQlNet-JHso9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Dataset**\n",
        "\n",
        "For this tutorial we will use the MNIST Fashion Dataset. This is a dataset that is included in keras.\n",
        "\n",
        "This dataset includes 60,000 images for training and 10,000 images for validation/testing."
      ],
      "metadata": {
        "id": "TolZZRdJJV7I"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ],
      "metadata": {
        "id": "7y3SOANYJVPP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#load dataset\n",
        "fashion_mnist = keras.datasets.fashion_mnist \n",
        "\n",
        "#Split into testing and training\n",
        "(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()"
      ],
      "metadata": {
        "id": "Bl1ww-wXJp36"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's have a look at this data to see what we are working with.\n",
        "\n",
        "Our labels are integers ranging from 0 - 9. Each integer represents a specific article of clothing. We'll create an array of label names to indicate which is which."
      ],
      "metadata": {
        "id": "jCTlQCZSKMwo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
        "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
      ],
      "metadata": {
        "id": "6u8OzK85Kgfq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Fianlly let's look at what some of these images look like!"
      ],
      "metadata": {
        "id": "H3A8L1dyKjzg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure()\n",
        "plt.imshow(train_images[1])\n",
        "plt.colorbar()\n",
        "plt.grid(False)\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "DRQbNsKrKkj6",
        "outputId": "77ee3298-fe31-42cb-8bfe-3acc1de536e9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAD4CAYAAACE9dGgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAb6UlEQVR4nO3df4xd9Xnn8fcz4xnbYxuwMTbGOECoUWqyiaFeSks2a8o2AZTKoFYEtKJul8ZsBNqwQtUS/ljYjajYKkBbKaFrFi9GAlIkYHEqGuK1ovyqYmK7CNu4KV4wwo7twUCw8Y/xzL3P/nHPhDu+c55zZu6vc8afFzqaO+e555wvd2Yen/M9z/l+zd0RESmrnm43QESkGUpiIlJqSmIiUmpKYiJSakpiIlJq0zp5sH6b7jOY1clDTg2zZobhaUtOpsaO/2pGvO2x+O60VTPuXmeERwbS/520M0fibU/Gv54zfjkUxn0k3v9UdIKjnPQha2YfX7x6lr/3fiXXe7e+NvSyu1/bzPGa1VQSM7Nrgb8GeoH/5e4PRu+fwSx+265p5pDtYxk/926Wonz6X4XhuY/sS43t+O6nwm0XbEtPgAC9Q/Evs52shvFDnx1I3/eX3gu3fW/P3DD+qW+8FcYrBwfD+FS02Tc1vY/33q/wysufyPXe3kVvzG/6gE2a9OWkmfUC3wKuA5YBt5jZslY1TES6w4Fqzv+ymNkSM/uBmb1uZjvN7GvJ+vvNbJ+ZvZos19dt83Uz221mvzCzL2Ydo5kzsSuA3e7+ZnLg7wCrgNeb2KeIdJnjDHu+y8kcRoC73X2bmc0BtprZxiT2iLt/s/7NyYnQzcClwHnA/zWzS9zTG9RMx/5i4J267/cm68YwszVmtsXMtgwT92GISDG06kzM3fe7+7bk9RFgF+PkiTqrgO+4+5C7vwXspnbClKrtdyfdfa27r3D3FX1Mb/fhRKRJjlPxfAswf/QkJVnWpO3XzC4ELgM2J6vuNLPXzGydmY12gOY6OarXTBLbByyp+/78ZJ2IlFwVz7UAh0ZPUpJl7Xj7M7PZwHPAXe5+GHgUuBhYDuwHHppsW5tJYj8HlprZRWbWT+06dkMT+xORAnCgguda8jCzPmoJ7Cl3fx7A3Q+6e8Xdq8BjfHzJOOGTo0l37Lv7iJndCbxMrcRinbvvnOz+mtZsiUQTJRSVlZeH8f/35fhj/m9XPx/GT3hcKnBh37upsQW3/0O47fLp3bvEf/zDc8P48Cd7w/hXbnwnjP90KP3f6K/+078Pt138cF8Yt5++GsbLrpozQWUxMwMeB3a5+8N16xe5+/7k2xuBHcnrDcDTZvYwtY79pcAr0TGaqhNz95eAl5rZh4gUiwPDrauLvAq4FdhuZqOZ/15qJVnLk8PtAW4HcPedZvYstSqHEeCO6M4kdLhiX0SKzydwqZi5L/efAONdJqWe/Lj7A8ADeY+hJCYiYzlUSjRWqpKYiIxRq9gvDyUxETmFURn3CrCYlMREZIxax76SmIiUVK1OTEms85q8Jdw7/+wwfvyZ2amxr17wXLhtv8UP0+45GY9mMnjyjDC+42j6UxkjHtdazeyJh+JZOvNgGN97cl4YHw6OX23yX/t7TiwI4/P7PkqN/fmlG1NjAGc9cSyM37fzD8L4uTfsCuNF1+zPppOmThITkZbQmZiIlJpjVEo0cr2SmIg00OWkiJSWY5zM6EstEiUxERmjVuyqy0kRKTF17JfQGS/GJRo3n/3T1NjmIxeH20ZlBgAze4fD+PFKPCxMj6W3vd/iacuibQFeO7okjE/LKB+J9DWxbR6DJ+ekxg4Np5fMQHaf0DcufTGMf+uKPwzjvLI9jneRu1FxnYmJSIlVdSYmImVV69gvT2ooT0tFpCPUsS8ipVdRnZiIlJUq9kWk9Kq6OykiZVV7AFxJrHBGfu+3wvj1Z8d1P9uOXpgaG8gYzmY6ca3Wgv7DYfz3Z8XDupzXm17r1WfxL+ORaty2gZ64xm3I44GMo6PP6ekPtz1Wjevn3hyJf33/4chn0vddiY+dVWFwwuPavX/5sxlh/JJwErLuciyztrFITpskJiL5uKNiVxEpM1Oxq4iUl6MzMREpOXXsi0hpOaZBEUWkvGpTtpUnNZSnpSLSIZo8t5D2/l5cF3T2tPTpvQDmTkufwiurpmZGT1zvdGg4fdwrgJu/fXcYn/XL9FqtOW8Phdt+tGR6GJ+9L97ee+Jf9p6T6W2rTI8/t+Ez4vjgZfGv73+/5anU2NajF4XbZtX+ZZ2pPHL1M2H8UX4jjHeTcxpV7JvZHuAIUAFG3H1FKxolIt11up2JXe3uh1qwHxEpAHc7fc7ERGTqqXXsnz6PHTnwfTNz4H+6+9pT32Bma4A1ADMYaPJwItJ+5Rpjv9mWfs7dLweuA+4ws8+f+gZ3X+vuK9x9RR9xJ7KIdF+tY99yLVnMbImZ/cDMXjeznWb2tWT9PDPbaGZvJF/nJuvNzP7GzHab2WtmdnnWMZpKYu6+L/k6CLwAXNHM/kSkGCr05FpyGAHudvdlwJXUTnaWAfcAm9x9KbAp+R5qJ0RLk2UN8GjWASadxMxslpnNGX0NfAHYMdn9iUgxjFbst+JMzN33u/u25PURYBewGFgFrE/eth64IXm9CnjSa34GnGVmi6JjNNMnthB4wcxG9/O0u3+vif211Zeu2xzGj1bjS92o1msoY1yr+dOOhPE3ji8M4+f95T+G8SNfvjI1dvCKmeG2ix6K973vnt8N4/O3xzVww/PTx93y3viPYOBAXKt1wX3xoFwnvpx+7Kw6sPl98c/sl8NnhfGvnrUzjP/tb61KjfnWeNtOmMBEIfPNbEvd92vH6xsHMLMLgcuAzcBCd9+fhA5QyydQS3Dv1G22N1m3nxSTTmLu/ibw2cluLyLF5A7D1dxJ7FCe+lAzmw08B9zl7oeTk5/keO7JzcFJUYmFiIxRu5xs3d1JM+ujlsCecvfnk9UHzWyRu+9PLhcHk/X7gPpp589P1qUqz31UEemYSvL8ZNaSxWqnXI8Du9z94brQBmB18no18GLd+j9O7lJeCXxYd9k5Lp2JicgYoyUWLXIVcCuw3cxeTdbdCzwIPGtmtwFvAzclsZeA64HdwDHgT7MOoCQmIqdo3eWku/+E9GlXrhnn/Q7cMZFjKImJSAONsV9AX1/w4zD+9xlDs0wPSizm9sXTlmX55Mx3w/gOzg7jP37426mxfZX0IYQA/u0l/zmMv/UH6fsG+Pz2G8P4xkv/LjU2kDFl233vXhrGf/bZeNq0Y0HZzPn974fbZk3JNlyN/3RePLo4jO//N2emxs7dGm7adrW7k6fPs5MiMsVoeGoRKT1dTopIabX47mTbKYmJSAMNiigipeVujCiJiUiZ6XJSREpLfWJd4lctD+Obh/45jGcNxdNnldTYDIuHozm378Mw/k/HLgjjWa7/wz9JjfUcj9v2iSXxL+v1//ULYXyOxXVofzT0xfRgxnRvv/p3l8TH5mdh/EcfpG+/ct4vwm2zxpjPir87Ek/Dd+J3gikC/yrctCOUxESktFQnJiKlpzoxESktdxjJPyhi1ymJiUgDXU6KSGmpT0xESs+VxESkzNSx3wUH/3wojJ/beziM7+GcMD5UTR9famFGHdjgyBlh/FglHldr5Jp4EuTj56S37fi8uIM2+N8C4Oi5F4fxYJg1AKadSJ/EptIf/6EMnRXHT/zH3wnjvzv7h6mxweH4Z3LJjHBYd3qJJ+c5s/doGF/9m+lTCP6QeJq9dnNXn5iIlJpR0d1JESkz9YmJSGnp2UkRKTev9YuVhZKYiDTQ3UkRKS1Xx76IlJ0uJ7tg5JW5Yfx/zL8ujH95wc/D+NL+wdTYkt543sn//eGnw/hQxhyGLz35t2F82NPHOhv2uG0nMuIzLP4XeaAnLjTrIX37IY+LzPosHrPrzeF4+3XvX5UaWzz9g3DbrDHi+mwkjP/wV58K4z99+TOpsQv4x3DbTijT3cnMc0YzW2dmg2a2o27dPDPbaGZvJF/jDCIipeFeS2J5liLIc+H7BHDtKevuATa5+1JgU/K9iEwRVbdcSxFkJjF3/xFw6pzvq4D1yev1wA0tbpeIdJF7vqUIJtsnttDdRx8uOwAsTHujma0B1gDMYGCShxORTnGMaonuTjbdUnd3SH8a1t3XuvsKd1/RRzwZh4gUg+dcimCySeygmS0CSL6m37oTkXKZgh3749kArE5erwZebE1zRKQQSnQqltknZmbPACuB+Wa2F7gPeBB41sxuA94GbmpnI/M4/y/i2poP/yLeft258dhUxz+zJDV2YM2JcNv7P/PdML7zo/PC+EPvxXVmbxxbkBqb1Xsy3HZ61oBgbdRj8V9BNNcnwHvDs8L4bwykXyCs331luO2CVfE8pdmCeSUpRi1YpChnWXlkJjF3vyUldE2L2yIiBeBAtdqaJGZm64AvAYPu/ulk3f3AV4B3k7fd6+4vJbGvA7cBFeA/ufvLWccozy0IEekMB9zyLdmeoLHOFOARd1+eLKMJbBlwM3Bpss23zTIe20BJTETG0ao6sZQ60zSrgO+4+5C7vwXsBq7I2khJTEQa5e/Yn29mW+qWNTmPcKeZvZY81jj62OJi4J269+xN1oWmzAPgItIqEyqfOOTuKyZ4gEeBb1BLg98AHgL+wwT38Ws6ExORRm0ssXD3g+5ecfcq8BgfXzLuA+rLAM5P1oV0JpYYOXAwjPcF8cXHLwu3nbEuLmPIGkXzzGnHwvii6elTxk3viYeMGfbMftNQr8VD+fQEv+lZx57fdySMHx6JpzY7Z1r69kOvzAu3Pa05eIvuTo7HzBbVPbZ4IzA6Qs4G4Gkzexg4D1gKvJK1PyUxERlHy0osxqszXWlmy6mdy+0Bbgdw951m9izwOjAC3OEeDJaXUBITkUYtqsZPqTN9PHj/A8ADEzmGkpiINCrII0V5KImJyFijxa4loSQmIg2KMuBhHkpiItKojXcnW01JTEQaZAwwUiinTxKz+F+WnunxqLPVE8FwOxnn3m+eTB8qB6C/yVquShM1y1l1XhUvbj10M8MIBaV1udi0+E/HKxmVAUW+XivQWGF5nD5JTERyyj1CRSEoiYlII52JiUipxb0MhaIkJiJjqU5MRMpOdydFpNxKlMSKe/9cRCSH0+dMLKMupzo0NOld9+14K4zvPrYwjM/sjeudPhiJpyaLZI1VFo33BbUpZ5oR1aFl1b9l/X/Pnjb5n1n/4SZPNXozxmEbiWv/ik6XkyJSXo4eOxKRktOZmIiUmS4nRaTclMREpNSUxESkrMx1OSkiZae7k+VjGXU/HtT9VA5/FG57OKPe6ay+42H8WKU/jA/0nkyNZdWBZdWRNTOvJECfpVeaVSyutf5gZCCML+qPBwXrCZ5itkqJTjW6oExnYpkV+2a2zswGzWxH3br7zWyfmb2aLNe3t5ki0lFtnAG81fI8dvQEcO046x9x9+XJ8lJrmyUiXeMf94tlLUWQmcTc/UfA+x1oi4gUxRQ7E0tzp5m9llxuzk17k5mtMbMtZrZlmMk/6yYinWPVfEsRTDaJPQpcDCwH9gMPpb3R3de6+wp3X9FHPBmHiMhETSqJuftBd6+4exV4DLiitc0Ska6a6peTZrao7tsbgR1p7xWRkilZx35mnZiZPQOsBOab2V7gPmClmS2nlov3ALe3sY0d4dUmfiLVeNStk9X4Y65mzO1YzRjvPKrFyjJc7QvjM5qY2xGgJ+g4yWp31v931nhk/cH+m+7Paeb3pQxK9L+XmcTc/ZZxVj/ehraISFFMpSQmIqcXozh3HvNQEhORsQrU35WHJgoRkUYtujuZ8tjiPDPbaGZvJF/nJuvNzP7GzHYnNaiX52mqkpiINGpdicUTND62eA+wyd2XApuS7wGuA5Ymyxpq9aiZlMREpEGrSixSHltcBaxPXq8Hbqhb/6TX/Aw465RyrnGpT6wDVs79RRh//dh5YXx6Tzz9VyUo0cgqY8gaaqebstp+pDIjjEflHRnVGdLePrGF7r4/eX0AGJ3TcDHwTt379ibr9hNQEhORsXxCdyfnm9mWuu/Xuvva3Idyd7PmbiMoiYlIo/xp5ZC7r5jg3g+a2SJ3359cLg4m6/cBS+red36yLqQ+MRFp0ObHjjYAq5PXq4EX69b/cXKX8krgw7rLzlQ6ExORRi3qE0t5bPFB4Fkzuw14G7gpeftLwPXAbuAY8Kd5jqEkJiJjtXCEipTHFgGuGee9Dtwx0WMoiYnIGEa5KvaVxESkgZJYGXn76qVOeDzcTZYzp8VTup0IhtPJnHLN49/Wpqd8C7Y/llGsNXtaPJz5B8PxlG7REEeVvibnVWzj70shKImJSKkpiYlIaZVsFAslMRFppCQmImVW4EdqGyiJiUgDXU6KSHkVaDq2PJTERKSRkpjUOzQ8J4xnjRd2rNofb2/p22dNa5ZV55U1ZduHlZlhvBLsf6A3rgPLmsruQPWMMB45eVaTdWJTmCr2RaT0rETzaiqJichY6hMTkbLT5aSIlJuSmIiUmc7ERKTclMREpLQmNttR1ymJdUBWrVazojHDqk0eO2vux6zxxiJZdWDRvJF5tj9anZ4aG4mnrMzkJSpBmKiy1YllznZkZkvM7Adm9rqZ7TSzryXr55nZRjN7I/k6t/3NFZGOcM+3FECeKdtGgLvdfRlwJXCHmS0D7gE2uftSYFPyvYhMAW2esq2lMpOYu+93923J6yPALmpTi68C1idvWw/c0K5GikgH+QSWAphQn5iZXQhcBmwGFtZNbHkAWJiyzRpgDcAM4jHRRaQYpmTHvpnNBp4D7nL3w2YfP0Dr7m42/smlu68F1gKcYfMKkrtFJFKmJJanTwwz66OWwJ5y9+eT1QfNbFESXwQMtqeJItJRTqk69jPPxKx2yvU4sMvdH64LbQBWU5uSfDXwYltaOAVklSlkjIaTqZJRatCMvmCYH8ieEi6S1e6sz63q8Qd3LCqxGCjGH2BRFaXTPo88l5NXAbcC283s1WTdvdSS17NmdhvwNnBTe5ooIh03lZKYu/+E9HOFa1rbHBHptrIVu6piX0TGctegiCJScuXJYUpiItJIl5MiUl4O6HJSREqtPDlMSezXuli4lzUtWjOyarGaGUoHYHoTbc+aLi5rKJ5pPXEd2QlP//Vu8+hIpafLSREptVbenTSzPcARoAKMuPsKM5sH/B1wIbAHuMndP5jM/ttX6i0i5dSeUSyudvfl7r4i+b5lQ3kpiYnIGLViV8+1NKFlQ3kpiYlIo2rOBeab2Za6Zc04e3Pg+2a2tS6eayivPNQnJiINJnCWdajuEjHN59x9n5ktADaa2T/XB6OhvPLQmZiIjNXiPjF335d8HQReAK6ghUN5KYmJyClqz07mWbKY2SwzmzP6GvgCsIOPh/KCJofy0uXkKMsY1KuJTszDGfODDfSfnPS+s2RNF5dVo3bC+8J41phfzUxXlzUlW2/GFchQNb3tTQ/B5iUa+nQyWlc3uRB4IRkJehrwtLt/z8x+TouG8lISE5GxWjh5rru/CXx2nPXv0aKhvJTERKRRQYaezkNJTEQalSeHKYmJSCOrlqfPT0lMRMZyRgtZS0FJTETGMJp+pKijlMREpJGSmExEX088t2NU7wTxmGBZdVxZ8d6MHt5KxphgWds3s+9mxkLTeGIZlMREpLTUJyYiZae7kyJSYq7LSREpMUdJTERKrjxXk0piItJIdWIiUm5TKYmZ2RLgSWrjAjmw1t3/2szuB74CvJu89V53f6ldDW27Nv7Qth5aEsaXnP9+GD9W6Q/j0ZhdWeN5ze4dmvS+88SjeS+HqvGv30Bvc8Vc0bG9t8mfd4n+yCfMHSrluZ7McyY2Atzt7tuSERq3mtnGJPaIu3+zfc0Tka4oUZLOTGLJjCT7k9dHzGwXsLjdDRORLipREpvQIL1mdiFwGbA5WXWnmb1mZuvMbG7KNmtGp3MaJr50EZECcKDq+ZYCyJ3EzGw28Bxwl7sfBh4FLgaWUztTe2i87dx9rbuvcPcVfUxvQZNFpL28NodAnqUAct2dNLM+agnsKXd/HsDdD9bFHwP+vi0tFJHOckrVsZ95Jma1aUoeB3a5+8N16xfVve1GatMwichU4J5vKYA8Z2JXAbcC283s1WTdvcAtZracWt7eA9zelhZOAUvm/CqO98UlFgM98ZRu/3rmm6mx/ozS676MaW3O7ImH6mnGMY+H2pmRMSXbdz/6zTC+uO+D1NjARYfDbTP1ZJR/VNv3uXVEQRJUHnnuTv4Exh3Yqbw1YSISKM5ZVh6q2BeRsRzQUDwiUmo6ExOR8pp6jx2JyOnEwQtSA5aHkpiINCpINX4eSmIi0kh9YiVkcc1SMz/UzTsuDuOvTL8o3sGH8ZRt3tfEqX9GuXPvRxlvyKj1Iqj1spF424wyMXqG4/jJM9N3cM6WjHZnKXsdWMRddydFpOR0JiYi5eV4pTxnmkpiIjLW6FA8JaEkJiKNSlRiMaFBEUVk6nPAq55rycPMrjWzX5jZbjO7p9XtVRITkbG8dYMimlkv8C3gOmAZtdFvlrWyubqcFJEGLezYvwLY7e5vApjZd4BVwOutOoB5B2+lmtm7wNt1q+YDhzrWgIkpatuK2i5Q2yarlW27wN3PaWYHZvY9am3KYwZwou77te6+tm5ffwRc6+5/lnx/K/Db7n5nM22s19EzsVM/XDPb4u4rOtmGvIratqK2C9S2ySpa29z92m63YSLUJyYi7bQPqJ89+vxkXcsoiYlIO/0cWGpmF5lZP3AzsKGVB+h2x/7a7Ld0TVHbVtR2gdo2WUVuW1PcfcTM7gReBnqBde6+s5XH6GjHvohIq+lyUkRKTUlMREqtK0ms3Y8hNMPM9pjZdjN71cy2dLkt68xs0Mx21K2bZ2YbzeyN5OvcArXtfjPbl3x2r5rZ9V1q2xIz+4GZvW5mO83sa8n6rn52QbsK8bmVVcf7xJLHEP4F+H1gL7W7F7e4e8sqeJthZnuAFe7e9cJIM/s88BHwpLt/Oln3l8D77v5g8g/AXHf/LwVp2/3AR+7+zU6355S2LQIWufs2M5sDbAVuAP6ELn52QbtuogCfW1l140zs148huPtJYPQxBDmFu/8IOHV68FXA+uT1emp/BB2X0rZCcPf97r4teX0E2AUspsufXdAuaUI3kthi4J267/dSrB+kA983s61mtqbbjRnHQnffn7w+ACzsZmPGcaeZvZZcbnblUreemV0IXAZspkCf3SntgoJ9bmWijv1Gn3P3y6k9dX9HctlUSF7rCyhSjcyjwMXAcmA/8FA3G2Nms4HngLvc/XB9rJuf3TjtKtTnVjbdSGJtfwyhGe6+L/k6CLxA7fK3SA4mfSujfSyDXW7Pr7n7QXeveG3Swsfo4mdnZn3UEsVT7v58srrrn9147SrS51ZG3UhibX8MYbLMbFbS4YqZzQK+AOyIt+q4DcDq5PVq4MUutmWM0QSRuJEufXZmZsDjwC53f7gu1NXPLq1dRfncyqorFfvJLeS/4uPHEB7oeCPGYWafpHb2BbVHsp7uZtvM7BlgJbVhUQ4C9wH/B3gW+AS1YY1ucveOd7CntG0ltUsiB/YAt9f1QXWybZ8DfgxsB0ZH7ruXWv9T1z67oF23UIDPraz02JGIlJo69kWk1JTERKTUlMREpNSUxESk1JTERKTUlMREpNSUxESk1P4/ni7gsVOyO9kAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Data Preprocessing**\n",
        "\n",
        "The last step before creating our model is to preprocess our data. This simply means applying some prior transformations to our data before feeding it the model. In this case we will simply scale all our greyscale pixel values (0-255) to be between 0 and 1. We can do this by dividing each value in the training and testing sets by 255.0. We do this because smaller values will make it easier for the model to process our values."
      ],
      "metadata": {
        "id": "4Caaj1JYKp5o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_images = train_images / 255.0\n",
        "\n",
        "test_images = test_images / 255.0"
      ],
      "metadata": {
        "id": "xArPS1UzKs-k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Building the Model**\n",
        "\n",
        "Now it's time to build the model! We are going to use a keras sequential model with three different layers. This model represents a feed-forward neural network (one that passes values from left to right). We'll break down each layer and its architecture below.\n",
        "\n",
        "**Layer 1**: This is our input layer and it will conist of 784 neurons. We use the flatten layer with an input shape of (28,28) to denote that our input should come in in that shape. The flatten means that our layer will reshape the shape (28,28) array into a vector of 784 neurons so that each pixel will be associated with one neuron.\n",
        "\n",
        "**Layer 2**: This is our first and only hidden layer. The dense denotes that this layer will be fully connected and each neuron from the previous layer connects to each neuron of this layer. It has 128 neurons and uses the rectify linear unit activation function.\n",
        "\n",
        "**Layer 3**: This is our output later and is also a dense layer. It has 10 neurons that we will look at to determine our models output. Each neuron represnts the probabillity of a given image being one of the 10 different classes. The activation function softmax is used on this layer to calculate a probabillity distribution for each class. This means the value of any neuron in this layer will be between 0 and 1, where 1 represents a high probabillity of the image being that class."
      ],
      "metadata": {
        "id": "m6D1uht0KwiP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.Sequential([\n",
        "    keras.layers.Flatten(input_shape=(28, 28)),  # input layer (1)\n",
        "    keras.layers.Dense(128, activation='relu'),  # hidden layer (2)\n",
        "    keras.layers.Dense(10, activation='softmax') # output layer (3)\n",
        "])"
      ],
      "metadata": {
        "id": "aLejI3bUK1Qz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Compile the Model**\n",
        "\n",
        "The last step in building the model is to define the loss function, optimizer and metrics we would like to track. I won't go into detail about why we chose each of these right now."
      ],
      "metadata": {
        "id": "-cM0W6UrK959"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "metadata": {
        "id": "Ju4TQgJJLAVN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training the Model**\n",
        "\n",
        "Now it's finally time to train the model. Since we've already done all the work on our data this step is as easy as calling a single method."
      ],
      "metadata": {
        "id": "53x55jKpLDCa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(train_images, train_labels, epochs=10)  # we pass the data, labels and epochs and watch the magic!"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YXx1ELqSLFca",
        "outputId": "ea822678-acf5-4a2e-ab46-ed08d1d305ae"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1875/1875 [==============================] - 8s 4ms/step - loss: 0.4979 - accuracy: 0.8258\n",
            "Epoch 2/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.3769 - accuracy: 0.8644\n",
            "Epoch 3/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.3378 - accuracy: 0.8773\n",
            "Epoch 4/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.3147 - accuracy: 0.8843\n",
            "Epoch 5/10\n",
            "1875/1875 [==============================] - 9s 5ms/step - loss: 0.2968 - accuracy: 0.8907\n",
            "Epoch 6/10\n",
            "1875/1875 [==============================] - 7s 4ms/step - loss: 0.2817 - accuracy: 0.8956\n",
            "Epoch 7/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.2712 - accuracy: 0.8992\n",
            "Epoch 8/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.2591 - accuracy: 0.9034\n",
            "Epoch 9/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.2493 - accuracy: 0.9065\n",
            "Epoch 10/10\n",
            "1875/1875 [==============================] - 6s 3ms/step - loss: 0.2403 - accuracy: 0.9105\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f4268687f10>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Evaluating the Model**\n",
        "\n",
        "Now it's time to test/evaluate the model. We can do this quite easily using another builtin method from keras.\n",
        "\n",
        "The verbose argument is defined from the keras documentation as: \"verbose: 0 or 1. Verbosity mode. 0 = silent, 1 = progress bar.\" (https://keras.io/models/sequential/)"
      ],
      "metadata": {
        "id": "hbzJ56oFLMIQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=1) \n",
        "\n",
        "print('Test accuracy:', test_acc)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_xr47MLVLhrj",
        "outputId": "fcea7da5-678a-49a3-f428-17f0fba92842"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "313/313 [==============================] - 1s 3ms/step - loss: 0.3438 - accuracy: 0.8813\n",
            "Test accuracy: 0.8812999725341797\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You'll likely notice that the accuracy here is lower than when training the model. This difference is reffered to as overfitting.\n",
        "\n",
        "And now we have a trained model that's ready to use to predict some values!"
      ],
      "metadata": {
        "id": "tQ6DH_F1Lqdq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Making Predictions**\n",
        "\n",
        "To make predictions we simply need to pass an array of data in the form we've specified in the input layer to .predict() method."
      ],
      "metadata": {
        "id": "MYPTwQ3LLrrT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "predictions = model.predict(test_images)"
      ],
      "metadata": {
        "id": "ww1ADoJpLwCU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "predictions[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ip66MkUnMA74",
        "outputId": "5ff7fffb-b93a-4f62-b74f-c191c23e5a6e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([3.1011901e-08, 6.9169789e-07, 1.1001375e-09, 4.7858252e-11,\n",
              "       4.7556750e-09, 1.4894539e-04, 2.3317522e-08, 3.5590318e-03,\n",
              "       1.4057159e-07, 9.9629116e-01], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.argmax(predictions[0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Khsz3-wLMDaD",
        "outputId": "68d9c9bb-8657-4931-81f3-5d8cd52c7419"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Verifying Predictions**\n",
        "\n",
        "I've written a small function here to help us verify predictions with some simple visuals."
      ],
      "metadata": {
        "id": "Sesae56jMW2Z"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "COLOR = 'white'\n",
        "plt.rcParams['text.color'] = COLOR\n",
        "plt.rcParams['axes.labelcolor'] = COLOR\n",
        "\n",
        "def predict(model, image, correct_label):\n",
        "  class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
        "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']\n",
        "  prediction = model.predict(np.array([image]))\n",
        "  predicted_class = class_names[np.argmax(prediction)]\n",
        "\n",
        "  show_image(image, class_names[correct_label], predicted_class)\n",
        "\n",
        "\n",
        "def show_image(img, label, guess):\n",
        "  plt.figure()\n",
        "  plt.imshow(img, cmap=plt.cm.binary)\n",
        "  plt.title(\"Excpected: \" + label)\n",
        "  plt.xlabel(\"Guess: \" + guess)\n",
        "  plt.colorbar()\n",
        "  plt.grid(False)\n",
        "  plt.show()\n",
        "\n",
        "\n",
        "def get_number():\n",
        "  while True:\n",
        "    num = input(\"Pick a number: \")\n",
        "    if num.isdigit():\n",
        "      num = int(num)\n",
        "      if 0 <= num <= 1000:\n",
        "        return int(num)\n",
        "    else:\n",
        "      print(\"Try again...\")\n",
        "\n",
        "num = get_number()\n",
        "image = test_images[num]\n",
        "label = test_labels[num]\n",
        "predict(model, image, label)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "UDVN0riTMrBH",
        "outputId": "e7909ca8-be0c-48d1-fbfc-90ce11bf153e"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Pick a number: 10\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAEWCAYAAADYaXqDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5hdVZnn8e+bO7kYEiqESBISMDRGJoLWAwjYxsGmg22Dtt0MkUZUNOCQAdSxm0YfZHD6eYKKaE8jGCUP2C1gbFEzGEUEGVqbS8JFQi5gDIm5k0ASEi658c4fa1dzUlVnrVN1TtXZq/L7PM956pz97r3PqlNVb6299rvXNndHRCQn/ZrdABGRrlLiEpHsKHGJSHaUuEQkO0pcIpIdJS4RyY4SlzTKJMCBAU1uhxwElLh63mrgVWBXxeOfm9mgKq4B/rWX3/MjwGLCZ7IR+DlwegP2+wDwyQbsR0pK/x17x18Cv2p2I0rms8CVwCXAPcAeYAZwDvCbJrZLMqAeV3PdBPyo4vV1wH2AAf2Bq4A/ADuBx4AJxXoOXAasArYCX+XAn+UngOXANkJSOKoi9jbgXuBFYHPxHjOKr/+N0Pv5XbHuSOAWQm9oPfC/i3ZRfP1a8f6rgL/owvc9ErgWuBS4C3gZ2Av8X+DzxTqDgW8AG4rHN4plAKOAu4Etxfd4NzC+iP0j8G5Cr7asvVupl7vr0bOP1e7+viqxoe7+rLt/zN3f7e5b3X18Efu8uy9x9z9xd3P3t7v7YUXM3f3X7j7a3ScW+/hkETvH3Ve6+1vdfYC7f9Hd/6OIjXD3je7+OXcfUrw+uYhd4+7/2q59P3b3b7v7MHc/3N0fdfeLi9gl7r7C3ScU7fh10a4BRfxKd7+7yvc9w933Vazb2eNad3+4eN8xxffw5SJ2mLt/uPj8Rrj7D939JxXbPlDxeejRBx9Nb8BB8Fjt7rvcfXvF41MV8ZPd/UV3X+PuMyuWP+MhCXW2T/fwx9/2+r+7+33F85+7+0UVsX7u/oq7H1Xs/4kq+7zGD0xcY919t7sfUrFspocEhbvf7yF5tcXOLNoVS0Ztj/PdfVNinT+4+/srXv958Vl2tu4J7r6t4vUDrsTVpx8a4+odH6T6GNcjhEOtw4H5FcsnEA4Tq1lb8XwN8Obi+VHAN4HrK+IGHFnDPisdBQwkHCa26Vfxvm/upA21egFoIYyx7quyzpvb7bPyexwK3EA4xB1VLBtBOHzd34V2SKY0xtV8lxLGbjYAf1exfC1wTGS7CRXPJxbbt213MXBoxeMQ4D+K2NFV9td+mpC1wG5Cgmnbz5sIY2QQElr7NtTqoWLfH4yss4EDx+Yqv8fPAX8CnFy06U+L5VZ81ZQnfZwSV3MdSxjw/lvgAkLiOqGIfRf4MjCF8Ac5DTisYtvPE3obE4DLgR8Uy28G/oE3EsxI4G+K53cD44ArCMlyBOGPH8JA/STe+J3YCPyS0HN7U7H8GOA9RXw+4QTB+KIdV3bh+94BXA3cSEheQwm9u7OArxTr3AF8ERhDSJ5X80a5xghCicl2YDTwpXb730z1BC19QbOPVQ+Cx2p3f9XDOFfb48cexoIe9TCI3bbupz0MyA929/4eBtafc/ed7r7I3xi4d3e/zN1XufsL7n59sX7bfi4o9vOSu69193kVseM9jIdt8zDO1Pb+h7n7b4rljxfLRrr7Te6+zt13eBgfO6+IDXD3G4r3f87dLy3a1TbGdZWH8bbYZ3O+uy9295eLtvzM3U8tYkPc/Z88nEzYWDwfUsTe7GEca5eHExMXt3vvdxXLtxXbNft3QI8GP8xdveoMOaEntrLZDRFpBh0qikh2lLhEpMeY2Twze97Mnq4SNzP7JzNbaWZPmdk7atmvEleeDB0mSh5uJZStVHMWYdhjCjCLcDVJkhKXiPQYd3+QcHlZNecA3wvnVvxh4FAzG5fab68WoLa0tPikSZN68y0PCmvXrq0ae/XVV6Pbjh49Ohp//fXXo3Ezi8a3bdtWNTZ27NjotiNHjozGpaPVq1ezdevW+A8lwcy6csZuKfBaxeu57j63C9sfyYGFzOuKZRs7Xz2oK3GZ2QxClXZ/4LvuPie2/qRJk1i8eHE9bymduPzyy6vGlixZEt32ggsuiMZ37doVjQ8YEP8Vuuuuu6rGYu0G+MAHPhCN1yOVkPv1y/NgpLW1tbff8jV37/U37fZPx8z6EwoIzwKmAjPNbGqjGiYizWNmNT0aYD0HXoExvlgWVc+/lZOAlR6KIPcAdxKOV0Ukc/369avp0QALgI8WZxdPAXa4e/QwEeo7VOzs2PTk9iuZ2SzC2QImTuzK5Wwi0iwN6k1hZncA04EWM1tHuDxrIIC73wwsBN5POEv+CvDxWvbb44PzxUDdXIDW1laV6YuUXAMPA3H3mYm4EyYa6JJ6Ele3jk1FpPwalbh6Sj0HqYuAKWY22cwGAecRjldFJHO9ODjfLd3ucbn7PjObTZjTvD8wz92XNqxlB5EHHnggGv/Wt74VjQ8ePLhq7MUXY7V/cNlll0Xj/fv3j8aHDh0ajZ9yyilVY/Pnz68aA1iwIP5/cM6caPVNtEYt13KH3lL2HlddY1zuvpAwuCYifYSZlT6xa+pmEemgT/e4RKRvUuISkewocYlIdpS4RCQrGpwXkSypx3UQeOaZZ6Lx6667Lhp/9tlno/Fp06ZF48uXL68aO+SQQ6LbtrS0RONbt26Nxo8//vhoPDYfV2pKnFh9GsAVV1wRjb/lLW+pGrvkkkui2x5++OHReF+nxCUi2VHiEpGsNPtynloocYlIB0pcIpIdnVUUkeyoxyUiWdEYV4ns378/Gk9N33LTTdXvU/nwww9Htx02bFg0ftJJJ0Xjw4cPj8Zfe+21qrEVK1ZEt02VS6TKAlKf66JFi6rGLrrooui2o0aNisZfeumlaHzjxupTl1988cXRbW+++eZoPHVrtdzvIqTEJSLZUeISkeyUvUeoxCUiB9AYl4hkSYlLRLKjxCUi2VHiEpHsKHGVRKpOK2XJkiVVY0cccURd752a3iU2NQzA2WefXTW2bNmy6LaxWieA66+/Phq/9tpro/Ezzzyzaiz1ucTq0yB9a7Q3velNVWOpOqvbb789Gv/MZz4TjZf9rFyMJhIUkSypxyUi2VHiEpHsKHGJSFZUgCoiWVLiEpHs6KyiiGRHPa5MpGqlYjVFY8aMqWvf+/bti8ZHjBgRjW/ZsqVqbPr06dFtN2/eHI3Pnz8/Gp88eXI0ftxxx1WNvfzyy9Ft9+zZE43v3bs3Go/NNZaqvVu3bl00Xu/8bmXW58e4zGw1sBPYD+xz99ZGNEpEmqvsiasRB7LvdfcTlLRE+o62XlfqUeO+ZpjZM2a20syu7CQ+0cx+bWZPmNlTZvb+1D51qCgiHTRqcN7M+gM3An8GrAMWmdkCd6+8Fu2LwHx3v8nMpgILgUnR9tXZLgd+aWaPmdmsKg2fZWaLzWxxbCxGRMqh1t5WjT2uk4CV7r7K3fcAdwLntFvHgbYLS0cCG1I7rbfHdbq7rzezw4F7zWyFuz94QIvc5wJzAVpbW73O9xORXtCFMa4WM1tc8Xpu8Tff5khgbcXrdcDJ7fZxDaED9D+AYcD7Um9aV+Jy9/XF1+fN7MeE7PpgfCsRKbsuJK6tDRjfngnc6u7Xm9m7gH8xs+PdveoUHt0+VDSzYWY2ou05cCbwdHf3JyLl0cBDxfXAhIrX44tllS4C5gO4+0PAEKAlttN6elxjgR8XjR8A3O7uv6hjf0313HPPdXvb1LxRu3fvjsZTNT+p+yr+8Y9/rBpL3Xtw3Lhx0XiqTmvTpk3R+OrVq6vGUvVpqXsXpv5wYrVWO3fujG6b+pnu2LEjGh89enQ0XnYNLIdYBEwxs8mEhHUe8JF26/wROAO41czeSkhc0QHxbicud18FvL2724tIOTVyIkF332dms4F7gP7APHdfambXAovdfQHwOeA7ZvYZwkD9x9w9Oh6ucggR6aCRBajuvpBQ4lC57OqK58uA07qyTyUuEemg7JXzSlwi0oESl4hkpc9fZC0ifZMSVybWr29fWnKg2OnxVElAagqVVMnC8uXLo/Ht27dXjaVuPxab+iW1b4AnnngiGm9pqV6OE5vyBmDt2rXReGpqmV27dlWNpX4mKStWrIjGTz311Lr232yaSFBEsqMel4hkRWNcIpIlJS4RyY4Sl4hkR4PzIpIVjXGJSJaUuDKRquMaPHhw1VjqNlup248ddthh0fiaNWui8djtz4YMGRLdNvZ9ARx++OHR+Fvf+tZofODAgVVjqbalppY59thjo/Ff/epXVWOpqYJi9WcAS5cujcZzr+NS4hKR7ChxiUh2lLhEJCuNnEiwpyhxiUgH6nGJSHaUuEQkO0pcIpIVFaBmJFUzFJvbaeXKldFtX3311Wh80qRJ0XiqzitWK/XCCy9Et43VgAG88sor0XjqNl9HH3101Vis3ZC+bVvqFmEPPfRQ1djxxx8f3fbMM8+MxlM/89wpcYlIdnRWUUSyokNFEcmSEpeIZEeJS0Syo8QlIlnRJT8ikiX1uDKRurdhbM6tWI0XwOTJk6Px1PbHHHNMNB6bU+vRRx+Nbrtly5ZofOrUqdF4qu179+6tGkvVtw0dOjQaT32ut9xyS9XYF77whei2qfq11BxsuSt74kr2B81snpk9b2ZPVywbbWb3mtnvi6+jeraZItKb2koiUo9mqeVA9lZgRrtlVwL3ufsU4L7itYj0EdknLnd/EHix3eJzgNuK57cBH2xwu0SkSWpNWs1MXN0d4xrr7huL55uAsdVWNLNZwCyAiRMndvPtRKQ3lf2sYt2tc3cHPBKf6+6t7t46ZsyYet9ORHpB2Xtc3U1cm81sHEDx9fnGNUlEmq2RicvMZpjZM2a20sw6HQ83s3PNbJmZLTWz21P77G7iWgBcWDy/EPhpN/cjIiXTyDEuM+sP3AicBUwFZprZ1HbrTAH+ATjN3d8GXJHab3KMy8zuAKYDLWa2DvgSMAeYb2YXAWuAc5PfQcmtXr06Go/VSqXmjTr//POj8Tlz5kTjqXmrYuMRqfq01Hxdzz8f70z/7ne/i8anTZtWNTZo0KDotqn7UabmAovNc5aqEUvVp4URkr6rgYeBJwEr3X1Vsd87CSf3llWs8yngRnffBuDuySO4ZOJy95lVQmekthWRPHVhcL7FzBZXvJ7r7nMrXh8JrK14vQ44ud0+jgUws98C/YFr3P0XsTdV5byIdNCFHtdWd2+t8+0GAFMIR3bjgQfN7L+4+/ZqG5T7nKeI9LoG13GtByZUvB5fLKu0Dljg7nvd/TngWUIiq0qJS0Q6aGDiWgRMMbPJZjYIOI9wcq/STwi9LcyshXDouCq2Ux0qikgHjRqcd/d9ZjYbuIcwfjXP3Zea2bXAYndfUMTONLNlwH7g8+4ePWukxCUiHTSyuNTdFwIL2y27uuK5A58tHjVR4ips3LgxGm9paaka27696hgikJ6+ZcqU6OF8sixgxYoVVWN79uyJbjty5MhoPFUmsmHDhmj8tNNO6/Z7r1mzJhofMWJENL5qVfWjjVQpxZAhQ6LxVLlEalqcVDlGM2kiQRHJUtnn41LiEpEOlLhEJDtKXCKSHSUuEclKs6esqYUSl4h0oLOKIpId9bhKIlXPlIrH/gOlanLqnUIlVSd21FFHdXvb1LQ1qbadeOKJ0fhrr73W7X3Hvi9IT9kzfPjwqrHRo0dHt926dWs0fsQRR0TjmzZtisaPPvroaLzZlLhEJCsa4xKRLClxiUh2NDgvItlRj0tEsqIxLhHJkhKXiGRHiaskVq5cGY3Hbj8GsHfv3qqxHTt2RLcdN25cND5gQPzHkJqP65BDDqkaS7UtdXuy9773vdH4s88+G42n6qFiUvVvqdvCxT631FxeqXjqc0vN91V2SlwikhVNJCgiWVKPS0Syo8QlItlR4hKR7ChxiUhWVIAqIlnSWcWSSM1LVU8d17Rp06LbpuZuWrduXTQem1cK4vNapb7v1H/WVNt///vfR+Oxzy3cB7S61Hxbqfq2MWPGVI2l/jBT98JM/UxS9XNlV/YeVzKtmtk8M3vezJ6uWHaNma03syeLx/t7tpki0pvaDhdTj2appT94KzCjk+U3uPsJxWNhJ3ERyVCtSauZiSt5qOjuD5rZpJ5vioiURfaHihGzzeyp4lByVLWVzGyWmS02s8Vbtmyp4+1EpLf069evpkfT2tfN7W4CjgFOADYC11db0d3nunuru7fGBktFpDyyP1TsjLtvbntuZt8B7m5Yi0SkqZqdlGrRrR6XmVXO0/Ih4Olq64pIfrLvcZnZHcB0oMXM1gFfAqab2QmAA6uBi3uwjQ2Run9gPXNepWqdYrVMEL/3IMDYsWOj8d27d1eNpeb6Su37/vvvj8aXLVsWjcfuHzhqVNWhUSD9ucR+JhCfr2vQoEHRbVN/lKmfaaoOrOzK3uOq5azizE4W39IDbRGRksg+cYnIwSWHiQTL3ToRaYpGjnGZ2Qwze8bMVprZlZH1PmxmbmatqX0qcYlIB41KXGbWH7gROAuYCsw0s6mdrDcCuBx4pJb2KXGJSAcN7HGdBKx091Xuvge4Ezink/W+DFwHxM/IFJS4RKSDLiSulrYrY4rHrHa7OhJYW/F6XbGs8r3eAUxw95/V2r6DZnA+dZus1O2oYqe/J0+eHN12+fLl0XhsWhqIlztAvBxj7dq1VWOQPm0/evToaDxVkjBs2LBub5sqUUlNRRSTKmdI7Ts1JU+qlKPMulijtdXdk2NSkffqB3wd+FhXtjtoEpeI1K6BZxXXAxMqXo8vlrUZARwPPFAkyyOABWZ2trsvrrZTJS4R6aCBdVyLgClmNpmQsM4DPtIWdPcdQEvF+z4A/M9Y0gKNcYlIJxo1OO/u+4DZwD3AcmC+uy81s2vN7Ozutk89LhE5QKOvQywmGl3YbtnVVdadXss+lbhEpANd8iMi2Sn7JT9KXCJygGZPWVOLgyZxpeqVhgwZ0u3tW1paqsYgPaXOyJEjo/HUrbBityBLTWvz8ssvR+OpKXtefPHFaDxWz7Rp06botoceemg0vnPnzmg8JlUjloqnPtc9e/Z0uU1losQlItlR4hKR7ChxiUh2lLhEJCs5TCSoxCUiHajHJSLZUeISkewocWUiNT9TrK4nVfOzdOnSaDw1npCKx+q4Ur+AqVuEpT6XgQMHRuOxObdSc17Fbi8G6VqpWJ1Y7LZptUjVcb3yyit17b+ZVIAqIlnS4LyIZEc9LhHJjhKXiGRFY1wikiUlLhHJjhKXiGQn+7OKZjYB+B4wFnBgrrt/08xGAz8AJgGrgXPdfVvPNbU+qf8gqXqk2JxYqfsmnnrqqdH4cccdF42n5q2K1Ttt2bIlum2qHmn//v11xWN1YDt27Ihum7p34aBBg6Lx119/vVvtgnQNWWr+tlRtX5nlMMZVS1rdB3zO3acCpwCXmtlU4ErgPnefAtxXvBaRPqBRd/npKcnE5e4b3f3x4vlOwi2GjgTOAW4rVrsN+GBPNVJEelfZE1eXxrjMbBJwIvAIMNbdNxahTYRDSRHpA8p+qFhz4jKz4cCPgCvc/aXKb8zd3cw6HZAws1nALICJEyfW11oR6RVlT1w1nTows4GEpPV9d7+rWLzZzMYV8XFAp3eEcPe57t7q7q1jxoxpRJtFpAe1TSRYy6NZku9sIfXeAix3969XhBYAFxbPLwR+2vjmiUgz9IUxrtOAC4AlZvZksewqYA4w38wuAtYA5/ZMExsjdWo9VRYQ+yGNHj06uu2nP/3paHzVqlXR+OOPPx6Nx3qyS5YsiW67bNmyaDz1vaXKIWK3N0uVoGzYsCEa/+hHPxqNn3LKKVVjqVKM1OeWUvY6qJSyHyomE5e7/wao9l2c0djmiEgZZJ+4ROTg0uzDwFoocYlIB2U/1FXiEpEO1OMSkewocYlIVjTGJSJZUuIqiXp/ELF6pdNPP72ufadulVXPrbTe8573dHtbiE8NA7B79+5oPHZ7smZKXcVR7+9L6nMru0YmLjObAXwT6A98193ntIt/FvgkYSaaLcAn3H1NbJ/lPnUgIk3RqEt+zKw/cCNwFjAVmFlMi1XpCaDV3acB/wZ8Jdm+Ln9HItKn1Xq5T429spOAle6+yt33AHcSpsT6T+7+a3dvu4Puw8D41E4PmkNFEaldFw4VW8xsccXrue4+t+L1kcDaitfrgJMj+7sI+HnqTZW4RKSDLiSure7e2qD3/FugFUgOzCpxiUgHDRycXw9MqHg9vljW/v3eB3wBeI+7x8/4oMQlIp1oYOJaBEwxs8mEhHUe8JF273Ui8G1ghrt3Oq9fe0pcInKAtokEG8Hd95nZbOAeQjnEPHdfambXAovdfQHwVWA48MMiYf7R3c+O7fegSVyDBw+Oxuv5D5OaVyolNadV6lZZsbnG6v3PmfoFbmadVmqOtdj3PmLEiOi2qc88Vae1Z8+eaLzsGlnH5e4LgYXtll1d8fx9Xd3nQZO4RKR2qpwXkewocYlIVnSRtYhkSRMJikh21OMSkewocYlIVjTGVSJbt26Nxvfu3RuNx+p6Uvdk7GmxX7J6ap3KLlVLFfuZpeq4UvOMpbavt7av2cr+e3HQJC4RqZ0Sl4hkR2cVRSQrGuMSkSwpcYlIdpS4RCQ7Slwikp3sE5eZTQC+B4wFnDAZ/jfN7BrgU4T7oAFcVcy7U0qpOa9SdTf79u2rGhs3bly32tQbevoXsJ46sXprzOqp40rNI5aq64v9PkC6zqvMGjmRYE+ppce1D/icuz9uZiOAx8zs3iJ2g7t/reeaJyLNkH2Py903AhuL5zvNbDnhlkMi0keVPXF1qT9oZpOAE4FHikWzzewpM5tnZqOqbDPLzBab2eItW7Z0toqIlEwDbwjbI2pOXGY2HPgRcIW7vwTcBBwDnEDokV3f2XbuPtfdW929dcyYMQ1osoj0pAbfybpH1HRW0cwGEpLW9939LgB331wR/w5wd4+0UER6XdkH55Ots5BWbwGWu/vXK5ZXnkr7EPB045snIs3QF3pcpwEXAEvM7Mli2VXATDM7gVAisRq4uEda2CCp/yA7d+6Mxrdv3141liq1SKnntH6z1fPL29Rf/MRURPWWzwwbNqzLbSqTsg/O13JW8TdAZ99FaWu2RKT7mt2bqoUq50WkAyUuEcmOEpeIZKWvXPIjIgcZ9bhEJDtKXCKSHSWukvj4xz8ejT/22GPReKyO653vfGe32tSm2bc3y1U94zCpqYhS8dTP7NBDD+1ym8pEiUtEsqI6LhHJks4qikh21OMSkeyUPXGVuz8oIr2u0fNxmdkMM3vGzFaa2ZWdxAeb2Q+K+CPFhKVRSlwi0kGjEpeZ9QduBM4CphJmlZnabrWLgG3u/hbgBuC61H6VuESkg379+tX0qMFJwEp3X+Xue4A7gXParXMOcFvx/N+AMyyRFS11i6hGMrMtwJqKRS3A1l5rQNeUtW1lbReobd3VyLYd5e51zZFuZr8gtKkWQ4DXKl7Pdfe5Ffv6a2CGu3+yeH0BcLK7z65Y5+linXXF6z8U61T9THp1cL79B2pmi929tTfbUKuytq2s7QK1rbvK1jZ3n9HsNqToUFFEetJ6YELF6/HFsk7XMbMBwEjghdhOlbhEpCctAqaY2WQzGwScByxot84C4MLi+V8D93tiDKvZdVxz06s0TVnbVtZ2gdrWXWVuW13cfZ+ZzQbuAfoD89x9qZldCyx29wWEm/H8i5mtBF4kJLeoXh2cFxFpBB0qikh2lLhEJDtNSVypSwCaycxWm9kSM3vSzBY3uS3zzOz5os6lbdloM7vXzH5ffB1VorZdY2bri8/uSTN7f5PaNsHMfm1my8xsqZldXixv6mcXaVcpPrec9PoYV3EJwLPAnwHrCGcdZrr7sl5tSBVmthpojRW/9WJb/hTYBXzP3Y8vln0FeNHd5xRJf5S7/31J2nYNsMvdv9bb7WnXtnHAOHd/3MxGAI8BHwQ+RhM/u0i7zqUEn1tOmtHjquUSAAHc/UHCWZZKlZdH3Eb4xe91VdpWCu6+0d0fL57vBJYDR9Lkzy7SLumiZiSuI4G1Fa/XUa4fngO/NLPHzGxWsxvTibHuvrF4vgkY28zGdGK2mT1VHEo25TC2UjHTwInAI5Tos2vXLijZ51Z2Gpzv6HR3fwfhavZLi0OiUiqK9MpUz3ITcAxwArARuL6ZjTGz4cCPgCvc/aXKWDM/u07aVarPLQfNSFy1XALQNO6+vvj6PPBjwqFtmWwuxkraxkyeb3J7/pO7b3b3/e7+OvAdmvjZmdlAQnL4vrvfVSxu+mfXWbvK9LnlohmJq5ZLAJrCzIYVg6aY2TDgTODp+Fa9rvLyiAuBnzaxLQdoSwqFD9Gkz66YEuUWYLm7f70i1NTPrlq7yvK55aQplfPF6d5v8MYlAP/Y643ohJkdTehlQbgc6vZmts3M7gCmE6YY2Qx8CfgJMB+YSJgi6Fx37/VB8iptm0443HFgNXBxxZhSb7btdODfgSXA68XiqwjjSU377CLtmkkJPrec6JIfEcmOBudFJDtKXCKSHSUuEcmOEpeIZEeJS0Syo8SVl7HA7cAqwgW6DxHqfpppOPBt4A+ENj0AnNzNfV3VoDZJH6fElQ8j1HA9CBwNvJNQvDu+mY0Cvku42HoKoU0fp/ZbW7WnxCU1UeLKx38F9gA3VyxbA/yf4vnHgH+uiN1NKAiFcAXAQ8DjwA8JvSSAOcAy4CmgbUqVvyFUbv+OkCRjjiH0rr7IGwWVzwE/K55/ttjX08AVFdv9hNA7Wwq0Xcg+BzgEeBL4fuJ95SDX7JtlSO3eRkg8XdVCSCzvA14G/p6QUG4kHGYeR6jYPrRY/2rgzwnXj7YtezOhZ9V+gru3ERLN/k7et633dTKht/gI8P+AJ4BPEHpphxAuAfsRcCUwm1BBLhKlHle+biT0ihYl1jsFmAr8lpBkLgSOAnYQ7kB8C/BXwCvF+r8FbgU+RbgkC2ADHZNWyumEy6deJkw4eBfw7iJ2WdH2hwkX3E/p4r7lIKceVz6WAh+ueH0poTfVNr30Pg78RzSk+GrAvYTr4do7CTiDcC+72SBHPIkAAAEqSURBVITD0UsIvaS/IBzOvZPqN+dcCrydkOA663V1Zjqh9/cuQrJ8oKKtIjVRjysf9xP+wD9dsWxoxfPVhMOsfoReTNvUKA8DpwFvKV4PA44ljHONBBYCnyEkIAjjVo8QDhm3cOAURO39gZA4/xchQQJMIiS9fyfMMDq0eM8PFctGAtsISes4Qo+wzV5gYOT9RAD1uHLihERwA/B3hKTSNmYF4RDvOcJg+3LeGA/bQhi4vwMYXCz7IrCTMK3LEELS+WwR+yrh0M2A+wiHdNXGuAA+SZj4biXwKrAV+Hzx/rcCjxbrfZcwvrWM0KtbDjxDSKxt5hJOFDwOnF/DZyIHKc0OISLZ0aGiiGRHiUtEsqPEJSLZUeISkewocYlIdpS4RCQ7Slwikp3/DxsP9X3AMkd1AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**CNN**\n",
        "\n",
        "In this guide we will learn how to peform image classification and object detection/recognition using deep computer vision with something called a convolutional neural network.\n",
        "\n",
        "The goal of our convolutional neural networks will be to classify and detect images or specific objects from within the image. We will be using image data as our features and a label for those images as our label or output."
      ],
      "metadata": {
        "id": "Q2gruTynQKrk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow.keras import datasets, layers, models\n",
        "import matplotlib.pyplot as plt"
      ],
      "metadata": {
        "id": "zb8j874wQZQV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#  LOAD AND SPLIT DATASET\n",
        "(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()\n",
        "\n",
        "# Normalize pixel values to be between 0 and 1\n",
        "train_images, test_images = train_images / 255.0, test_images / 255.0\n",
        "\n",
        "class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',\n",
        "               'dog', 'frog', 'horse', 'ship', 'truck']"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VVgQKZzTQcA-",
        "outputId": "73098c91-783a-4589-9052-62239cf82991"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
            "170500096/170498071 [==============================] - 2s 0us/step\n",
            "170508288/170498071 [==============================] - 2s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Let's look at a one image\n",
        "IMG_INDEX = 7  # change this to look at other images\n",
        "\n",
        "plt.imshow(train_images[IMG_INDEX] ,cmap=plt.cm.binary)\n",
        "plt.xlabel(class_names[train_labels[IMG_INDEX][0]])\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 280
        },
        "id": "7xa3MA6iQfAk",
        "outputId": "0a06f367-aa0f-44fd-fbb6-c92764e3b539"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEHCAYAAABoVTBwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deZBkV3Xmv5Nb7Wuvpd5Kaq0tgQQUWkDIsjEaIRsL7LEGYoJhJhiacZgJM+GJGIKJACZi/sDjAYY/HDjEoEA4MIgxyMgYm0VmUIBB0BJakdHard6rl1qyltxenvkjs2dKmvvdKnV1ZTXc7xfR0Vn35H3vvpvvvJd5v3fOMXeHEOJXn9x6D0AI0Rnk7EIkgpxdiESQswuRCHJ2IRJBzi5EIhRW09nMbgXwaQB5AP/T3T8ee38un/dCsRjellukY9hW6g5vq7VBbqpV6tTmkY75fPjayNoBOnQAQJHMBQBkzSa1NbIGtRUK4Y+02eDba9YzaosdW7FU4ttEeH9Zg489y/gYLfK5xOTjLAsfWy5yXA6+vdi+zlbGNgsfW460x/ZVq9bQqDeCHW0VA8wDeBrAWwAcAvBTAO9y95+zPqXubt+yfTxoyzk/8fO9+WD7jsvGIuOjJux/7gi1NZv8+jcwNEDau2mf/lJ47AAwNraV2qbnytR2anqK2kY3bAy216YWaZ+546eobWQgfMwAsHXXNr7NRiXYPnOK72uuPE9t+ch9qV7lF6uZ2Zlge89ID99exm8G9Tq3ZU0+Do/YSsXwsfV08/OqVqsF25959GkszC0Ez/7VfI2/FsCz7v68u9cAfBnA7avYnhBiDVmNs28DcHDJ34fabUKI85BV/WZfCWa2F8BeAMiT35NCiLVnNXf2wwB2LPl7e7vtJbj7ne4+4e4TuTz//SqEWFtW4+w/BXCJmV1oZiUA7wRw37kZlhDiXHPW36vdvWFmHwDwLbSkt7vc/cl4J8Dr4dX/2ErmIlkdPXaUr0pv3thHbd2FmFTGV2mLzfA3k+rUAu0zsqmX2rZv2UBtfT38o1mYPU1tqM4Fm6+4gi+nbH3D5dTW39NFbV393FZthleLq9XttM/sNFcgisbn48SRE9T2woGwnFcaHaR98t38G2hm4eMCgJ5Bvnre3cVlyoHu8LlajPzsbTbDfnT8wP/35fr/sqof0e7+TQDfXM02hBCdQU/QCZEIcnYhEkHOLkQiyNmFSAQ5uxCJ0NFH2swMXaXwLj3jkStZRoJ1Glwi2TwSDggBgMppLpUtzvGorO58WJbr7eXy2hWXXUxtl1w6Tm0zkUCYYnfkGp0Lz9WeV/F9XTh+AbXVqjw4xXN8rnLko2FRjwDQrHH5tT7PJa/aPA8our5yRbDdilwmy5HAKwDISjwQJsdPA+SK/PwuWXhOzibq7a8///d8DNQihPiVQs4uRCLI2YVIBDm7EIkgZxciETq6Gp/PG/qGw7ssNPl1ZyALr5z2dPEV1Ui8AnoLvF+lMkttC3Mng+3ey8c+eYTv62cZVwUqtSq1bdi8mdrGtodXpscu4OpEzzAfIw/fACKxHegm6bicKSsA6vP8mNHDd1YtRfLJVcOBMLkscup38VXwns1D1Nbo4cdWjZyQbuF+zUgewqaT48rzsevOLkQiyNmFSAQ5uxCJIGcXIhHk7EIkgpxdiEToqPRW6ilg/MotQVtXJVLuqByWJg4fnqZ9fvEYrzySc37Y1Vkuh1kjXFUlR+QdAHhhX7giCQC8SIKCAKBBpBUA2LiFS29TRHrra76a9tk8GA4WAYCtkao1vV1cauoiclKtHKlMU+OBNbVZLl3N7ec56GYnw3kKa+VwxRoAWAQPdtl46Q5qy0WqzHRv7qc2Gw7LlBapHVYkkUaRQki6swuRCnJ2IRJBzi5EIsjZhUgEObsQiSBnFyIRViW9mdl+AGUAGYCGu0/E3j80PIBb3/6moG1+/yTt96O/+3GwPR/Jj7Ywy/OZZRm/xvWAy0lDveFcYX1Fvq8NeZ6YbLiXR1ChECmCWee23OFw1N4j3/gh7XPgkZ9T2823vIHarrp8nNr6iuExlma4vGYn+TyeepGXvKr801Fqmz8WluUqVS4BHpnlku6BZw5SW2ED/zx7d45Q2563vCrYXuzl5bXqWViajSi250Rn/3V3D8d+CiHOG/Q1XohEWK2zO4Bvm9lDZrb3XAxICLE2rPZr/I3uftjMNgP4jpn9k7s/sPQN7YvAXgAY3RT5jSqEWFNWdWd398Pt/ycB3Avg2sB77nT3CXef6B/kNdOFEGvLWTu7mfWZ2cCZ1wBuAfDEuRqYEOLcspqv8VsA3GutEjUFAH/p7rz2DICe3iKuumZb0PbsIk82ODMVjkTb0DtA+zTqPHLpZJnLOGPDPLHhxcPh/RXAJaOi8SkeGYwkeuzh34KyyDW6uzscedXXx+OhZib5fPziG9+jtuFjkUi6kcFge6PCo9eatUiU12Ikwq7JbQvTRCiKSFTZDI98nD7Jy3L1nuBScH2a96u+5qJge36cnzsZP70pZ+3s7v48gKvPtr8QorNIehMiEeTsQiSCnF2IRJCzC5EIcnYhEqHjtd6GhsKRYydP8gSRxVxYhurPc+lqqsmjmuA82WDJufyzcyA8jp4uHoVWi1xOqzU+xnJE/in1cMnRi+Hx9xqfq80beR24UiEiax08Rm1HJ8PRZo2MS2+5HE/YCOdzXIjUZhsYDW+zOsul3t5IDcHTczyB6MJxLmEODfBj67dwdFuWiyTgJB+LR6I2dWcXIhHk7EIkgpxdiESQswuRCHJ2IRKho6vxZjn0lMIrj9bgwSTlqXBOsFxkNb5gPFLAG/wa12jwMj31OslB18ujKop5vq9ymQdOlEhACwAM9PPjLpbCq9bz83O0DzJ+GowO84CcSpWvaGfk46xXucpQmeer2eUy79fbx4OXRvrDn+dkpJxUdzfPG+hNHtBSqfFz7uCLXLm48GBYudg8vp32yZrhuXfXarwQySNnFyIR5OxCJIKcXYhEkLMLkQhydiESoaPSG9yBevjh/kgFJRTJNWl4iAeE9Da5PHVwlkte1YgMVa6EB1ksclmo0MVL+DTqXP7ZvoPLLkMbRqnt5KlwQFE9sq9G5Cyo13i/riKXvCokp2C2yOdqIRKcMns6XNYKALwRCTLZFC67VCfnIQDMzXMJbaHKT9R6g8telUjuuheeDpeU2njDBbRPgZTXaueEDKI7uxCJIGcXIhHk7EIkgpxdiESQswuRCHJ2IRJhWenNzO4C8NsAJt39qnbbKIB7AIwD2A/gDnefWm5bzUYDs6fCb5sn7QAwQso8dZMIOgCoVbl80ixw+WTBeF64qWr42jgwGI6GA4BiRAoZ7OOS0fAQj7wa6OeS18x0+NhOzfLcaXnwSL9No1zejFGpEBmNJU8DUKvx6MG5OZ43cC4S0dfVFZ6rLMc/l5NlLpNNseMCUKnz8VfqvN+Rw+ESVfFzODyPq81B93kAt76s7UMA7nf3SwDc3/5bCHEes6yzt+utvzzQ+HYAd7df3w3g7ed4XEKIc8zZ/mbf4u5H26+PoVXRVQhxHrPqBTpvpcagPxTMbK+Z7TOzfVOnI9lShBBrytk6+3EzGwOA9v+T7I3ufqe7T7j7xMgoXwgSQqwtZ+vs9wF4T/v1ewB8/dwMRwixVqxEevsSgJsBbDSzQwA+CuDjAL5iZu8FcADAHSvZmbujSZLy1SMJBUf7w/LPzDSPhDqxyKWmjbvCkVAAMNLHZbRjh8JJAwcrY7RPV4Fvb8PoMLX190aSaea5xDM4GO535EUuXc3Pcxmq2YzJYZHkkQthW5MH0WFqlo9xusw7Np3bCsfCslaJlPICgLkmj4ibaXBbNVI6rNrktkozHMHWaHIZLWNRjJGEk8s6u7u/i5jevFxfIcT5g56gEyIR5OxCJIKcXYhEkLMLkQhydiESobO13mAokOtL0fhQaiR54WyZP5G36Dxi6Ma3vIHartzDZbQffPGbwfaTh3mk3NjQILUNDfCHjGo1LkNVI/JPMwsfd7Ua0bwyLq+dOs3rr4HUGwMAb4aj7+bn+L6mZ/gxZ8YjHHMRefPYqbA8OzbMPxf08mjEcqTWW7UZqSFoYXkNAPK94fMg42odzLjExtCdXYhEkLMLkQhydiESQc4uRCLI2YVIBDm7EInQYekthy4PJ1Lcumk37fdQdjzYPgUedXXBlZup7Q0376G2y6/g9bU29Ian6++/dD/tMzvN5cGFeR55dfokj+irRZIXeiF8/S5XuY4zRyIRAWCEyJ4A0AWeuDMj8uB0JLqxFqmVVizxKMBKnY9/qhKW+oqRxJeLeS6JLoLXCayBy4oLDX4e5AfCsmJvHz/mjES3WSSRpu7sQiSCnF2IRJCzC5EIcnYhEkHOLkQidHQ1vpk5FmbDK6e5Lh6YUCVxCRfs2kH73Povrqe2iy/bSG2lHr5Ke+WN4VX8RmQWf/DZv6G2R557ntqsyjeaNfiqL0rhgIvTkVX10ZFIvrseXmpqcZYHhZRnwqvP85F4nHyeH3O1wTvOVHgAzUIuPB9PHT5B+7x4ku+rHAkaakbyv1URKQO2cSjY3t/HS4CdnmOqwOrKPwkhfgWQswuRCHJ2IRJBzi5EIsjZhUgEObsQibCS8k93AfhtAJPuflW77WMA3gfgjH7xYXcPJ2hbQr1Rx6FT4RJK//j4P9J+m3aHpYk79v4u7XPRHi6vWYHnjKtWI4EOtXDgx1Wvu4L2OfDwc9T23Xv+gdpKNR4kU6/yAJSmhwNQhrq59LNjbBu1IZLrbK7G5TwWgDJdjeSS46NAscjHUS7ycRSHw/LVwUOnaJ9jZb69jTt5gNWRQ1zOa9R5DrqcheXN2SkubVYa4TE2IyWjVnJn/zyAWwPtn3L3a9r/lnV0IcT6sqyzu/sDACIpRoUQvwys5jf7B8zsMTO7y8x4WVQhxHnB2Tr7ZwDsBnANgKMAPsHeaGZ7zWyfme2bneGJC4QQa8tZObu7H3f3zN2bAD4L4NrIe+909wl3nxgc4s/6CiHWlrNydjNbWjblHQCeODfDEUKsFSuR3r4E4GYAG83sEICPArjZzK5BK8RmP4D3r2Rnxa4Stu7eHrQ1+nmk0TUTVwfbL756K+2TOc/5Vc94lFSNlE8CAOTD8lWpn0/jzlddQm1z936P2gp1LqHMznNpqERy0F1z+UW0z/iF3DYzz+dxfpJLmMcWwvN4fIFHjeXzXFLMF7gM1b+Vy1pvvC1c6uv43/yE9jlSP0Jtt//L36S2B/7hR9T24+8foLbDRLKrV3fSPkbLSXGJdVlnd/d3BZo/t1w/IcT5hZ6gEyIR5OxCJIKcXYhEkLMLkQhydiESoaMJJ/PFPIbHRoO2f/sf/jXtV+oJX5PqOS7H5CKliXKRw+7pGaA29/A2G00uhV2wi8uDl17BZblDj/MIKs/4/vLFcHbOWoEnlXzkOS4LTU7PUNuxE1yWOzETllJnqWQE5PJcyuvv5pLodb/+Jmq79q3XBdt/9OgLtM/CsweprW+YJ+B82+/eRG1PP3kvtT2yL/yYys1v4+fH1vHwE+r5HL9/684uRCLI2YVIBDm7EIkgZxciEeTsQiSCnF2IROhsrTdvYr4alsv6Rrk01ERYdmFSGABYnl/HGlUeeeUeu/6FI9FqdR5FN7yFS3lv+723UtuXj91HbQvTkVpvCEtbp3I8qnDj5nBCTwCYa3DprRpJolggdcp68uGEmACwedMWarvuhnCdPQC4/jdfR202HP48L7gwLAEDQLNZpLZnn+WS3dt+i6Z1wGWXjVHbQw//Ith+aP9R2mfXxRcE280kvQmRPHJ2IRJBzi5EIsjZhUgEObsQidDR1Xj3JhqN8KpwM7oIHl51L0RWgxvOc7h55LDdua3eCK+6e46vjjcipYl2vHqc2nq2DlLbzFOHqc0K4ZXkHdddSPv8zh23UNvR43xFeHJymtrK82EFpWF8NX7bGC/ZtTNSdqlW4EEyU4vhMk/bd/HV+EKOl956/mk+932/z8+DiddeTG0/e/iZYPviPFdQsjrZFz/tdWcXIhXk7EIkgpxdiESQswuRCHJ2IRJBzi5EIqyk/NMOAF8AsAWthf073f3TZjYK4B4A42iVgLrD3aeW2RqMlKdp1Ll8UiiEJbZmJB5kYYFLXjF5DeAbzRrhMRa7eeBELXI57Rnm0mH/BcPUdmye594bGgpLdpt386raQ+P91NZ9wS5qu9i4rb4Ylo3mKvxzaWZclsvlIkFPzj+zrnxXsH3jpg20z8AgD8oqFbks1zvAA4quvpbnkxu59/vB9makEllPV/gcNuPln1ZyZ28A+GN33wPgegB/aGZ7AHwIwP3ufgmA+9t/CyHOU5Z1dnc/6u4Pt1+XATwFYBuA2wHc3X7b3QDevlaDFEKsnlf0m93MxgG8BsCDALa4+5nHq46h9TVfCHGesmJnN7N+AF8F8EF3n11qc3cHeVDPzPaa2T4z2zd9iv/WFEKsLStydjMrouXoX3T3r7Wbj5vZWNs+BmAy1Nfd73T3CXefGN7As7YIIdaWZZ3dWst7nwPwlLt/conpPgDvab9+D4Cvn/vhCSHOFSuJensjgHcDeNzMHmm3fRjAxwF8xczeC+AAgDuW21DTHYu1cFhOPpIzrlQID7MRCfFZqPKIocVKpGxUpHwOCynqy3PpKovlBMtFcteNcamskedSX64YlppGR/n26hHJq0by/wFArsFlNGP9IhJarc4/M3MuKXnkPCjlw+Wa+ge59Daykc/v2LZw7jcAyCLRcht28jHu3B0ei2f8mAtEYuM9VuDs7v6DyDbevFx/IcT5gZ6gEyIR5OxCJIKcXYhEkLMLkQhydiESocMJJ4EKU2QiIWx1hCWZej0i/VhEjukKyzEAkDW4NNRshrdZich8lVrkuCKzPzDE5bx8iUfLFbt7gu1dRZ7MsboQSZiZi0SpVReordAkkYp8euER4ahR5/LgwiIfRzUX/qxPn56nfRZrfHu9feH5BYCTp3mprEadH3gfiZabn+d9FhbCjsTOUUB3diGSQc4uRCLI2YVIBDm7EIkgZxciEeTsQiRCR6W3rAnM18ISSiMS8VQohq9J5TKvNTbQx5MGbtrAI568GKkRR+rHLVYiEXYLi9SW5SPJLZuR5IslLlFNz80G2w+8wHOBjozxPAP5njlq84xHxDVJHb5yhc9HpRZLEso/l3okWWmDfJ4vHuQ17GbK4TkEgBw5FwFgdo7PVc653LtYCY/xmWd5XbmZ2fAxZ5LehBBydiESQc4uRCLI2YVIBDm7EInQ0dX4ZjNDmaxYlop8tbKrEM4JViqF860BQM74oVnEVqvxvHALC+EAiXokyCGSHi1mQt35any+m1+jp6fDq+5/+83v0j6DG26jtvGLIvn1IvnpGiSv3cIiX3Fn5wYANBp8PoqlSE6+Zth29Pgp2qcWCYYqkLJLy/XLIkpDgwSBHXnxCO1z6lR4rhqRMejOLkQiyNmFSAQ5uxCJIGcXIhHk7EIkgpxdiERYVnozsx0AvoBWSWYHcKe7f9rMPgbgfQBOtN/6YXf/ZmxbOTP0kPxv3d1ceiuR4IPukXDuLgDoKkQCDxa5vDYzzfOILZJcZ/39g7SPR5KuMSkPQPQy3DfUS22vef1rg+37Dz5D+3z2z/6C2n7tpmup7fJX76C2oS1hWdSd588r5HnwkoHPY4MEVwHAiZlwsNSzz+2nfWJzn0Uk0azJA5QWazxYqqc/vMNimbvn/GJ4e7EcdCvR2RsA/tjdHzazAQAPmdl32rZPuft/X8E2hBDrzEpqvR0FcLT9umxmTwHYttYDE0KcW17Rb3YzGwfwGgAPtps+YGaPmdldZsbLhAoh1p0VO7uZ9QP4KoAPuvssgM8A2A3gGrTu/J8g/faa2T4z2zc7zXN1CyHWlhU5u5kV0XL0L7r71wDA3Y+7e+buTQCfBRBcyXH3O919wt0nBod5/WohxNqyrLObmQH4HICn3P2TS9rHlrztHQCeOPfDE0KcK1ayGv9GAO8G8LiZPdJu+zCAd5nZNWjJcfsBvH+5DRmAIpFQchmXJrrz4ZI7Hokb80g5qWbG+3V1cfmnVArLeT09/BtLucwjubKMS2/dvXwcDXD5Z/dlu4Ltl75qC+3zt/d8n9ru/csfUtst82GZDwAm3hweRzPHT7lYiSQzfl9y55LX5GQ4uq08x+XXHbt2Ult5rkxtxyZPUFshctxDG8K2XHEz7TM3H/5J3Iyc9ytZjf8BECzCFdXUhRDnF3qCTohEkLMLkQhydiESQc4uRCLI2YVIhI4mnHRvokESOjZqkWgdEijV2xuW5ACgGElgmY/IILHEl6wEUbXCkwk2a5EEgBlPlNio8n71Ot/f6amw1HTDTVfQPtfdOEFtP/7+k9T2woFD1Lb1YDjqraufJ7AcGhqltlqkPNjsLH8yszwXljcv2bOb9hke3kptgyM8am96hpeNyud4v52XhENNKgv8XrxQe+XSm+7sQiSCnF2IRJCzC5EIcnYhEkHOLkQiyNmFSISOSm9Z0zG/EK4PVm/wumH1RviaVKvxaKfeHi7lZVmsNhvfZj4fnq4sIq/VF/lxLczx6LXjh3ktsi2bNlLbyNBweF8RuW7XqzZR21SF20oFfq+YIypUPcePudQTSebYiEizXTwB55Zt24Pt4xfxOoG1SALLSPAdanUur83M8kSmff1hCbmnO3LMvUS2zfPzV3d2IRJBzi5EIsjZhUgEObsQiSBnFyIR5OxCJEJnpbesiemZxbPoF454WliMJChscvmkWuFjYPIaAHR1h5NAlkpcxplb4IkN6xE5aWB0gNpu+LXXUdvO8bFge67I52NglCfMvOb1e6itt8Qlr8HBcP27KiJzH4lGtIjM1xWJKGM5SSsk+hIA6nUul3b38EjLgQH+mZW6+DmSL4WPu1blcinbXi6iDerOLkQiyNmFSAQ5uxCJIGcXIhHk7EIkwrKr8WbWDeABAF3t9/+Vu3/UzC4E8GUAGwA8BODd7s4ThQEAcmginOOtWOD52JAL2+bm+cpuVuMrmfNzPGdZPrLqOzIcXvXNF3ipJkRWYbtZMAOArWSFFgD6NvKSUj0D4fFnTX5chSYfY2GEj7Gvi6/iFwvh8dcX+eeSy3gQR6w01GyZB5lUyXkQW90vRObeeYo3dHVH5rHI53F+ITzGXC6i8pTDakKWrS4HXRXAb7j71WiVZ77VzK4H8CcAPuXuFwOYAvDeFWxLCLFOLOvs3uLMraTY/ucAfgPAX7Xb7wbw9jUZoRDinLDS+uz5dgXXSQDfAfAcgGl3P/OkxiEA4Xy4QojzghU5u7tn7n4NgO0ArgVw+Up3YGZ7zWyfme2bj+T3FkKsLa9oNd7dpwF8D8ANAIbN7MxKxnYAh0mfO919wt0n+gb5go4QYm1Z1tnNbJOZDbdf9wB4C4Cn0HL6f95+23sAfH2tBimEWD0rCYQZA3C3meXRujh8xd2/YWY/B/BlM/uvAH4G4HPLbcjdUauHIxMakeCDRZLHbX4+XNoHALpi5Z8K/BtGJA4GbmHprdrgslA1IoXUSQkfAHDwbXYN8kE2LCzJ1Cp8e1mVj7E6z6WyWp4rrUxKPXl6kvYZHQnnzwOAJim9BQAnj56gtkotPMaNY7zEU2ZcAjw9O0VtNOoGQC5yYh09Et5msxnJo9gMf56NyLm4rLO7+2MAXhNofx6t3+9CiF8C9ASdEIkgZxciEeTsQiSCnF2IRJCzC5EI5hFJ45zvzOwEgAPtPzcCONmxnXM0jpeicbyUX7Zx7HL3YM2ujjr7S3Zsts/dJ9Zl5xqHxpHgOPQ1XohEkLMLkQjr6ex3ruO+l6JxvBSN46X8yoxj3X6zCyE6i77GC5EI6+LsZnarmf3CzJ41sw+txxja49hvZo+b2SNmtq+D+73LzCbN7IklbaNm9h0ze6b9/8g6jeNjZna4PSePmNltHRjHDjP7npn93MyeNLM/ard3dE4i4+jonJhZt5n9xMwebY/jv7TbLzSzB9t+c4+Z8dDOEO7e0X8A8miltboIQAnAowD2dHoc7bHsB7BxHfZ7E4DXAnhiSdt/A/Ch9usPAfiTdRrHxwD8xw7PxxiA17ZfDwB4GsCeTs9JZBwdnRMABqC//boI4EEA1wP4CoB3ttv/HMAfvJLtrsed/VoAz7r7895KPf1lALevwzjWDXd/AMDplzXfjlbiTqBDCTzJODqOux9194fbr8toJUfZhg7PSWQcHcVbnPMkr+vh7NsAHFzy93omq3QA3zazh8xs7zqN4Qxb3P1o+/UxAFvWcSwfMLPH2l/z1/znxFLMbByt/AkPYh3n5GXjADo8J2uR5DX1Bbob3f21AN4K4A/N7Kb1HhDQurIjlvZkbfkMgN1o1Qg4CuATndqxmfUD+CqAD7r77FJbJ+ckMI6Oz4mvIskrYz2c/TCAHUv+pskq1xp3P9z+fxLAvVjfzDvHzWwMANr/8/xNa4i7H2+faE0An0WH5sTMimg52Bfd/Wvt5o7PSWgc6zUn7X2/4iSvjPVw9p8CuKS9slgC8E4A93V6EGbWZ2YDZ14DuAXAE/Fea8p9aCXuBNYxgecZ52rzDnRgTszM0Mph+JS7f3KJqaNzwsbR6TlZsySvnVphfNlq421orXQ+B+A/r9MYLkJLCXgUwJOdHAeAL6H1dbCO1m+v96JVM+9+AM8A+C6A0XUax18AeBzAY2g521gHxnEjWl/RHwPwSPvfbZ2ek8g4OjonAF6NVhLXx9C6sHxkyTn7EwDPAvhfALpeyXb1BJ0QiZD6Ap0QySBnFyIR5OxCJIKcXYhEkLMLkQhy9rQYx/o+SyDWETm7WCkrKQIqzmPk7OmRR+uRzycBfBtAD1rPfP8YrYc47gVwJtDjfwP4HwD2AfgjAL+P1jeDRwE8sGR7f4rWk5GPAXh/B45BnAVy9vS4BMCfAbgSwDSA3wPwBQD/Ca0ntx4H8NEl7y8BmEAr+OMjAP4ZgKsB/E7b/l4AMwBe3/73PgAXrvVBiFeOnD09XkDrMVAAeAitaK5hAN9vt92NVlKLM9yz5PUPAXweLYc+U6z+FgD/qr3NB9F6xPWSNRi3WCX6HZYe1SWvM7QcPVmxrFcAAACmSURBVMb8ktf/DsB1AH4LrQvF69DKqvLvAXzrHI5RrAG6s4sZAFMA3tT++934f3f5l7Mbrbv3RwCcQCtU+VsA/gCtbCoAcCmAvrUarDh7dGcXQCtc8s8B9AJ4HsC/Ie/7U7S+ohta0WiPorUoNw7g4Xb7CXQgpZZ45SjqTYhE0Nd4IRJBzi5EIsjZhUgEObsQiSBnFyIR5OxCJIKcXYhEkLMLkQj/B7ZT8Pg9QfvTAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**CNN Architecture**\n",
        "\n",
        "A common architecture for a CNN is a stack of Conv2D and MaxPooling2D layers followed by a few denesly connected layers. To idea is that the stack of convolutional and maxPooling layers extract the features from the image. Then these features are flattened and fed to densly connected layers that determine the class of an image based on the presence of features.\n",
        "\n",
        "We will start by building the **Convolutional Base.**\n",
        "\n",
        "**Layer 1**\n",
        "\n",
        "The input shape of our data will be 32, 32, 3 and we will process 32 filters of size 3x3 over our input data. We will also apply the activation function relu to the output of each convolution operation.\n",
        "\n",
        "**Layer 2**\n",
        "\n",
        "This layer will perform the max pooling operation using 2x2 samples and a stride of 2.\n",
        "\n",
        "**Other Layers**\n",
        "\n",
        "The next set of layers do very similar things but take as input the feature map from the previous layer. They also increase the frequency of filters from 32 to 64. We can do this as our data shrinks in spacial dimensions as it passed through the layers, meaning we can afford (computationally) to add more depth."
      ],
      "metadata": {
        "id": "VcZhy-WCQjVd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))\n",
        "model.add(layers.MaxPooling2D((2, 2)))\n",
        "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
        "model.add(layers.MaxPooling2D((2, 2)))\n",
        "model.add(layers.Conv2D(64, (3, 3), activation='relu'))"
      ],
      "metadata": {
        "id": "ANjS1kefQnxn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.summary()  # let's have a look at our model so far"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "blIgIHV4QxGW",
        "outputId": "d056bdaa-3634-4685-a480-dbdc763bdcb8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " conv2d (Conv2D)             (None, 30, 30, 32)        896       \n",
            "                                                                 \n",
            " max_pooling2d (MaxPooling2D  (None, 15, 15, 32)       0         \n",
            " )                                                               \n",
            "                                                                 \n",
            " conv2d_1 (Conv2D)           (None, 13, 13, 64)        18496     \n",
            "                                                                 \n",
            " max_pooling2d_1 (MaxPooling  (None, 6, 6, 64)         0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_2 (Conv2D)           (None, 4, 4, 64)          36928     \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 56,320\n",
            "Trainable params: 56,320\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "After looking at the summary you should notice that the depth of our image increases but the spacial dimensions reduce drastically."
      ],
      "metadata": {
        "id": "tQpFLHm8Q0xI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Adding Dense Layers**\n",
        "So far, we have just completed the convolutional base. Now we need to take these extracted features and add a way to classify them. This is why we add the following layers to our model."
      ],
      "metadata": {
        "id": "1e-YnAEiQ2nw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.add(layers.Flatten())\n",
        "model.add(layers.Dense(64, activation='relu'))\n",
        "model.add(layers.Dense(10))"
      ],
      "metadata": {
        "id": "d3F3o7D-REEo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vPZBODeZRF5n",
        "outputId": "582888cd-3c24-4007-a665-f26b0930bf9c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " conv2d (Conv2D)             (None, 30, 30, 32)        896       \n",
            "                                                                 \n",
            " max_pooling2d (MaxPooling2D  (None, 15, 15, 32)       0         \n",
            " )                                                               \n",
            "                                                                 \n",
            " conv2d_1 (Conv2D)           (None, 13, 13, 64)        18496     \n",
            "                                                                 \n",
            " max_pooling2d_1 (MaxPooling  (None, 6, 6, 64)         0         \n",
            " 2D)                                                             \n",
            "                                                                 \n",
            " conv2d_2 (Conv2D)           (None, 4, 4, 64)          36928     \n",
            "                                                                 \n",
            " flatten_1 (Flatten)         (None, 1024)              0         \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 64)                65600     \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 10)                650       \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 122,570\n",
            "Trainable params: 122,570\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can see that the flatten layer changes the shape of our data so that we can feed it to the 64-node dense layer, follwed by the final output layer of 10 neurons (one for each class)."
      ],
      "metadata": {
        "id": "JP-3rBdERLOB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training**\n",
        "Now we will train and compile the model using the recommended hyper paramaters from tensorflow.\n",
        "\n",
        "Note: This will take much longer than previous models!"
      ],
      "metadata": {
        "id": "a-w_Bp8VRQE4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(optimizer='adam',\n",
        "              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "history = model.fit(train_images, train_labels, epochs=4, \n",
        "                    validation_data=(test_images, test_labels))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sX2iQceJRRYa",
        "outputId": "651feafe-b0a3-48b0-fc40-957d6820f755"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/4\n",
            "1563/1563 [==============================] - 89s 56ms/step - loss: 1.5519 - accuracy: 0.4317 - val_loss: 1.2586 - val_accuracy: 0.5448\n",
            "Epoch 2/4\n",
            "1563/1563 [==============================] - 79s 51ms/step - loss: 1.1790 - accuracy: 0.5791 - val_loss: 1.1477 - val_accuracy: 0.5946\n",
            "Epoch 3/4\n",
            "1563/1563 [==============================] - 79s 50ms/step - loss: 1.0285 - accuracy: 0.6381 - val_loss: 0.9980 - val_accuracy: 0.6466\n",
            "Epoch 4/4\n",
            "1563/1563 [==============================] - 80s 51ms/step - loss: 0.9342 - accuracy: 0.6706 - val_loss: 0.9678 - val_accuracy: 0.6617\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Evaluating the Model**\n",
        "\n",
        "We can determine how well the model performed by looking at it's performance on the test data set."
      ],
      "metadata": {
        "id": "h_7MB_2yRUsV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)\n",
        "print(test_acc)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WVOWhrt9RWoR",
        "outputId": "90a27a75-3341-4ff5-a1a0-5d36a8382d26"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "313/313 - 4s - loss: 0.9678 - accuracy: 0.6617 - 4s/epoch - 13ms/step\n",
            "0.6617000102996826\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**NLP**\n",
        "\n",
        "Natural Language Processing (or NLP for short) is a discipline in computing that deals with the communication between natural (human) languages and computer languages. A common example of NLP is something like spellcheck or autocomplete. Essentially NLP is the field that focuses on how computers can understand and/or process natural/human languages.\n",
        "\n",
        "Recurrent Neural Networks\n",
        "In this tutorial we will introduce a new kind of neural network that is much more capable of processing sequential data such as text or characters called a recurrent neural network (RNN for short).\n",
        "\n",
        "We will learn how to use a reccurent neural network to do the following:\n",
        "\n",
        "Sentiment Analysis\n",
        "Character Generation\n",
        "RNN's are complex and come in many different forms so in this tutorial we wil focus on how they work and the kind of problems they are best suited for.\n",
        "\n"
      ],
      "metadata": {
        "id": "qGmxSB1UUJTA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Movie Review Dataset**\n",
        "\n",
        "Well start by loading in the IMDB movie review dataset from keras. This dataset contains 25,000 reviews from IMDB where each one is already preprocessed and has a label as either positive or negative. Each review is encoded by integers that represents how common a word is in the entire dataset. For example, a word encoded by the integer 3 means that it is the 3rd most common word in the dataset."
      ],
      "metadata": {
        "id": "hNfw4PnsUUML"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.datasets import imdb\n",
        "from keras.preprocessing import sequence\n",
        "import keras\n",
        "import tensorflow as tf\n",
        "import os\n",
        "import numpy as np\n",
        "\n",
        "VOCAB_SIZE = 88584\n",
        "\n",
        "MAXLEN = 250\n",
        "BATCH_SIZE = 64\n",
        "\n",
        "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words = VOCAB_SIZE)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jMCrskldUIKN",
        "outputId": "8638f2fb-ec8f-45fd-960a-ca034eb971ed"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n",
            "17465344/17464789 [==============================] - 0s 0us/step\n",
            "17473536/17464789 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Lets look at one review\n",
        "train_data[1]"
      ],
      "metadata": {
        "id": "Exnj2PC8UgRH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**More Preprocessing**\n",
        "\n",
        "If we have a look at some of our loaded in reviews, we'll notice that they are different lengths. This is an issue. We cannot pass different length data into our neural network. Therefore, we must make each review the same length. To do this we will follow the procedure below:\n",
        "\n",
        "if the review is greater than 250 words then trim off the extra words\n",
        "if the review is less than 250 words add the necessary amount of 0's to make it equal to 250.\n",
        "Luckily for us keras has a function that can do this for us:"
      ],
      "metadata": {
        "id": "C14HxJm6UkoE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_data = sequence.pad_sequences(train_data, MAXLEN)\n",
        "test_data = sequence.pad_sequences(test_data, MAXLEN)"
      ],
      "metadata": {
        "id": "7NYowASVUm8v"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Creating the Model**\n",
        "\n",
        "Now it's time to create the model. We'll use a word embedding layer as the first layer in our model and add a LSTM layer afterwards that feeds into a dense node to get our predicted sentiment.\n",
        "\n",
        "32 stands for the output dimension of the vectors generated by the embedding layer. We can change this value if we'd like!"
      ],
      "metadata": {
        "id": "XYQOf4WBUox_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Embedding(VOCAB_SIZE, 32),\n",
        "    tf.keras.layers.LSTM(32),\n",
        "    tf.keras.layers.Dense(1, activation=\"sigmoid\")\n",
        "])\n",
        "\n",
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k6RgSKY6UrQz",
        "outputId": "42b7487b-4cbe-4640-fe48-62e0e511c34f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " embedding (Embedding)       (None, None, 32)          2834688   \n",
            "                                                                 \n",
            " lstm (LSTM)                 (None, 32)                8320      \n",
            "                                                                 \n",
            " dense_4 (Dense)             (None, 1)                 33        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 2,843,041\n",
            "Trainable params: 2,843,041\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training**\n",
        "Now it's time to compile and train the model."
      ],
      "metadata": {
        "id": "z3G9RJxmUwyz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(loss=\"binary_crossentropy\",optimizer=\"rmsprop\",metrics=['acc'])\n",
        "\n",
        "history = model.fit(train_data, train_labels, epochs=10, validation_split=0.2)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Mb18Cq0ZUvyg",
        "outputId": "cd348af0-b778-42b5-e960-61181eadbb3f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "625/625 [==============================] - 80s 124ms/step - loss: 0.4192 - acc: 0.8091 - val_loss: 0.2824 - val_acc: 0.8854\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 71s 113ms/step - loss: 0.2397 - acc: 0.9101 - val_loss: 0.2991 - val_acc: 0.8766\n",
            "Epoch 3/10\n",
            "625/625 [==============================] - 65s 104ms/step - loss: 0.1883 - acc: 0.9307 - val_loss: 0.2669 - val_acc: 0.8938\n",
            "Epoch 4/10\n",
            "625/625 [==============================] - 65s 104ms/step - loss: 0.1527 - acc: 0.9452 - val_loss: 0.3311 - val_acc: 0.8850\n",
            "Epoch 5/10\n",
            "625/625 [==============================] - 66s 105ms/step - loss: 0.1313 - acc: 0.9536 - val_loss: 0.2956 - val_acc: 0.8810\n",
            "Epoch 6/10\n",
            "625/625 [==============================] - 65s 103ms/step - loss: 0.1148 - acc: 0.9610 - val_loss: 0.3628 - val_acc: 0.8774\n",
            "Epoch 7/10\n",
            "625/625 [==============================] - 65s 104ms/step - loss: 0.1010 - acc: 0.9661 - val_loss: 0.3037 - val_acc: 0.8856\n",
            "Epoch 8/10\n",
            "625/625 [==============================] - 64s 103ms/step - loss: 0.0858 - acc: 0.9714 - val_loss: 0.3616 - val_acc: 0.8834\n",
            "Epoch 9/10\n",
            "625/625 [==============================] - 64s 102ms/step - loss: 0.0764 - acc: 0.9742 - val_loss: 0.3292 - val_acc: 0.8838\n",
            "Epoch 10/10\n",
            "625/625 [==============================] - 65s 104ms/step - loss: 0.0669 - acc: 0.9785 - val_loss: 0.3808 - val_acc: 0.8810\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And we'll evaluate the model on our training data to see how well it performs."
      ],
      "metadata": {
        "id": "p2Semn2GU1SQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "results = model.evaluate(test_data, test_labels)\n",
        "print(results)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MejihhrhU20C",
        "outputId": "3210b3db-e58c-4f7e-f19c-cc547d5b1656"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "782/782 [==============================] - 21s 27ms/step - loss: 0.4533 - acc: 0.8550\n",
            "[0.45329999923706055, 0.8550400137901306]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Making Predictions**\n",
        "\n",
        "Now let’s use our network to make predictions on our own reviews.\n",
        "\n",
        "Since our reviews are encoded well need to convert any review that we write into that form so the network can understand it. To do that well load the encodings from the dataset and use them to encode our own data."
      ],
      "metadata": {
        "id": "5wYAbfAHU5Ex"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "word_index = imdb.get_word_index()\n",
        "\n",
        "def encode_text(text):\n",
        "  tokens = keras.preprocessing.text.text_to_word_sequence(text)\n",
        "  tokens = [word_index[word] if word in word_index else 0 for word in tokens]\n",
        "  return sequence.pad_sequences([tokens], MAXLEN)[0]\n",
        "\n",
        "text = \"that movie was just amazing, so amazing\"\n",
        "encoded = encode_text(text)\n",
        "print(encoded)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IpkO6YRlVAPA",
        "outputId": "62542de0-4e53-4fc2-8252-ab22bb145190"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
            "1646592/1641221 [==============================] - 0s 0us/step\n",
            "1654784/1641221 [==============================] - 0s 0us/step\n",
            "[  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
            "   0   0   0   0   0   0   0   0   0  12  17  13  40 477  35 477]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# while were at it lets make a decode function\n",
        "\n",
        "reverse_word_index = {value: key for (key, value) in word_index.items()}\n",
        "\n",
        "def decode_integers(integers):\n",
        "    PAD = 0\n",
        "    text = \"\"\n",
        "    for num in integers:\n",
        "      if num != PAD:\n",
        "        text += reverse_word_index[num] + \" \"\n",
        "\n",
        "    return text[:-1]\n",
        "  \n",
        "print(decode_integers(encoded))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cA3WcSIJVH-D",
        "outputId": "cdf4ef22-f95f-4b35-8cae-5b3c2d2a9db2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "that movie was just amazing so amazing\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# now time to make a prediction\n",
        "\n",
        "def predict(text):\n",
        "  encoded_text = encode_text(text)\n",
        "  pred = np.zeros((1,250))\n",
        "  pred[0] = encoded_text\n",
        "  result = model.predict(pred) \n",
        "  print(result[0])\n",
        "\n",
        "positive_review = \"That movie was! really loved it and would great watch it again because it was amazingly great\"\n",
        "predict(positive_review)\n",
        "\n",
        "negative_review = \"that movie really sucked. I hated it and wouldn't watch it again. Was one of the worst things I've ever watched\"\n",
        "predict(negative_review)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K81sz-dzVKMR",
        "outputId": "3fc52c7f-8496-49db-f92f-85c0eb0b5be6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0.9134413]\n",
            "[0.49175972]\n"
          ]
        }
      ]
    }
  ]
}