2a - unit 05.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "edd76bd1",
   "metadata": {},
   "source": [
    "# Exercise: Assessing a logistic regression model\n",
    "\n",
    "In the previous exercise, we fit a simple logistic regression model to predict the chance of an avalanche. This time, we will create the same model and take a deeper look at how to best understand the mistakes that it makes.\n",
    "\n",
    "## Data visualization\n",
    "\n",
    "Let's remind ourselves of our data. Remember we are planning to train a model that can predict avalanches based on the number of weak layers of snow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4eec3f24",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "   Unnamed: 0  avalanche  no_visitors  surface_hoar  fresh_thickness  \\\n0           0          1            2      6.624345         4.388244   \n1           1          1            2      3.927031         5.257594   \n2           2          1            1      2.707691         3.584448   \n3           3          1            9      5.631902         5.376657   \n4           4          1            4      6.704904         5.924346   \n\n        wind  weak_layers  tracked_out  \n0 -11.126870            6            1  \n1 -68.375784           10            0  \n2  45.433176            6            1  \n3  25.045401            3            1  \n4 -42.194450            8            1  \n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "alignmentgroup": "True",
         "hovertemplate": "avalanche=%{x}<br>weak layers=%{y}<extra></extra>",
         "legendgroup": "",
         "marker": {
          "color": "#636efa"
         },
         "name": "",
         "notched": false,
         "offsetgroup": "",
         "orientation": "v",
         "showlegend": false,
         "type": "box",
         "x": [
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          0
         ],
         "x0": " ",
         "xaxis": "x",
         "y": [
          6,
          10,
          6,
          3,
          8,
          0,
          7,
          6,
          10,
          3,
          4,
          0,
          1,
          7,
          5,
          10,
          6,
          7,
          8,
          2,
          0,
          10,
          8,
          3,
          7,
          2,
          6,
          9,
          4,
          9,
          6,
          2,
          2,
          7,
          9,
          4,
          2,
          2,
          10,
          9,
          4,
          7,
          2,
          9,
          10,
          9,
          0,
          5,
          5,
          10,
          5,
          1,
          2,
          2,
          1,
          6,
          2,
          3,
          10,
          3,
          0,
          10,
          9,
          0,
          6,
          6,
          0,
          0,
          3,
          1,
          3,
          1,
          10,
          7,
          2,
          5,
          1,
          4,
          9,
          8,
          4,
          1,
          5,
          0,
          1,
          7,
          3,
          9,
          10,
          5,
          8,
          3,
          5,
          5,
          9,
          0,
          3,
          1,
          8,
          6,
          2,
          3,
          1,
          8,
          9,
          3,
          4,
          2,
          9,
          8,
          6,
          9,
          10,
          8,
          5,
          4,
          10,
          9,
          9,
          5,
          9,
          7,
          8,
          5,
          7,
          2,
          10,
          10,
          3,
          7,
          4,
          6,
          5,
          10,
          2,
          7,
          2,
          10,
          0,
          2,
          8,
          7,
          4,
          5,
          4,
          10,
          1,
          5,
          8,
          9,
          1,
          7,
          8,
          10,
          5,
          10,
          8,
          2,
          5,
          0,
          4,
          0,
          8,
          6,
          3,
          2,
          6,
          1,
          1,
          8,
          5,
          3,
          5,
          10,
          8,
          5,
          7,
          2,
          3,
          9,
          4,
          5,
          4,
          5,
          9,
          10,
          5,
          7,
          7,
          0,
          8,
          5,
          7,
          7,
          8,
          8,
          2,
          9,
          6,
          0,
          4,
          5,
          5,
          10,
          6,
          6,
          1,
          7,
          1,
          5,
          6,
          2,
          9,
          9,
          6,
          9,
          9,
          10,
          6,
          5,
          8,
          0,
          7,
          0,
          9,
          3,
          2,
          8,
          10,
          0,
          5,
          5,
          3,
          9,
          2,
          5,
          7,
          4,
          0,
          9,
          0,
          2,
          1,
          8,
          6,
          2,
          5,
          0,
          10,
          4,
          10,
          9,
          8,
          8,
          8,
          3,
          10,
          0,
          8,
          0,
          3,
          2,
          3,
          3,
          10,
          8,
          9,
          0,
          5,
          5,
          3,
          10,
          2,
          5,
          10,
          7,
          3,
          3,
          9,
          9,
          7,
          8,
          9,
          7,
          1,
          6,
          0,
          5,
          3,
          7,
          5,
          7,
          5,
          10,
          6,
          5,
          0,
          5,
          0,
          10,
          2,
          2,
          10,
          6,
          7,
          0,
          9,
          8,
          5,
          7,
          9,
          7,
          10,
          10,
          4,
          3,
          5,
          2,
          2,
          4,
          2,
          4,
          9,
          0,
          7,
          10,
          0,
          7,
          1,
          9,
          2,
          7,
          7,
          9,
          8,
          2,
          2,
          5,
          3,
          4,
          8,
          1,
          9,
          2,
          2,
          6,
          10,
          9,
          3,
          2,
          0,
          2,
          0,
          7,
          5,
          10,
          3,
          10,
          10,
          7,
          6,
          10,
          3,
          9,
          1,
          8,
          7,
          3,
          9,
          1,
          4,
          9,
          3,
          6,
          7,
          6,
          8,
          3,
          10,
          2,
          8,
          5,
          5,
          4,
          3,
          10,
          4,
          1,
          10,
          7,
          9,
          2,
          3,
          9,
          6,
          0,
          0,
          5,
          4,
          6,
          4,
          9,
          5,
          0,
          8,
          3,
          1,
          5,
          5,
          7,
          8,
          4,
          1,
          9,
          3,
          8,
          6,
          5,
          6,
          7,
          5,
          8,
          7,
          7,
          8,
          6,
          0,
          2,
          1,
          6,
          10,
          1,
          4,
          1,
          10,
          8,
          8,
          9,
          2,
          6,
          9,
          4,
          9,
          3,
          10,
          9,
          8,
          4,
          9,
          6,
          9,
          0,
          8,
          10,
          0,
          6,
          2,
          10,
          4,
          9,
          2,
          0,
          0,
          2,
          7,
          7,
          6,
          5,
          6,
          6,
          4,
          1,
          8,
          4,
          6,
          5,
          9,
          7,
          8,
          9,
          0,
          2,
          6,
          10,
          8,
          6,
          6,
          3,
          10,
          10,
          4,
          0,
          2,
          2,
          5,
          4,
          4,
          0,
          4,
          4,
          8,
          1,
          10,
          5,
          3,
          5,
          5,
          5,
          3,
          10,
          1,
          1,
          7,
          10,
          7,
          2,
          7,
          3,
          3,
          6,
          0,
          8,
          0,
          4,
          8,
          9,
          9,
          8,
          3,
          4,
          5,
          7,
          1,
          2,
          6,
          8,
          2,
          6,
          8,
          3,
          3,
          5,
          9,
          2,
          9,
          0,
          5,
          5,
          9,
          7,
          10,
          5,
          10,
          9,
          10,
          5,
          9,
          1,
          8,
          10,
          7,
          8,
          8,
          2,
          0,
          7,
          4,
          6,
          6,
          5,
          0,
          8,
          2,
          4,
          1,
          2,
          5,
          0,
          1,
          9,
          5,
          4,
          8,
          8,
          5,
          3,
          8,
          7,
          5,
          3,
          10,
          7,
          1,
          5,
          6,
          3,
          10,
          8,
          3,
          10,
          3,
          5,
          4,
          9,
          3,
          9,
          5,
          5,
          6,
          0,
          3,
          1,
          7,
          6,
          5,
          8,
          10,
          1,
          8,
          7,
          3,
          7,
          0,
          8,
          3,
          2,
          3,
          6,
          10,
          3,
          8,
          8,
          9,
          6,
          5,
          10,
          0,
          0,
          0,
          3,
          10,
          7,
          7,
          10,
          2,
          6,
          5,
          5,
          5,
          7,
          1,
          9,
          5,
          5,
          4,
          0,
          8,
          9,
          1,
          7,
          5,
          7,
          9,
          2,
          9,
          9,
          10,
          9,
          7,
          4,
          7,
          9,
          8,
          8,
          10,
          10,
          9,
          2,
          0,
          5,
          10,
          0,
          10,
          9,
          6,
          8,
          3,
          6,
          2,
          10,
          10,
          6,
          4,
          3,
          2,
          7,
          5,
          2,
          6,
          7,
          0,
          1,
          9,
          6,
          0,
          3,
          9,
          10,
          0,
          4,
          0,
          7,
          6,
          7,
          1,
          5,
          8,
          9,
          3,
          3,
          3,
          2,
          5,
          0,
          9,
          8,
          6,
          8,
          4,
          9,
          8,
          4,
          8,
          3,
          9,
          6,
          3,
          9,
          2,
          2,
          4,
          5,
          9,
          5,
          6,
          1,
          3,
          8,
          6,
          2,
          8,
          3,
          6,
          5,
          5,
          4,
          2,
          5,
          0,
          2,
          1,
          10,
          6,
          7,
          3,
          10,
          8,
          7,
          2,
          4,
          6,
          4,
          7,
          2,
          9,
          5,
          2,
          9,
          5,
          9,
          7,
          1,
          8,
          6,
          10,
          4,
          3,
          8,
          1,
          2,
          0,
          4,
          7,
          1,
          1,
          5,
          3,
          10,
          0,
          2,
          2,
          9,
          7,
          4,
          5,
          8,
          1,
          2,
          0,
          5,
          0,
          9,
          4,
          10,
          1,
          9,
          2,
          1,
          8,
          10,
          3,
          0,
          2,
          9,
          5,
          2,
          3,
          4,
          2,
          3,
          4,
          7,
          1,
          3,
          10,
          5,
          1,
          4,
          5,
          9,
          1,
          4,
          1,
          8,
          9,
          6,
          1,
          0,
          9,
          5,
          7,
          2,
          8,
          7,
          10,
          2,
          1,
          3,
          9,
          8,
          10,
          5,
          10,
          10,
          3,
          6,
          4,
          3,
          0,
          9,
          5,
          3,
          0,
          10,
          4,
          2,
          6,
          2,
          10,
          9,
          1,
          5,
          7,
          4,
          2,
          5,
          0,
          7,
          0,
          0,
          4,
          8,
          8,
          6,
          7,
          6,
          10,
          9,
          10,
          8,
          10,
          10,
          7,
          9,
          0,
          5,
          4,
          2,
          8,
          4,
          2,
          2,
          9,
          7,
          3,
          0,
          4,
          7,
          10,
          6,
          7,
          10,
          0,
          1,
          9,
          3,
          9,
          7,
          7,
          2,
          2,
          8,
          6,
          7,
          9,
          5,
          10,
          6,
          0,
          1,
          9,
          9,
          4,
          9,
          5,
          4,
          1,
          9,
          1,
          4,
          10,
          4,
          9,
          7,
          8,
          10,
          7,
          4,
          0,
          4,
          3,
          10,
          6,
          0,
          8,
          7,
          7,
          1,
          5,
          8,
          8,
          10,
          2,
          3,
          6,
          7,
          2,
          7,
          8,
          4,
          0,
          8,
          1,
          0,
          8,
          10,
          9,
          10,
          8,
          3,
          7,
          10,
          9,
          3,
          0,
          6,
          8,
          5,
          7,
          9,
          10,
          4,
          3,
          8,
          9,
          8,
          6,
          5,
          2,
          2,
          7,
          8,
          10,
          7,
          5,
          8,
          8,
          1,
          3,
          8,
          6,
          1,
          10,
          3,
          2,
          4,
          4,
          7,
          6,
          0,
          5,
          8,
          7,
          5,
          10,
          5,
          0,
          9,
          8,
          5,
          0,
          6,
          9,
          9,
          1,
          7,
          9,
          9,
          8,
          8,
          10,
          1,
          3,
          7,
          4,
          3,
          3,
          10,
          8,
          8,
          1,
          6,
          3,
          4,
          1,
          3,
          10,
          5,
          8,
          7,
          8,
          8,
          9,
          4,
          3,
          1,
          1,
          6,
          5,
          8,
          0,
          0,
          2,
          5,
          9,
          1,
          10,
          5,
          5,
          5,
          3,
          2,
          5,
          5,
          2
         ],
         "y0": " ",
         "yaxis": "y"
        }
       ],
       "layout": {
        "boxmode": "group",
        "legend": {
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "avalanche"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "weak layers"
         }
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "import pandas\n",
    "import graphing # custom graphing code. See our GitHub repo for details\n",
    "\n",
    "#Import the data from the .csv file\n",
    "dataset = pandas.read_csv('Data/avalanche.csv', delimiter=\"\\t\")\n",
    "\n",
    "#Let's have a look at the data and the relationship we are going to model\n",
    "print(dataset.head())\n",
    "\n",
    "graphing.box_and_whisker(dataset, label_x=\"avalanche\", label_y=\"weak_layers\")"
   ]
  },
  {
   "source": [
    "It seems that avalanches are associated with having more weak layers of snow. That said, some days many weak layers have been recorded, but no avalanche occurred. This means our model will have difficulty being extremely accurate using only this feature. Let's continue though, and come back to this in a future exercise.  \n",
    "\n",
    "Before we begin, we need to split our dataset into training and test sets. We will train on the _training_ set, and test on (you guessed it) the _test_ set."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Split the dataset in an 75/25 train/test ratio. \n",
    "train, test = train_test_split(dataset, test_size=0.25, random_state=10)\n",
    "\n",
    "print(\"Train size:\", train.shape[0])\n",
    "print(\"Test size:\", test.shape[0])"
   ],
   "cell_type": "code",
   "metadata": {},
   "execution_count": 2,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Train size: 821\nTest size: 274\n"
     ]
    }
   ]
  },
  {
   "source": [
    "## Fitting a model\n",
    "\n",
    "Let's fit a simple logistic regression model using *log-loss* as a cost function. This is a very standard way to fit a classification model - so standard, in fact that we don't need to specify it at all."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.666964\n",
      "         Iterations 4\n",
      "Model trained\n",
      "C:\\Users\\leere\\Anaconda3\\envs\\ml_crash_course\\lib\\site-packages\\statsmodels\\tools\\_testing.py:19: FutureWarning:\n",
      "\n",
      "pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import statsmodels.formula.api as smf\n",
    "\n",
    "# Perform logistic regression.\n",
    "model = smf.logit(\"avalanche ~ weak_layers\", train).fit()\n",
    "\n",
    "print(\"Model trained\")"
   ]
  },
  {
   "source": [
    "## Assessing the model with summary information\n",
    "\n",
    "If we use `statsmodels`, we can get a detailed summary about the model by simply calling `summary()`"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                           Logit Regression Results                           \n",
       "==============================================================================\n",
       "Dep. Variable:              avalanche   No. Observations:                  821\n",
       "Model:                          Logit   Df Residuals:                      819\n",
       "Method:                           MLE   Df Model:                            1\n",
       "Date:                Thu, 17 Jun 2021   Pseudo R-squ.:                 0.02713\n",
       "Time:                        17:53:24   Log-Likelihood:                -547.58\n",
       "converged:                       True   LL-Null:                       -562.85\n",
       "Covariance Type:            nonrobust   LLR p-value:                 3.276e-08\n",
       "===============================================================================\n",
       "                  coef    std err          z      P>|z|      [0.025      0.975]\n",
       "-------------------------------------------------------------------------------\n",
       "Intercept      -0.4050      0.138     -2.925      0.003      -0.676      -0.134\n",
       "weak_layers     0.1259      0.023      5.436      0.000       0.080       0.171\n",
       "===============================================================================\n",
       "\"\"\""
      ],
      "text/html": "<table class=\"simpletable\">\n<caption>Logit Regression Results</caption>\n<tr>\n  <th>Dep. Variable:</th>       <td>avalanche</td>    <th>  No. Observations:  </th>  <td>   821</td>  \n</tr>\n<tr>\n  <th>Model:</th>                 <td>Logit</td>      <th>  Df Residuals:      </th>  <td>   819</td>  \n</tr>\n<tr>\n  <th>Method:</th>                 <td>MLE</td>       <th>  Df Model:          </th>  <td>     1</td>  \n</tr>\n<tr>\n  <th>Date:</th>            <td>Thu, 17 Jun 2021</td> <th>  Pseudo R-squ.:     </th>  <td>0.02713</td> \n</tr>\n<tr>\n  <th>Time:</th>                <td>17:53:24</td>     <th>  Log-Likelihood:    </th> <td> -547.58</td> \n</tr>\n<tr>\n  <th>converged:</th>             <td>True</td>       <th>  LL-Null:           </th> <td> -562.85</td> \n</tr>\n<tr>\n  <th>Covariance Type:</th>     <td>nonrobust</td>    <th>  LLR p-value:       </th> <td>3.276e-08</td>\n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n       <td></td>          <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>  \n</tr>\n<tr>\n  <th>Intercept</th>   <td>   -0.4050</td> <td>    0.138</td> <td>   -2.925</td> <td> 0.003</td> <td>   -0.676</td> <td>   -0.134</td>\n</tr>\n<tr>\n  <th>weak_layers</th> <td>    0.1259</td> <td>    0.023</td> <td>    5.436</td> <td> 0.000</td> <td>    0.080</td> <td>    0.171</td>\n</tr>\n</table>"
     },
     "metadata": {},
     "execution_count": 4
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "source": [
    "This summary provides some detailed information. \n",
    "\n",
    "Two useful pieces of information are in the bottom row. The coef for `weak_layers` is positive, which means that as `weak_layers` increases, so does the probability of an avalanche. The the `P` value column is less than 0.05, which means the model is confident that `weak_layers` is a useful predictor of avalanches. \n",
    "\n",
    "The rest of this table, however, is difficult to understand for beginners and so it's not clear how well our model is working. Let's try another way. \n",
    "\n",
    "## Assessing model visually\n",
    "\n",
    "Sometimes, but not always, we can visually assess a logistic regression mode. Let's plot our model against the actual data in the test dataset."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "weak layers=%{x}<br>avalanche=%{y}<extra></extra>",
         "legendgroup": "",
         "marker": {
          "color": "#636efa",
          "size": 9,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          10,
          1,
          5,
          5,
          6,
          7,
          9,
          5,
          0,
          2,
          9,
          8,
          3,
          3,
          6,
          3,
          10,
          9,
          10,
          10,
          9,
          3,
          0,
          8,
          9,
          7,
          2,
          10,
          6,
          7,
          4,
          8,
          8,
          3,
          9,
          2,
          7,
          7,
          10,
          0,
          2,
          6,
          4,
          5,
          0,
          4,
          5,
          6,
          8,
          6,
          7,
          3,
          9,
          5,
          10,
          8,
          10,
          2,
          4,
          3,
          9,
          5,
          9,
          5,
          10,
          6,
          6,
          6,
          10,
          6,
          2,
          2,
          2,
          6,
          8,
          0,
          5,
          0,
          9,
          0,
          8,
          9,
          8,
          0,
          5,
          6,
          6,
          6,
          2,
          3,
          10,
          4,
          9,
          5,
          2,
          5,
          1,
          2,
          0,
          2,
          8,
          6,
          9,
          3,
          9,
          10,
          6,
          2,
          6,
          7,
          10,
          8,
          10,
          4,
          8,
          6,
          4,
          7,
          2,
          2,
          10,
          5,
          3,
          8,
          9,
          7,
          6,
          9,
          9,
          7,
          7,
          0,
          2,
          4,
          2,
          10,
          7,
          1,
          1,
          4,
          5,
          7,
          3,
          3,
          6,
          2,
          2,
          8,
          1,
          9,
          8,
          10,
          6,
          3,
          5,
          8,
          4,
          10,
          10,
          3,
          8,
          10,
          9,
          10,
          3,
          7,
          7,
          9,
          9,
          9,
          4,
          4,
          8,
          4,
          9,
          1,
          5,
          8,
          10,
          10,
          10,
          4,
          4,
          8,
          10,
          4,
          4,
          8,
          0,
          8,
          1,
          2,
          0,
          10,
          0,
          4,
          8,
          7,
          1,
          1,
          5,
          8,
          7,
          0,
          9,
          9,
          1,
          0,
          5,
          7,
          1,
          1,
          4,
          7,
          10,
          8,
          5,
          5,
          7,
          4,
          6,
          7,
          10,
          5,
          2,
          0,
          10,
          8,
          8,
          7,
          1,
          4,
          3,
          3,
          10,
          0,
          4,
          7,
          3,
          6,
          7,
          10,
          2,
          5,
          6,
          3,
          6,
          9,
          10,
          0,
          9,
          6,
          2,
          7,
          8,
          0,
          1,
          0,
          5,
          1,
          0,
          5,
          9,
          10,
          5,
          5,
          5,
          4,
          9,
          6,
          3,
          10,
          0,
          9
         ],
         "xaxis": "x",
         "y": [
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "tracegroupgap": 0
        },
        "shapes": [
         {
          "line": {
           "color": "rgb(228,26,28)"
          },
          "path": "M0.0 0.40012240116488373 L 0.05025125628140704 0.40164161990315156 L 0.10050251256281408 0.4031627302108776 L 0.15075376884422112 0.40468570513578583 L 0.20100502512562815 0.40621051758478605 L 0.2512562814070352 0.40773714032576275 L 0.30150753768844224 0.4092655459893842 L 0.35175879396984927 0.41079570707093377 L 0.4020100502512563 0.4123275959321635 L 0.45226130653266333 0.4138611848031672 L 0.5025125628140704 0.4153964457842758 L 0.5527638190954774 0.41693335084797317 L 0.6030150753768845 0.4184718718408318 L 0.6532663316582915 0.42001198048546906 L 0.7035175879396985 0.42155364838252335 L 0.7537688442211056 0.42309684701264993 L 0.8040201005025126 0.42464154773853546 L 0.8542713567839196 0.4261877218069329 L 0.9045226130653267 0.4277353403507135 L 0.9547738693467337 0.4292843743909387 L 1.0050251256281408 0.43083479483894954 L 1.0552763819095479 0.43238657249847356 L 1.105527638190955 0.43393967806775025 L 1.155778894472362 0.4354940821416729 L 1.206030150753769 0.43704975521394795 L 1.256281407035176 0.4386066676792701 L 1.306532663316583 0.44016478983551466 L 1.35678391959799 0.4417240918859454 L 1.407035175879397 0.4432845439414378 L 1.4572864321608041 0.4448461160227178 L 1.5075376884422111 0.4464087780626154 L 1.5577889447236182 0.447972499908333 L 1.6080402010050252 0.4495372513237275 L 1.6582914572864322 0.4511030019916065 L 1.7085427135678393 0.4526697215160382 L 1.7587939698492463 0.45423737942467407 L 1.8090452261306533 0.4558059451710842 L 1.8592964824120604 0.4573753881371055 L 1.9095477386934674 0.45894567763520133 L 1.9597989949748744 0.46051678291083326 L 2.0100502512562817 0.46208867314484364 L 2.0603015075376887 0.46366131745584876 L 2.1105527638190957 0.46523468490264375 L 2.1608040201005028 0.46680874448661586 L 2.21105527638191 0.46838346515416845 L 2.261306532663317 0.46995881579915405 L 2.311557788944724 0.47153476526531585 L 2.361809045226131 0.47311128234873867 L 2.412060301507538 0.4746883358003063 L 2.462311557788945 0.4762658943281682 L 2.512562814070352 0.4778439266002126 L 2.562814070351759 0.4794224012465462 L 2.613065326633166 0.48100128686198074 L 2.663316582914573 0.4825805520085253 L 2.71356783919598 0.4841601652178843 L 2.763819095477387 0.4857400949939606 L 2.814070351758794 0.48732030981536306 L 2.864321608040201 0.4889007781379188 L 2.9145728643216082 0.49048146839718926 L 2.9648241206030153 0.4920623490109889 L 3.0150753768844223 0.4936433883819088 L 3.0653266331658293 0.49522455489984024 L 3.1155778894472363 0.4968058169445031 L 3.1658291457286434 0.49838714288797376 L 3.2160804020100504 0.4999685010972159 L 3.2663316582914574 0.501549859936611 L 3.3165829145728645 0.5031311877704907 L 3.3668341708542715 0.5047124529656671 L 3.4170854271356785 0.5062936238939646 L 3.4673366834170856 0.5078746689347502 L 3.5175879396984926 0.5094555564774617 L 3.5678391959798996 0.5110362549241356 L 3.6180904522613067 0.5126167326919322 L 3.6683417085427137 0.5141969582156573 L 3.7185929648241207 0.5157768999502826 L 3.7688442211055277 0.5173565263734615 L 3.819095477386935 0.5189358059880411 L 3.869346733668342 0.5205147073245704 L 3.919597989949749 0.5220931989438032 L 3.969849246231156 0.5236712494391972 L 4.020100502512563 0.525248827439405 L 4.07035175879397 0.5268259016107625 L 4.120603015075377 0.5284024406597676 L 4.170854271356784 0.529978413335555 L 4.2211055276381915 0.5315537884323611 L 4.2713567839195985 0.533128534791984 L 4.3216080402010055 0.534702621306233 L 4.371859296482413 0.5362760169193715 L 4.42211055276382 0.5378486906305501 L 4.472361809045227 0.5394206114962302 L 4.522613065326634 0.5409917486325992 L 4.572864321608041 0.5425620712179748 L 4.623115577889448 0.5441315484951985 L 4.673366834170855 0.5457001497740189 L 4.723618090452262 0.5472678444334642 L 4.773869346733669 0.5488346019242019 L 4.824120603015076 0.5504003917708872 L 4.874371859296483 0.5519651835744998 L 4.92462311557789 0.553528947014666 L 4.974874371859297 0.5550916518519703 L 5.025125628140704 0.5566532679302506 L 5.075376884422111 0.5582137651788824 L 5.125628140703518 0.559773113615047 L 5.175879396984925 0.5613312833459856 L 5.226130653266332 0.5628882445712384 L 5.276381909547739 0.5644439675848698 L 5.326633165829146 0.5659984227776753 L 5.376884422110553 0.567551580639375 L 5.42713567839196 0.5691034117607899 L 5.477386934673367 0.5706538868360014 L 5.527638190954774 0.5722029766644938 L 5.577889447236181 0.5737506521532807 L 5.628140703517588 0.5752968843190124 L 5.678391959798995 0.5768416442900661 L 5.728643216080402 0.5783849033086188 L 5.778894472361809 0.5799266327326995 L 5.8291457286432165 0.5814668040382251 L 5.8793969849246235 0.583005388821016 L 5.9296482412060305 0.5845423587987923 L 5.9798994974874375 0.5860776858131509 L 6.030150753768845 0.5876113418315223 L 6.080402010050252 0.5891432989491077 L 6.130653266331659 0.590673529390795 L 6.180904522613066 0.5922020055130554 L 6.231155778894473 0.5937286998058169 L 6.28140703517588 0.5952535848943195 L 6.331658291457287 0.5967766335409462 L 6.381909547738694 0.5982978186470345 L 6.432160804020101 0.5998171132546642 L 6.482412060301508 0.6013344905484254 L 6.532663316582915 0.6028499238571612 L 6.582914572864322 0.6043633866556909 L 6.633165829145729 0.6058748525665084 L 6.683417085427136 0.607384295361458 L 6.733668341708543 0.6088916889633883 L 6.78391959798995 0.6103970074477814 L 6.834170854271357 0.6119002250443583 L 6.884422110552764 0.613401316138663 L 6.934673366834171 0.6149002552736201 L 6.984924623115578 0.6163970171510694 L 7.035175879396985 0.6178915766332768 L 7.085427135678392 0.6193839087444202 L 7.135678391959799 0.6208739886720512 L 7.185929648241206 0.6223617917685323 L 7.236180904522613 0.6238472935524491 L 7.28643216080402 0.6253304697099975 L 7.336683417085427 0.6268112960963469 L 7.386934673366834 0.6282897487369767 L 7.437185929648241 0.6297658038289891 L 7.4874371859296485 0.6312394377423954 L 7.5376884422110555 0.6327106270213777 L 7.5879396984924625 0.634179348385525 L 7.63819095477387 0.6356455787310428 L 7.688442211055277 0.6371092951319385 L 7.738693467336684 0.6385704748411795 L 7.788944723618091 0.6400290952918272 L 7.839195979899498 0.6414851340981431 L 7.889447236180905 0.6429385690566712 L 7.939698492462312 0.6443893781472927 L 7.989949748743719 0.6458375395342547 L 8.040201005025127 0.6472830315671748 L 8.090452261306533 0.6487258327820168 L 8.14070351758794 0.6501659219020428 L 8.190954773869347 0.6516032778387373 L 8.241206030150755 0.6530378796927067 L 8.291457286432161 0.654469706754551 L 8.341708542713569 0.6558987385057111 L 8.391959798994975 0.6573249546192885 L 8.442211055276383 0.65874833496084 L 8.492462311557789 0.6601688595891443 L 8.542713567839197 0.6615865087569466 L 8.592964824120603 0.6630012629116712 L 8.643216080402011 0.6644131026961141 L 8.693467336683417 0.6658220089491048 L 8.743718592964825 0.6672279627061454 L 8.793969849246231 0.6686309452000218 L 8.84422110552764 0.6700309378613897 L 8.894472361809045 0.6714279223193352 L 8.944723618090453 0.6728218804019095 L 8.99497487437186 0.6742127941366366 L 9.045226130653267 0.6756006457509978 L 9.095477386934673 0.6769854176728883 L 9.145728643216081 0.6783670925310503 L 9.195979899497488 0.6797456531554796 L 9.246231155778895 0.6811210825778067 L 9.296482412060302 0.6824933640316542 L 9.34673366834171 0.6838624809529671 L 9.396984924623116 0.6852284169803207 L 9.447236180904524 0.6865911559552006 L 9.49748743718593 0.6879506819222612 L 9.547738693467338 0.6893069791295565 L 9.597989949748744 0.6906600320287494 L 9.648241206030152 0.6920098252752938 L 9.698492462311558 0.6933563437285949 L 9.748743718592966 0.6946995724521439 L 9.798994974874372 0.6960394967136292 L 9.84924623115578 0.6973761019850244 L 9.899497487437186 0.6987093739426518 L 9.949748743718594 0.7000392984672222 L 10.0 0.7013658616438533",
          "type": "path"
         }
        ],
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "weak layers"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "avalanche"
         }
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "def predict(weak_layers):\n",
    "    return model.predict(dict(weak_layers=weak_layers))\n",
    "\n",
    "graphing.scatter_2D(test, label_x=\"weak_layers\", label_y=\"avalanche\", trendline=predict)"
   ]
  },
  {
   "source": [
    "It's hard to see the s-shape of the trendline, because the number of weak layers of snow, and the likelihood of an avalanche, are only weakly related. If we zoom out, we can get a slightly better view:"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "weak layers=%{x}<br>avalanche=%{y}<extra></extra>",
         "legendgroup": "",
         "marker": {
          "color": "#636efa",
          "size": 9,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          10,
          1,
          5,
          5,
          6,
          7,
          9,
          5,
          0,
          2,
          9,
          8,
          3,
          3,
          6,
          3,
          10,
          9,
          10,
          10,
          9,
          3,
          0,
          8,
          9,
          7,
          2,
          10,
          6,
          7,
          4,
          8,
          8,
          3,
          9,
          2,
          7,
          7,
          10,
          0,
          2,
          6,
          4,
          5,
          0,
          4,
          5,
          6,
          8,
          6,
          7,
          3,
          9,
          5,
          10,
          8,
          10,
          2,
          4,
          3,
          9,
          5,
          9,
          5,
          10,
          6,
          6,
          6,
          10,
          6,
          2,
          2,
          2,
          6,
          8,
          0,
          5,
          0,
          9,
          0,
          8,
          9,
          8,
          0,
          5,
          6,
          6,
          6,
          2,
          3,
          10,
          4,
          9,
          5,
          2,
          5,
          1,
          2,
          0,
          2,
          8,
          6,
          9,
          3,
          9,
          10,
          6,
          2,
          6,
          7,
          10,
          8,
          10,
          4,
          8,
          6,
          4,
          7,
          2,
          2,
          10,
          5,
          3,
          8,
          9,
          7,
          6,
          9,
          9,
          7,
          7,
          0,
          2,
          4,
          2,
          10,
          7,
          1,
          1,
          4,
          5,
          7,
          3,
          3,
          6,
          2,
          2,
          8,
          1,
          9,
          8,
          10,
          6,
          3,
          5,
          8,
          4,
          10,
          10,
          3,
          8,
          10,
          9,
          10,
          3,
          7,
          7,
          9,
          9,
          9,
          4,
          4,
          8,
          4,
          9,
          1,
          5,
          8,
          10,
          10,
          10,
          4,
          4,
          8,
          10,
          4,
          4,
          8,
          0,
          8,
          1,
          2,
          0,
          10,
          0,
          4,
          8,
          7,
          1,
          1,
          5,
          8,
          7,
          0,
          9,
          9,
          1,
          0,
          5,
          7,
          1,
          1,
          4,
          7,
          10,
          8,
          5,
          5,
          7,
          4,
          6,
          7,
          10,
          5,
          2,
          0,
          10,
          8,
          8,
          7,
          1,
          4,
          3,
          3,
          10,
          0,
          4,
          7,
          3,
          6,
          7,
          10,
          2,
          5,
          6,
          3,
          6,
          9,
          10,
          0,
          9,
          6,
          2,
          7,
          8,
          0,
          1,
          0,
          5,
          1,
          0,
          5,
          9,
          10,
          5,
          5,
          5,
          4,
          9,
          6,
          3,
          10,
          0,
          9
         ],
         "xaxis": "x",
         "y": [
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          1,
          0,
          1,
          1,
          1,
          1,
          1,
          0,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          0,
          1,
          1,
          1,
          0,
          0,
          1,
          0,
          1,
          0,
          0,
          1,
          1,
          0,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          1,
          1,
          1,
          1,
          0
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "tracegroupgap": 0
        },
        "shapes": [
         {
          "line": {
           "color": "rgb(228,26,28)"
          },
          "path": "M-20.0 0.05105311316674535 L -19.798994974874372 0.05229291974610313 L -19.597989949748744 0.05356113519269421 L -19.396984924623116 0.0548583272080182 L -19.195979899497488 0.05618507075720858 L -18.99497487437186 0.057541947950556914 L -18.79396984924623 0.05892954791119006 L -18.592964824120603 0.06034846662830356 L -18.391959798994975 0.06179930679534582 L -18.190954773869347 0.06328267763253778 L -17.98994974874372 0.06479919469310724 L -17.78894472361809 0.06634947965260861 L -17.587939698492463 0.06793416008069546 L -17.386934673366834 0.06955386919470873 L -17.185929648241206 0.07120924559444118 L -16.984924623115578 0.07290093297743917 L -16.78391959798995 0.0746295798342025 L -16.582914572864322 0.0763958391226478 L -16.381909547738694 0.0782003679212054 L -16.180904522613066 0.08004382705992695 L -15.979899497487438 0.08192688072899156 L -15.77889447236181 0.08385019606401033 L -15.577889447236181 0.0858144427075438 L -15.376884422110553 0.08782029234626557 L -15.175879396984925 0.08986841822322565 L -14.974874371859297 0.09195949462469194 L -14.773869346733669 0.09409419634107483 L -14.57286432160804 0.0962731981014717 L -14.371859296482413 0.09849717398140302 L -14.170854271356784 0.10076679678334878 L -13.969849246231156 0.1030827373897385 L -13.768844221105528 0.10544566408809321 L -13.5678391959799 0.10785624186806934 L -13.366834170854272 0.11031513169021007 L -13.165829145728644 0.11282298972626825 L -12.964824120603016 0.11538046657103129 L -12.763819095477388 0.11798820642564613 L -12.56281407035176 0.12064684625251683 L -12.361809045226131 0.12335701490192649 L -12.160804020100503 0.1261193322106188 L -11.959798994974873 0.12893440807266235 L -11.758793969849245 0.13180284148301552 L -11.557788944723617 0.13472521955430777 L -11.356783919597989 0.13770211650745603 L -11.15577889447236 0.14073409263684272 L -10.954773869346733 0.14382169325089542 L -10.753768844221105 0.1469654475890236 L -10.552763819095476 0.15016586771599072 L -10.351758793969848 0.15342344739492306 L -10.15075376884422 0.1567386609402873 L -9.949748743718592 0.16011196205229986 L -9.748743718592964 0.16354378263436686 L -9.547738693467336 0.16703453159529053 L -9.346733668341708 0.1705845936381189 L -9.14572864321608 0.17419432803765633 L -8.944723618090451 0.1778640674087945 L -8.743718592964823 0.1815941164679685 L -8.542713567839195 0.1853847507901828 L -8.341708542713567 0.18923621556419612 L -8.140703517587939 0.19314872434859406 L -7.939698492462311 0.1971224578316141 L -7.738693467336683 0.20115756259772602 L -7.537688442211055 0.2052541499040974 L -7.3366834170854265 0.20941229447020293 L -7.135678391959798 0.213632033283952 L -6.93467336683417 0.2179133644278251 L -6.733668341708542 0.22225624592861185 L -6.532663316582914 0.22666059463443955 L -6.331658291457286 0.23112628512286817 L -6.130653266331658 0.23565314864390033 L -5.92964824120603 0.24024097210181886 L -5.7286432160804015 0.24488949707981447 L -5.527638190954773 0.2495984189113999 L -5.326633165829145 0.2543673858026308 L -5.125628140703517 0.25919599800915444 L -4.924623115577889 0.26408380707209894 L -4.723618090452261 0.2690303151167851 L -4.522613065326633 0.2740349742181931 L -4.321608040201005 0.27909718583705334 L -4.1206030150753765 0.28421630033034107 L -3.9195979899497466 0.2893916165398499 L -3.7185929648241185 0.2946223814623909 L -3.5175879396984904 0.2999077900050191 L -3.3165829145728623 0.30524698482851764 L -3.115577889447234 0.31063905628218264 L -2.914572864321606 0.3160830424327413 L -2.713567839195978 0.3215779291900071 L -2.5125628140703498 0.32712265053162104 L -2.3115577889447216 0.33271608882896536 L -2.1105527638190935 0.3383570752760405 L -1.9095477386934654 0.3440443904227955 L -1.7085427135678373 0.34977676481407677 L -1.5075376884422091 0.35555287973502164 L -1.306532663316581 0.3613713680633735 L -1.105527638190953 0.3672308152288273 L -0.9045226130653248 0.3731297602791404 L -0.7035175879396967 0.3790666970523598 L -0.5025125628140685 0.3850400754541225 L -0.3015075376884404 0.3910483028385916 L -0.10050251256281229 0.3970897454911875 L 0.10050251256281584 0.40316273021087773 L 0.30150753768844396 0.4092655459893842 L 0.5025125628140721 0.4153964457842758 L 0.7035175879397002 0.42155364838252335 L 0.9045226130653283 0.42773534035071353 L 1.1055276381909565 0.43393967806775025 L 1.3065326633165846 0.4401647898355147 L 1.5075376884422127 0.4464087780626154 L 1.7085427135678408 0.4526697215160383 L 1.909547738693469 0.45894567763520133 L 2.110552763819097 0.46523468490264375 L 2.311557788944725 0.47153476526531596 L 2.5125628140703533 0.4778439266002127 L 2.7135678391959814 0.4841601652178843 L 2.9145728643216096 0.49048146839718926 L 3.1155778894472377 0.4968058169445031 L 3.316582914572866 0.5031311877704907 L 3.517587939698494 0.5094555564774617 L 3.718592964824122 0.5157768999502826 L 3.91959798994975 0.5220931989438033 L 4.120603015075378 0.5284024406597677 L 4.321608040201006 0.534702621306233 L 4.5226130653266345 0.5409917486325994 L 4.723618090452263 0.5472678444334642 L 4.924623115577891 0.5535289470146662 L 5.125628140703519 0.559773113615047 L 5.326633165829147 0.5659984227776753 L 5.527638190954775 0.5722029766644938 L 5.728643216080403 0.5783849033086188 L 5.929648241206031 0.5845423587987923 L 6.1306532663316595 0.590673529390795 L 6.331658291457288 0.5967766335409462 L 6.532663316582916 0.6028499238571613 L 6.733668341708544 0.6088916889633883 L 6.934673366834172 0.6149002552736201 L 7.1356783919598 0.6208739886720512 L 7.336683417085428 0.6268112960963469 L 7.537688442211056 0.6327106270213778 L 7.7386934673366845 0.6385704748411795 L 7.939698492462313 0.6443893781472927 L 8.14070351758794 0.6501659219020428 L 8.341708542713569 0.6558987385057111 L 8.542713567839197 0.6615865087569466 L 8.743718592964825 0.6672279627061454 L 8.944723618090453 0.6728218804019095 L 9.145728643216081 0.6783670925310503 L 9.34673366834171 0.6838624809529671 L 9.547738693467338 0.6893069791295565 L 9.748743718592966 0.6946995724521439 L 9.949748743718594 0.7000392984672222 L 10.150753768844222 0.7053252470030832 L 10.35175879396985 0.7105565601996856 L 10.552763819095478 0.7157324324443651 L 10.753768844221106 0.7208521102162123 L 10.954773869346734 0.7259148918421632 L 11.155778894472363 0.7309201271680277 L 11.35678391959799 0.7358672171478597 L 11.557788944723619 0.7407556133552111 L 11.758793969849247 0.745584817419945 L 11.959798994974875 0.7503543803943872 L 12.160804020100507 0.7550639020526831 L 12.361809045226131 0.7597130301272933 L 12.562814070351763 0.7643014594866091 L 12.763819095477388 0.7688289312576984 L 12.96482412060302 0.773295231898207 L 13.165829145728644 0.7777001922214324 L 13.366834170854275 0.7820436863785675 L 13.5678391959799 0.7863256308020777 L 13.768844221105532 0.790545983114126 L 13.969849246231156 0.7947047410038902 L 14.170854271356788 0.7988019410775558 L 14.371859296482413 0.8028376576846686 L 14.572864321608044 0.8068120017244456 L 14.773869346733669 0.8107251194355299 L 14.9748743718593 0.8145771911725734 L 15.175879396984925 0.8183684301728973 L 15.376884422110557 0.8220990813163718 L 15.577889447236181 0.8257694198815099 L 15.778894472361813 0.8293797503006485 L 15.979899497487438 0.8329304049169426 L 16.18090452261307 0.8364217427457653 L 16.381909547738694 0.8398541482429583 L 16.582914572864325 0.8432280300822407 L 16.78391959798995 0.8465438199439344 L 16.98492462311558 0.8498019713170298 L 17.185929648241206 0.8530029583164636 L 17.386934673366838 0.856147274517352 L 17.587939698492463 0.8592354318077754 L 17.788944723618094 0.8622679592615816 L 17.98994974874372 0.8652454020325377 L 18.19095477386935 0.8681683202710387 L 18.391959798994975 0.871037288064444 L 18.592964824120607 0.8738528924020091 L 18.79396984924623 0.8766157321652422 L 18.994974874371863 0.8793264171444238 L 19.195979899497488 0.8819855670819007 L 19.39698492462312 0.884593810742677 L 19.597989949748744 0.8871517850127172 L 19.798994974874375 0.8896601340252882 L 20.0 0.8921195083155742",
          "type": "path"
         }
        ],
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "range": [
          -20,
          20
         ],
         "title": {
          "text": "weak layers"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "avalanche"
         }
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "graphing.scatter_2D(test, label_x=\"weak_layers\", label_y=\"avalanche\", x_range=[-20,20], trendline=predict)"
   ]
  },
  {
   "source": [
    "Checking the earlier graph, we can see that our model will predict an avalanche when the number of weak layers of snow is greater than 5. We can tell this because the value of the line is `0.5` at `x=5` (remember that in the previous unit we defined a *classifier threshold*, so that probabilities over `0.5` would be classified as `True`).\n",
    "\n",
    "How this relates with points is hard to tell - the points overlap and so it is difficult to see how many points are at 0 or at 1. How else can we assess the model?\n",
    "\n",
    "## Assess with cost function\n",
    "\n",
    "Let's assess our model with a log-loss cost function.\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Log loss 0.6559884464717545\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import log_loss\n",
    "\n",
    "# Make predictions from the test set\n",
    "predictions = model.predict(test)\n",
    "\n",
    "# Calculate log loss\n",
    "print(\"Log loss\", log_loss(test.avalanche, predictions))"
   ]
  },
  {
   "source": [
    "0.66 - what does that mean? This could be useful to compare two different models, but it's hard to get a grasp on exactly what this means for real-world performance. \n",
    "\n",
    "## Assess accuracy\n",
    "\n",
    "Let's instead assess _accuracy_. Accuracy refers to the _proportion of predictions the model got correct_, after predictions are converted from probabilities to `avalanche` or `no-avalanche`.\n",
    "\n",
    "The first thing to do is convert probabilities to hard predictions:"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "First three predictions (probabilities): 0.7013658616438533, 0.43067969123705807, 0.5558725978674339\nFirst three predictions (categories): True, False, True\n"
     ]
    }
   ],
   "source": [
    "import numpy\n",
    "\n",
    "# Print a few predictions before we convert them to categories\n",
    "print(f\"First three predictions (probabilities): {predictions.iloc[0]}, {predictions.iloc[1]}, {predictions.iloc[2]}\")\n",
    "\n",
    "# convert to absolute values\n",
    "avalanche_predicted = predictions >= 0.5\n",
    "\n",
    "# Print a few predictions converted into categories\n",
    "print(f\"First three predictions (categories): {avalanche_predicted.iloc[0]}, {avalanche_predicted.iloc[1]}, {avalanche_predicted.iloc[2]}\")"
   ]
  },
  {
   "source": [
    "Now we can calculate accuracy:"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Accuracy for whole test dataset: 0.6058394160583942\n"
     ]
    }
   ],
   "source": [
    "# Calculate what proportion were predicted correctly\n",
    "guess_was_correct = test.avalanche == avalanche_predicted\n",
    "accuracy = numpy.average(guess_was_correct)\n",
    "\n",
    "# Print the accuracy\n",
    "print(\"Accuracy for whole test dataset:\", accuracy)"
   ]
  },
  {
   "source": [
    "It looks like it's predicting the correct answer 61% of the time. This is helpful information. What kind of mistakes is it making, though? Let's take a look at whether it is guessing avalanche when there are none (false positives), or failing to guess 'avalanche' when one actually occurs (false negative):"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Wrongly predicted an avalanche 9.854014598540147% of the time\nFailed to predict avalanches 29.56204379562044% of the time\n"
     ]
    }
   ],
   "source": [
    "# False Positive: calculate how often it guessed avalanche when none actually occurred\n",
    "false_positive = numpy.average(numpy.logical_not(guess_was_correct) & test.avalanche)\n",
    "\n",
    "# False negative: calculate how often it guessed no avalanche, when one actually happened\n",
    "false_negative = numpy.average(numpy.logical_not(guess_was_correct) & numpy.logical_not(test.avalanche))\n",
    "\n",
    "\n",
    "print(f\"Wrongly predicted an avalanche {false_positive * 100}% of the time\")\n",
    "print(f\"Failed to predict avalanches {false_negative * 100}% of the time\")"
   ]
  },
  {
   "source": [
    "I think we can agree that's a lot more understandable than what the cost function or the graph!\n",
    "\n",
    "## Summary\n",
    "\n",
    "We've walked through different ways to assess a logistic regression model. We've seen that detailed summaries can provide rich information, but this can be difficult to digest. Metrics for these kinds of models also are not necessarily intuitive or detailed enough to understand the model. With a little extra work, we can look at actual predictions versus actual outcomes and get an intuition for how the model might work in the real world."
   ],
   "cell_type": "markdown",
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.7.10 64-bit ('ml_crash_course': conda)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "interpreter": {
   "hash": "a970c7e51787c6ef7f3c805442fbb5a30ae2745635e1d2514e62d936ba85c55c"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}