supervised_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Supervised Learning\n",
    "This notebook demonstrates a variety of supervised learning approaches--from simple linear regression to random forests.\n",
    "\n",
    "If you are interested in learning from data, specifically labelled data, then this notebook will help guide you through learning from data with Python. We'll use some key Python statistics and machine learning libaries:\n",
    "\n",
    "* `sci-kit learn`\n",
    "* `statsmodels`\n",
    "\n",
    "We'll use basic information about baseball players such as height, weight, and handedness to try to predict average runs scored per season. This could be used in player recruiting models (e.g. \"moneyball\"). \n",
    "\n",
    "The data we will use comes from [Sean Lahman](http://www.seanlahman.com/baseball-archive/statistics/). He has provided a [data dictionary](http://www.seanlahman.com/files/database/readme2017.txt) as well, which is critical for understanding the data sources.\n",
    "\n",
    "Let's get started by importing some packages we'll need:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import statsmodels.api as sm\n",
    "import statsmodels.formula.api as smf\n",
    "\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.linear_model import LinearRegression, LogisticRegression\n",
    "from sklearn.svm import LinearSVC, LinearSVR\n",
    "from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor\n",
    "from sklearn.metrics import mean_squared_error, mean_absolute_error, accuracy_score, precision_score, recall_score\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's load the Lahman players data and batting data. Note that these are two separate `.csv` files. The first we'll load as a dataframe called `players` and the second as a dataframe called `batting`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "players = pd.read_csv('data/People.csv')\n",
    "batting = pd.read_csv('data/Batting.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 19370 players in the dataset.\n",
      "There are 104324 batting records in the dataset.\n"
     ]
    }
   ],
   "source": [
    "print(\"There are {0} players in the dataset.\".format(players.shape[0]))\n",
    "print(\"There are {0} batting records in the dataset.\".format(batting.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that there are far more batting records than there are players. That is because each batting record corresponds to a specific player _in a specific year_. We can handle this by calculating the average runs per season per player. This will give us a sense as to how the player generally does overall, instead of in any specific year."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "avg_runs = pd.DataFrame(batting.groupby(['playerID']).R.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check on a player we recognize. I know that `harperbr03` is the `playerID` for Bryce Harper of the Washington Nationals. Let's take a look at his average runs (per season):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "R    84.5\n",
       "Name: harpebr03, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "avg_runs.loc['harpebr03']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merging player data with batting data\n",
    "Now, let's join the two datasets. We want to know, for each player, what were his average runs per season as well as his vital stats. So let's do a left join:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = players.set_index('playerID').join(avg_runs, how='left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's focus on just players who actually have scored runs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[df['R'] > 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 13596 players who scored runs.\n"
     ]
    }
   ],
   "source": [
    "print(\"There are {0} players who scored runs.\".format(df.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generating dummy variables\n",
    "We can't just feed algorithms categorical information such as handedness. How will our model understand the difference between \"left hand\" and \"right hand\"? To get around this we can create dummy variables.\n",
    "\n",
    "Dummy variables take each of the categories and convert it into its own column with binary values. So a right handed player would have values `1` for the `right_handed` column and `0` for the `left_handed` column. That's something we can work with! So, let's generate dummy variables for whether the player bats and throws righthanded, lefthanded, or both."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "bats_dummies = pd.get_dummies(df.bats, prefix='bats')\n",
    "throws_dummies = pd.get_dummies(df.throws, prefix='throws')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to add these dummy variable columns to our dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.join(bats_dummies)\n",
    "df = df.join(throws_dummies)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Assessing average runs\n",
    "Before we get started learning from data, let's try to understand a typical player's performance. We can use a kernel density estimation plot on average runs to get a sense for the distribution of runs we can expect from players:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0, 193.91677018633538)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD8CAYAAACGsIhGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt0XOV57/Hvo9H9bkuybGQb29gGDOFW1ZCE0KaEFNIEtw0B06QhKS3JaWhPy8rpIck5LMrpOqvknIbVLjhN3UJLSFIgBFqnIaGhNOQGjuULF9sxloXBMr5IsizrYksa6Tl/7C0xFiPN6LI1o9Hvs5bW7Hn3OzOPtmT/9O53X8zdERERmUhepgsQEZHsp7AQEZGUFBYiIpKSwkJERFJSWIiISEoKCxERSUlhISIiKSksREQkJYWFiIiklJ/pAmZKbW2tr1ixItNliIjMKdu2bWt397pU/XImLFasWEFTU1OmyxARmVPM7I10+mk3lIiIpKSwEBGRlBQWIiKSksJCRERSUliIiEhKCgsREUlJYSEiIinlZFh8/9XD7D3SnekyRERyRs6FxY9ea+OzX9/O9ff/hP74UKbLERHJCTkXFt99+TAA/fFhmg50ZrgaEZHckHNhsf3NTt69qoY8gy2vH890OSIiOSGnwqI/PkRLey+/dPYCVtaWsefwyUyXJCKSE3LmQoIAr7f3MjTsrKkv5/WOSl5p7cp0SSIiOSGnRhatx08BcHZNGcsXlvLWiVMMDXuGqxIRmftyKyw6+wBYuqCEpQtKiA87x7pPZ7gqEZG5L9KwMLNrzWyvmTWb2Z1J1heZ2WPh+i1mtiJh3UVm9oKZ7TKzV8ysONXntXaeorggj5qyQhqqS0bbRERkeiILCzOLAQ8A1wHrgJvNbN2YbrcCne6+GrgPuDd8bT7wdeCz7n4B8KvAYKrPPNbdz6KKYsyMpQuCsDiksBARmbYoRxbrgWZ3b3H3AeBRYMOYPhuAh8PlJ4CrzcyADwIvu/tLAO7e4e4pz7Dr6O2ntrwQgIbqUgAOnVBYiIhMV5Rh0QAcTHjeGrYl7ePucaALqAHWAm5mz5jZdjP7s2QfYGa3mVmTmTW1tbXR3j1AbXkRACWFMWrKCkfnMUREZOqydYI7H7gS+Hj4+FtmdvXYTu6+yd0b3b2xrq6O9p5+asKwADiruoS3TmiCW0RkuqIMi0PAsoTnS8O2pH3CeYoqoINgFPIjd2939z7gaeCyVB94vG+AunA3FEBdRRHtPf3T+R5ERIRow2IrsMbMVppZIbAR2Dymz2bglnD5BuA5d3fgGeBdZlYahsivALsn+rD4sOMOtRVvjyxqywsVFiIiMyCyM7jdPW5mtxP8xx8DHnL3XWZ2D9Dk7puBB4FHzKwZOE4QKLh7p5l9hSBwHHja3b870efFh4YBqClLDIsiOnoGGB528vJsxr9HEZH5ItLLfbj70wS7kBLb7kpYPg18bJzXfp3g8Nm0xMMztWsTdkPVlhcRH3ZOnBpkYVnheC8VEZEUsnWCe9LiQ2FYJO6GCpe1K0pEZHpyJyyGg91QC0sTRxbBcnu3wkJEZDpyJixGLhhYWVIw2lYXHkbbppGFiMi05FRYVBTlE0uYyB45Qa+9ZyBTZYmI5ITcCQv3M0YVANWlBeTnmeYsRESmKWfCYnjYqRoTFmZGdWkhnb0aWYiITEfOhMVQkrAAWFBaQGefwkJEZDpyJiziw05lyTtPG1lQWkhnX8qrm4uIyARyJizGG1lUlxZwQiMLEZFpyZmwGHaShsXCMo0sRESmK4fCYryRRSEn+gYIrk8oIiJTkTNhAbzj0FkIJrgHh5zegZQ32hMRkXHkVFgkPxoquOSHDp8VEZm6nAqLZCOL6tKg7YTmLUREpiy3wqI4yaGz4aXJda6FiMjU5VRYlBYmO88iGFkoLEREpi6nwqIsSVhUh3MW2g0lIjJ1uRUWRbF3tFWXaGQhIjJdORYW7xxZ5MfyqCjO18hCRGQaciosivKTfzvB9aE0shARmaqcCYs8M8ws6brgyrMaWYiITFXOhEVsnKCAty/5ISIiUxNpWJjZtWa218yazezOJOuLzOyxcP0WM1sRtq8ws1NmtjP8+mqqz8qb4DtZUFqgOQsRkWl454zwDDGzGPAAcA3QCmw1s83uvjuh261Ap7uvNrONwL3ATeG6/e5+Sbqfl6eRhYhIZKIcWawHmt29xd0HgEeBDWP6bAAeDpefAK628SYeUpgoLKpKCjh5Ok58aHgqby0iMu9FGRYNwMGE561hW9I+7h4HuoCacN1KM9thZs+b2fuSfYCZ3WZmTWbWFI+PP3IYOYv75On4VL4PEZF5L1snuA8Dy939UuAO4JtmVjm2k7tvcvdGd28sKSoa983ePotbu6JERKYiyrA4BCxLeL40bEvax8zygSqgw9373b0DwN23AfuBtRN9WF7eBLuhRq48e0qT3CIiUxFlWGwF1pjZSjMrBDYCm8f02QzcEi7fADzn7m5mdeEEOWa2ClgDtEz0YRNOcIeX/OjSEVEiIlMS2dFQ7h43s9uBZ4AY8JC77zKze4Amd98MPAg8YmbNwHGCQAG4CrjHzAaBYeCz7n58os+bYGDx9m6oU9oNJSIyFZGFBYC7Pw08PabtroTl08DHkrzu28C3J/NZE40sRi9T3quRhYjIVGTrBPekTTRnUVFcgJnmLEREpipnwiI2wW6oWJ5RWVxAl46GEhGZkpwJi4l2Q0FwL26NLEREpiZ3wmKiGW6CI6J0fSgRkanJnbBIObLQ9aFERKZqHoWFdkOJiExV7oRFiu9Eu6FERKYuZ8JiopsfAVSVFnLy9CBDwz5LFYmI5I6cCYtUu6EWlBbgDt2nNboQEZmsnAmLVHfBqB45i1u7okREJi1nwiKV6hJdplxEZKrmTVjoMuUiIlM3b8JClykXEZm6eRMWC3S3PBGRKZs3YVFZogluEZGpmjdhEVx5Np8uzVmIiEzavAkL0PWhRESmap6Fha4PJSIyFfMsLAp1fSgRkSmYX2FRUqDdUCIiUzC/wkK7oUREpmR+hUVJAV2nBhnWlWdFRCYl0rAws2vNbK+ZNZvZnUnWF5nZY+H6LWa2Ysz65WbWY2afn4l6qkoLwyvPxmfi7URE5o3IwsLMYsADwHXAOuBmM1s3ptutQKe7rwbuA+4ds/4rwPdmqqYFo9eH0ryFiMhkRDmyWA80u3uLuw8AjwIbxvTZADwcLj8BXG0WXGzczH4TeB3YNVMFjVymXEdEiYhMTpRh0QAcTHjeGrYl7ePucaALqDGzcuC/A38+kwVVhZcp79QRUSIik5KtE9x3A/e5e89EnczsNjNrMrOmtra2lG86MrLQJT9ERCYnP8L3PgQsS3i+NGxL1qfVzPKBKqADuBy4wcy+DFQDw2Z22t3vT3yxu28CNgE0NjamPMTp7SvPKixERCYjyrDYCqwxs5UEobAR+J0xfTYDtwAvADcAz7m7A+8b6WBmdwM9Y4NiKiqLg29XYSEiMjmRhYW7x83sduAZIAY85O67zOweoMndNwMPAo+YWTNwnCBQIpMfy6OiOF9HQ4mITFKUIwvc/Wng6TFtdyUsnwY+luI97p7JmqpLCzSyEBGZpGyd4I5MdYkuUy4iMlnzLyx0fSgRkUmbh2FRSJd2Q4mITMr8C4sSjSxERCYrrbAwsyfN7DfMbM6HSzDBPaArz4qITEK6//n/P4JzJPaZ2V+a2bkR1hSpqpIChh26+3XlWRGRdKUVFu7+rLt/HLgMOAA8a2Y/M7NPm1lBlAXOtLfP4tYRUSIi6Up7t5KZ1QCfAn4f2AH8NUF4/CCSyiKysDwIi45ehYWISLrSOinPzJ4CzgUeAT7i7ofDVY+ZWVNUxUWhtqwIgI4ehYWISLrSPYP778OzsUeZWZG797t7YwR1RaYmHFm09/RnuBIRkbkj3d1Qf5Gk7YWZLGS2jIRFh8JCRCRtE44szGwxwQ2KSszsUsDCVZVAacS1RaIoP0ZFcT7t2g0lIpK2VLuhfp1gUnspwf2wR3QDX4yopsjVlhdpN5SIyCRMGBbu/jDwsJl91N2/PUs1Ra6mrFAT3CIik5BqN9Qn3P3rwAozu2Psenf/SpKXZb3a8iJa2ie8Y6uIiCRItRuqLHwsj7qQ2VRTXsjPD2hkISKSrlS7of4ufPzz2SlndtSUF9HZN0B8aJj82Jy/3JWISOTSvZDgl82s0swKzOw/zKzNzD4RdXFRqS0vxB06dalyEZG0pPtn9Qfd/STwYYJrQ60G/ltURUWtJjyLW0dEiYikJ92wGNld9RvAt9y9K6J6ZkXt6Il5mrcQEUlHupf7+Dcz+wVwCvgvZlYHnI6urGjVlIfXh+rVyEJEJB3pXqL8TuA9QKO7DwK9wIYoC4tSXUUQFsdOKixERNIxmUOBzgNuMrNPAjcAH0z1AjO71sz2mlmzmd2ZZH2RmT0Wrt9iZivC9vVmtjP8esnMfmsSdaZUWZxPSUGMIyfn7OBIRGRWpXuJ8keAc4CdwFDY7MDXJnhNDHgAuAZoBbaa2WZ3353Q7Vag091Xm9lG4F7gJuBVglFM3MyWAC+Z2XfcfUZub2dmLK4qVliIiKQp3TmLRmCdu0/mxtXrgWZ3bwEws0cJdl0lhsUG4O5w+QngfjMzd+9L6FNMEEwzqr6yiKNdCgsRkXSkuxvqVWDxJN+7ATiY8Lw1bEvaJxw1dAE1AGZ2uZntAl4BPjtTo4oRiys1shARSVe6I4taYLeZ/RwYnRV29+sjqSp47y3ABWZ2PsHFDL/n7mf8725mtwG3ASxfvnxS719fVcyxk/0MDzt5eZb6BSIi81i6YXH3FN77ELAs4fnSsC1Zn1YzyweqgI7EDu6+x8x6gAuBpjHrNgGbABobGye1q2pxZTEDQ8Mc7xugNjyUVkREkkv30NnnCc7cLgiXtwLbU7xsK7DGzFaaWSGwEdg8ps9m4JZw+QbgOXf38DX5AGZ2NsGRWAfSqTVdiyuLATiieQsRkZTSvTbUHxBMQP9d2NQA/MtErwnnGG4HngH2AI+7+y4zu8fMRnZfPQjUmFkzcAcwcnjtlQRHQO0EngL+0N3b0/+2UltcFYTFUc1biIiklO5uqM8RHN20BcDd95nZolQvcvengafHtN2VsHwa+FiS1z0CPJJmbVMyEhaa5BYRSS3do6H63X30QkrhLqIZP5x1NtWVF5Fn6PBZEZE0pBsWz5vZF4ESM7sG+BbwnejKil5+LI/a8iKNLERE0pBuWNwJtBGc8/AZgl1L/yOqombLWdUlHDpxKtNliIhkvbTmLNx92Mz+BfgXd2+LuKZZs3xhKTsPnsh0GSIiWW/CkYUF7jazdmAvsDe8S95dE71urli+sJRDJ04RHxrOdCkiIlkt1W6oPwXeC/yyuy9094XA5cB7zexPI68uYssXljI07BzWJLeIyIRShcXvAje7++sjDeGFAT8BfDLKwmbDsoWlALx5vC9FTxGR+S1VWBQkOxkunLcoiKak2bO8RmEhIpKOVGEx0U2q5/wNrBdXFlMQM4WFiEgKqY6GutjMTiZpN4L7TMxpsTyjobpEYSEiksKEYeHusdkqJFOWLSzloMJCRGRCk7kHd046u6aUA+29TO4mgCIi88u8D4vVdeWcPB2nrbs/dWcRkXlq3ofF2voKAF472pPhSkREste8D4s1o2HRneFKRESy17wPi9ryQhaUFrDvmMJCRGQ88z4szIw19RXaDSUiMoF5HxYAa+vLee1ot46IEhEZh8KCYJK7+3RcN0ISERmHwgJYt6QSgFdauzJciYhIdlJYABc2VJGfZ7oRkojIOBQWQHFBjPOWVCgsRETGobAIXbKsmpdbuxga1iS3iMhYkYaFmV1rZnvNrNnM7kyyvsjMHgvXbzGzFWH7NWa2zcxeCR9/Lco6AS5ZtoCe/jj723QIrYjIWJGFhZnFgAeA64B1wM1mtm5Mt1uBTndfDdwH3Bu2twMfcfd3AbcAj0RV54hLllUDsOPNzqg/SkRkzolyZLEeaHb3FncfAB4FNozpswF4OFx+ArjazMzdd7j7W2H7LqDEzIoirJVz6sqoLS/iZ/s7ovwYEZE5KcqwaAAOJjxvDduS9nH3ONAF1Izp81Fgu7u/47KwZnabmTWZWVNbW9u0ijUzrlxdw0/2tTOseQsRkTNk9QS3mV1AsGvqM8nWu/smd29098a6urppf9771tTR0TvAniPJbg4oIjJ/RRkWh4BlCc+Xhm1J+5hZPlAFdITPlwJPAZ909/0R1jnqyjW1APxkX/tsfJyIyJwRZVhsBdaY2UozKwQ2ApvH9NlMMIENcAPwnLu7mVUD3wXudPefRljjGeorizm3voLnfnFstj5SRGROiCwswjmI24FngD3A4+6+y8zuMbPrw24PAjVm1gzcAYwcXns7sBq4y8x2hl+Loqo10bUXLubnB45zTNeJEhEZZblypdXGxkZvamqa9vu8drSbD973I/78+gu45T0rpl+YiEgWM7Nt7t6Yql9WT3Bnwtr6CtbWl/NvL7+VurOIyDyhsEji+ovPYuuBTl5v7810KSIiWUFhkcSNjcuI5Rn//PM3M12KiEhWUFgksaiymA+uq+dbTQc5PTiU6XJERDJOYTGO33332XT2DfLEttZMlyIiknEKi3G8e1UNly2v5m9/uJ+B+HCmyxERySiFxTjMjD++eg2HTpzi29s1uhCR+U1hMYFfWVvHxcuquf+5Zk4NaO5CROYvhcUEzIwvXHceh06c4qvPz8rlqUREspLCIoUrVtXwkYvP4m+f38/B432ZLkdEJCMUFmn44ofOIz/P+OJTr5Arl0cREZkMhUUallSV8IUPnc+P97XzyItvZLocEZFZp7BI0ycuX86vrK3jfz+9h/1tPZkuR0RkViks0mRmfPmGiyguiPFH39yho6NEZF5RWExCfWUx9914CXuOnORLmr8QkXlEYTFJ7z9vEX9y9Vqe3HFI8xciMm8oLKbgj35tNR84fxH3fGc3P2vW/bpFJPcpLKYgL8/4yk2XsKqujM98fRuvHe3OdEkiIpFSWExRZXEB//jp9ZQUxPj0P27VPbtFJKcpLKahobqEhz71y3T2DfDpf9pK9+nBTJckIhIJhcU0XdhQxQMfv4y9R7r5vX/aSt9APNMliYjMOIXFDHj/uYu476ZL2PZGJ595ZBv9cZ2DISK5JdKwMLNrzWyvmTWb2Z1J1heZ2WPh+i1mtiJsrzGz/zSzHjO7P8oaZ8pHLj6Lv/zoRfx4Xzuf+8YOBYaI5JTIwsLMYsADwHXAOuBmM1s3ptutQKe7rwbuA+4N208D/xP4fFT1ReHGxmX8rw0X8Oyeo/zB17bpLG8RyRlRjizWA83u3uLuA8CjwIYxfTYAD4fLTwBXm5m5e6+7/4QgNOaU3333Cr780Yv4yb42PvnQFk5q0ltEckCUYdEAHEx43hq2Je3j7nGgC6iJsKZZceMvL+Nvbr6UHW+e4OZNL3K461SmSxIRmZY5PcFtZreZWZOZNbW1tWW6nDN8+KKz+IdbGnmjo4/r7/8pO97szHRJIiJTFmVYHAKWJTxfGrYl7WNm+UAV0JHuB7j7JndvdPfGurq6aZY783713EU8+Yfvobggj5s2vciT21szXZKIyJREGRZbgTVmttLMCoGNwOYxfTYDt4TLNwDPeY5dynVtfQX/+rkruXRZNXc8/hKf/9ZL9PTrXAwRmVsiC4twDuJ24BlgD/C4u+8ys3vM7Pqw24NAjZk1A3cAo4fXmtkB4CvAp8ysNcmRVHPGwrJCvvH7l/PHv7aaJ7e38uG/+TFNB45nuiwRkbRZrvwh39jY6E1NTZkuI6UtLR3c8fhLHDpxipvXL+O/X3se1aWFmS5LROYpM9vm7o2p+s3pCe656PJVNfz7n17FbVet4vGmVt7/f3/IP/y4hdODOidDRLKXwiIDyory+eKHzuc7t1/JhQ1V/MV393D1Xz3PY1vf1JnfIpKVtBsqC/y0uZ17v/8LXm7tor6yiE+/dyW/c/lyKosLMl2aiOS4dHdDKSyyhLvz433tbPpRCz9pbqe8KJ8bG5fx8SuWc05deabLE5EcpbCYw1491MWmH7XwvVcPMzjkvHd1DZ+4/Gw+sK6egpj2HIrIzFFY5IC27n4ebzrIN7e8yaETp1hUUcTG9cu5sXEpSxeUZro8EckBCoscMjTs/HDvMb7+4hv88LU23OHdq2r47csauO5dSygvys90iSIyRyksclRrZx9PbT/EkzsO8Xp7L0X5eVyxqoar1tZx+cqFrKkvpyg/lukyRWSOUFjkOHdnx8ETfOelt3j+tTZa2noByM8zVi8qZ92SStYurmBtfTlr6ytoqC7BzDJctYhkm3TDQvsv5igz47LlC7hs+QIgGHHsPHiC3W+dZNdbJ3mhpYMnd7x93cbyonze1VDFe86p4T2ra7h4aTX5miwXkTRpZJHDuk4Nsu9oN3uPdrP3SDdNBzrZffgkANWlBVxzfj3XXriY966upbhAu65E5iONLISqkgIaVyykccXC0bbjvQO8sL+DZ/cc5fu7jvCtba2UFcZ4/3mLuGZdPb967iKqSnQyoIicSSOLeWwgPswLLR18/9Uj/GD3Edp7BsjPM9avXMgHzq/nilU1nLu4glie5jpEcpUmuGVShoedna0neHb3UZ7dc5TXjvYAUFYY46Kl1Zy7uIKVtWWjXw3VJeQpRETmPIWFTMvB431se6OT7W92svPgCfYf66F34O2LHJYWxlhTX8F59RWcv6SCCxuqWHdWJaWF2rMpMpcoLGRGuTttPf283tZLS3sve48Ek+Z7j3ZzvHcAgDyDc+rKubChigsbqnhXGCA6aVAke2mCW2aUmbGoophFFcVcvqpmtN3dOXqyn1cPdfHKoS52vdXFz/a381TCYbsN1SWsrS9nTX0FaxaVc/6SSs5bXKFDd0XmEIWFTIuZsbiqmMVVxXxgXf1o+7Hu0+w6dJLdh0/y2tFuXjvaw0/3dzAQHwaC8z4aVyzgilU1vHtVDRc2VGkiXSSLKSwkEosqill0XjHvP2/RaFt8aJg3j/fx6lsn2dLSwYstHfxwbxsAFcX5XLGqJjhp8Jxa1taX64xzkSyisJBZkx/LY1VdOavqyrn+4rOAYATyYstxXtjfzs/2d/CD3UcBqC0v5NLlC7jgrErOX1LJuiWVLF2gS5aIZIrCQjJqUUUx11981mh4tHb28cL+Dn62v4OXW0/w7J6jjByDUV6Uzzl1ZayqK094LOfsmlKdgS4SMR0NJVmtbyDO3iPdwdzHkW5a2nvZf6yHt7pOj/bJM1i6oPSMABlZri0v1GhEZAJZcTSUmV0L/DUQA/7B3f9yzPoi4GvALwEdwE3ufiBc9wXgVmAI+GN3fybKWiU7lRbmc+nyBVwaXjBxRN9AnJbwMN79x3rY39ZDS1svL7R0cHpweLRfZXH+aICsqisbfVxUUURVSYGCRCRNkYWFmcWAB4BrgFZgq5ltdvfdCd1uBTrdfbWZbQTuBW4ys3XARuAC4CzgWTNb6+5DiBCEyMj5HImGh53DJ0+fESD723r4aXM7397eekbf/DxjQVkhNWWF1JQXsrCsKFguK2RhefhYVsSC0gIWlBVSXVKgw31l3opyZLEeaHb3FgAzexTYACSGxQbg7nD5CeB+C/7U2wA86u79wOtm1hy+3wsR1is5IC/PaKguoaG6hKvW1p2xrqc/TktbD6+399LeM8Dx3n46egbo6B2go6efVzpP0NEzQHd/fNz3LyuMUZifN/pVEMujMJZHSWGM4vwYJYUxSgpiFBfEKC0c+coPHovefl5WmE9JYR6FsRj5MaMgZhTE8siP5VGQN7IcPBbE8nRYsWRclGHRABxMeN4KXD5eH3ePm1kXUBO2vzjmtQ3RlSrzQXlRPhctreaipdUT9uuPD9HZO0h7Tz8n+gY53jfAib4BjvcO0H06zkB8mMGhYQbiwwyEj6fjw5weGKKtu59Tg0OcGhji1OAQfQPxM3aLTZUZFOTlYRYsGxY+Bue6GARPSGgbs97CTm+3v/N90q1loufhp6R+XdL3TvK6lA3pvVcuxG1xQYzv/NGVGfnsOX00lJndBtwGsHz58gxXI7miKD/G4qoYi6uKZ+T9hoadvoE4pwaG6B0Yorc/Tt9AECTxIWdwaJjBYSc+FITQ4NDIsjM4PMxg3IkPB8/dHSc4c96dcBkcJ/FYlbf7vb1u5Dkjz8esGyvZsS8+tmfSPsney9Pok/q9kh2Qk/QQnXeUmRsH8mTylslRhsUhYFnC86VhW7I+rWaWD1QRTHSn81rcfROwCYKjoWascpEZFMszKooLqCjWfUJk7opytm4rsMbMVppZIcGE9eYxfTYDt4TLNwDPefCnw2Zgo5kVmdlKYA3w8whrFRGRCUQ2sgjnIG4HniE4dPYhd99lZvcATe6+GXgQeCScwD5OECiE/R4nmAyPA5/TkVAiIpmjk/JEROaxdE/K00HjIiKSksJCRERSUliIiEhKCgsREUlJYSEiIinlzNFQZtYN7M10HWmqBdozXcQkzKV6VWs0VGs0sqHWs929LlWnOX25jzH2pnP4VzYws6a5UivMrXpVazRUazTmUq3aDSUiIikpLEREJKVcCotNmS5gEuZSrTC36lWt0VCt0ZgztebMBLeIiEQnl0YWIiISkZwICzO71sz2mlmzmd2Z6XoSmdkyM/tPM9ttZrvM7L+G7Xeb2SEz2xl+fSjTtQKY2QEzeyWsqSlsW2hmPzCzfeHjgiyo89yEbbfTzE6a2Z9k03Y1s4fM7JiZvZrQlnRbWuBvwt/hl83ssiyo9f+Y2S/Cep4ys+qwfYWZnUrYxl/NglrH/bmb2RfC7brXzH49C2p9LKHOA2a2M2zP6HZNKbjj1tz9Irj8+X5gFVAIvASsy3RdCfUtAS4LlyuA14B1BPce/3ym60tS7wGgdkzbl4E7w+U7gXszXWeS34EjwNnZtF2Bq4DLgFdTbUvgQ8D3CO7+eQWwJQtq/SCQHy7fm1DrisR+WbJdk/7cw39rLwFFwMrw/4pYJmsds/6vgLuyYbum+sqFkcV6oNndW9x9AHgU2JDhmka5+2F33x4udwO5eJ8bAAADDElEQVR7mHv3E98APBwuPwz8ZgZrSeZqYL+7v5HpQhK5+48I7tOSaLxtuQH4mgdeBKrNbMnsVJq8Vnf/d3ePh09fJLhjZcaNs13HswF41N373f11oJng/4xZMVGtFtwk/Ebgn2ernunIhbBoAA4mPG8lS/8zNrMVwKXAlrDp9nCI/1A27NoJOfDvZrYtvMc5QL27Hw6XjwD1mSltXBs58x9cNm7XEeNty2z/Pf49gpHPiJVmtsPMnjez92WqqDGS/dyzebu+Dzjq7vsS2rJxuwK5ERZzgpmVA98G/sTdTwJ/C5wDXAIcJhiOZoMr3f0y4Drgc2Z2VeJKD8bLWXMInQW37L0e+FbYlK3b9R2ybVuOx8y+RHDHym+ETYeB5e5+KXAH8E0zq8xUfaE583NPcDNn/pGTjdt1VC6ExSFgWcLzpWFb1jCzAoKg+Ia7Pwng7kfdfcjdh4G/ZxaHxhNx90Ph4zHgKYK6jo7sEgkfj2Wuwne4Dtju7kche7drgvG2ZVb+HpvZp4APAx8Pw41wl05HuLyNYB5gbcaKZMKfe7Zu13zgt4HHRtqycbsmyoWw2AqsMbOV4V+ZG4HNGa5pVLhf8kFgj7t/JaE9cX/0bwGvjn3tbDOzMjOrGFkmmOB8lWB73hJ2uwX418xUmNQZf51l43YdY7xtuRn4ZHhU1BVAV8Luqowws2uBPwOud/e+hPY6M4uFy6uANUBLZqocrWm8n/tmYKOZFZnZSoJafz7b9SXxAeAX7t460pCN2/UMmZ5hn4kvgiNJXiNI4i9lup4xtV1JsKvhZWBn+PUh4BHglbB9M7AkC2pdRXDkyEvArpFtCdQA/wHsA54FFma61rCuMqADqEpoy5rtShBih4FBgn3lt463LQmOgnog/B1+BWjMglqbCfb3j/zefjXs+9Hw92MnsB34SBbUOu7PHfhSuF33Atdlutaw/Z+Az47pm9HtmupLZ3CLiEhKubAbSkREIqawEBGRlBQWIiKSksJCRERSUliIiEhKCgsREUlJYSEiIikpLEREJKX/D/X6seG90rIMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot = df['R'].plot(kind='kde')\n",
    "plot.set_xlim(0,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that this looks like a log-normal distribution. This makes sense and is actually the case for many natural phenomenon. Consider height, for example, there is a natural limit to the height a person can grow to so density at its higher bounds is very small. \n",
    "\n",
    "We can check this by looking at the distribution of `log(runs)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['R_log'] = np.log(df.R)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0, 9.557392426547839)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD8CAYAAACGsIhGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VfWd//HXJztZISSENSSBoIAiYERca12xtmhbrdpKnY7WTmt1rOPMOG2ni+NvZrqMtTPSVqY6tdbWfaHVqrhU686O7ISwJWwJRJYAWT+/P+6ljRi4N5Cbc3Pv+/l45JF7zz0neXMfJO+c8/2ec8zdEREROZKUoAOIiEj8U1mIiEhEKgsREYlIZSEiIhGpLEREJCKVhYiIRKSyEBGRiFQWIiISkcpCREQiSgs6QE8pKirysrKyoGOIiPQp8+fPb3D34kjrJUxZlJWVMW/evKBjiIj0KWa2IZr1dBhKREQiUlmIiEhEKgsREYlIZSEiIhGpLEREJCKVhYiIRKSyEBGRiBLmPAuJb/ta2lhSu4uNO/axo6mF9o4OzIz+2ekMzMmkrCibsoE5ZKWnBh1VRLqgspCY2dfSxh+WbOHJBbXMW99IW8eR7/eeYjByYA4nDS9g8sgBTC4dwNgh+aSmWC8lFpHDUVlIj2tt7+ChdzZwz6tradjbTEVRDl8+u4IpZYVUFOdQnJdJWkoKHe58sK+Vhr3N1DQ0Ub19L6u27ubNtTt4etFmAApzMjjnuGLOH1vCuccP0p6HSEBUFtKjVm7dzW2PLWZp3W6mVhRyz+cncWp5IWZd7x0MLkhlcEEWJwwr+Msyd6e2cT/zNzTy2up6Xlm5nScX1NE/O53PTh7Ol84oY/iA7N76J4kIYO5HPjTQV1RVVbmuDRWspxbWcvsT75Obmcadl53AtBMGH7YkuqOtvYN31+3kt+9t5IWlW0kxY8ZpI7n1gjHkZOrvHZFjYWbz3b0q0nr6SZNj5u78+MVVzHx1bXhvYjJFuZk99vXTUlM4Y3QRZ4wuYsuu/dw9Zw33v7mOOcu3cfdVE5lcOqDHvpeIdE1TZ+WYdHQ43//9cma+uparp4zgwetO7dGiONSQgn784PIJPHLDabR3OFfNeocXlm2N2fcTkRCVhRw1d+dbT7/Pr95az/VnlvPvnz6R9NTe+S81pbyQP9x0JuOH5vPV38znmUV1vfJ9RZKVykKO2g+eX8Xv3tvEjR8fxbcuGdsj4xPdMSAng4euP5Up5YXc9thi3lrb0KvfXySZxLQszGyama0ys2ozu72L1281s+VmtsTMXjazkZ1eu9bM1oQ/ro1lTum+X/65hl+8tpZrppZy24XH9XpRHJSdkca9M6oYOTCHm367kO27DwSSQyTRxawszCwVmAlcDIwDrjazcYesthCocvcJwOPAD8PbFgLfBU4FpgDfNTONYsaJF5Zt5c5nV/CJEwfz/eknBFYUBxX0S+fnX5hMU0sbtz2+hESZ4ScST2K5ZzEFqHb3GndvAR4GLu28gru/6u77wk/fAYaHH18EzHH3ne7eCMwBpsUwq0Rp1dY93PrIIk4a0Z+7Pjcxbs6urizJ4/Zpx/P66npmL94cdByRhBPLshgGbOr0vDa87HCuA/54lNtKL2hsauH6X88lJzONWTNOjruzqWecVsZJwwu489kV7GtpCzqOSEKJiwFuM7sGqAJ+1M3tbjCzeWY2r76+PjbhBAhdwuNrDy1g2+5m7p1xMiX5WUFH+ojUFONfPzmO+j3N/N+b64OOI5JQYlkWdcCITs+Hh5d9iJmdD3wLmO7uzd3Z1t1nuXuVu1cVFxf3WHD5qDv/sJy3a3bwH58+kUlxfBJcVVkh548dxC9eW8ueA61BxxFJGLEsi7lApZmVm1kGcBUwu/MKZjYJuJdQUWzv9NILwIVmNiA8sH1heNkRtbR16PBDDDz83kYeeHsDXz6rnM+ePDzyBgG7+bxK9hxo43fvbQw6ikjCiFlZuHsb8HVCv+RXAI+6+zIzu8PMpodX+xGQCzxmZovMbHZ4253AvxEqnLnAHeFlh7W3uY1T//0lTvzei8x8tTpG/6rkM3/DTv71maWcVVnE7RePDTpOVCYM78/powZy3xvraGnrCDqOSEJImAsJZg8b46fdOovyolxeWrGNX1wzmWknDAk6Vp+2ddcBPnXPG+RkpPLMjWdSkJ0edKSovbxiG9c9ME//D0QiiPZCgnExwN0T2jucez4/mZ9fM5mxQ/K589kVtLbrr8qjdaC1na88OI99zW3M+mJVnyoKgI+NKWZwfhaPzN0UeWURiShhyiI/K42xQ/JJT03h1gvGUNu4n5eWbws6Vp/k7nzrqaUsrt3FT66cyJiSvKAjdVtaagqXnzyc11bXs2XX/qDjiPR5CVMWA7Iz/vL43OMHMax/P3799oYAE/Vds16v4YkFtdxyfiUXjh8cdJyj9rmqEXQ4PD6vNugoIn1ewpRFXtZfD5OkphhfmFrK2zU7WN/QFGCqvueZRXX8xx9XcsmEIdx8bmXQcY5J6cBsTqsYyFML63QJEJFjlDBlcejliS6dGDrh+9n3t8T8e+9tbmPu+p08v3QrTy+s47n3t/Dqyu0srdtFY1NLn/lF9dbaBm57bDFTygv5rytOIiVOLuVxLD4xYchf7u8tIkcvYe+UN6x/PyaV9ue597dw48dH9/jXb+9wnlpYx2/e2cDi2g84Uh+U5GcyuXQAZ1YW8YkThjAgJ+PwKwdkad0uvvLgfMoG5vC/M6ri7lIeR+uicSV855mlPL90K5V9cOxFJF4kbFkAXHLiEO58dgXrG5ooK8rpsa9b98F+vvbQAhZv+oAxJbn8/XmVTBheQEl+Fv3SU2ltd5pa2ti++wC1jft5v24X8zc08selW/ne7GVcNnEYN358dI9mOhZL63bxhV++S35WOr/62yl9bubTkQzKz2Jy6QCeX7aVm87r24fVRIKU0GVx0fjB3PnsCl5cvpUbzh7VI1+zevserpr1Ds2tHdx95UQunTg0qkt0uzvLNu/m0XmbeGTuJp5aWMeXz67g5nMr6ZcR3F/x79fuYsb975KbmcbDN0xlWP9+gWWJlWnjB/P/nlvBxh37KB2YHXQckT4pYcYsujKiMJuxQ/KZ00NTaLfvPsCM+94DjCe/djqXTRoW9b0czIwThhVwx6Un8Od//jifnjSMn/9pLZf8z59ZuXV3j+TrrldXbefKWW+Tk5HG7748lRGFifmLdNoJoRldc1ZoKrXI0UrosgC4YFwJ8zc0smNvc+SVj6C9w7nlkUV8sK+VB/72lGM6/j0oL4sfXXESD11/KnsOtHHZzDd5dF7vnTzm7vzmnQ1c/8A8yotyeOprpyf0X9wjCrOpKMrhz2t0ZWKRo5XwZXHhuBI6HF5euT3yykfw67fX89baHXx/+njGDy3okWxnjC7i2ZvPZNKIAfzT40v47jNLY37W+b6WNv7h0cV8++nQ9Z4e+cppDIrDy433tLMqi3i3ZifNbe1BRxHpkxK+LMYPzWdoQRYvLjv6QxANe5u5a85qzqos4oqqnr3q6qC8LB68bgrXn1nOA29v4Iv3vUdjU0uPfo+DVm/bw6dnvsVTi+q45fxK7rv2FHIzE3rY6i/OrCxmf2s7CzZ8EHQUkT4p4cvCzLhgXAlvVNezv+Xo/qr84fMr2d/Sznc/NT4m95tOS03h258cx4+vOIn5GxqZPvONHh3HaGvvYOar1Xzyv9+gfm8zD3xpCrecPyZubonaG6ZWFJKWYjoUJXKUEr4sAC4YN5gDrR1H9Yti9bY9PDa/lr89s5zRg3JjkO6vLj95OI98ZSrNrR185mdv8cKyrcf8NZfUfsBnfv4WP3phFeePG8SL3zibs8ck342i8rLSmVTanzeqG4KOItInJUVZnFpRSF5WGi8exayoe16pJjs9la9+rGem3kYyqXQAv7/pTCpL8vjKg/P56Utr6Ojo/hngDXub+efHl3DpzDfZ/MF+Zn5+Mj/7wskU5WbGIHXfcOboYt6v28UH+2JzmE8kkSVFWaSnpnDu8YN4ZeV22rvxi7d6+15+v2QzM04r69Wzrkvys3jkhql8ZtIwfvLSav7mV3PZsCO6a1w1NrXwoxdWcs6P/sQTC2q5/sxyXrntHC6ZoHs6TK0oxB3mb2gMOopIn5Mco5uEptA+s2gz8zc0MqW8MKptfvZqNVlpqVx/VnmM031UVnoq//W5k5hY2p8fPr+KC37yOp+dPJwZU0cydkjeh8ZOWts7WLChkacW1jF78Wb2t7bziROH8I3zx8T80FlfctKI/qSnGnPXN3Le2JKg44j0KUlTFh8bU0x6qjFn+daoymJ9QxNPL6rjujPLAzt0Y2Z88bQyLho/mLtfWs0TC2r53XsbKcrNZPSgHLIz0tixt5nq7XtpammnX3oqnzppCNefVdEn70ERa1npqZwwrIB56494h14R6ULSlEVeVjqnjSrixeXb+OYnxkac1fSzP1WTnprCl8+u6KWEh1eSn8V/fGYCt114HC+t2MY7NTupa9zP1l0HGJibwWdPHs7po4o4Y/TAD12qXT7qlLJCfvXmeg60tifMxRJFekPSlAWETtD79tNLWbN97xH/8t60cx9PLqjjmqkjGZQXPyesDczN5MpTSrnylNKgo/RZVSMHMOv1GpbW7aKqLLrDkSKSJAPcB10wLnScOtK1ov775TWkpBh/10szoKT3nDxyAABz12uQW6Q7kqosSvKzmFzanycW1B52Oura+r08saCWGVNHMrggfvYqpGcMzM1kVHGOxi1EuimpygLgi6eVUVPfxOuHOUHvrjmryUpP5avnaK8iUU0cMSB8w6q+cQdDkXiQdGXxiROHMCgvk1mv13zkl8U7NTt4dsmWQGdASeydNKKAhr0tbNl1IOgoIn1G0pVFRloKXz1nFG+t3cEfl/71chp7DrRy+xNLGDkwm6+d0/O3YZX4ceKw0FWDl9TqooIi0Uq6sgCYMXUk44bk851nllK9fS97DrRy/QPzqG3czw8/OyHQO9dJ7I0dkk9airGkdlfQUUT6jKSaOntQWmoK/331RD537ztcdPfrpKcaLW0d/OTKiZxaMTDoeBJjWempHDc4T2Uh0g1JWRYAowfl8dzNZ/HgO+tpam7nM5OHMWF4/6BjSS+ZMLw/zy7ZjLvH5LLzIokmacsCYHBBFv940fFBx5AATBhewO/e28iGHfsoK8oJOo5I3EvKMQuRCcNDg9yLNcgtEhWVhSSlMSV5ZKSmsHxLz92RUCSRqSwkKaWnpjB6UC4rtuwJOopIn6CykKQ1dkg+K7RnIRIVlYUkrbFD8qjf00z9nuago4jEPZWFJK1xQ/MBtHchEgWVhSStcUNUFiLRUllI0uqfncGQgiyVhUgUVBaS1EKD3JoRJRJJTMvCzKaZ2Sozqzaz27t4/WwzW2BmbWZ2+SGvtZvZovDH7FjmlOQ1dkgea+v30tzWHnQUkbgWs7Iws1RgJnAxMA642szGHbLaRuBvgN928SX2u/vE8Mf0WOWU5Hb84HzaOpya+qago4jEtVjuWUwBqt29xt1bgIeBSzuv4O7r3X0J0BHDHCKHVVmSC8DqbToUJXIksSyLYcCmTs9rw8uilWVm88zsHTO7rKsVzOyG8Drz6uu7vk2qyJGUF+WQmmJUb98bdBSRuBbPA9wj3b0K+Dxwt5l95KbY7j7L3avcvaq4uLj3E0qfl5mWysiB2dqzEIkglmVRB4zo9Hx4eFlU3L0u/LkG+BMwqSfDiRw0ZlAea7RnIXJEsSyLuUClmZWbWQZwFRDVrCYzG2BmmeHHRcAZwPKYJZWkVlmSy4Yd+zQjSuQIYlYW7t4GfB14AVgBPOruy8zsDjObDmBmp5hZLXAFcK+ZLQtvPhaYZ2aLgVeB/3R3lYXERGVJHu2aESVyRDG9U567Pwc8d8iy73R6PJfQ4alDt3sLODGW2UQOGhOeEbVm+17Ghi8BIiIfFs8D3CK94uCMqDUa5BY5LJWFJL2DM6LWbNMgt8jhqCxEgMpBuazerj0LkcNRWYgQuie3ZkSJHJ7KQgQYPSiX9g5nXYNmRIl0RWUhQmjPAtC4hchhqCxECM2IMkPnWogchspCBMhKT2VoQT9qGrRnIdIVlYVIWEVxjvYsRA5DZSESNqo4l5r6vbh70FFE4o7KQiSsvCiHppZ2tu9pDjqKSNxRWYiEVRTnALC2XuMWIodSWYiEVRSHLiiocQuRj1JZiIQNyc8iKz1FZSHSBZWFSFhKilFelKvpsyJdUFmIdKLpsyJdU1mIdDKqKIfaRl1QUORQKguRTiqKc+lw2LBjX9BRROJKVGVhZk+a2SVmpnKRhHZw+myNps+KfEi0v/x/BnweWGNm/2lmx8Uwk0hgyosOnmuhcQuRzqIqC3d/yd2/AEwG1gMvmdlbZvYlM0uPZUCR3pSXlc6gvEwNcoscIurDSmY2EPgb4HpgIfBTQuUxJybJRAJSUZyj6bMih4h2zOIp4M9ANvApd5/u7o+4+01AbiwDivS2iuJcauqbdEFBkU7Solzvf939uc4LzCzT3ZvdvSoGuUQCU1GUw679rexsamFgbmbQcUTiQrSHoe7sYtnbPRlEJF6MCl8jSvfjFvmrI+5ZmNlgYBjQz8wmARZ+KZ/QISmRhHNwRlRNQxNVZYUBpxGJD5EOQ11EaFB7OHBXp+V7gG/GKJNIoIYP6Ed6qmnPQqSTI5aFuz8APGBmn3X3J3opk0ig0lJTKC3MZp2mz4r8RaTDUNe4+2+AMjO79dDX3f2uLjYT6fPKi3K1ZyHSSaTDUDnhz5oeK0mlojiH19fU097hpKZY5A1EElykw1D3hj9/v3fiiMSH8qIcWto62PzBfkYUai6HSLQn5f3QzPLNLN3MXjazejO7JtbhRIJycEaUDkWJhER7nsWF7r4b+CSha0ONBv4xVqFEgnbw6rMqC5GQaMvi4OGqS4DH3H1XjPKIxIXi3ExyM9NUFiJh0V7u4w9mthLYD3zVzIqBA7GLJRIsM6O8KIe1uq+FCBD9JcpvB04Hqty9FWgCLo1lMJGglRflaM9CJCzaPQuA4wmdb9F5m1/3cB6RuFFelMPvl2zmQGs7WempQccRCVS0s6EeBH4MnAmcEv6IeLVZM5tmZqvMrNrMbu/i9bPNbIGZtZnZ5Ye8dq2ZrQl/XBvVv0akB1UU5+AOG3fqftwi0e5ZVAHjvBsX+DezVGAmcAFQC8w1s9nuvrzTahsJXXvqtkO2LQS+G/6+DswPb9sY7fcXOVYVRaFzUWvqmxhTkhdwGpFgRTsbaikwuJtfewpQ7e417t4CPMwh4xzuvt7dlwAdh2x7ETDH3XeGC2IOMK2b31/kmJQVhU7G013zRKLfsygClpvZe0DzwYXuPv0I2wwDNnV6XgucGuX362rbYVFuK9Ij8rLSKc7L1AUFRYi+LL4XyxBHy8xuAG4AKC0tDTiNJKIKzYgSAaKfOvsaoTO308OP5wILImxWB4zo9Hx4eFk0otrW3We5e5W7VxUXF0f5pUWiV1GsshCB6GdDfRl4HLg3vGgY8HSEzeYClWZWbmYZwFXA7ChzvQBcaGYDzGwAcGF4mUivKi/KYUdTC7v2tQYdRSRQ0Q5w3wicAewGcPc1wKAjbeDubcDXCf2SXwE86u7LzOwOM5sOYGanmFktcAVwr5ktC2+7E/g3QoUzF7gjvEykV5UfnBGlQW5JctGOWTS7e4tZ6Lr+4RPzIk6jdffngOcOWfadTo/nEjrE1NW29wP3R5lPJCY6X312UumAgNOIBCfaPYvXzOybQD8zuwB4DPh97GKJxIfSwmxSU3Q/bpFoy+J2oB54H/gKob2Fb8cqlEi8yEhLYcSAftSoLCTJRXUYyt07zOxp4Gl3r49xJpG4Ul6UQ43OtZAkd8Q9Cwv5npk1AKuAVeG75H3nSNuJJJLyolzWNzTR0RH11W5EEk6kw1DfIDQL6hR3L3T3QkJnYZ9hZt+IeTqROFBenMP+1na27dEtXCR5RSqLGcDV7r7u4AJ3rwGuAb4Yy2Ai8WLUwRlROhQlSSxSWaS7e8OhC8PjFumxiSQSX8rD9+PWILcks0hl0XKUr4kkjJK8LPqlp2qQW5JapNlQJ5nZ7i6WG5AVgzwicSclxSgrymGdzuKWJHbEsnB33UtShNAFBZfV7Qo6hkhgoj0pTySpVRTlsKlxPy1th96nSyQ5qCxEolBelEN7h7OpUffjluSkshCJwsELCmqQW5KVykIkChXhS5VrkFuSlcpCJAoF2ekMzMnQ1WclaaksRKKkCwpKMlNZiESpvEj345bkpbIQiVJ5cQ7b9zSz54Duxy3JR2UhEqWK8Iyo9Q2aPivJR2UhEqWK4tCMqBrNiJIkpLIQiVJpYTZmaNxCkpLKQiRKWempDOvfT2UhSUllIdINmj4ryUplIdINo4pzWdfQhLvuxy3JRWUh0g3lRTnsbW6jfm9z0FFEepXKQqQbdEFBSVYqC5FuGDUoNH12bb2mz0pyUVmIdMPQgixyM9NYvXVP0FFEepXKQqQbzIzKklxWbVNZSHJRWYh003EleazZpsNQklxUFiLdVFmSx46mFho0I0qSiMpCpJuOK8kD0LiFJBWVhUg3jSkJzYharXELSSIqC5FuKs7LpH92Oqs0biFJRGUh0k1mxpiSPO1ZSFJRWYgchTEluazetkfXiJKkobIQOQrHleSx50AbW3cfCDqKSK9QWYgchTHhGVGrNCNKkkRMy8LMppnZKjOrNrPbu3g908weCb/+rpmVhZeXmdl+M1sU/vhFLHOKdNfBstDJeZIs0mL1hc0sFZgJXADUAnPNbLa7L++02nVAo7uPNrOrgB8AV4ZfW+vuE2OVT+RYDMjJYFBeJiu27A46ikiviOWexRSg2t1r3L0FeBi49JB1LgUeCD9+HDjPzCyGmUR6zPih+SzbrLKQ5BDLshgGbOr0vDa8rMt13L0N2AUMDL9WbmYLzew1Mzurq29gZjeY2Twzm1dfX9+z6UUiGD+0gOr6vRxobQ86ikjMxesA9xag1N0nAbcCvzWz/ENXcvdZ7l7l7lXFxcW9HlKS27ih+bR3uAa5JSnEsizqgBGdng8PL+tyHTNLAwqAHe7e7O47ANx9PrAWGBPDrCLdNn5o6O8XHYqSZBDLspgLVJpZuZllAFcBsw9ZZzZwbfjx5cAr7u5mVhweIMfMKoBKoCaGWUW6bcSAbPIy01i2eVfQUURiLmazody9zcy+DrwApAL3u/syM7sDmOfus4H7gAfNrBrYSahQAM4G7jCzVqAD+Dt33xmrrCJHIyXFGDs0n+WaESVJIGZlAeDuzwHPHbLsO50eHwCu6GK7J4AnYplNpCeMH5rPw+9tor3DSU3RRD5JXPE6wC3SJ4wfWsD+1nZq6nVyniQ2lYXIMZgwvACAxbUat5DEprIQOQajinPJzUxj0abGoKOIxJTKQuQYpKYYE4YXsGjTB0FHEYkplYXIMZo4oj8rt+zRmdyS0FQWIsdo4oj+tHU4S+s0biGJS2UhcowmlvYH0KEoSWgqC5FjNCgvi2H9+7Fwo8pCEpfKQqQHTCztz8KNmhEliUtlIdIDppQVsnnXATbt3Bd0FJGYUFmI9ICpFaHbsLxTsyPgJCKxobIQ6QGVg3IpzMng3XW63qUkJpWFSA9ISTGmlBVqz0ISlspCpIdMrSiktnE/tY0at5DEo7IQ6SGnhsct3q3RoShJPCoLkR5yXEkehTkZvFndEHQUkR6nshDpISkpxtmVRfxpdT3tHR50HJEepbIQ6UEfP34QO5taWFyrs7klsagsRHrQx8YUk2Lw6srtQUcR6VEqC5Ee1D87g5NHDuAVlYUkGJWFSA8757hBLNu8m227DwQdRaTHqCxEetiF40oAeO79LQEnEek5KguRHlZZksfYIfk8s2hz0FFEeozKQiQGLps4lEWbPmB9Q1PQUUR6hMpCJAamTxyKGcxerL0LSQwqC5EYGFLQjyllhTy1sI4OnaAnCUBlIRIjV00ZwbqGJv6sy39IAlBZiMTIJScOpTgvk/vfWBd0FJFjprIQiZGMtBS+OHUkr62up3r73qDjiBwTlYVIDH3+1FIy0lKY9fraoKOIHBOVhUgMDczN5AunlvL4/FrW1mvvQvoulYVIjN348dFkpafyn39cGXQUkaOmshCJsaLcTG46t5I5y7fx4rKtQccROSoqC5FecP1Z5Rw/OI9vPb2Uhr3NQccR6TaVhUgvSE9N4SdXTmTX/lZueXgRre0dQUcS6RaVhUgvGTsknzsvO4E3qhv45yeW6Nar0qekBR1AJJl8rmoEW3cd4K45q9lzoI2fXjWR7Az9GEr8i+mehZlNM7NVZlZtZrd38XqmmT0Sfv1dMyvr9Nq/hJevMrOLYplTpDfdfF4l358+npdXbGP6PW8yd/3OoCOJRBSzsjCzVGAmcDEwDrjazMYdstp1QKO7jwZ+AvwgvO044CpgPDAN+Fn464kkhGtPL+NXX5rC/pZ2rvjF21z3q7m8saaBNo1lSJyK5f7vFKDa3WsAzOxh4FJgead1LgW+F378OHCPmVl4+cPu3gysM7Pq8Nd7O4Z5RXrV2WOKmXPr2cx6vYZfv72Bl+97lwHZ6UytGMgJwwqoHJTL4IIsSvKzKOiXTmZaCqEfD5HeF8uyGAZs6vS8Fjj1cOu4e5uZ7QIGhpe/c8i2w2IXVSQY2Rlp3HL+GP7uY6N4deV25izfxoKNjfxx6UfPx0gx6JeeSr+MNNJSjBQDM8MMUjp/Biz8mvRN/3P1JMYOyQ86xof06ZE1M7sBuAGgtLQ04DQiRy8rPZWLTxzCxScOAWD3gVbW1TexfU8zW3cfYPf+Vg60trOvJfTR3tGBO3Q4OB5+/OHP0nf1S4+/o+6xLIs6YESn58PDy7pap9bM0oACYEeU2+Lus4BZAFVVVfrxkISRn5XOSSP6Bx1D5C9iORtqLlBpZuVmlkFowHr2IevMBq4NP74ceMXdPbz8qvBsqXKgEngvhllFROQIYrZnER6D+DrwApAK3O/uy8zsDmCeu88G7gMeDA9g7yRUKITXe5TQYHgbcKO7t8cqq4iIHJl5ghzcrKqq8nnz5gWr8fS0AAADEklEQVQdQ0SkTzGz+e5eFWk9Xe5DREQiUlmIiEhEKgsREYlIZSEiIhGpLEREJKKEmQ1lZnuAVUHniFNFQEPQIeKQ3peu6X3pWqK+LyPdvTjSSn36ch+HWBXN9K9kZGbz9N58lN6Xrul96Vqyvy86DCUiIhGpLEREJKJEKotZQQeIY3pvuqb3pWt6X7qW1O9Lwgxwi4hI7CTSnoWIiMRIQpSFmU0zs1VmVm1mtwedJx6Y2Qgze9XMlpvZMjP7+6AzxRMzSzWzhWb2h6CzxAsz629mj5vZSjNbYWanBZ0pHpjZN8I/Q0vN7HdmlhV0piD0+bIws1RgJnAxMA642szGBZsqLrQB/+Du44CpwI16Xz7k74EVQYeIMz8Fnnf344GT0PuDmQ0Dbgaq3P0EQrdbuCrYVMHo82UBTAGq3b3G3VuAh4FLA84UOHff4u4Lwo/3EPrB133MATMbDlwC/DLoLPHCzAqAswndYwZ3b3H3D4JNFTfSgH7hu3lmA5sDzhOIRCiLYcCmTs9r0S/FDzGzMmAS8G6wSeLG3cA/AR1BB4kj5UA98H/hw3O/NLOcoEMFzd3rgB8DG4EtwC53fzHYVMFIhLKQIzCzXOAJ4BZ33x10nqCZ2SeB7e4+P+gscSYNmAz83N0nAU1A0o//mdkAQkcqyoGhQI6ZXRNsqmAkQlnUASM6PR8eXpb0zCydUFE85O5PBp0nTpwBTDez9YQOWZ5rZr8JNlJcqAVq3f3g3ufjhMoj2Z0PrHP3endvBZ4ETg84UyASoSzmApVmVm5mGYQGn2YHnClwZmaEjj+vcPe7gs4TL9z9X9x9uLuXEfq/8oq7J+Vfip25+1Zgk5kdF150HrA8wEjxYiMw1cyywz9T55GkA/99/kKC7t5mZl8HXiA0U+F+d18WcKx4cAYwA3jfzBaFl33T3Z8LMJPEt5uAh8J/dNUAXwo4T+Dc/V0zexxYQGiG4UKS9ExuncEtIiIRJcJhKBERiTGVhYiIRKSyEBGRiFQWIiISkcpCREQiUlmIiEhEKgsREYlIZSEiIhH9f0Krz6ChiEYvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot = df['R_log'].plot(kind='kde')\n",
    "plot.set_xlim(0,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It does look more \"normal\" (though not perfectly so). I most cases, we would want to identify the functional form of the distribution and convert it to a normal (or gaussian) distribution prior to learning from it. However, that complicates interpretation of our data so we will continue using the non-transformed variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "On average, players we're considering scored 6.0 runs per season.\n"
     ]
    }
   ],
   "source": [
    "total_median_runs = df['R'].median()\n",
    "print(\"On average, players we're considering scored {0} runs per season.\".format(round(total_median_runs,2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's focus just on the features we think will be predictive of runs:\n",
    "\n",
    "- birth month\n",
    "- weight\n",
    "- height\n",
    "- batting hand\n",
    "- throwing hand\n",
    "\n",
    "We can subset our data to reflect these features of interest. We can also set our X (explanatory variable) and y (response variables)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.reset_index()\n",
    "df = df[['birthMonth','weight','height','bats_B','bats_L','bats_R','throws_L','throws_R', 'R']].dropna()\n",
    "X = df[['birthMonth','weight','height','bats_B','bats_L','bats_R','throws_L','throws_R']]\n",
    "y = df['R']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's split our data into training and test sets. We will specify that 20% of that data should be held back for testing purposes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can fit the ordinary least squares regression model to our data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                      R   R-squared:                       0.398\n",
      "Model:                            OLS   Adj. R-squared:                  0.398\n",
      "Method:                 Least Squares   F-statistic:                     861.9\n",
      "Date:                Tue, 07 Aug 2018   Prob (F-statistic):               0.00\n",
      "Time:                        12:35:22   Log-Likelihood:                -45507.\n",
      "No. Observations:               10420   AIC:                         9.103e+04\n",
      "Df Residuals:                   10412   BIC:                         9.109e+04\n",
      "Df Model:                           8                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "birthMonth     0.0661      0.054      1.222      0.222      -0.040       0.172\n",
      "weight        -0.0194      0.011     -1.751      0.080      -0.041       0.002\n",
      "height         0.1381      0.034      4.050      0.000       0.071       0.205\n",
      "bats_B        12.4639      1.662      7.500      0.000       9.206      15.722\n",
      "bats_L        13.5614      1.575      8.610      0.000      10.474      16.649\n",
      "bats_R         6.3940      1.544      4.141      0.000       3.367       9.421\n",
      "throws_L      -5.8218      1.744     -3.338      0.001      -9.240      -2.403\n",
      "throws_R       0.8867      1.675      0.529      0.597      -2.397       4.170\n",
      "==============================================================================\n",
      "Omnibus:                     2988.970   Durbin-Watson:                   2.004\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):             7012.318\n",
      "Skew:                           1.629   Prob(JB):                         0.00\n",
      "Kurtosis:                       5.354   Cond. No.                     3.47e+03\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
      "[2] The condition number is large, 3.47e+03. This might indicate that there are\n",
      "strong multicollinearity or other numerical problems.\n"
     ]
    }
   ],
   "source": [
    "results = sm.OLS(y_train, X_train).fit()\n",
    "print(results.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that a non-trivial amount of variance (40%) is explained by our model. This isn't overwhelming, to say the least, but we also learned that the majority of our features have some bearing on average runs as well. It seems that being able to bat with your left hand (or both hands) is associated with a significant increase in runs. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generating predictions\n",
    "Remember the 20% of data we held back for testing purposes. Now we get to use it! We can generate predictions on our test data and score the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = results.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can evaluate our model's performance by calculating the Mean Absolute Error (MAE) using `sci-kit learn`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14.636058993229732"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_error(y_test, preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can calculate it ourself if we'd like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14.636058993229723"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(abs(preds - y_test))/len(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classification\n",
    "To perform classification, we need to convert runs, which is a continuous variable, into a categorical or dichotomous variable. To do this, we want to categorize a certain number of runs as _good_ and anything below that as _bad_.\n",
    "\n",
    "We'll use the median runs as a cutoff point. Anything above the median should be considered _good_ (or a `1`) and anything below _bad_ (a `0`). This way we can use classification algorithms to determine whether we expect a given player, based on his stats, to be a good or bad scorer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def above_avg(runs, median):\n",
    "    if runs > median:\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "\n",
    "df['R_'] = df['R'].apply(lambda x: above_avg(x, total_median_runs))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we have to swap our y variable for this new binary variable called `R_`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df[['birthMonth','weight','height','bats_B','bats_L','bats_R','throws_L','throws_R']]\n",
    "y = df['R_']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Naive Bayes\n",
    "Now we can begin running classifiers on this data. We can start with Naive Bayes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.551\n",
      "Precision: 0.585\n",
      "Recall: 0.473\n",
      "0.004 seconds\n"
     ]
    }
   ],
   "source": [
    "nb = MultinomialNB()\n",
    "start_time = time.time()\n",
    "nb.fit(X_train, y_train)\n",
    "y_pred = nb.predict(X_test)\n",
    "end_time = time.time()\n",
    "acc = accuracy_score(y_test, y_pred)\n",
    "pre = precision_score(y_test, y_pred)\n",
    "rec = recall_score(y_test, y_pred)\n",
    "print('Accuracy: {}'.format(round(acc, 3)))\n",
    "print('Precision: {}'.format(round(pre, 3)))\n",
    "print('Recall: {}'.format(round(rec, 3)))\n",
    "print('{} seconds'.format(round(end_time - start_time, 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can compare accuracy, precision, recall, and time to run for each of the algorithms below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.591\n",
      "Precision: 0.632\n",
      "Recall: 0.517\n",
      "0.04 seconds\n"
     ]
    }
   ],
   "source": [
    "logreg = LogisticRegression()\n",
    "start_time = time.time()\n",
    "logreg.fit(X_train, y_train)\n",
    "y_pred = logreg.predict(X_test)\n",
    "end_time = time.time()\n",
    "acc = accuracy_score(y_test, y_pred)\n",
    "pre = precision_score(y_test, y_pred)\n",
    "rec = recall_score(y_test, y_pred)\n",
    "print('Accuracy: {}'.format(round(acc, 3)))\n",
    "print('Precision: {}'.format(round(pre, 3)))\n",
    "print('Recall: {}'.format(round(rec, 3)))\n",
    "print('{} seconds'.format(round(end_time - start_time, 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Support Vector Machine (SVM)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.521\n",
      "Precision: 0.521\n",
      "Recall: 1.0\n",
      "0.489 seconds\n"
     ]
    }
   ],
   "source": [
    "svmclf = LinearSVC()\n",
    "start_time = time.time()\n",
    "svmclf.fit(X_train, y_train)\n",
    "y_pred = svmclf.predict(X_test)\n",
    "end_time = time.time()\n",
    "acc = accuracy_score(y_test, y_pred)\n",
    "pre = precision_score(y_test, y_pred)\n",
    "rec = recall_score(y_test, y_pred)\n",
    "print('Accuracy: {}'.format(round(acc, 3)))\n",
    "print('Precision: {}'.format(round(pre, 3)))\n",
    "print('Recall: {}'.format(round(rec, 3)))\n",
    "print('{} seconds'.format(round(end_time - start_time, 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that SVM did not predict any positive classes. In other words, no players in the test set were predicted to be above the median. This isn't a terrible prediction given what we know about our data; it is log-normal but we are not transforming it. Also, based on our regression, less than half the variance is explained by the features we've chosen. A conservative model might not be willing to predict that any players are above median!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.549\n",
      "Precision: 0.571\n",
      "Recall: 0.537\n",
      "0.789 seconds\n"
     ]
    }
   ],
   "source": [
    "rfclf = RandomForestClassifier(n_estimators=100)\n",
    "start_time = time.time()\n",
    "rfclf.fit(X_train, y_train)\n",
    "y_pred = rfclf.predict(X_test)\n",
    "end_time = time.time()\n",
    "acc = accuracy_score(y_test, y_pred)\n",
    "pre = precision_score(y_test, y_pred)\n",
    "rec = recall_score(y_test, y_pred)\n",
    "print('Accuracy: {}'.format(round(acc, 3)))\n",
    "print('Precision: {}'.format(round(pre, 3)))\n",
    "print('Recall: {}'.format(round(rec, 3)))\n",
    "print('{} seconds'.format(round(end_time - start_time, 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the benefits of random forests is that we can determine the relative importance of each feature:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>weight</td>\n",
       "      <td>0.460055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>birthMonth</td>\n",
       "      <td>0.259751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>height</td>\n",
       "      <td>0.208208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>bats_R</td>\n",
       "      <td>0.019312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>throws_R</td>\n",
       "      <td>0.015576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>throws_L</td>\n",
       "      <td>0.014530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>bats_L</td>\n",
       "      <td>0.013850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>bats_B</td>\n",
       "      <td>0.008720</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      feature  importance\n",
       "1      weight    0.460055\n",
       "0  birthMonth    0.259751\n",
       "2      height    0.208208\n",
       "5      bats_R    0.019312\n",
       "7    throws_R    0.015576\n",
       "6    throws_L    0.014530\n",
       "4      bats_L    0.013850\n",
       "3      bats_B    0.008720"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = [col for col in X_train]\n",
    "\n",
    "feature_imp = pd.DataFrame({'feature':features, 'importance':rfclf.feature_importances_}) \\\n",
    "                .sort_values(by='importance', ascending = False)\n",
    "feature_imp.head(10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}