DM_class_07-09.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# 데이터마이닝\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# k-NN & Naive Bayes"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## k-NN 분류기(범주형 결과)\n",
    "* 모델 중심이 아닌 데이터 중심\n",
    "* 데이터에 대한 가정을 하지 않음\n",
    "* 간단한 아이디어: 레코드를 유사한 레코드로 분류\n",
    "* 분류하고자 하는 새로운 레코드의 유사한 k개의 이웃 레코드를 식별\n",
    "* 이웃 레코드를 사용해 새로운 레코드를 특정 클래스로 분류. 예측 변수 값 X1, X2, ... Xp가 유사한 레코드를 의미\n",
    "* 이웃 레코드들이 속한 클래스들을 참조하여 분류하고자 하는 레코드에 클래스 할당"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 이웃 결정하기\n",
    "* 가장 보편적으로 사용되는 거리 측도는 유클리드 거리\n",
    "\n",
    "\n",
    "## 분류 규칙\n",
    "* 가장 단순한 경우는 k = 1\n",
    "* k > 1의 이웃을 갖는 경우\n",
    "* ** 분류될 레코드와 가장 가까운 k 개의 이웃을 찾는다\n",
    "* ** 레코드를 분류하기 위해 다수결 결정 규칙을 사용한다. 즉, 레코드는 k개 이웃들의 다수가 속하는 클래스로\n",
    "분류된다."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import preprocessing\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.neighbors import NearestNeighbors, KNeighborsClassifier\n",
    "import matplotlib.pylab as plt\n",
    "import koreanize_matplotlib\n",
    "import numpy as np"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "data": {
      "text/plain": "   Income  Lot_Size Ownership  Number\n0    60.0      18.4     Owner       1\n1    85.5      16.8     Owner       2\n2    64.8      21.6     Owner       3\n3    61.5      20.8     Owner       4\n4    87.0      23.6     Owner       5",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Income</th>\n      <th>Lot_Size</th>\n      <th>Ownership</th>\n      <th>Number</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>60.0</td>\n      <td>18.4</td>\n      <td>Owner</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>85.5</td>\n      <td>16.8</td>\n      <td>Owner</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>64.8</td>\n      <td>21.6</td>\n      <td>Owner</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>61.5</td>\n      <td>20.8</td>\n      <td>Owner</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>87.0</td>\n      <td>23.6</td>\n      <td>Owner</td>\n      <td>5</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mower_df = pd.read_csv('https://raw.githubusercontent.com/reisanar/datasets/master/RidingMowers.csv')\n",
    "mower_df['Number'] = mower_df.index + 1\n",
    "mower_df.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14, 4) (10, 4)\n"
     ]
    },
    {
     "data": {
      "text/plain": "   Income  Lot_Size\n0      60        20",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Income</th>\n      <th>Lot_Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>60</td>\n      <td>20</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainData, validData = train_test_split(mower_df, test_size = 0.4, random_state= 26)\n",
    "print(trainData.shape, validData.shape)\n",
    "newHousehold = pd.DataFrame([{'Income':60, 'Lot_Size':20}])\n",
    "newHousehold"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAHFCAYAAADosxNlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABpuUlEQVR4nO3dd1xT1/sH8E8mG1SG4N4b90LFuvWnVdHWUVDrXjjRqv26cO9RrQNR1Lpw1lGtWrXuUa1arXVitQ4UBdkrJPf3R0pqZAVISAKf9+vFS3Puzb1PHgJ5OPfcc0SCIAggIiIiMkNiYwdARERElFMsZIiIiMhssZAhIiIis8VChoiIiMwWCxkiIiIyWyxkiIiIyGyxkCEiIiKzxUKGiIiIzBYLGSIiE8X5SomyJjV2AETmLjY2Fg0bNoRSqcxwn6JFi2Lz5s0oX768Vnvv3r1x69YtrTZnZ2dcvHgRADBo0CB89dVXaNOmDRYuXAg7Ozv4+vrmONYNGzbg3bt3mDp1Kn755RcEBwdj06ZNmu2CIKT7OqRS7V8VN27cwJQpU3Dq1Kkcx6IvMTExaNWqFU6cOIEiRYpkuf+yZcuwadOmTL9fAHDy5EmULl06030GDRqk+V59zNnZGfv370fRokUBAFOnTkXp0qUxdOjQLONLdenSJcyaNQsnT55Ms23Dhg149uwZ5s+fr9X+/fffY/Xq1VnGc+PGDUybNg3Hjx/X2u/t27fo1q0bwsPDM41NKpXit99+g42Njc6vh8hQWMgQ5ZKtrS1u376d7l/PCQkJ8PPzw7Nnz+Ds7Ky1TaVS4datWzhy5IjWB6ZIJNL8Pzk5GUlJSZr/JycnZxrL9u3bsXDhQigUCq32OXPmoGfPnkhKSsr0eBs2bMDy5cvTHFcul2PDhg3w8PDQOZa8cvv2bURHR8PS0lKn/f/44w/4+fmhf//+Ge4jEokgkUiyPFZgYCBUKpVWW2hoKNq0aaP1Pfg477qKjIzM8DUlJSWlm39fX18MHz5cq+369esYMGAAxOL/OuAz+v49ffoUIpEId+/e1dr/U7rmhygvsJAh0gO5XJ6m7f79+5g4cSIkEgl27twJe3t7re2pH4DW1tawsLDQSxz37t1D3759MWHCBK32T3tUMjJs2DAMGzZMq+3Dhw9o3LgxHBwc9BKjvh09ehQAsGfPnkyLk1SCIMDGxkbnnGRGLBan+cC/e/cuSpUqhRIlSuTq2E+ePMn2MUQiUZrXtWvXLjRq1ChNIZ0eQRBgYWGR7vuZyFRxjAyRnkVERMDf3x9ffPEF6tSpgz179sDV1VWn5yYnJyMmJibH5xYEAXK5HFKpVOsrN44dO4bKlSujWrVquTqOITx79gzHjx/HwoULsXbtWrx69cqo8ahUKmzatAk9evTI9bEuX76M9+/f5+oYf/zxB06ePIlBgwZBEASkpKQgJSUlTS8SkTljIUOkJ//88w/mzJmD1q1b48WLF2jTpg1++eUXLF26FI8ePcrwee3atUO1atVQuXJl1K9fHwMGDNDaPnv2bDRv3hw//vijoV9CGkqlEsHBwfDx8cl0vwMHDuDLL79M03727Fm0b98eABAWFoYxY8agfv36qF27Nry8vBAaGprj2OLj4/HNN9+gZ8+e6NatGwYNGoSvv/4ar1+/zvR5IpEIMTExmss9GX3lxMaNG/Hhwwf07ds3zTaVSoWUlJQsx+YA6t6Y27dv46+//sK9e/fS3UcQhAwvMQFAeHg4xo4di3bt2qF58+ZYuXIlqlevjurVq2PQoEHZe2FEJoyXlohyQaVSYePGjThx4gSeP3+Otm3bIjg4GJUrVwYAvHr1Cjt27NCMW2jfvj0mT56sdYyTJ09meglh5MiRaNGiBb7//nvDvZAMbN26FUqlEl988UWm+3l6emLq1KkICwuDi4uLpv3kyZNo2bIlAPWAV3t7exw4cADW1tZ49uxZji9XxcbGYujQoZBIJJg4cSIA9WUxhUKBL774AjNnzkSHDh3SfW7t2rWxcuVKLFu2LNNzBAYGonnz5jrHtHfvXqxbtw5btmyBlZVVmu1r167F2rVrIZFIcOvWrQwvJwqCgHnz5sHDwwONGzeGv78/goOD04xJ+emnn/DTTz8BUPeafTyQ/N27dxg+fDikUilu3ryJO3fuYPz48Rg/fjwAdW/PtGnTdH5tRKaMhQxRLojFYkRFRWHkyJHw9PRMM7agePHimDRpEiZNmoQHDx7g+fPn2T6Hk5MTSpcuDTs7O73EvHv3buzevRsA0LBhwwz3CwkJwapVq/DZZ59leXnK2dkZNWvWxLlz5zSXVVQqFc6dO4dVq1YBUN8ps3v3bpQqVQqA+nXlxMWLFzFv3jw4ODhg7dq1WjkfNWoUatWqhenTpyMgIAAjR45E27ZttZ7v5+cHPz+/HJ07PbGxsVixYgUOHjyI77//HrVq1Up3v1GjRmH06NFZHm/nzp24efMm9u/fjzJlyuDKlSv49ttvsXDhQq3xOJ07d8bSpUvTPP/mzZsYP3486tevj/nz5+Pnn39G//790bVrV4wcOTLTsTIikUjTy8PBvmQueGmJKJe++eYbtG7dOssBklWqVNFcZgEAiUQCsViMiIgIxMXFISIiAo8fP8bp06cREBCA2NhYg8Tbo0cP3LlzB4sWLcpwn7CwMAwbNgyDBw/G3bt3sWPHjiyP26pVK5w7d07z+M6dOxCJRKhTpw4AoGrVqti+fXuOL9sA6ruqRowYgc8++ww//PADihQpoimYUsd9eHp64uTJk+jTpw/i4+M1r6dy5co5+goICEg3lpSUFGzduhVt2rTBvXv3sG/fPjRt2jTHrw1QD1xetGgRFixYgAoVKkAqlWL16tV4/vw5vv76a7x48SLD53748AGTJ0/GoEGDMGDAACxbtgwWFhbw8vLC3r178ezZs3TvSPtYuXLlIAgC3N3dNZeh0vuqVq1ajopyIkNgjwxRDmU050pWUv+aFYlEaN26NXx8fCCXy2FlZQUHBweUKVMG5cqVQ0pKCmQyGWQymV7jFovFsLCwyPC4L1++xKBBg9CpUyeMGjUKbdu2Rb9+/eDo6Jjh5RoAaN26NQICAqBQKCCTyfDrr7+idevWmr/sV65ciRkzZqBdu3bw9fVF9+7dsz0QuWPHjujSpYvW4OnUy0xXrlzRzCMjl8u1Loe5uLjg7t27OZpgLqM8JSUl4dSpU5g+fTo6duyoddt8Tty7dw/+/v5YsWIFWrdurWm3t7fHli1bsGLFCrx9+xYlS5ZM9/kJCQmwsrLC0aNHUaxYMa1t5cuXx+bNmzXvV2tr63Tn3ClatCguX76s1TZmzBjUqFEjW3PgEOUlFjJEORQYGJjlOIv0iEQinD9/Hi4uLlmOe9mwYYOmC79JkyZZzpUiFosRHR2NyMhIKJVKKBQKvHr1Cn///bdmMrTM3LhxA2PHjsWXX36pGU+R2isxfPhwvH//Hn369En3uRUqVICTkxNu3LgBDw8PnDlzBlOmTNFsd3Fxwfr163Ht2jXMnTsXwcHB2LZtW7YmVcvNLc36vqXYxsYG27Zt09vxqlevjlOnTqU7bsjKygr/+9//Mn1+sWLF4O/vr3mc0cDilJQUVKtWDfv27ct1zESmgIUMUQ4NHTo0w79Su3fvrunV0MWMGTM041YyYmFhkeWlgSZNmmDGjBk4cOAALCwsYGFhAUdHR5QuXRqff/55ps999uwZRowYgZkzZ6bZt3bt2tixYwdu3LiR6TFatWqFs2fPomTJknj79i0aNWqUZp9GjRph165d6NatG/bu3avT3C/68uzZM63LexkpXrw4jhw5onORNWvWLLRp0ybXl5ZSi5gVK1agWLFi6NWrV46Os3HjRixdujTTHqimTZsiKCgoR8cnMiUsZIhMwMyZMzF16tRM95k8eTLu3buHNm3aZLjP559/nmnB8uDBgwy3lSlTJtNp/suXL59miYVPtW7dGtOnT0fx4sXRsmXLDC8d2draomLFitma90WlUqU7/0lqr0Pq7c2f+njSujJlyuCvv/7K9ANeqVSiXr16CA0NRYUKFXSK7cGDB3B3d89wu4+PT7YGa7969SrTy25eXl6Ii4vLcHtISAgGDBiQ5g65VL/++ivmzJmj1ZZe7gD1JdSMcgvoPtkikaHwHUhkAiQSSZZ3gVhaWhp8IjNd1irKTN26dfHhwwfs3LlTc1s0ACgUCuzfvx8NGzaEra0tLly4gPPnz2PdunU6H3vIkCHprmuUKqPekLJly2qtKZRVnqVSKUQikV5znTrgWV8yGieTKnVixIx8euv30aNHM72T6+TJk1ixYkW623bv3o3atWtnGg+RIbGQISK9kUgkaN++PU6cOAFPT09Ne1JSEnbt2oU5c+ZAJBKhbNmyWLBggWYfpVKJwYMHw9fXF/Xr10/32B8vbmlqxGIxEhISsrwjS9elKMRiMRITE7M8nlwuT3eQsUgkyjSeT++I69Spk86XQYlMDQsZIgOQSqUmOc+GhYWF5lJAbtbUkcvlGT539uzZmD17tlabra0tDh06lOHxlEolnj59isjIyBzFY2w1a9bEvHnz0rzuTwUHB+vUO+Pu7o5FixZh48aNme63aNEieHl5pWkvV64cVqxYga1bt6b7PJFIlOklSiJzIhJycj8iEeW5kJAQyOXyLC8rUO4dPHgQ7dq1g7W1tbFDIaIssJAhIiIis8WZfYmIiMhssZAhIiIis2XUwb7nzp1DUFAQIiIiIAgC6tati2+//TbdlWP79u2LV69e4cyZM0aIlIiIiEyRUXtkrK2tsXjxYhw5cgQHDx5EXFycZqXcjx0+fBiWlpYZTshEREREBZNRC5kGDRpo1n+RSqUYPHhwmgmvYmNjsWHDBowdO9YYIRIREZEJM6l5ZCIjI9NMGLVmzRr07NkThQoVyvFxIyJiYOAJUU2aSAQ4OtohPDwGBfkeNebhP8yFGvOgxjyoMQ9qppCH1Bh0YVKDfYODg7UmdwoJCcHly5fh7e1tvKCIiIjIZJlMj8yFCxfw4MEDLFmyRNM2Z84cTJw4MdeLkhUpovtibfmZrtVtfsc8/Ie5UGMe1JgHNeZBzVzyYBKFTGhoKGbMmIFVq1Zppj3/+eefYWlpqbVeS07x0pLxuwlNAfPwH+ZCjXlQYx7UmAc1U8hDdi4tGb2QiY+Ph6+vL8aOHQt3d3cAQHJyMpYvX47AwEC9nEMQUKDflKmYBzXm4T/MhRrzoMY8qDEPauaSB6MWMkqlEn5+fvD09NQaGxMXFweFQqF1p5JCoUBERAS6du2K4cOH4//+7/+MEDERERGZEqMWMvPmzYOFhQXGjRun1V64cGGcPXtWq+3ly5fw9vbOdAVdIiLK2okTx7By5VIULeqqabO2tsT33wdCLDa9VduJMmO0QiYqKgo7duxA2bJltXpjRCIRNm7cCCcnJ639pVJprgf9EhGRujfcw6MpZsyYA0A9HsHJyQ7v3xfssSFknoxWGTg4OODhw4c67+/q6srlCYiIiEiLSc0jQ0RERJQdLGSIiIjIbHHQCRGROVEpIQu9BnFcGFQ2LlC4NQKyOUBXJBLh9u2bGDFiEKKjo1CiREmMHu2LkiUrGChoIsNhIUNEZCbkIcdge2EmJHGhmjaljRtiPWchuXxHnY/TokVrNG/eAjY2thAEAVevXsKIESOwbt0mlChRyhChExkMLy0REZkBecgx2B8fBvFHRQwAiOPewP74MMhDjul8LCsrK9jY2AJQ9840adIMrVu3xtWrl/UaM1FeYCFDRGTqVErYXpgJQIDok00iqO+Xtr3oD6iUOT+FSgWJhHPIkPlhIUNEZOJkodcgiQtNU8SkEkGAJPY1ZKHXdDreu3dhSElJ0Tz+9dfTuHDhApo3b6mHaInyFsfIEBGZOHFcmF73u3btMnbu3AaZTL1Ib+nSpbF161Y4ODhxQjwyOyxkiIhMnMrGRa/7ff65Fz7/3Evz+OOZfYnMDS8tERGZOIVbIyht3NIZIaMmQASlbTH1rdhEBQwLGSIiUyeWINZzFoC0w31TH8c288/2fDJE+QELGSIiM5BcviOiOwRAZeOq1a6ydUN0h4BszSNDlJ9wjAwRkZlILt8REWXb53pmX6L8hIUMEZE5EUugKN7E2FEQmQxeWiIiIiKzxUKGiIiIzBYLGSIiIjJbLGSIiIjIbLGQISIiIrPFQoaIiIjMFgsZIiIiMlssZIiIiMhssZAhIiIis8VChoiIiMwWCxkiIiIyWyxkiIiIyGyxkCEiIiKzxUKGiIiIzBYLGSIiIjJbLGSIiIjIbLGQISIiIrPFQoaIiIjMFgsZIiIiMlssZIiIiMhssZAhIiIis8VChoiIiMwWCxkiIiIyWyxkiIiIyGyxkCEiIiKzJTV2AEREBV1cXCwCA9fh1q2bEIlEsLKyxMCBw9CgQSNjh0Zk8ljIEBEZ2fTp36JOnbrYvHkHxGIxHjy4j8mTxyMgYDNcXd2MHR6RSeOlJSIiI7t58zq6desBsVj9K7lKlaqoXLkKHj68b+TIiEwfCxkiIiOrXt0du3fv0Dy+ffsm7t69g6pVqxsxKiLzwEtLRERGNnWqPyZOHIN79+6iVKnS+OWX45gxYzZcXIoaOzQik8ceGSKi3FApIXt1GRaPDkL26jKgUmb7EK6ubujevQd+//069u3bjQYNGqNKFfbGEOmCPTJERDkkDzkG2wszIYkL1bQpbdwQ6zkLyeU76nyc2bOn4+XLF1ixYg2KFnXF5s2B6N+/NwIDf2CvDFEW2CNDRJQD8pBjsD8+DOKPihgAEMe9gf3xYZCHHNPpOC9fvsDVq5ewcuVa1K1bH8WLl8C0abPQsKEHfvxxnyFCJ8pXWMgQEWWXSgnbCzMBCBB9skkEAQBge9Ffp8tMcXFxcHJyhq2trVZ7uXIVEBMTrZ94ifIxFjJERNkkC70GSVxomiImlQgCJLGvIQu9luWxKlSoCCsra+zevQMqlQoA8OrVSxw58iPatfs/PUZNlD9xjAwRUTaJ48L0tp9EIsGSJd9hw4Y1GDDAG2KxGJaWlhg5cgxq1qydy0iJ8j8WMkRE2aSycdHrfoUKFcKkSVNzExJRgcVLS0RE2aRwawSljVs6I2TUBIigtC0GhRvXSiIyNBYyRETZJZYg1nMWgLTDfVMfxzbzB8SSvI6MqMBhIUNElAPJ5TsiukMAVDauWu0qWzdEdwjI1jwyRJRzHCNDRJRDyeU7IqJse8hCr0EcFwaVjYv6chJ7YojyDAsZIqLcEEugKN7E2FEQFVi8tERERERmi4UMERERmS2jX1o6d+4cgoKCEBERAUEQULduXXz77bewsrJCWFgYVqxYgT///BMikQj29vb49ttvUb06V4UlIiIiE+iRsba2xuLFi3HkyBEcPHgQcXFxWLVqFQBAEAR0794dR44cweHDhzF48GD4+voiKSnJyFETERGRKTB6IdOgQQMULapepl4qlWLw4MG4ePEiAKBo0aJo0KCBZt8WLVrAwcEBT548MUqsREREZFqMXsh8KjIyEhYWFhluj46OznQ7ERERFRxGHyPzqeDgYHh5eaW77dy5cyhSpAgqVKiQrWOKROqvgir1tRfkHADMw8eYCzXmQY15UGMe1EwhD9k5t0gQBMFwoWTPhQsXMHfuXBw5cgRyuVxrW0JCArp27YpZs2bBw8PDSBESERGRKTGZHpnQ0FDMmDEDq1atSlPEAMC0adPQunXrHBUxERExUKn0EaV5EokAR0c7hIfHwHTK1rzHPPyHuVBjHtSYBzXmQc0U8pAagy5MopCJj4+Hr68vxo4dC3d39zTbAwICEBkZicWLF+fo+IKAAv2mTMU8qDEP/2Eu1JgHNeZBjXlQM5c8GL2QUSqV8PPzg6enZ7pjY44ePYqffvoJO3fuhETC9UuIiIjoP0YvZObNmwcLCwuMGzcuzbbff/8dS5Yswfbt22Fnp1sXE1F+tXdvMH766SAAERSKZFStWg3Dh4+Gs7OLsUMjIjIaoxYyUVFR2LFjB8qWLavVGyMSibBx40Zs3LgRSUlJ8PX11Xpenz590KNHjzyOlsi4mjb1RJcu3WBhYYGUlBRs3hyISZPGYfPmncYOjYjIaIxayDg4OODhw4cZbl+3bl0eRkNk2ooVK675v1QqxaBBw7BvXzDev38HJydnI0ZGRGQ8JjchHhHpJjEx8d81yByMHQoRkdEYfYwMEWXf06chWLduFQYMGJLudAVERAUFCxmivKBSQhZ6DeK4MKhsXKBwawSIs38X3po13+HEiWOIiAhH585e6NHjKwMES0RkPnhpicjA5CHHUOSHxih0sCfsfxmFQgd7osgPjSEPOZbtY/n6jsXhwydw9OgpyOVyzJ8/ywARExGZDxYyRAYkDzkG++PDII4L1WoXx72B/fFhOSpmAMDBoRDGjp2I8+fPIjY2Vh+hEhGZJRYyRIaiUsL2wkwAAj5d/0wE9XSZthf9AZUyR4dXKJKRkqKAKofPJyLKD1jIEBmILPQaJHGhaYqYVCIIkMS+hiz0WpbHUigUCAt7q3kcExODuXP90aJFa961REQFGgf7EhmIOC5Mb/tFRn7AlCkTkJiYALncAmKxCG3adEDPnhzsS0QFGwsZIgNR2ei2dIAu+zk7uyAoaHtuQyIiynd4aYnIQBRujaC0cUtnhIyaABGUtsXUt2ITEVGOsJAhMhSxBLGe6tujPy1mUh/HNvPP0XwyRESkxkKGyICSy3dEdIcAqGxctdpVtm6I7hCA5PIdjRQZEVH+wDEyRAaWXL4jIsq218vMvkREpI2FDFFeEEugKN7E2FEQEeU7vLREREREZouFDBEREZktFjJERERktljIEBERkdliIUNERERmi4UMERERmS0WMkRERGS2WMgQERGR2WIhQ0RERGaLhQwRERGZLRYyREREZLZYyJBR/PTTIbRs6YHQ0NcZ7jNq1FD06NElD6PSv+fPn6FlSw8EBW0wdihERPkSF42kPBcYuA4PHtyHnZ09lEpluvucPPkzLC0tkZKSksfR6dd33y1F3boNzP51EBGZKvbIUJ5SqVRwdHTC4sUrIJfL090nPj4O27ZtxuDBI/I4Ov06e/Y0ChcugmrVqhs7FCKifIuFDGVbilKV4+eKxWJ0794DEokkw32CggLRpUt3ODg45Pg8xpaYmIiNGwMwYsRoY4dCRJSvsZChbDl89w2ar76Ew3ffGOT4z579jRs3fkO3bl8a5Ph55YcfgtCuXQc4OTkbOxQionyNY2RIZ4fvvsGck48AQPNvF3dXvZ5jxYolGDFiNKRSI701VUrIQq9BHBcGlY0LFG6NAHHGvUfpefXqJc6dO4OgoB0GCpKIiFKxkCGdfFzEpNJ3MXPmzClYWFigUSMPvRwvu+Qhx2B7YSYkcaGaNqWNG2I9ZyG5fEedj/Pdd0sxZMgIWFhYGCJMIiL6CC8tUZbSK2JSzTn5SC+XmRQKBQICvsfo0eNzfayckIccg/3xYRB/VMQAgDjuDeyPD4M85JhOx7l69TISExPRokVrQ4RJRESfYI8MZSqzIiaVPnpm4uPjoFAoMH36FE1bSooCHz5EoH9/b/TrNxCtWrXJ8fEzpVLC9sJMAAJEn2wSQYAAEWwv+iOibPssLzOFhr5GWFgY+vf31rRFRIQDAK5du4I1awJhaWmp5xdARFRwsZChDOlSxKTKbTHj4FAIBw4c1WoLDX2NkSMHY8uWnTk6pq5kode0Lid9SgQBktjXkIVeg6J4k0yP1a3bl2kGKm/aFAClUomhQ0fqJV4iIvoPLy1RulKUKiw8/Thbz1l4+nG2bs2WSmWZDuqVSCR5MuhXHBem1/0+JZVKjTd4mYgon+NvV0qXVCLGlNYVde6RAYAprStCKtG9Ng4OPpDpdheXoti797DOx8splY2LXvf71NdfD8rR84iIKGvskaEMdXF3xfR2lXTad3q7Snq/FTuvKNwaQWnjls4IGTUBIihti6lvxSYiIpPCQoYypUsxY85FDABALEGs5ywAaYf7pj6Obeaf7flkiIjI8FjIUJYyK2bMvoj5V3L5jojuEACVjfZrUdm6IbpDQLbmkSEiorzDMTKkk9Ri5eMxM/mliEmVXL4jIsq2z/XMvkRElHdYyJDOUouWhacfY0rrivmqiNEQS7K8xZqIiEwHCxnKli7uruhYzSVbdycREREZCj+NKNtYxBARkangJxIRERGZLRYyREREZLZYyBAREZHZYiFDREREZitHdy2FhYXh0qVL+PvvvxEZGYlChQqhbNmyaNKkCYoWLarvGImIiIjSla1C5vHjx1i9ejUiIiLQuHFjVKlSBQ4ODoiKisLTp08xfvx4ODk5wdfXF5UrVzZUzEREREQAslHIBAYGIjQ0FOPGjUO5cuXS3WfUqFEICQnB1q1bUbJkSQwZMkRvgRIRERF9SudCxtPTE1WqVMlyv/Lly2P27Nm4f/9+rgIjIiIiyorOg311KWI+VrVq1WwHQ0RERJQdOvXIPHz4ECkpKbofVCrlGBkiIiIyOJ0KmUWLFkGhUOh8ULlcjk2bNuU4KCJDiI6OwvTpk/DNN1NhZ+dg7HCIiEgPdCpkgoKCDB0HkcEdPHgA69evR6VK1dCnT39jh0NERHqglwnxNm7ciNjYWH0cishgDh36UetfIiIyfzoXMufOncONGzfStG/YsAG3b9+Gra2tXgMj0qfIyA+4dOkCAODixfOIjPxg5IiIiEgfdL79eu7cubC3t0d0dDSGDBkCLy8vLF26FM+fP8fq1atzHMC5c+cQFBSEiIgICIKAunXr4ttvv4WVlRUAICQkBDNnzkRMTAxEIhFGjhyJdu3a5fh8VDAdP34MSqUSAKBUKnHixM/o1cvbyFEREVFu6dwjY2lpif3792Pt2rW4fPkyGjdujHfv3mHNmjWQy+U5DsDa2hqLFy/GkSNHcPDgQcTFxWHVqlUAgKSkJIwcORJjxozBoUOHEBgYiGXLluHBgwc5Ph8VTIcP/wiJRAIAkEgkOHyYl5eIiPKDbK+1VLFiRaxcuRKnTp3CggUL8PDhQ1SvXj3HATRo0OC/YKRSDB48GJMmTcLkyZNx8eJFVK1aFQ0bNgQAODs7Y+DAgdi/fz+mTp2a43NS/vTPP88xcGAfxMXFpdn2999PoVKpAKh7ZE6f/gUeHnXT7GdjY4OgoO0oVaq0weMlIqLcy9GikQDQpk0bFC1aFOPGjcPq1auzPWFeRiIjI2FhYQEAuHLlilahA6gLnx9++EEv56L8pXDhwhCJxAgJeZLlviqVKt39ateui8KFCxsiPCIiMoAcFzIA4O7ujsWLF2PMmDHYvXu3Xj4AgoOD4eXlBUC9ynaTJk20tru5ueHFixfZOqZIpP4qqFJfe37Pgb29PX7++RQWL16A775bBpFIpOmFyYxYLIYgCBg7dgImTfoWMpksD6I1roLynsgK86DGPKgxD2qmkIfsnFskCIKgy463bt1CnTp10t22d+9eREVFYfDgwbqfOR0XLlzA3LlzceTIEcjlcvTv3x9DhgxB06ZNNfuoVCpUrVoVDx48gKigv9soQ6dPn8ZXX32FiIgIzSDf9EgkEhQpUgTBwcFo1apVHkZIRET6oHOPTEZFDAD06NEj14GEhoZixowZWLVqlWbwsFwuR1JSktZ+iYmJkMvl2SpiIiJioMMf5vmWSAQ4OtohPDwGupWt5q9WrYY4f/4aBg7siytXLmW4X8OGjbF583Y4Ojri/fuYPIzQuArieyI9zIMa86DGPKiZQh5SY9BFri4t6Ut8fDx8fX0xduxYuLu7a9pdXV0RGhqqtW9oaChcXV2zdXxBQIF+U6YqaHlwdHSCq6sbJBIplMq0a4VJJFK4uRVDkSKOBSovHyto74mMMA9qzIMa86BmLnnQqZCZPn16thaNlMlkmD17tk77KpVK+Pn5wdPTUzM2JlWdOnVw9uxZ+Pj4aNquX7+eae8QUaqkpKR/54/5770rFos/unspBcePH0VSUpJmgDkREZkXnQqZjh07ahUygiBg5syZGRYrUqnuHT3z5s2DhYUFxo0bl2Zb+/btsWrVKvz2229o2LAh3r17h6CgICxZskTn41PBde7cGSQkxAMARCKRZsLFGzduaB7Hx8fj/Plf0bZtByNHS0REOaFTxeHh4ZGmzcrKCp6enrk6eVRUFHbs2IGyZctq9caIRCJs3LgRTk5OWLduHfz9/REfHw9BEDB69GjUqlUrV+elguGnnw4DUA/otbS0xHffrcWgQf2wceNWjBvni8TERCiVShw5cpiFDBGRmdLprqWWLVsiMTFRqy06Ohr29vZabYIgwNraGmfOnNFvlLkUHs7Bvk5Odnj/vuAMYFMoFKhcuQxiY2NQt259BAZuQalSpTR5eP78OYYM6Y9bt36HnZ09Hjz4u0Dcdp2qIL4n0sM8qDEPasyDminkITUGXejUI/Prr79muC0mJgavX79G5cqVdYuOKA+Eh7+HUpkCP79vMHHit2kud5YqVRpHj/6CpUsXYN267xEe/h6urm5GipaIiHJK57WWbt++nW7727dv8d133+krHiK9cHV1w9OnrzFlyvQMx2xJpVJMmTIdT5++ZhFDRGSmdC5kpk+fnm57qVKlsj3TLlFeSF0kUl/7ERGR6dG5kMmIXC6HjpMDExEREelVrgsZANmaY4aIiIhIX3Se8EUkEuHq1atpuuFVKhXi4uL0HhgRERFRVnQuZLp3744tW7aku61Xr176ioeIiIhIZzoXMv3790f//v0NGAoRERFR9uhljAwRERGRMbCQISIiIrPFQoaIiIjMlu7LVBMRERHpICkpEdu3b8WlSxcgCCokJyfDz28y6tVroPdzsZAhIiIivUlJScGECWNQt259rF8fpJk4V6lUGuR8OhUygwYNgkKhSHebm5sbFi1ahDlz5uDJkyeaWX7Hjh2LevXq6S9SIiIiMnknThyDra0tBg4cqmkTiUQZrnuXWzoddfr06ZpCRhAEjBw5EuvWrQMAWFpaAgDOnDmD1atXa55TtmxZfcdKREREJu706ZPo0aN3np1Pp0KmWLFiWl1CMpkMJUuWhCAImkLG0tISNWrUMEyUREREZBaePHkMCwtLTJ36DV6+fAEHh0Lw9u6Hxo2bGOR8OhUyXl5eSEpK0jwWBAEdO3aESCTC+PHj8fnnnxskOCIiIspjKiXw9wXIQ59BZe0ChVsjQCzJ+nn/ioqKxNatm+DnNxmlS5fBkyePMWnSOEybNgt169bXe7g6FTLHjh1L05acnIy3b9+iZMmSANTXv4iIiMh8yUOOwfbiTCA2FPb/tilt3BDrOQvJ5TvqdAyxWAxv734oXboMAKBChYro1csbR48eNkgho/M8Mi9evND8/+3bt3j69KnWmBgiIiIyX/KQY7A/Pgzi2FCtdnHcG9gfHwZ5SNpOjfQULlwEJUuW0morXrwkIiMj9RWqFp0LmWHDhgEA4uLi4Ofnh7JlyyIkJMQgQREREVEeUilhe2EmAAGfXl8RQX03su1Ff/VlpyxUqVINT58+0Wp7+fIflChRQj+xfkLnQib1tmobGxvExcXBwsICiYmJabYTERGReZGFXoMkLjRNEZNKBAGS2NeQhV7L8ljdun2JgIA1CA9/DwB49uxv7Nu3G92799RjxP/R+abu9MbAiMX/1UGcM4aIiMg8iePC9LZfgwaN0LOnN3x9h0IsFsHS0goTJ36rGTOjbzoXMgkJCfjrr7+gUqmQlJSEe/fuITY2VrN97ty5BgmQiIiIDEtl46LX/Tp39kLnzl65iEh3OhcyHh4eWLBgAQDAyckJixYtQvv27Q0WGBEREeUNhVsjKG3cII57oxkT8zEBIqhs3dS3YpsYnQuZ+fPnGzIOIiIiMhaxBLGes2B/fBgEiLSKmdThv7HN/LM1n0xe0Xmwb0ZmzpypjziIiIjIiJLLd0R0hwCobF212lW2bojuEKDzPDJ5LUcrOAUFBaFz585wdnbGzZs39R0TERERGUFy+Y74UK49nOLuIjqHM/vmNZ16ZP744w8kJydrHu/btw+FCxc2WFBERERkJGIJUNYTyZW8oCjexKSLGEDHQmbw4MEYPHgwUlJScOfOHVSoUMFgy3ETERER6UqnasTR0RElS5bEsmXLEBcXBy8vLwOHRURERJQ1nQoZmUyGadOmwcvLCxYWFpg9e7bW9oiICK2ZfcViMS89ERERkcHpfH3IysoKfn5+CA4OTrOtR48eWoWMTCbDiRMn9BMhERERUQayNdClffv2CAwMxKNHj1CpUiVN++nTp/UeGBEREVFWsj2PTL9+/bBjxw5DxEJERESULdkuZP7v//4PV69ehUqlMkQ8RERERDrTqZAZNWqU5v8ymQy1atXC9evXDRYUERERkS50GiPz6eKQkyZNgr29vUECIiIiItJVjma1c3Jy0vx/2LBheguGiIiIKDt0HiOzevVqxMbGpmn//PPP07RFR0djxYoVuYuMiIiIKAs6FzJeXl6YNGkSVqxYgRcvXqS7zz///IMlS5Zg0qRJ+PLLL/UWJJGp+emnQ2jZ0gOhoa+NHQoRUYGm86WlkiVLYu3atfj111/h7++PN2/eoGTJkrCzs0NUVBRevHiBYsWKwcfHB998840hYyYyqsDAdXjw4D7s7OyhVCqNHQ4RUYGW7TEyLVu2RMuWLZGUlISXL18iOjoa9vb2KFGiBCwsLAwRI5HJUKlUcHR0wuLFK9Crl5exwyEiKvByvIS1hYUFypcvr89YiEyeWCxG9+49jB0GERH9K8eFTFRUFH799VfExcXBx8dHnzERERER6STbM/sCwJUrV9CtWzfcvHkTP/zwg6b91atXmDRpkt6CI9IblRKyl5eBu/vU/6o4toWIKD/IUY/MkiVLEBQUhDJlysDLy0vTXrx4cTx58kRfsRHphTzkGGwvzIQkLhQA4ABAaeOGWM9ZSC7f0bjBERFRruSoRyYxMRFlypQBAIhEIq1tCoUi10ER6Ys85Bjsjw+D+N8iJpU47g3sjw+DPOSYkSIjIiJ9yFEhY2FhgVevXqVpv3nzJqysrHIdFJFeqJSwvTATgADRJ5tEEAAAthf9eZmJiMiM5ejS0qRJkzB06FD4+PggLi4Ox48fx507d7B//36sXLlSzyES5Yws9JrmclJ6RBAgiX0NWeg1KIo3ycPIiIhIX3LUI+Ph4YFNmzbh7du3qFSpEvbt24fk5GQEBQXBw8ND3zES5Yg4Lkyv+31KKpVBKs3xjX9ERKQHOf4t7OrqivHjx+szFiK9Utm46HW/TwUHH8jR84iISH9y1CNTu3ZtLF26FCkpKWm2ffXVV7kOikgfFG6NoLRxS2eEjJoAEZS2xaBwa5THkRERkb7kqJApVqwYEhMT0bt37zQLSCYkJOglMKJcE0sQ6zkLQNrhvqmPY5v5A2JJXkdGRER6kuO7lqZNmwZfX18MGDAABw8e1Gz79HZsImNKLt8R0R0CoLJx1WpX2bohukMA55EhIjJzuRqp2LJlS9SoUQNTpkzBxYsX4e/vr6ewiPQnuXxHRJRtD3noNThIYhCltEOyWyP2xBAR5QM56pERBEHzf2dnZ2zatAlVqlTBl19+iXfv3uktOCK9EUugKNEEcP9S/S+LGCKifCFHPTJDhw5N0zZ48GA0btwYO3fuzHVQRERERLrIUY9Mx47pjyuoUaMG5s+fn6uAiIiIiHSlc4/M2rVrMXLkSADA//73v3RvvQYAmUyGefPmZTuQffv2wd/fH8ePH0eJEiU07YcOHcKOHTuQnJwMpVKJtm3bYtSoURCLc1SDERERUT6icyFTs2ZNzf+bNGmS4eKQMpks20GsXLkSf/75JxwcHKBU/rfuzdGjR7Fjxw6sX78eRYoUQWxsLCZMmIBNmzZhyJAh2T4PERER5S86FzLNmjXT/P/zzz/XWwAqlQrOzs4ICAhA27ZttbZduXIFnTp1QpEiRQAAtra26NatGw4ePMhChoiIiHQfIxMZGZlu+/Pnz7F06VLMnz8fT548yX4AYjF8fHwgkaS9i6R27drYv38/oqOjAQAxMTHYvHkzGjZsmO3zEBERUf6jc4/MgAEDsHPnTlhZWWna7t+/j379+sHb2xvOzs4YNWoU5s+fj7p16+oluO7du+Pu3bvo3LkzunfvjoMHD8LT0xP9+/fP1nFEIvVXQZX62gtyDgDm4WPMhRrzoMY8qDEPaqaQh+ycW+dCRhAErSImOTkZEydOxOjRo9GvXz8AQLt27TBjxgxs3bpV9wgyIRaL0aNHD1y7dg1r165F8eLF0aVLl2wP9C1SxE4v8Zg7R0fmAWAePsZcqDEPasyDGvOgZi550LmQ+XTpgS1btsDGxkZTxABA6dKlERMTo7fgzp49i8mTJ2PEiBHo2rUrjh07Bl9fX0yYMAE9e/bU+TgRETFQqfQWltkRidRvyPDwGHw0l2GBwzz8h7lQYx7UmAc15kHNFPKQGoMusnXX0p49e9CzZ0+cP38eGzZswK5du9Lsl5SUpHukWQgICMDkyZPRvXt3AICPjw8qVKiQ7UJGEFCg35SpmAc15uE/zIUa86DGPKgxD2rmkgedr9H4+fnhxIkTqFu3LiZMmIBly5ahYsWKWvs8ePAAhQoV0ltwsbGxqFChglZbpUqV9NrrQ0REROZL5x4ZBwcHbNq0CREREbC1tYVcLk+zT5UqVbB582a9BderVy8sW7YMy5cvh6OjI5KSkrBixQp4eXnp7RxERObip58OYdmyhdi5cz/c3IoBADZvDsS5c79q7RcbG4Nq1Wpg9uwFxgiT8gmlUolhwwammTfu9etXmDVrPjw8mhopMm3ZXmspdU6XjKQWOFOmTMHChQuzdWyZTAap9L+Q+vTpA7lcjiFDhkCpVEIQBDRv3hxjxozJbthERGYtMHAdHjy4Dzs7e62JQwcMGIIBA7Tn1Vq6dAEqVKiU1yFSPiORSBAYqH3zTlJSEnr18kKNGjUzeFbey9Gikbp4+PBhtp9z4sSJNG09e/bM1ngYIqL8RqVSwdHRCYsXr0CvXl6Z7hsfH48LF85i5Ej+wUf6d+bML6hXrwHs7EznjiYuWERElAdSlDm/dVIsFqN79x7pThz6qV9+OY7GjZvC2tomx+cjysihQwfQpUs3Y4ehhYUMEZGBHb77Bs1XX8Lhu28Mfq5Dhw6gc2fT+qCh/CEk5AliYqJRq1YdY4eixWCXloiISF3EzDn5CAA0/3ZxdzXIue7fv4eUFAVq1HA3yPHJzKiUkIVegzguDCobFyjcGgHirHv1MnLw4H58/rmX/uLTExYyREQG8nERk8qQxYwpdvuTcchDjsH2wkxI4kI1bUobN8R6zkJy+Y7ZPl5CQgJ+/fUUtm/fq88w9cJgl5YEc5hFh4jIQNIrYlLNOflI75eZ4uJiceHCWbRrl/0PKcpf5CHHYH98GMQfFTEAII57A/vjwyAPOZbtY54+fQL16tXX61xx+qL3QiYhIQEA0L59e30fmojILGRWxKTSdzFz/PgxNG7cBPb29no7JpkhlRK2F2YCEPDpuosiqDsYbC/6Ayrlp8/M1MGDpjv2KkeFzLx58zLcNnjwYADAiBEjchYREZEZ06WISaXPYubwYdP9oKG8Iwu9BklcaJoiJpUIAiSxryELvabzMR8+fIDo6CjUq9dAP0HqWbYLGZVKhRs3bkAQBKhUKq2vFy9eICwszBBxEhGZvBSlCgtPP87WcxaefpytW7OlUu2JQwHgr7/+hCAIqF27brbOTfmPOE63z2Bd9wOAI0cOwsvryzSLR5uKbA323bBhA5YvXw6RSIRq1appbROLxShUqBAGDRqk1wCJiMyFVCLGlNYVde6RAYAprStCKtH9b8rg4ANp2qpVq4Efftit8zEo/1LZuOh1PwCYOHFKTsPJE9kqZIYOHYqhQ4fCy8sLBw8eNFBIRETmK/VuJF2KmentKhnsVmwqmBRujaC0cYM47o1mTMzHBIigsnVT34qdT+RojMyAAQP0HQcRUb7Rxd0V09tlvtYRixgyCLEEsZ6zAKQd7pv6OLaZf67mkzE1OZpHpmvXrgCA69ev4+HDhxCLxahWrRpq166tz9iIiMxWZj0zLGLIkJLLd0R0h4A088iobN0Q28w/R/PImLIcFTIRERHw9fVFYmIi6tSpA0EQsGfPHtjZ2WHNmjW8/Y+ICOkXMyxiKC8kl++IiLLt9Tqzr6nKUSGzYMECtGrVCkOGaC8dv379eixcuBDz58/XS3BEROYutWhZePoxprSuyCKG8o5YAkXxJsaOwuByVMjcvn0bS5YsSdM+fPhwtGvXLtdBERHlJ13cXdGxmku27k4iIt3k6Kcqs+UHuDQBEVFaLGKIDCNHP1kVKlTA4cOH07T/+OOPqFQp85H6RERERPqSo0tL//vf/9C/f3+cOHEC9erVA6C+gykkJARbtmzRZ3xEREREGcpRIVOqVCkcP34cR44cwaNHjyAWi9GpUye0b98eMplM3zESERERpStHhQwAyOVyfPHFF2naT506hTZt2uQqKCIiIiJd6H302Zo1a/R9SCIiIqJ06b2Q4V1LRERElFf0XsiY6jLfRERElP/oPEbmxo0bUCgUWe4XHx+fq4CIiIiIdKVzIbNu3TokJydnuZ+bm1uuAiIiIiLSlc6FzKZNmwwZBxER5dKVKxcRHLwDkZEfoFKpULNmbYwe7QdLS0tjh0ZkMJwzm4gon7Cyssa0abOwdWswNm/eifj4eGzcuN7YYREZFAsZIqJ8onbtunB2dgEASKVSeHv3xfXrV40cFZFhsZAhIsqnoqOjIZfLjR0GkUGxkCEiyqcOHtyPDh06GTsMIoNiIUNEZApUSsheXYbFo4OQvboMqJS5Oty1a1fw+PEjdO7cTU8BEpmmHK+1RERE+iEPOQbbCzMhiQvVtClt3BDrOQvJ5Ttm+3hv377B4sXzMG/eYl5aonyPPTJEREYkDzkG++PDIP6oiAEAcdwb2B8fBnnIsWwdLyEhAd9+OxFDhoxAlSrV9BkqkUliIUNEZCwqJWwvzAQg4NPFXURQr1tne9Ff58tMSqUSM2f+D40bN+HYGCowWMgQERmJLPQaJHGhaYqYVCIIkMS+hiz0mk7H++67pbCwsMCQISP0FySRieMYGSIiIxHHheltv+joaBw4sBelSpXGgAE+mnaRCFi2bDWKFHHMcZxEpoyFDBGRkahsXPS2n729PS5evJHbkIjMDi8tEREZicKtEZQ2bumMkFETIILSthgUbo3yODIi88FChojIWMQSxHrOApB2uG/q49hm/oBYkteREZkNFjL53JUrFzFmzAh07twZffr0xJIl85GYmKi1z6tXL+Ht/QU2bQowUpREBVdy+Y6I7hAAlY2rVrvK1g3RHQJyNI8MUUHCMTL5nJWVNaZPn4WqVcvjzZsPmDvXHxs3rseoUeMAAH/+eRcLF85B8eIloFTmbiZRIsqZ5PIdEVG2PWSh1yCOC4PKxkV9OYk9MURZYo9MPpfVargfPoRj8eIVnDiLyNjEEiiKN0FSJS8oijdhEUOkI/bImIEUpQpSiX5qzk9Xw/X0bKGX4xIRERkDe2RM3OG7b9B89SUcvvtGL8fjarhERJSfsJAxYYfvvsGck4+gUAqYc/JRrosZroZLRET5DS8tmajUIuZjqY+7uLum95RMhYaGYtEiroZLRET5C3tkTFB6RUyqnPTMJCQkwNfXF0OHcjVcIiLKX1jImJjMiphU2SlmUlfD9fT05NgYIiLKd1jImBBdiphUuhYzqavhjhs3LpfRERERmR6OkTERKUoVFp5+nK3nLDz9GB2ruWR4a/bHq+F6eXkhJUUFIP3VcGUyGcRi1rVERGReWMiYCKlEjCmtK+rcIwMAU1pXzHR+mdTVcEUiwMnJDu/fx0AQ0t+3X7+B2Q2ZiIjI6PgnuAnp4u6K6e0q6bTv9HaVcnT3EhERUX7CQsbE6FLMsIghIiJSYyFjgjIrZljEGMdPPx1Cy5YeCA19rdV+8+YNjBgxEF9/3Rs+Pl/ip58OGidAIqICimNkTFRqsfLxmBkWMcYRGLgODx7ch52dvdYK4SEhTzBvnj+WLPkO5cqVR0xMDCZNGgdX12KoX7+hESMmIio42CNjwlJ7ZmQSEYsYI1GpVHB0dMLixSvSzIh88OB+9OrljXLlygMA7Ozs4Os7DgcP7jNGqEREBRJ7ZExcF3fXTG+xpszlduVwsViM7t17pLvt1auXaNy4iVZbuXLlcP/+Xzk+HxERZQ8/Hc0Ai5ic0ffK4Z8qXLgQXr9+pdX28uULfPgQYZDzERFRWibxCblv3z7UqFEDL1++TLPt119/Rd++fdG9e3d06tQJs2fPNkKEZG70vXJ4ejp16oo9e3bh77+fAgDevQtDYOA6iEQivZ+LiIjSZ/RLSytXrsSff/4JBwcHrYGUALB3717s378fS5cuRYkSJQAACoXCGGGSGclw5XARMKilnd7OU7dufUyYMBkrVy5BZOQHODgUQv/+g/Hw4QO9nYOIiDJn1EJGpVLB2dkZAQEBaNu2rda2mJgYLFu2DEeOHIGzs7OmXSaT5XWYZEYyXTn8xCPY2VqiVdlCejtf48ZNtMbJ3L9/D2XLltfb8YmIKHNGvbQkFovh4+MDiUSSZtu5c+fQqFEjrSKGKDO6LLo5ef8dHDLQmBkAOHLkIFq0aGWw4xMRkTajX1rKyIMHD1CuXDl8//33OHXqFMRiMVq2bIlhw4aluQ02KyKR+qugSn3t+TkHh3RcOVyAumcGALrm4Hb2j99Lwr8LV4lEIigUCuzfvxuPHj2En98kk891QXhP6IJ5UGMe1JgHNVPIQ3bObbKFTGRkJM6dO4eJEydi//79SEpKwtSpU+Hv74/58+dn61hFiuhvXIQ5c3TMn3lQKFVYeCqbK4efeox+zctDlo07wiwtLeDs7AAnJ3UeHz16hMmTJwMAUlJS4OHhgW3btsLe3j5bsRhTfn1PZBfzoMY8qDEPauaSB5EgZLQect5q1aoVNm/ejNKlSwMAZsyYgdjYWCxfvlyzT2RkJDw9PXHjxg1YWFjofOyIiBioVHoP2WyIROo3ZHh4xqtfm7tDd99oelp0Mb19pRz1yOQXBeE9oQvmQY15UGMe1EwhD6kx6MJke2SKFCkCJycnrbZChQrB2toaMTEx2SpkBAEF+k2ZKj/noUsNV0BAlpeXRACmta+ELjVc820usiM/vyeyg3lQYx7UmAc1c8mDScwjkx53d3c8eqT9ofThwwcolUo4OjoaKSoyZbqsHL7oi5oFuieGiCi/MdlCxtPTE48fP8apU6cAAMnJyZg9ezb69u3LCccoQ5muHN6+Eno2KJnHERERkSGZTCEjk8kglf53pUsul2Pt2rXYsmUL2rdvj06dOqF48eLw9fU1YpRkDtIrZqa3K9hjYoiI8iuTGSNz4sSJNG3ly5fH9u3bjRANmbvUlcIXnn6MKa0rcuVwIqJ8ymQKGSJ948rhRET5H3/DU77GIoaIKH/jb3kiIiIyWyxkiIiIyGyxkCEiIiKzxUKGiIiIzBYLGSIiIjJbLGSIiIjIbHEeGSIi0olKpYJSmWLsMAxGJAISExOhUCSbxWKJhpIXeZBIpBCL9dOXwkKGiIgyJQgCoqMjkJAQa+xQDC4iQgyVSmXsMIwuL/JgZWULe/siuV4/kYUMERFlKrWIsbUtDLncIl8v3CuRiKBUFuDumH8ZMg+CICA5OQmxsR8AAA4Ojrk6HgsZIiLKkEql1BQxtrb2xg7H4KRSMVJS2CNj6DzI5RYAgNjYD7CzK5yry0wc7EtERBlSKpUA/vvgIdKX1PdUbsddsZAhIqIs5efLSWQc+npPsZAhIiIis8UxMkRElG8pFArs2rUNZ86cgiAIUKmU8PBoiv79B8Pa2sbY4ZEesJAhIqK8oVJCFnoN4rgwqGxcoHBrBIglBjudIAiYNGkcnJycsWbNBtjY2CIpKQlbtmzEqFHDsHbtRlhaWhrs/JQ3WMgQEZHByUOOwfbCTEjiQjVtShs3xHrOQnL5jgY55/nzvyI2NgbLlq3W3BVjYWGBYcN88eLFc+zevQNffz3IIOemvMMxMkREZFDykGOwPz4M4o+KGAAQx72B/fFhkIccM8h5DxzYhx49vkr31t6ePb3x889HAQDz58/Czz//pLV9xIhBmDNnulZbYOA6HD78I5RKJbp374Q9e3bBx+dL9O3bE+PGjcTbt280+6pUKgQErEHv3t3g7f0Fpk2bjKioSM32Ll3a4+rVyxg8uB/mzJmhx1dd8LCQISIiw1EpYXthJgABn96jIoJ6wjXbi/6ASqn3Uz969ADVq7unu61q1ep4/fol4uPj0LhxU5w/f1azLSIiHIIg4M6dPzS3nwPAhQtn0aRJM0gkEoSHv8f9+/ewZcsubNu2B/XqNcTKlUs1++7evRPh4e+xffte7Ny5H9Wq1cCKFUs022NjY3D58gVs2LAF06fP1vdLL1BYyBARkcHIQq9BEheapohJJYIASexryEKv6f3c8fFxKFIk/VljZTIZ7OzsEBsbi8aNPfDHH7egUCgAAJcvX0Tz5i1QuXIV3Lt3FwDw6tVLWFhYwMnJGYB6fp3Bg4dDJpMBADp2/Bx//HFLc/y9e3dhzJgJkErVIzh69fLGlSsXNYVRcnIy2rfvpLf1hgoyjpEhIiKDEceF6XW/7LC2tkF4+HuUKFEyzTaFQoHo6GjY2NjA2toGVapUxe3bv8PDowkuX76IYcN84eBQCFeuXELNmrVx8eI5NGv2mdYxihZ11fzfwaEQoqOjAACxsbEID3+PUaOGau1vZWWN6OgoFC5cBABQpkwZPb/igomFDBERGYzKxkWv+2VH+fIV8Ndff6ZbyNy/fw/Ozi6wsbEFADRt2hxXrlxCvXr18ebNa5QuXQa2trb45ptxGDbMFxcvnsf48ZN0Om9SUiJkMhm2bNmZ6X4WFrxjSh/Yp0VERAajcGsEpY1bOiNk1ASIoLQtpr4VW8/atGmPvXt3pbuK8549O9GqVVvN42bN1IXM779fR5069QEAjo5OEIvFePbsb0REhKNcufI6nbdw4SKQSCR4+jREPy+EMsVChoiIDEcsQaznLABph/umPo5t5m+Q+WQ+/7wr5HILzJvnj9jYWABAUlISAgLW4PHjR+jXb6Bm36JFXWFpaYldu3agaVNPTXujRh5YuXIJGjduqvN5xWIxOnbsgpUrlyAmJgYAkJKSgnfv9H/5jFjIEBGRgSWX74joDgFQ2bhqtats3RDdIcBg88hIpVIsXboKLi5FMXz4AHz99Vfo3/8rREVFYt26TbC3117Nu3nzlnj8+CFq1qytaWvWrDl+//06WrRopbWvpaWl1lpBIpFIa3I9X9+xcHevhREjBuLrr7/C0KFfaw0GtrKy4vpVeiISBEEwdhCGFh4eg3R6FgsMkQhwcrLD+/cxyP/f7YwxD/9hLtSYB7XM8qBQJCM8PBSOjm6QyeS5O1Eez+ybE1KpGCkpBfgD4195kYfM3lup70ldcLAvERHlDbEEiuJNjB0F5TO8tERERERmi4UMERERmS0WMkRERGS2WMgQERGR2WIhQ0RERGaLhQwRERGZLRYyRESUp1KUeTNPy9ixIzFoUF98Ol3au3dh6Nmza57EQIbHQoaIiPLM4btv0Hz1JRy++8bg51IqU/D+/TucPPmzVntKSgoUCoXBz095g4UMERHlicN332DOyUdQKAXMOfkoT4qZfv0GYvPmQCQnJxv8XGQcnNmXiIgMLrWI+Vjq4y7uruk9RS9KlSqNevUa4MCBPejdu0+6++zevQMHD+6HSCSCTCZDnz790bZtBwDA69evMGfOdNSsWQcXLpyFWCxBmTJlMGnSNM1aTSkpKdi0KQCnT5+EVCqFtbUNhg0biQYNGuPNm1CMHj0ce/ce0pzv+vWrmDhxLLZt24NSpUoDAOLj4zBwYB8EB/+ITZsCIAgC7t69g/DwdwAAL68v8OWXvTXHePz4IZYvX4To6GiIxWL06zdQE/MPPwRBKpXi0aOHePbsb0ycOAU1atTUd2pNBgsZIiIyqPSKmFR5UcwMHDgUQ4Z8jU6dusLOTnv9ngMH9uLMmVNYu3YTChcujLCwNxg/fhTs7R3QqJEHRCIR/vrrHtzda2HHjn0QiURYunQBtmwJxJgxEwAAGzaswT//PMfWrcGwsrLC48cPMXmyH5YuXYVy5crDwsICjx8/QsWKlQAA586dRb16DXHlykVNIXP16hXUrVtfE9eePbvw3XdrUbVqdURGRmLgQB/UqlUHFStWRkJCAmbO/B9mzpyHypWrIDz8PXx9h6JSpSooXboMFAoFfvrpEObMWYTKlasYLK+mgpeWiIjIYDIrYlIZ+jKTo6MTOnTohG3bNqfZtn37FkyYMBmFCxcGABQrVgxDh/qm2Xfw4BGa1ao7dPgct2+rV7JOSEjA4cM/YtKkqbCysgIAVKxYGb16eWPXrm0A1CtoX7lyUXOsu3dvY/hwX1y5cknTdunSOXh6fqZ57On5GapWrQ4AKFSoEDw8mmpWz/7ll+No2rS5pkhxdHRCp06dcfr0Sc3zS5YsXSCKGICFDBERGYguRUwqQxczPj79cOrUCbx9+985YmNjERMTjUqVtD/wa9asjZCQJ5rHhQsXgVz+3+rMhQoVQkxMNADg5csXcHJyQZEijukc4zEA7ULmyZPHKFOmLCpVqoI3b94gISEBSqUSt2/fQr16DTXPd3EpqnU8B4dCiI5Wn/PZs6c4ffok+vf31nwdO3YEiYkJmv3LlCmT7RyZK15aIiIivUtRqrDw9ONsPWfh6cfoWM0FUon+/8a2sbFF7959sGlTAAYMGAIAEItFGewtQPJRDKk9MVp7/HtLtySDWAVBgFgsAQBUq1YDoaGhiI6OwqVL59G0qbrnpX79Bvj9999gY2OLqlWraxVLmZ0zKSkJX3zREz4+X2f4ei0sLDPclt+wR4aIiPROKhFjSuuK2XrOlNYVDVLEpOrW7UvcuXNb09tibW0DR0dnPHz4QGu/O3f+QMWKlXU6ZokSpRAREY7w8Pda7Xfv/ncMsViMRo08cPXqFVy9ehmNGzcBADRu3BRXrlzCpUsXtC4r6XLO+/f/0nn//I6FDBERGUQXd1dMb1dJp32nt6tk0AG/ACCTyTBw4DBs2LBG0+bt3RdLly7Ahw8RANR3KQUEfJ9pb8fH5HI5unbtjkWL5iE+Ph4A8PDhA+zevRO9e/to9mvWrDmOHj0MmUymudupfv2G+P33G7h27TI8PJrp/Dpat26L69ev4ty5M5q2N29C00z8V1Dw0hIRERlManGS2VgZQxUxMpkMMplMq61t2/bYvXsHpFL1x1+XLt2QkpKCUaOGAlAXJkOHjkT9+urxKlKpNM0xpFLt4w4ePBxbt27CwIE+EIvFsLW1w/Tps1G6dBnNPg0aNMbs2dMxZMhITZulpSVKlSoFhUKhKW5S4xaLtfsZ5HK5ps3FpShWrQrA6tXLsWHDWsjlcjg4FMKyZashkUggl8uhVCpzmjazIxIKQAkXHh4DVd7MiG2SRCLAyckO79/HIP9/tzPGPPyHuVBjHtQyy4NCkYzw8FA4OrpBJpOnfwAdZDTwNy96YrJDKhUjJaUAf2D8Ky/ykNl7K/U9qQteWiIiIoNL7zKTqRUxZJ54aYmIiPJEatGy8PRjTGldkUUM6QULGSIiyjNd3F0Ndos1FUx8JxERUZ5iEUP6xHcTERERmS0WMkRERGS2WMgQEVGeiY6OwjffjEd0dJSxQ6F8goN9KVeuXLmI4OAdiIz8AJVKhZo1a2P0aD9YWlri6dMnWLp0IWJjYyAIAqytbdC//6BszWBJRPnLwYMHsHXrJtSsWQt9+/Y3djiUD7BHhnLFysoa06bNwtatwdi8eSfi4+OxceN6AEDRoq6YNWs+fvhhN7Zt24PRo/0wZ87MNOuaEFHBcfjwj1r/GtLYsSMxaFDfNFP3v3sXhp49uxr8/ABw8uRxzJ8/K0/OlV2LFs3Dzz//lKtjjB/vi9u3b2a4/c6d21i0aG6uzpEVFjKUK7Vr14WzswsA9VTe3t59cf36VQDq1WZTtwFAjRruaNOmHa5du2yUWInIuCIjP+DSpQsAgIsXzyMy8oNBz6dUpuD9+3c4efJnrfaUlBQoFAqDnvu/cymQkpKSJ+fKLn3EplAoMs1lSkqKwV8/CxlCilJ/01BHR0drLUX/qZiYGK3ihogKjuPHj2nWAFIqlThx4ucsnpF7/foNxObNgUhOTjb4ucg4OEamgDt8941eZ9k8eHA/OnTolKY9KioSx48fxevXr9CmTftcn4eIzM/hwz9CIpFAqVRCIpHg8OEf0auXt0HPWapUadSr1wAHDuxB79590t1HpVIhMHAdfv31FCQSCcqWLY9vvvkWDg6F0Lt3NyxbthrFi5cAAMTFxaJLl/YYP/4bfP65l+YYI0cOxrffzkDJkqXSHP/9+3fw8xuNsLA3UKlUaNq0OUaOHAORSAQAiIgIx6pVy3Hv3l2IxWKULl0WY8dO0Jzzhx+CoFQqMWDAEM0xt2zZCLFYjH79BiI6Ohpz587A27dvIZVK0bx5C3z99SAAwI0bv2HNmpVITk6GhYUFRowYgwYNGmmO88cft3Do0AEkJiYCENCv3yC0a9dBs/3Bg7/w/fcr8f79OyiVKjRu3AQjR46BlZVVurk8duwIdu/eCUCAnZ09WrZsnfU3KZdMopDZt28f/P39cfz4cZQoUSLdffr27YtXr17hzJkz6W6n7Pt4EbfUf3NTzFy7dgWPHz/C9OmzNW1//HEbCxbMQmjoaxQrVhxLlnyXZiVZIso//vnnOQYO7IO4uLg02/7++ylU/67gq1Qqcfr0L/DwqJtmPxsbGwQFbUepUqX1EtPAgUMxZMjX6NSpK+zs0i5EuHv3ToSHv8f27XthaSnHDz9sxYoVS+DvPw8NGjTGpUsX0LPnVwCAq1evoFaturh48bymkPnwIQJRUZHpFjEAcPfuH1i+/HvUqVMPiYmJGDFiIC5fvoimTT0BAJMn+6FpU0/MnDkXIpEIJ08ex/jxvti+fS/kcjkUCkWa1ayTk5MhkUgAAHv37kKdOvXx1VfqQi11TFBY2FusWLEYixevRPHiJfDPP88wfvwoBAVth4NDIU1sq1atR9Girnj16iUGD+6HBg0awdnZERER4Zg0aTymTvVHo0YeUCqVWLlyKRYtmgt//3lpXue9e38iMHAd1q0LgqurK/7++ynGjRuJhg0bZ/M7lj1Gv7S0cuVKHD9+HA4ODhkuO3748GFYWlqa7HVGc5TeSrRzTj7C4btvcnS8t2/fYPHiefD3n6t1aalWrdoIDv4RZ85cxogRY+DnNwofPkTkKnYiMl2FCxeGSCRGSMiTNF+pRUwqlUqV7n5isQSFCxfWW0yOjk7o0KETtm3bnO72vXt3YcyYCZBK1X/b9+rljStXLkKpVKJZs+a4cuWiZt/Lly9g0KBhCAkJ0YwNuXTpvKYoSU+9eg1Qp049AIClpSVatGiNP/64BQD4/ffrSE5ORv/+gzU9NO3adUC5cuVx6tQJnV6fSqWCUvnf52PqcX78cR+6d++p6dkpVaoMGjXywOXL/72eLl26oWhR9R+wxYuXQOXKVfDgwV8AgAMH9qJVq7Zo1MgDACCRSDB69HjcvHkDr1+/ShPHyZPH0L17T7i6qo9Xtmw5dO7spdNryA2jFjIqlQrOzs4ICAiAhYVFuvvExsZiw4YNGDt2bB5Hl3+lV8Skykkxk5CQgG+/nYghQ0agSpVq6e4jkUjQvHkLNGrUBKdOncx2zERkHuzs7HHs2CmMGzcRIpEIYrFuHzNisRgikQjjxk3E0aO/wM7OXq9x+fj0w6lTJ/D2rfbvt9jYWISHv8eoUUPRv783+vbtjUGD+sLKyhrR0VGoW7c+Hj9+iISEBKhUKjx58gjVqlVHrVq1cfv27wDUA5ebNfssw3Pb2ztoPS5cuIhmoHNIyBPUrFk7zXNq1qyNkJDHOr22nj2/wu3bNzF+vC/u3v1D0/7s2d/YvXsH+vf31nz9/vt1xMXF5jg2uVyOypWr4OnTJ2niePMmFGXLltNqq1y5ik6vITeMemlJLBbDx8cn033WrFmDnj17olChQnkTVD6XWRGTKjuXmZRKJWbO/B8aN26S7tiYT8XFxUIQ9De4mIhMj0wmw//+NwPNmjXHsGEDERn5IcMed0D9h06hQoWxYcNmeHpmXBDkho2NLXr37oNNmwK0xpokJSVCJpNhy5adAACpVIyUFO3fUbVr18ONG9fg4FAI1aq5QyQSwcOjKa5evfxvwfEENWrU1DkWkUikufwjyWDdKUEQIBZLMjxGUlIirK1tAAAODoWwdOkq/PHHLSxaNBedOnXFV1/1QVJSEoYNG4XWrdvmSWzqolX7VneVSkizn74Z/dJSZkJCQnD58mV4e+duMJhIxC8AOPxn1kVMqjknH+Hwn2+yPO533y2FpaUFhg4dkWbbmzevIQiqf88v4MSJo/j999/Qpk07o+bB2N8LU/liLpgHXfOQU82bt8CFC79lOUaiYcPGuHjxusGKmFTdun2JO3duIyTkv96EwoWLQCKR4OnTkAyf16xZc1y+fAmXLl3QXEJq2NADv/12FTduXEe9eg107nn6VKVKVXDnzu007Xfv/oGKFSsDAOzs7BAVFam1/dGjh2meU6tWHSxf/j2CggIAACVLltRcJsp5bLe02pKTk/H48SNUqFAxzf4lS5ZKk8f0Xtuncvu+M4nBvhmZM2cOJk6cqLlumVNFiqQd3FXQKJQqLPhFt27KVAtPPUa/5uUhy6Aqj4qKwoEDe1G2bFkMHtxX0y4SibBx40YcOBCMS5cuwcLCAmKxGJUqVcLu3btRsmTJXL2W3HJ05PshFXOhxjyopZeHxMRERESIIZGIIJVm/8Pa1dUFbm5ukEql6Y5zlEqlKFasGIoWdc5RzJkRiUSQSMSauKVSCwwZMhyBgWv/fSwGIMbnn3fBd98twcKFy/4dDKxCREQEXFzUU0V4ejbHxo3rUKhQYQwZMgxSqRiFCzvAwcEB+/YFo2fP3hnmRiwWQSSC1nZ1mzqfderUgbW1NbZu3YiBA4dAJBLh559/wvPnz9CuXVtIpWK4u7tj167tiIuLhoNDIZw79ytevnyBunXrQSoVIzo6Gvb26ktxz549hbOzC6RSMTp37oLRo0fgs89aoHbtOgCA169fo1ixYpr8iMXa31eRSB0fAHTv/gV8fHqhadNm8PBoipSUFKxevRwNGzZCsWJuaXLs5fUFxowZgY4dO8HV1Q1//nkX586dRt26DdLNj0qlvvRYuLANLC0tc/ptNt1C5ueff4alpSU8PTMeQKWriIgYqArw1QyRSP0L6tu2FTH7uG49MgAwpU1FRH1Ie+fBf8S4dOlGhlt9ff3g6+uXpv39+xidY9Cn1DyEh8dAMHxvp0ljLtSYB7XM8qBQJP87mFRIc8lFF0lJSfj552NaRYxYLNYM/E1JScHPPx9FXFxChmMlc0oqlUIslmjF3apVO+zcuf3fwkrdPmLEWGzeHIghQ/r/+xwxvL37aaaKsLGxQ7FiJWBjYwupVK55XpMmzbF160bUrdsww9yIxVJIJFKt7RKJdtvChcuwevUKdO/eBWKxCOXLV8SKFWsAqGOvWtUdXl5fYMiQAbC0tES5chXQq5c3kpOTkZKiwrp1a3D16iVYWVnB2toGM2bMRUqKChUqVMHs2Qvx3XfLkZCQAJlMirJly2PGjDmaOD7Nj1Qqg1isLg1sbR2wdOkqfP/9CixbthgqlQrNmjXHqFHjNM/5OMfFipWAr+9YjBs3ClKpDEWKOGLEiLG4ceNauvlRKgWoVCp8+BAHmUx7Ur3U96QuRMKnczcbSatWrbB582aULl0aycnJ6NSpEwIDA1GmTBkAwMuXL+Ht7Y3z589n+9jh4SxknJzs8P59DA7d0e3y0vR2lfQyr4wp+TgPpvGuNx7mQo15UMssDwpFMsLDQ+Ho6AaZLOPJLjNy8uTP6NOn17/nUY+/qF27Lm7fvomPx2Ps2LEHbdt2yOxQeSK9MTIFUV7kIbP3Vup7UhcmOUYmLi4OCoUCY8eORdeuXdG1a1cMHToUERER6Nq1K37+2fCzQeZXXdxdMb1dpUz3yY9FDBEZx08/HQagHtBrbW2NjRu34uTJswgM3AJra2vNXChHjhw2Zphkxkzy0lLhwoVx9uxZrbbUHplDhw4ZJ6h8JLVISa9nhkUMEemLQqHQFDK1atVBYOAWzaRxXbt2R5069TBkSH/cuvU7jh49jGXLOGEmZZ/J9MjIZLJMB/VKpdJcD/ql/6TXM8Mihoj0KTz8PZTKFPj5fYOffjqZZubbUqVK4+jRX+Dn9w1SUhQID39vpEjJnJnMGBlD4hiZjK9/63utJVPG8RD/YS7UmAc1Q46RSV1XSV/7GRrHyKiZ0xgZdnEUcF3cXdGxmgukGdxiTUQE/Ld+T3bpWpyYQhFDeUtf/Sj89CIWMUSUodQCIzk5yciRUH6T+p6SSHLXp8IeGSIiypBYLIGVlS1iY9Xr78jlFhBlZ9pVM6NSiaBUFuDrjP8yZB4EQUBychJiYz/Ayso2x7Mip2IhQ0REmbK3LwIAmmImP/t4sr6CLC/yYGVlq3lv5QYLGSIiypRIJIKDgyPs7ApDqUy7zEB+IRIBhQvb4MOHuAI/+NvQeVDPKqyfYQ0sZIiISCdisRhicfbvXDIXIhFgaWkJmUxR4AsZc8oDR3kSERGR2WIhQ0RERGaLhQwRERGZrQIxRkYkUn8VVKmvvSDnAGAePsZcqDEPasyDGvOgZgp5yM65C8QSBURERJQ/8dISERERmS0WMkRERGS2WMgQERGR2WIhQ0RERGaLhQwRERGZLRYyREREZLZYyBAREZHZYiFDREREZouFDBEREZmtfLdEQUhICLp27Yrhw4dj1KhRAICwsDBMmzYNb968gSAI8Pb2xldffWXkSPXv0KFDmDdvHtzc3DRtcrkcwcHBkEgkBSYPAJCYmIgNGzbg7NmzUKlUSEpKwowZM+Dh4QGgYLwnlEolevfujeTkZK32Fy9eYMWKFfjss88QEhKCmTNnIiYmBiKRCCNHjkS7du2MFLHhxMbGYuXKlfjtt98gEolgZWWFMWPGoEmTJgBQYPKgUCgQEBCAkydPIiUlBY6Ojpg1axbKlSsHIP//XOzbtw/+/v44fvw4SpQooWnP6vuvUCiwaNEiXLlyBQDQuHFjTJ48GXK5PM9fgz5klAcA+OeffzB06FB06tQJo0eP1tpmsnkQ8pmBAwcKgwYNEpYvX65p69Wrl3Do0CFBEAQhJiZG6NGjh3D27FljhWgw+/fvFyZMmJDh9oKSB4VCIfTp00dYvXq1kJSUJAiCIKhUKkGhUGj2KSi5+FRiYqLQrFkzISoqSkhMTBTatWsnXLt2TRAEQQgLCxPatWsn3L9/38hR6t/AgQOF9evXC0qlUhAEQbh7967QtGlT4eXLlwUqD4sXLxbGjRsnJCYmCoIgCL/99pvQqVMnITk5WRCE/P1zsWLFCmHQoEFCkyZNhGfPnmnadfn+L126VJg+fbqgVCoFpVIpzJo1S1i0aFGevwZ9yCgPgiAIt27dEjp27CgMGTJE6zM0lanmIV9dWjpx4gQcHR1Rq1YtTduDBw+gVCrRpUsXAICtrS3GjBmD3bt3GytMoyhIeTh06BBsbW0xatQozV8KIpEIUqm6A7Ig5eJTx44dg4eHB+zt7XHx4kVUrVoVDRs2BAA4Oztj4MCB2L9/v5Gj1L9r167B29sbYrH6V16NGjVQvXp13Lt3r8DkQRAE7Nq1CzNnzoSFhQUAoEGDBqhbty4uXryYr38uVCoVnJ2dERAQoHntqbL6/qtUKhw6dAgTJ06EWCyGWCyGn58fjhw5AqVSmeevJTcyywMAvH//HuvXr4e7u3u6zzXVPOSbQiYhIQGrVq3ChAkTtNqvXLmCBg0aaLXVr18fV69ehVCA1sssSHk4duwYevfuneH2gpSLT+3evRs9e/YEkH4eGjRogMuXLxsjNIOqVasWNm/erHl8/fp13Lp1CzVr1iwweXj//j1kMhkKFSqk1V6xYkXcvXs3X/9ciMVi+Pj4QCKRpNmW1ff//v37cHFxgb29vWa7ra0tihUrhnv37hk2cD3LLA8A0KZNG5QsWTLdbaach3xTyKxfvx6dO3dG0aJFtdrDwsK0xowAgKWlJSwsLBAeHp6XIRpVQcrDgwcPYGFhgdGjR6Nz587o168fzp8/r9lekHLxsYcPHyIqKgr169cHkH4e3Nzc8OLFC2OEZ1ALFy7EsWPHMGjQIMydOxejRo3CkiVL4OrqWmDyYGdnh/j4eERHR2u1P3/+HO/fvy+wPxdZff/T2w4Arq6uePnyZZ7EaApMOQ/5opD5559/cPLkSQwYMCDNtujo6HS70CwsLJCQkJAX4eUZkUiEGzdu4KuvvsL//d//Yfjw4bh16xaAgpWHyMhIrFu3DuPGjcORI0fwv//9DzNmzMC1a9cAFKxcfCw4OBg9evTQPE4vDxYWFkhKSjL7v8A/Vbx4cfj4+ODKlSvYtm0bmjVrpuk+Lyh5sLS0xOeff4758+cjISEBgiDg/Pnz+OWXXyAIQoH9ucjq+x8dHZ3uYNb8npdPmXIe8sVdS/PmzcO4cePS/SGUy+VISkpK056YmAhLS8u8CC/PtG/fHm3btoWtrS0EQcC5c+cwYsQIBAcHF6g8iEQiDB48GOXLlwcAVKlSBf3798f+/fvRqFGjApWLVPHx8Th+/DiOHj2qaUsvD4mJiZDL5RCJRHkdokF98803ePbsGTZv3oxixYrh+++/R5cuXbBv374ClYdZs2YhICAAPj4+UCgUqFu3Lnx8fBAdHY24uLgC93MBZP1zIJfL09z5l7pPfs7Lp0w5D2ZfyJw/fx4JCQlo3759uttdXV3x+vVrrbbExETEx8fD0dExL0LMM9bW1pr/i0QitGjRAq1bt8b58+cLVB4cHR1RpkwZrbZSpUrh4sWLAArWeyLV0aNH0bhxYxQpUkTT5urqitDQUK39QkND4erqmtfhGdTz589x7tw5/Prrr7CzswMALFq0CFOmTMHOnTsLTB4A9YfR6NGjtW6rnTVrFmrXro03b94UuJ8LIOufg/R+XwDAmzdv0gxlyM9MOQ9mf2np5cuXePv2Lbp27ar5Cg4Oxr59+9C9e3fUqVMH169f13rO9evX4e7urrmDIT9TqVSQSCQFKg/u7u549OiRVtvz589RunRpAChQuUgVHBysGeSbqk6dOvjtt9+02q5fv446derkZWgGFxsbCxcXF00Rk6pSpUqIiooqMHlIT2xsLM6cOYPPPvusQP5cAFn/HFStWhXPnz/XGlsUGxuLp0+fonr16nkaqzGZch7M/t3p7e2NEydO4NChQ5qv3r1748svv8SBAwfQoEEDpKSk4PDhwwDUiV+9ejX69Olj5Mj17+3bt0hJSdE8PnHiBC5cuIC2bdsWqDx4e3tjxYoVePfuHQD1ZFfbtm2Dt7c3ABSoXADAvXv3EBUVhcaNG2u1t2/fHnfu3NH8En/37h2CgoLg4+NjjDANpkqVKrCxscGWLVugUqkAqMfV7dmzB126dCkweQCgdZvs27dvMXr0aPTv3x+FChUqcD8XqbL6/ltaWsLLywtLly6FSqWCIAhYvnw5OnfuDCsrK2OGnqdMOQ8iIT+NZvvXunXroFQqNTP7vnr1CjNmzEBYWBiUSiV69OiR7sBgc7d3715s2rRJMyCrbNmyGDNmjGasSEHJA6DOxcaNGyESiWBtbY1x48ahefPmmu0FKRf+/v4oWbIkBg0alGbbgwcP4O/vj/j4eAiCgKFDh6Jz585GiNKwIiIisHLlSty6dQsSiQSWlpYYMmQIWrduDaDg5GHNmjU4c+YMUlJSYGlpCW9vb3Tt2lWzvSD8XLRv3x5BQUEoXry4pi2r739SUhLmzZun6bGqV68epk2bZvSxIbmRXh5SrV+/HikpKZrP0FSmmod8WcgQERFRwWD2l5aIiIio4GIhQ0RERGaLhQwRERGZLRYyREREZLZYyBAREZHZYiFDREREZsvslyggIjKkkydPYvr06ShcuHC626tUqYKVK1fmbVBEpMF5ZIgoQwMHDsSIESPQoEEDY4eSI//88w9+/PFHjB07Ns02pVKJVq1a4ejRo7C1tc3wGBs2bIBCoYCvr2+W5wsLC0NQUBCmTJmSq7iJSHe8tEREGVIoFFAoFMYOI8cWLFiQ4cy0v/32G968eYOnT59meRxdV8F2cXGBg4MDzp8/n604iSjnWMgQUb505swZVKxYEfb29mm2JScnY8GCBRg7dixmzpyJhISETI+VukaTLry9vbF+/fpsx0tEOcMxMkSUpVWrVkEQBNy6dQthYWEAgK+++gp9+/bV7BMTE4PFixfj0qVLsLGxgZOTEzZv3gwAuHjxIpYvX464uDioVCp07NgRY8aMgUQiAQC0aNECfn5+CAwMhFKpRP369TF58mQsWLAAN2/ehFwux7hx49CiRQvN+a5cuYJFixYhKSkJlpaW+Oabb9CkSRPN9h07dmDGjBlpXktiYiImTZqEZs2aYeTIkbCyssLw4cOxZs2adC8xVatWDXPmzMHx48fTzY2LiwuCgoI0jx0cHODm5oZbt24ViBW0iYxOICLKQJ8+fYRLly4Jq1atEurUqSP88ccfgiAIQnh4uNC8eXPhr7/+EgRBEFJSUoRu3boJ27ZtE1QqldYx7t+/LzRr1ky4e/euIAiCEB8fL/j6+grLli3T7NOyZUthxIgRQlJSkiAIgjB9+nShe/fuwt69ezXna9++vRAbGysIgiCEhoYKHTp0EP755x9BEAQhJCREaNGihRARESEIgiAkJCQIHTp0SPN6Lly4IHTr1k1YuXKlVvuOHTuEdu3aCUePHhWSk5NzlzRBEPbv3y989913uT4OEWWNPTJEpJM2bdqgZs2aAIAiRYrgs88+w40bN1C1alUcPXoULi4u6NOnT5rnBQUFYdCgQahRowYAwMrKCv7+/vi///s/jBgxAlZWVgDUA4tTV25v3bo1/vjjD3z55Zea81WpUgVPnjxBrVq1sGvXLvTp0wclS5YEAJQrVw6enp44e/YsunXrhlevXmm2pTp48CAOHjyIqVOnol69eprYfHx84O3tjSZNmmDt2rU4f/48Fi5ciEOHDmHdunU656d+/fqYO3cuAKB8+fK4cOGCzs8lopxjIUNEOnFzc9N6XLhwYURFRQEAbt26leGdTY8ePUpT4Dg5OcHFxQXPnz9HlSpVAADOzs6a7ZaWlqhQoYLWcywtLZGYmAgAePLkCY4ePYo9e/ZotsfHx6NixYoAgKioKNjZ2Wk938vLC15eXlpt27dvh5eXFywsLFCmTBksXrxYs61r167o2rVr+snIgr29vSY3RGRYLGSIKMeEj2ZvUCqV6e4jFqd/T4EgCJoxMumRyWQZbktMTISfnx86duyY7nYHBwdER0dn+HxDi46OhoODg9HOT1SQsJAholyrXbs2Dhw4gKFDh6bZVr16dfz++++ay1IA8P79e4SHh6N06dI5Ol+ZMmVw9+7dDAuZ4sWL48WLFzk69qdGjx6Nx48fp7stKSkJ9erVw9KlS7XaQ0JCULZsWb2cn4gyx0KGiHKtU6dO2LBhA4KCgjBw4ECtbf369cPAgQNRv359uLu7IyEhATNmzIC3t7dmTEx2devWDQMGDEDr1q1Rv359AMDLly9RokQJAOrLUMWKFcOzZ89QpkyZDMe7yOVyeHt7p2n/eLzL6tWrM4wjJCQEEyZMSNN+6dIl+Pj45Oi1EVH2sJAhogzJZDLN16eTwsnlcs2lIalUim3btmHu3Llo3749rKysYG9vjx9++AEVK1bE0qVLMXv2bERHR0MQBHTt2hUjRoxIc56PH0ul2r+epFKpZp8aNWrgu+++w6JFixAfHw+ZTIYKFSpo9Yx4e3tjz549mDRpUq7Gu2RF+GRy9MjISLx69Qp169Y1yPmISBuXKCCifEkQBAwbNgyLFy9GoUKFDHKOkJAQ+Pn54dChQ5q277//Hu7u7vjss88Mck4i0saZfYkoXxKJRJg6darWZHX6VrRoUQwZMkTz+O3bt4iOjmYRQ5SH2CNDREREZos9MkRERGS2WMgQERGR2WIhQ0RERGaLhQwRERGZLRYyREREZLZYyBAREZHZYiFDREREZouFDBEREZktFjJERERktv4fuzM9Pas6OG8AAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Scatter Plot\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "subset = trainData.loc[trainData['Ownership']=='Owner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'o', label = 'Owner', color = 'C1')\n",
    "\n",
    "subset = trainData.loc[trainData['Ownership']=='Nonowner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'D', label = 'Nonowner', color = 'C0')\n",
    "\n",
    "ax.scatter(newHousehold.Income, newHousehold.Lot_Size, marker = '*', label = 'New household', color = 'black', s = 150)\n",
    "\n",
    "plt.title('주택 대지vs. 소득 간 산점도')\n",
    "plt.xlabel('Income(소득)')\n",
    "plt.ylabel('Lot_Size(주택 대지)')\n",
    "for _, row in trainData.iterrows():\n",
    "    ax.annotate(row.Number, (row.Income + 2, row.Lot_Size))\n",
    "\n",
    "handles, labels = ax.get_legend_handles_labels()\n",
    "ax.set_xlim(40, 115)\n",
    "ax.legend(handles, labels, loc = 4)\n",
    "\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     zIncome  zLot_Size Ownership  Number\n",
      "3  -0.409776   0.743358     Owner       4\n",
      "13 -0.804953   0.743358  Nonowner      14\n",
      "0  -0.477910  -0.174908     Owner       1\n"
     ]
    }
   ],
   "source": [
    "# 정규화된 학습, 검증 및 전체 데이터 프레임 초기화, 훈련 데이터를 사용하여 변환을 학습\n",
    "scaler = preprocessing.StandardScaler()\n",
    "scaler.fit(trainData[['Income', 'Lot_Size']])\n",
    "\n",
    "# 전체 데이터 반환\n",
    "mowerNorm = pd.concat([pd.DataFrame(scaler.transform(mower_df[['Income', 'Lot_Size']]),\n",
    "                                    columns = ['zIncome', 'zLot_Size']),\n",
    "                       mower_df[['Ownership', 'Number']]], axis = 1)\n",
    "trainNorm = mowerNorm.iloc[trainData.index]\n",
    "validNorm = mowerNorm.iloc[validData.index]\n",
    "newHouseholdNorm = pd.DataFrame(scaler.transform(newHousehold), columns = ['zIncome', 'zLot_Size'])\n",
    "\n",
    "# Use k-nearest neighbor\n",
    "knn = NearestNeighbors(n_neighbors=3)\n",
    "knn.fit(trainNorm[['zIncome', 'zLot_Size']])\n",
    "distances, indices = knn.kneighbors(newHouseholdNorm)\n",
    "print(trainNorm.iloc[indices[0],:])    # indices is a list of lists, we are only intereseted in the first element"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 값 선택\n",
    "* K가 너무 작으면 데이터의 노이즈를 적합할 위험\n",
    "* k가 너무 크면 이 알고리즘의 주된 장점 중 하나인 데이터의 지역적 구조를 파악할 수 있는 능력을 놓칠 수 있음\n",
    "* 학습 데이터셋을 사용해 검증 데이터셋의 레코드를 분류한 다음 다양한 값에 대해 오차율을 계산하여 선택 가장 븐루 성능이 좋은 k 선택\n",
    "* k 가 선택되면 새로운 레코드를 분류하기 위해 학습 데이터셋과 테스트 데이터셋을 훔치고 알고리즘 반복"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     k  accuracy\n",
      "0    1       0.6\n",
      "1    2       0.7\n",
      "2    3       0.8\n",
      "3    4       0.9\n",
      "4    5       0.7\n",
      "5    6       0.9\n",
      "6    7       0.9\n",
      "7    8       0.9\n",
      "8    9       0.9\n",
      "9   10       0.8\n",
      "10  11       0.8\n",
      "11  12       0.9\n",
      "12  13       0.4\n",
      "13  14       0.4\n"
     ]
    }
   ],
   "source": [
    "# 다양한 k에 대한 k-NN의 검증 정확도\n",
    "train_X = trainNorm[['zIncome', 'zLot_Size']]\n",
    "train_y = trainNorm['Ownership']\n",
    "valid_X = validNorm[['zIncome', 'zLot_Size']]\n",
    "valid_y = validNorm['Ownership']\n",
    "\n",
    "# Train a classifier for different values of k\n",
    "results = []\n",
    "for k in range(1,15):\n",
    "    knn = KNeighborsClassifier(n_neighbors = k).fit(train_X, train_y)\n",
    "    results.append({\n",
    "        'k': k,\n",
    "        'accuracy':accuracy_score(valid_y, knn.predict(valid_X))\n",
    "    })\n",
    "\n",
    "# Convert results to a pandas data frame\n",
    "results = pd.DataFrame(results)\n",
    "print(results)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### k=4에 대한 분류\n",
    "* K가 선택되면 새로운 레코드를 분류하기 위해 학습/테스트 데이터셋을 합치고 알고리즘 반복"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Owner']\n",
      "Distances [[0.31358009 0.40880312 0.44793643 0.61217726]]\n",
      "Indices [[ 3  8 13  0]]\n",
      "     zIncome  zLot_Size Ownership  Number\n",
      "3  -0.409776   0.743358     Owner       4\n",
      "8  -0.069107   0.437269     Owner       9\n",
      "13 -0.804953   0.743358  Nonowner      14\n",
      "0  -0.477910  -0.174908     Owner       1\n"
     ]
    }
   ],
   "source": [
    "# 전체 데이터 셋에 대한 재훈련\n",
    "mower_X = mowerNorm[['zIncome', 'zLot_Size']]\n",
    "mower_y = mowerNorm['Ownership']\n",
    "knn = KNeighborsClassifier(n_neighbors = 4).fit(mower_X, mower_y)\n",
    "distances, indices = knn.kneighbors(newHouseholdNorm)\n",
    "print(knn.predict(newHouseholdNorm))\n",
    "print('Distances', distances)\n",
    "print('Indices', indices)\n",
    "print(mowerNorm.iloc[indices[0],:])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 레코드 분류의 기본 : 다수결의 원칙\n",
    "* 근접 이웃들 중 다수가 속한 클래스의 member가 되도록 새로운 레코드 분류\n",
    "* 즉, 다수 == 클래스 멤버십 확률(레코드가 클래스에 속할 확률에 대한 컷오프 값)\n",
    "* 소득($60,000) & 주택 크기(20,000m^2)에 대한 근접 이웃\n",
    "* Owner = 3, Non-owner = 1 -> 0.75\n",
    "* 코드에서는 'Owner'로 할당"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Owner']\n",
      "Distances [[0.31358009 0.40880312 0.44793643 0.61217726]]\n",
      "Indices [[ 3  8 13  0]]\n",
      "     zIncome  zLot_Size Ownership  Number\n",
      "3  -0.409776   0.743358     Owner       4\n",
      "8  -0.069107   0.437269     Owner       9\n",
      "13 -0.804953   0.743358  Nonowner      14\n",
      "0  -0.477910  -0.174908     Owner       1\n"
     ]
    }
   ],
   "source": [
    "# 전체 데이터셋에 대한 재훈련\n",
    "mower_X = mowerNorm[['zIncome', 'zLot_Size']]\n",
    "mower_y = mowerNorm['Ownership']\n",
    "knn = KNeighborsClassifier(n_neighbors = 4).fit(mower_X, mower_y)\n",
    "distances, indices = knn.kneighbors(newHouseholdNorm)\n",
    "print(knn.predict(newHouseholdNorm))\n",
    "print('Distances', distances)\n",
    "print('Indices', indices)\n",
    "print(mowerNorm.iloc[indices[0],:])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## k-NN 알고리즘의 장단점\n",
    "### 장점\n",
    "* 단순하다는 것과 모수에 대한 가정이 거의 없다\n",
    "* 학습 데이터셋이 충분히 많을 때, 특히 각 클래스의 특성이 예측 변수 값들의 여러 조합으로 결정될 때 탁월한 성능\n",
    "\n",
    "### 단점\n",
    "* 학습 데이터셋으로부터 모수를 추정하는 데 걸리는 시간은 없어도 학습 데이터 셋이 클 경우에는 근접 이웃을 찾는 데 매우 많은 시간이 소요\n",
    "* - 극복\n",
    "* - ** 주성분 분서과 같은 차원 축소 방법으로 차원을 감소시킴으로써 거리 계산 시간을 줄인다\n",
    "* - ** 근접 이웃을 빨리 찾기 위해 검색 트리와 같은 정교한 데이터 구조를 사용한다\n",
    "* 학습 데이터셋으로 필요한 레코드의 개수는 예측 변수의 개수 p가 증가함에 따라 기하급수적으로 증가\n",
    "* 많은 학습 시간이 소요되는 계산이 예측에 집중된 '나태 학습 방법(Lazy Learner)'"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 요약\n",
    "* 분류할 레코드와 다른 모든 레코드 사이의 거리 찾기\n",
    "* k-가장 가까운 레코드 선택\n",
    "* - 가장 가까운 이웃의 다수결에 따라 분류\n",
    "* - 또는 예측을 위해 가장 가까운 이웃의 평균\n",
    "* 차원의 저주 - 예측 변수의 수를 제한해야 함"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 07-2. Naive Bayes\n",
    "\n",
    "## 나이브 베이즈\n",
    "* 모델 중심이 아닌 데이터 중심\n",
    "* 데이터에 대한 가정을 하지 않음\n",
    "\n",
    "## 나이브 베이즈 기본 원리\n",
    "1. 예측 변수 ㅍ로파일이 동일한(즉, 예측 변수 값들이 동일한) 다른 모든 레코드를 찾음\n",
    "2. 그 레코드들이 어떤 클래스에 속하고 어떤 클래스가 가장 일반적인지 결정\n",
    "3. 그 클래스를 새로운 레코드의 답\n",
    "\n",
    "## 컷오프 확률 방법과 조건부 확률\n",
    "### 컷오프 확률\n",
    "1. 어떤 레코드가 해당 클래스에 속한다고 간주하는 값 이상으로 관심 클래스의 컷오프 확률을 설정\n",
    "2. 새로운 레코드와 도일한 예측 변수 프로파일을 갖는(즉, 예측 변수 값들이 동일한) 모든 학습 레코드를 찾음\n",
    "3. 그 레코드들이 관심 클래스에 속할 확률을 결정\n",
    "4. 그 확률이 컷오프 확률보다 크면 새로운 레코드의 답을 관심 클래스로 함\n",
    "\n",
    "### 조건부 확률\n",
    "* P(Ci|x1, x2, ..., xp)\n",
    "\n",
    "## 완전한(정확한) 베이지안 분류기의 적용\n",
    "* 가장 가능성 있는 클래스에 배정하는 방법의 사용\n",
    "* 컷오프 확률 방법의 사용\n",
    "* 완전한(정확한) 베이즈 절차의 실질적 어려움\n",
    "- 예측 변수의 수가 많아지면(20개 정도만 되어도) 분류할 많은 레코드들과 정확하게 일치하는 것은 없을 것임\n",
    "\n",
    "## 나이브 베이즈\n",
    "\n",
    "## 컷오프 확률 방법의 사용"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from dmba import classificationSummary, gainsChart\n",
    "from sklearn.model_selection import train_test_split"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "outputs": [
    {
     "data": {
      "text/plain": "   schedtime carrier  deptime dest  distance      date  flightnumber origin  \\\n0       1455      OH     1455  JFK       184  1/1/2004          5935    BWI   \n1       1640      DH     1640  JFK       213  1/1/2004          6155    DCA   \n2       1245      DH     1245  LGA       229  1/1/2004          7208    IAD   \n3       1715      DH     1709  LGA       229  1/1/2004          7215    IAD   \n4       1039      DH     1035  LGA       229  1/1/2004          7792    IAD   \n\n   weather  dayweek  daymonth  tailnu   delay  \n0        0        4         1  N940CA  ontime  \n1        0        4         1  N405FJ  ontime  \n2        0        4         1  N695BR  ontime  \n3        0        4         1  N662BR  ontime  \n4        0        4         1  N698BR  ontime  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>schedtime</th>\n      <th>carrier</th>\n      <th>deptime</th>\n      <th>dest</th>\n      <th>distance</th>\n      <th>date</th>\n      <th>flightnumber</th>\n      <th>origin</th>\n      <th>weather</th>\n      <th>dayweek</th>\n      <th>daymonth</th>\n      <th>tailnu</th>\n      <th>delay</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1455</td>\n      <td>OH</td>\n      <td>1455</td>\n      <td>JFK</td>\n      <td>184</td>\n      <td>1/1/2004</td>\n      <td>5935</td>\n      <td>BWI</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n      <td>N940CA</td>\n      <td>ontime</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1640</td>\n      <td>DH</td>\n      <td>1640</td>\n      <td>JFK</td>\n      <td>213</td>\n      <td>1/1/2004</td>\n      <td>6155</td>\n      <td>DCA</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n      <td>N405FJ</td>\n      <td>ontime</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1245</td>\n      <td>DH</td>\n      <td>1245</td>\n      <td>LGA</td>\n      <td>229</td>\n      <td>1/1/2004</td>\n      <td>7208</td>\n      <td>IAD</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n      <td>N695BR</td>\n      <td>ontime</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1715</td>\n      <td>DH</td>\n      <td>1709</td>\n      <td>LGA</td>\n      <td>229</td>\n      <td>1/1/2004</td>\n      <td>7215</td>\n      <td>IAD</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n      <td>N662BR</td>\n      <td>ontime</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1039</td>\n      <td>DH</td>\n      <td>1035</td>\n      <td>LGA</td>\n      <td>229</td>\n      <td>1/1/2004</td>\n      <td>7792</td>\n      <td>IAD</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n      <td>N698BR</td>\n      <td>ontime</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "delays_df = pd.read_csv('https://raw.githubusercontent.com/niharikabalachandra/Logistic-Regression/master/FlightDelays.csv')\n",
    "delays_df.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "outputs": [],
   "source": [
    "# 범주형으로 변환\n",
    "delays_df.dayweek.astype('category')\n",
    "\n",
    "# 시간별 bin 출발 시간 생성\n",
    "delays_df.deptime = [round(t / 100) for t in delays_df.deptime]\n",
    "delays_df.deptime = delays_df.deptime.astype('category')\n",
    "\n",
    "predictors = ['dayweek', 'deptime', 'origin', 'dest', 'carrier']\n",
    "outcome = 'delay'\n",
    "\n",
    "X = pd.get_dummies(delays_df[predictors])\n",
    "y = (delays_df[outcome] == 'delayed').astype(int)\n",
    "classes = ['ontime', 'delayed']\n",
    "\n",
    "# 훈련과 검증 분할\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size = 0.40, random_state = 1)\n",
    "\n",
    "# NB 알고리즘 실행\n",
    "delays_nb = MultinomialNB(alpha = 0.01)\n",
    "delays_nb.fit(X_train, y_train)\n",
    "\n",
    "# 확률 예측\n",
    "predProb_train = delays_nb.predict_proba(X_train)\n",
    "predProb_valid = delays_nb.predict_proba(X_valid)\n",
    "\n",
    "# 클래스 예측\n",
    "y_valid_pred = delays_nb.predict(X_valid)\n",
    "y_train_pred = delays_nb.predict(X_train)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "outputs": [
    {
     "ename": "OptionError",
     "evalue": "'Pattern matched multiple keys'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mOptionError\u001B[0m                               Traceback (most recent call last)",
      "Input \u001B[1;32mIn [52]\u001B[0m, in \u001B[0;36m<cell line: 13>\u001B[1;34m()\u001B[0m\n\u001B[0;32m     10\u001B[0m \u001B[38;5;66;03m# reset all options\u001B[39;00m\n\u001B[0;32m     11\u001B[0m pd\u001B[38;5;241m.\u001B[39mreset_option(\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mprecision\u001B[39m\u001B[38;5;124m'\u001B[39m)\n\u001B[1;32m---> 13\u001B[0m \u001B[43mpd\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mset_option\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;124;43m'\u001B[39;49m\u001B[38;5;124;43mprecision\u001B[39;49m\u001B[38;5;124;43m'\u001B[39;49m\u001B[43m,\u001B[49m\u001B[38;5;241;43m4\u001B[39;49m\u001B[43m)\u001B[49m\n\u001B[0;32m     14\u001B[0m \u001B[38;5;66;03m# probability of delay status\u001B[39;00m\n\u001B[0;32m     15\u001B[0m \u001B[38;5;28mprint\u001B[39m(train_df[\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mdelay\u001B[39m\u001B[38;5;124m'\u001B[39m]\u001B[38;5;241m.\u001B[39mvalue_counts() \u001B[38;5;241m/\u001B[39m \u001B[38;5;28mlen\u001B[39m(train_df))\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\pandas\\_config\\config.py:256\u001B[0m, in \u001B[0;36mCallableDynamicDoc.__call__\u001B[1;34m(self, *args, **kwds)\u001B[0m\n\u001B[0;32m    255\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21m__call__\u001B[39m(\u001B[38;5;28mself\u001B[39m, \u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwds):\n\u001B[1;32m--> 256\u001B[0m     \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[38;5;18;43m__func__\u001B[39;49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwds\u001B[49m\u001B[43m)\u001B[49m\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\pandas\\_config\\config.py:149\u001B[0m, in \u001B[0;36m_set_option\u001B[1;34m(*args, **kwargs)\u001B[0m\n\u001B[0;32m    146\u001B[0m     \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mTypeError\u001B[39;00m(\u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m_set_option() got an unexpected keyword argument \u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mkwarg\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m'\u001B[39m)\n\u001B[0;32m    148\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m k, v \u001B[38;5;129;01min\u001B[39;00m \u001B[38;5;28mzip\u001B[39m(args[::\u001B[38;5;241m2\u001B[39m], args[\u001B[38;5;241m1\u001B[39m::\u001B[38;5;241m2\u001B[39m]):\n\u001B[1;32m--> 149\u001B[0m     key \u001B[38;5;241m=\u001B[39m \u001B[43m_get_single_key\u001B[49m\u001B[43m(\u001B[49m\u001B[43mk\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43msilent\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    151\u001B[0m     o \u001B[38;5;241m=\u001B[39m _get_registered_option(key)\n\u001B[0;32m    152\u001B[0m     \u001B[38;5;28;01mif\u001B[39;00m o \u001B[38;5;129;01mand\u001B[39;00m o\u001B[38;5;241m.\u001B[39mvalidator:\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\pandas\\_config\\config.py:116\u001B[0m, in \u001B[0;36m_get_single_key\u001B[1;34m(pat, silent)\u001B[0m\n\u001B[0;32m    114\u001B[0m     \u001B[38;5;28;01mraise\u001B[39;00m OptionError(\u001B[38;5;124mf\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mNo such keys(s): \u001B[39m\u001B[38;5;132;01m{\u001B[39;00m\u001B[38;5;28mrepr\u001B[39m(pat)\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m\"\u001B[39m)\n\u001B[0;32m    115\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mlen\u001B[39m(keys) \u001B[38;5;241m>\u001B[39m \u001B[38;5;241m1\u001B[39m:\n\u001B[1;32m--> 116\u001B[0m     \u001B[38;5;28;01mraise\u001B[39;00m OptionError(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mPattern matched multiple keys\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n\u001B[0;32m    117\u001B[0m key \u001B[38;5;241m=\u001B[39m keys[\u001B[38;5;241m0\u001B[39m]\n\u001B[0;32m    119\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m silent:\n",
      "\u001B[1;31mOptionError\u001B[0m: 'Pattern matched multiple keys'"
     ]
    }
   ],
   "source": [
    "# 목적지 공항에 따른 항공기 상태의 피벗 테이블(학습 데이터셋)\n",
    "\n",
    "# frequency 테이블 구성 후 확률 테이블로 변환\n",
    "# raw 테이터 프레임을 동일한 random_state를 활용하여 훈련 시험으로 분할\n",
    "# First cell\n",
    "# frequency 테이블 구성 후 확률 테이블로 변환\n",
    "# raw 테이터 프레임을 동일한 random_state를 활용하여 훈련 시험으로 분할\n",
    "train_df, valid_df = train_test_split(delays_df, test_size=0.4, random_state=1)\n",
    "\n",
    "# reset all options\n",
    "pd.reset_option('precision')\n",
    "\n",
    "pd.set_option('precision',4)\n",
    "# probability of delay status\n",
    "print(train_df['delay'].value_counts() / len(train_df))\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dayweek         1         2         3         4         5         6         7\n",
      "delay                                                                        \n",
      "delayed  0.191571  0.149425  0.114943  0.126437  0.187739  0.068966  0.160920\n",
      "ontime   0.124646  0.141643  0.144476  0.179415  0.169027  0.135977  0.104816\n",
      "\n",
      "deptime    0\n",
      "delay       \n",
      "delayed  1.0\n",
      "ontime   1.0\n",
      "\n",
      "origin        BWI       DCA       IAD\n",
      "delay                                \n",
      "delayed  0.080460  0.521073  0.398467\n",
      "ontime   0.060434  0.647781  0.291785\n",
      "\n",
      "dest          EWR       JFK       LGA\n",
      "delay                                \n",
      "delayed  0.379310  0.199234  0.421456\n",
      "ontime   0.266289  0.155807  0.577904\n",
      "\n",
      "carrier        CO        DH        DL        MQ        OH        RU        UA  \\\n",
      "delay                                                                           \n",
      "delayed  0.057471  0.314176  0.095785  0.222222  0.007663  0.218391  0.015326   \n",
      "ontime   0.034939  0.229462  0.203966  0.117092  0.010387  0.169027  0.016997   \n",
      "\n",
      "carrier        US  \n",
      "delay              \n",
      "delayed  0.068966  \n",
      "ontime   0.218130  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "for predictor in predictors:\n",
    "    # frequency 테이블 구성\n",
    "    df = train_df[['delay', predictor]]\n",
    "    freqTable = df.pivot_table(index='delay', columns=predictor, aggfunc=len)\n",
    "\n",
    "    # 조건부 화률을 얻기 위해 각 행을 행의 합으로 나눠줌\n",
    "    propTable = freqTable.apply(lambda x: x / sum(x), axis=1)\n",
    "    print(propTable)\n",
    "    print()\n",
    "pd.reset_option('precision')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 새로운 항공기 분류를 위한 계산\n",
    "* 연착 확률 & 정시 확률 계산 -> 두 값은 분모가 같기 때문에 분자만 비교해도 됨\n",
    "* 분자는 관련된 예측 변수 값들의 조건부 확률을 곱하고 해당 클래스의 비율을 곱함\n",
    "* ex) 일요일 오전 10-11시 사이에 DCA-> LGA로 가는 델타 항공기 분류"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P_hat_delayed ~  0.000669340146792383\n",
      "P_hat_ontime ~  0.00642128403158402\n",
      "P(delayed|...) =  0.09439791617127381\n",
      "P(ontime|...) =  0.9056020838287262\n"
     ]
    }
   ],
   "source": [
    "P_hat_delayed = 0.0958 * 0.1609 * 0.4215 * 0.5211 * 0.1977\n",
    "\n",
    "P_hat_ontime = 0.2040 * 0.1048 * 0.5779 * 0.6478 * 0.8023\n",
    "\n",
    "print('P_hat_delayed ~ ', P_hat_delayed)\n",
    "print('P_hat_ontime ~ ', P_hat_ontime)\n",
    "\n",
    "print('P(delayed|...) = ', P_hat_delayed / (P_hat_delayed + P_hat_ontime))\n",
    "print('P(ontime|...) = ', P_hat_ontime / (P_hat_delayed + P_hat_ontime))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### NB를 활용해 데이터의 점수를 구하는 코드"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      actual  predicted         0         1\n",
      "708        0          0  0.939586  0.060414\n",
      "1228       0          0  0.939586  0.060414\n",
      "181        0          0  0.939586  0.060414\n",
      "1226       1          0  0.939586  0.060414\n",
      "1229       1          0  0.939586  0.060414\n",
      "705        0          0  0.939586  0.060414\n",
      "712        0          0  0.939586  0.060414\n",
      "713        0          0  0.939586  0.060414\n",
      "1225       0          0  0.939586  0.060414\n",
      "1754       0          0  0.939586  0.060414\n",
      "707        0          0  0.939586  0.060414\n",
      "1749       0          0  0.939586  0.060414\n",
      "1755       0          0  0.939586  0.060414\n"
     ]
    }
   ],
   "source": [
    "# 데이터 세트에서 검색하여 특정 항공편을 분류\n",
    "# 동일한 예측 값을 가진 항공편의 경우\n",
    "# 특정 집합을 부분 집합화\n",
    "df = pd.concat([pd.DataFrame({'actual': y_valid, 'predicted': y_valid_pred}),\n",
    "                pd.DataFrame(predProb_valid, index = y_valid.index)], axis = 1)\n",
    "mask = ((X_valid.carrier_DL == 1) & (X_valid.dayweek == 7) & (X_valid.deptime_0 == 1) & (X_valid.dest_LGA == 1) & (X_valid.origin_DCA == 1))\n",
    "\n",
    "print(df[mask])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### NB를 활용한 항공편 연착에 대한 confusion matrix"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix (Accuracy 0.8023)\n",
      "\n",
      "        Prediction\n",
      " Actual  ontime delayed\n",
      " ontime    1059       0\n",
      "delayed     261       0\n",
      "\n",
      "Confusion Matrix (Accuracy 0.8104)\n",
      "\n",
      "        Prediction\n",
      " Actual  ontime delayed\n",
      " ontime     714       0\n",
      "delayed     167       0\n"
     ]
    }
   ],
   "source": [
    "# training\n",
    "classificationSummary(y_train, y_train_pred, class_names = classes)\n",
    "print()\n",
    "\n",
    "\n",
    "# validation\n",
    "classificationSummary(y_valid, y_valid_pred, class_names = classes)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### NB에 대한 향상 차트"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 400x400 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAGGCAYAAAB/gCblAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABaxklEQVR4nO3deVhU5RfA8e8MMIOIK4pgaiJmkqnhnmW2mOZuJrmmpaaluWSuSS6pqWm55ZZrWq7gAmnlL8stMdHcUnDfUBBZZBFmBmbu7w9yckKLQeCynM/z9Dx537uce5jhcO977/tqFEVREEIIIf6iVTsAIYQQ+YsUBiGEEDakMAghhLAhhUEIIYQNKQxCCCFsSGEQQghhQwqDEEIIG1IYhBBC2JDCIIQQwoYUBiGEEDYc1Q4gP4iNTcKegUE0GnBzK2H3doWJ5CCD5EFyAAUjB/dizAopDICikK0fZna3K0wkBxkkD5IDKDw5kFtJQgghbEhhEEIIYUMKgxBCCBtSGIQQQtiQwiCEEMKGFAYhhBA2pDAIIYSwIYVBCCGEDSkMQgghbEhhEEIIYUMKgxBCFCBpZkuuH0MKgxBCFAAWRaGP/ywajF7GuqMRuXosGURPCCHyuTupabw94QsOfvMZGidn4l+uC1TKteOpVhjMZjPdunXDZDLZLL9+/Tpz5syhefPmXLx4kYkTJ5KUlIRGo2HQoEG0bNnSum5aWhozZ84kJCQEgCZNmjBmzBh0Ol2enosQQuSW+BQTbUZ9wcXNMwBo1PIN3m/dKFePqVphcHBwYPPmzTbLjEYjLVq0wNfXF6PRyKBBg5gyZQqNGjXi9u3b9OrViypVqlCzZk0A5s+fj8lkIjg4GICpU6cyd+5cRo8enefnI4QQOS06yUj7j+dzOWAmoNCsXVc2L1+IVpu7vQD5qo9h586dPPvss5QsWZIDBw7g4+NDo0YZlbF8+fL07duXwMBAACwWC9u3b2fkyJFotVq0Wi0jRowgODgYs9ms5mkIIcQjSzNbGLhgM5c3TwfFQqsOXdi8fGmuFwXIZ4Vh48aNvPnmmwCEhITQsGFDm/aGDRty8OBBAMLCwnB3d6dkyZLWdldXVypWrMjp06fzLmghhMhhd03pDNvyJ9cdPClRoxGvtu7AqiVf50lRgHzU+Xz27FkSEhJo0KABANHR0TRt2tRmHU9PT65fv25t9/T0zLQfDw8PIiIiqFOnTpaPrdHYF+u99e3drjCRHGSQPEgOIGdzcOhKHGOCwrhrMqNxdGLF8tU8X80NJ6dH+3VtT2z5pjBs2LABPz8/678TExPR6/U26+j1eoxGI4qikJiY+MBOZr1eT2pqql3Hzuo8qDm1XWEiOcggeZAcwKPnYGPoNYZ/tYWUcwcp07wPX/WoR7s6FXMouqzLF4UhJSWFH3/8kR07dliX6XQ6jEajzXoGgwGdTodGo0Gn02V6ouneOs7OznYd394JvAvCxN+5TXKQQfIgOYCcycGGP27w2Yb/Eb1pAhZDMu+3qEuTis2JiUnK0RizIl8Uhh07dtCkSRPKli1rXebh4UFkZKTNepGRkXh4eFjbb968mWlfUVFRVKhQwa7jZ3cC78Iy8fejkBxkkDxIDiD7Odh97jafbfqVWxs/wWJIpkGDRgzs21e1fOaLzucNGzZYO53v8fX15fDhwzbLQkND8fX1BcDHx4erV6+SmJhobU9OTubSpUvUqlUr94MWQogccDk2Bf91e4je6I8lJYHadeqyfn0Arq7q3ZpTvTCcPn2ahIQEmjRpYrO8VatWnDx50locbt++zcqVK+nZsycAzs7OdOrUidmzZ2OxWFAUhS+//JL27dtTrFixPD8PIYSwV9itJHou/JEr347FnBxHzZpPsXnTNkqVKq1qXKrfStq8eTPdu3dH848ucxcXFxYvXsykSZNISUlBURSGDBlC3bp1reuMGjWKadOm0bZtWwDq16+Pv79/nsYvhBDZEZNsZNjm41xZNx5z4m2qenkTEBBE2bJuaoeGRlGK+l1BiImxv/O5XLkSdm9XmEgOMkgeJAdgfw7SzRYGbT7JsRuJOF0JIf3wBrZt/Z6KFR/L9RizQvUrBiGEKEpuJhgYGniKq/GpFNc58M3kIXi6fpivxnhTvY9BCCGKipM3E+m8eA9HvplMemIMk157ksfLuuSrogByxSCEEHniSlwKH6w/zI2NEzHeCKOKQyLNJ72udlgPJFcMQgiRy1JMZkYGHuPy+kkYb4RRsmQpFs77KtNDN/mFFAYhhMhFiqIwecefHF7uj/HaSYq7urJp01Zq1876eG55TQqDEELkou9Cr7Jh9igMl4+idy7G+nWB1KvXQO2w/pUUBiGEyCXHIxKYPGUSqecP4eik49u1G2jS5Fm1w/pPUhiEECIXxCQbGft9GK4NOuFWpQarV31L8+YvqR1WlshTSUIIkcN2nrnFxB/OAlDj8UqsOHAQV+f89Ujqv5ErBiGEyCGKorDt5E2GjRlD8p+/UNLZkc87PFWgigLIFYMQQuSIG3dSeX/9CfZtWEjC71vQaLQEjOnO42Vd1A7NblIYhBDiEV2PT6X3d8eI+HUdCSEbAZj06XRq1XxS5ciyR24lCSHEIzCkmRkddIYb+wO4s28NABMmTOH9ge+rHFn2SWEQQohsMlsUJv94lj92bSb+l+UAjB79MR98MEzlyB6N3EoSQohs+P1KPBN+CCfy/Cnidi0CYOjQEXz00RiVI3t0UhiEEMIOsXdNjA46w8mbGdMK6yo+yatv9sPHozj+/hOB/Dn+kT2kMAghRBbF3DXR59s/iE42AVDbswTT2vngWfIFypUrQWxscqGYrEgKgxBCZEHIlTiGBv4JgPnqHzwWdZBFa9bg7OyMRkO+HSk1O6TzWQgh/sPZ6GRGbT8DgCXiFNFbP+P3PT+ydOlClSPLHVIYhBDiX5yNTmbgxhMY0y14pFwhdutUTEYDrVq15v33h6gdXq6QW0lCCPEQUYkGBm8+yV2TmZJJVzm9dhwpKXdp3vwlli37Jt9NyZlT5IpBCCEewJBm5sOtp0kwpFMy5QZXvxtPclISzz77HN98sx5nZ2e1Q8w1csUghBD/kGhI4+3vjnH9joGSOg131k8nMeEO9es34LvvNuHiUvDGP7KHXDEIIcR9LIrCxB/Ocv2OAYDpHZ5m6ZLlNGvWnA0btuDqWkLlCHOfXDEIIcR9Vv1+jQOX4nDSwsoevtSsUAIeb0RAQFCheiT138gVgxBC/CXkShxLf7tKelIs5q0fY4g8b20rKkUBpDAIIQQA1+JT8d8RTvrdO6Rsm8jl8JOMGDEUi8Widmh5Tm4lCSGKvB/DovlkZzjm1CQSAieSFHmFihUfY9Wqb9Fqi97fz1IYhBBF2qErcUzYGY7FeJfYgImkRl7E3b0CgYFBPP54VbXDU0XRK4VCCPGXn8/eZkjgn5hNBlKDp5F68xxly5YlICAIb+8n1A5PNapeMRgMBr7++mv27NmDxWLBaDQyYcIEnn32WQCio6Px9/cnKioKRVHo0aMH3bt3t26vKAqLFy9mx44daDQaatSowaeffoqrq6tapySEKCAOXYnjk53hAOhOBHL94klKlSrN5s3bqVnTR+Xo1KVaYUhPT+fdd9+lcePGbNiwAZ1Oh6IomM1m6zpDhw6lR48edOjQgeTkZPr27UvFihVp3rw5ABs3buTEiRNs3boVnU7HsmXLGD9+PPPmzVPrtIQQBcDGP24w+9eLADSoXIrPB37JsKEpDB48lNq166ocnfpUu5W0fft2XF1d+eCDD6zjjWg0GhwdM2pVeHg4ZrOZDh06AODq6srQoUPZuHGjdR8bN25k9OjR1u379evHyZMniY+Pz+OzEUIUBHEpJvqvP87sXy+iKBZ8Krgyo/1TlHAtzsqVa6lfv6HaIeYLqhWGnTt30q1bt4e2h4SE0LCh7Q+pQYMGHDp0CEVRiI+P59atW3h7e1vbtVotvr6+HDp0KNfiFkIUTGlmCyO3neHEzUQUixn9voXUuPY9JZ3lGZx/Ui0j4eHh6PV6hgwZwpUrVyhTpgz9+/fnhRdeADL6FypWrGizjbOzM3q9ntjYWOLi4vDw8Mi0Xw8PD65fv25XLPa+t3Jv/SL0vksmkoMMkoeCk4N5ey9xKjIRR61CtbDv+N+hn7h8ZDdvdnmTJ5+s+Uj7Lgg5sCc21QrDnTt3WLx4MRMmTMDb25vw8HDee+89Zs6cSePGjUlMTMTLyyvTdnq9ntTUVBITEx845K1er8dgMNgVi5tb9sY+ye52hYnkIIPkIX/nYNuxG2w8dhNFUah1JZCgoE1otVrWr1/Pc8/l3O2j/JwDe6hWGDQaDf3797feCqpZsyZvv/02gYGBNG7cGJ1Oh9FozLSdwWDA2dkZnU6HyWR6YHuZMmXsiiU2NsmueVo1mowPgL3bFSaSgwySh/ydgzspaSw8cJmtJzOebKx0djNB29eg0Wj46qslvPhiK2Jikh75OPk5B/fcizErVCsMbm5uVK1a1WZZlSpVOHDgAJBxS+jmzZs27QaDgZSUFNzc3FAUJVM7QFRUFDVr2ndZqChk64eZ3e0KE8lBBslD/stBsjGdfuuPczU+FYASfwZycOcaAL74Yj5dunTL8XjzWw6yS7XO59q1a3Pu3DmbZVevXuXxxx8HwNfXl9DQUJv20NBQateujVarxd3dHRcXFy5cuGBtt1gsHD16lHr16uX+CQgh8i1FUZjy0zmuxqei1UAnz1RO71wNwGeffU6vXn3UDTCfU60w9OjRgzlz5nD79m0ALl68yNq1a+nRowcADRs2JD09naCgIACSk5NZsGABvXr1su6jd+/ezJo1y3pLacWKFdSsWZPKlSvn8dkIIfKLdLOFEdtO88v5GBy1GpZ3e4bxPVoxb94iJkyYQv/+76kdYr6nURT1Lnw2b97M8uXL0Wg0uLi4MHz4cOtTSQA3btxgwoQJREdHYzab8fPz45133rG2K4rC3Llz2bVrF1qtFm9vbyZNmkTZsmXtiiMmxv4+hnLlSti9XWEiOcggechfOUgxmRm5/TSh1+4AMOalqnSpVyXXj5ufcvAw92LM0rpqFob8QgqD/SQHGSQP+ScHiqLw8fdh/HwuBoCW2jMcDFrLpk3bKF++fK4eO7/k4N/YUxhkED0hRKGw7ugNfj4XgwZoX+wiK2aO5fTpU6xbt0bt0AocKQxCiALvj4g7LNh3CYCWzldZMvUjLBYLvXr1YciQD1WOruCRd8GFEAWWoijsPBPNZ/87h1mBp9Mvsnr6KNLT03njjTeZNWtukZxo51FJYRBCFFjr/7jBnD0ZVwpuCef5efU4TCYT7dp1ZMGCJTg4OKgcYcEkpVQIUSD9EXGH+XszikLTqqVI2L0Eo8FAy5avsWTJCutIzcJ+UhiEEAVOTLKRj78Px6xAq5rlmdu5Dps2BNKzZ2+WL1/zwHHURNZJSRVCFCjpZgtjg8OIvWuiailHxresgUajoXLlKsyZ85Xa4RUKcsUghChQ5u27zImbiTgm3iR8fl92/7RD7ZAKHSkMQogC46ewaDb8cYO0+EjiNn/CraibLFjwpc2UwOLRya0kIUSBcCHmLlN3nSM9MZq7WyeSEHOLJ5+sybp1gfL0UQ6TwiCEyPeSjemMCTpD8p0YEgImkHz7JtWqeRMQEISbm5va4RU6UhiEEPmaoihM/vEsl29EEbf5E1JvR1ClyuMEBgZToULm6X3Fo5M+BiFEvrbq9+vsuRBLyokfSI2+iqdnRQICgnjssUpqh1ZoyRWDECLfWvzbFVYeugbAVP/xnN1Rga5de1C1aub54EXOkcIghMiXfgyLZvn+c2gcnOhWvzJv1K2I5pmJaodVJMitJCFEvnPh9l2m7PyT24FTcT2wkGHNHkej0agdVpEhhUEIka8kG9MZueU4EQHTMFw9zpUTv3Hhwnm1wypSpDAIIfINi6Iw4fvTHF87hdSLoTg7F2Pdus34+DyldmhFihQGIUS+sSrkClsW+JNy7iBOTjq++WYdTZs+r3ZYRY50PgshVGW2KBy6Gs+usFus+cKflDN70To4snLlWl566RW1wyuSpDAIIVSjKAr+O8L5+dxtTLevkhK2F41Gy9Ily2nVqrXa4RVZUhiEEKpIN1v49Kdz/HzuNg5aDc/Xr4tb9aXULpVOx46d1Q6vSJPCIIRQxYL9l/khLBpzaiKj2tbjTd+KQB21wxJI57MQIo9l3D4KY93RGyQc2kzC2qHUdr6jdljiPnLFIITIMxF3UhkSeIqIOwYSjwRxZ+83AOzb96s8kpqPSGEQQuSJK3EpvLfpJLF3TSQd/5H43V8D8NFHYxg4cLDK0Yn7SWEQQuSqZGM6C/dfJuBEJADpZ/cQv2shAIMGDWX06I/VDE88gBQGIUSuibiTytvfHSPBkA6A9vIhIoO+RFEU+vZ9l4kTp8gYSPmQdD4LIXJF+K0kuq4+Yi0K/RpXotyVn7FYLPTs2ZvPPpslRSGfkisGIUSOi7iTytDAPzGZFcoUc2LtW/WoUEJPz/UBrFjxNUOGfIhWK3+X5leqFYbt27czbdo0PD09rct0Oh0bNmzAwcGB6Oho/P39iYqKQlEUevToQffu3a3rKorC4sWL2bFjBxqNhho1avDpp5/i6uqqxukIIcgYBO/7P28xY/d50swKniX1zGrhSYUSegBKlCjJ8OEjVY5S/BfVCoPZbOaFF15g9uzZD2wfOnQoPXr0oEOHDiQnJ9O3b18qVqxI8+bNAdi4cSMnTpxg69at6HQ6li1bxvjx45k3b15enoYQ4j5z9lxiwx83ACjr4kR/LyOvvdyEDz8cxdChH6ocnciqfHktFx4ejtlspkOHDgC4uroydOhQNm7caF1n48aNjB49Gp1OB0C/fv04efIk8fHxqsQsRFE38+cL1qLwYnU3xtdzZMR7Pbl7N5m9e38lPT1d5QhFVuXLwhASEkLDhg1tljVo0IBDhw6hKArx8fHcunULb29va7tWq8XX15dDhw7ldbhCFGmGNDP+206x+fhNAPo3qULfGhoG9PEjMTGBxo2fZc2a9Tg6SpdmQZEvf1LR0dFUrFjRZpmzszN6vZ7Y2Fji4uLw8PDItJ2HhwfXr1+3+3j2Phhxb/2i/ECF5CBDUc/D+dvJ9F9/grsmMwDvPfc4L5Y30aFDB+Li4qhXrz7r12/G1bW4ypHmroLwObAntkcuDCaTiT179lChQgXq1q2b5e00Gg1Hjhyhe/fu3Llzh8cff5yBAwfi6+tLYmIiXl5embbR6/WkpqaSmJhovYX0z3aDwWD3Obi5lbB7m0fZrjCRHGQoinlISE1j7Koj1qIw9JUn6ODlwIsvvsbt29E888wz/Pzz/yhTpozKkeadwvI5sLswdO/enfXr1wOQlpbGW2+9hYuLC5GRkXTv3p0+ffpkaT+tWrXi1VdfxdXVFUVR2Lt3L++//z4bNmxAp9NhNBozbWMwGHB2dkan02EymR7Ynp0PYWxsEoqS9fU1mowPgL3bFSaSgwxFNQ8WReGjbae5GpuCZ0k9O4e9gMVgZNWqFdy4cYMaNZ5k/fotmM2OxMQkqR1urisIn4N7MWaF3YXh/l/YAQEBPPHEE0ydOpWkpCS6deuW5cLg4uJi/X+NRsOLL77IK6+8wr59+/Dw8ODmzZs26xsMBlJSUnBzc0NRlEztAFFRUdSsWdPeU0JRyNYPM7vbFSaSgwxFLQ+rf7/O/otx6Bw0fN7hKcoU1xGTaqRPn37odHpefrkFbm7lilROoPB8DuzufDaZTJjNZkwmE2vXrmXEiBEAlChR4pFfWLFYLDg4OODr60toaKhNW2hoKLVr10ar1eLu7o6LiwsXLlyw2fbo0aPUq1fvkWIQQvy7+XsvsejAFQDGvPIE7k4mEhMTre3du/eiQoXMfYCi4LD7N/nrr7+On58f3bp1o0uXLpQtWxbIuK109+7dLO/n1q1bNo+v/fTTT+zfv59XX32Vhg0bkp6eTlBQEADJycksWLCAXr16Wdfv3bs3s2bNst5SWrFiBTVr1qRy5cr2npIQIgvSzRaGBJxi7ZEIALr6VuSFynr8/Drx6quvcueOPCpeWNh9K6lfv360bNkSwOaXsNlsZvr06Vnez759+1ixYoW1E9nLy4tvvvkGd3d3ABYuXMiECRNYtmwZZrMZPz8/Wrf+ew7YPn36EB8fT8eOHdFqtXh7ezNjxgx7T0cIkQWXY1MYuPEE8alpAAxs+jjdapfFz68jp06dwN3dndjYGEqVKjodzYWZRlEKwx2xRxMTY3/nc7lyJezerjCRHGQoCnlINqbT57tjXItPBeDzDk/R+DEXund/g5CQ3yhTpgx79+7F07Nqoc3BfykIn4N7MWaF3VcMSUlJLFy4kNDQUBISEri/ruh0On744Qd7dymEyKdM6RaGb/mTa/GplHVxYmWPZ3DTa+jVqyshIb9RokRJNm/eRu3atYvE00dFhd2Fwd/fnzJlyjBhwoRMj4Y6OTnlWGBCCHWZ0i3033CcsFvJODlo+PL1pylfzIF33unJvn2/4uJSnPXrA6lb11ftUEUOs7swXLp0ieDg4NyIRQiRTyQb0+m7/jiXY1MAmN7Oh1oeJbhy5TLHjx/D2dmZ777bRKNGjVWOVOQGuwuDDIQlROGmKAqTfzxrLQrzOj9NU6+Mpw+rVvUiKOgHIiIieO65ZmqGKXKR3Y+rtmjRgi+++EIKhBCF1NrQCPZciMXJQcOqHs/Q5PHSnDt31tru7f0EzZu/pGKEIrfZfcVw5MgRTp48yapVq3Bzc7NOzacoCnq9nl27duV4kEKIvBF6LZ6FBy4DMPIlb2p5lGD8+NF8++03rF69jpdfbqFyhCIv2F0YZs16+DytMqyuEAXXrSQj478Px6JAu1oV6FTbgylTJrJ8+VIAbt+OVjlCkVfs/k1eqVKl3IhDCKGi1DQzQwNPEZ+aRo3yxRnzSnW++GImX301F4BZs+bStWsPdYMUeSZLhWHz5s34+fkBsGjRItLS0h64npOTE4MGDcq56IQQue7+F9hK6B2Z2eEpli1ZwKxZGSMZTJkynT59+qocpchLWSoM998icnBwwGKxPHA9BweHnIlKCJEnLIrCpB/Oci0+Fa0m463mHwO+YcqUCQCMHz+RgQMHqxylyGtZKgyvv/669f8HDhyYa8EIIfJOmtnChJ3h7L2Y8QTS8m7P4FPBlSWHQgAYMWIUw4Z9pHKUQg3Z6i1OTk7m8uXLmWZLM5vNNGnSJEcCE0LknmRjOoMDTnEmKmMYi1EvV+cpj4xxdJYsWUGbNu14/fUuaoYoVGR3Yfjf//7HJ598QqVKlbh06RLVqlXj2rVrODg48PLLL0thECKfi08xMTTwT8Kjk4GMolBNicJiqYBWq8XR0ZHOnf1UjlKoye4X3BYuXMjmzZsJCAjgscceIyAggAMHDtCpUyeqVKmSGzEKIXJIoiGNARtPEB6djKNWw/w3nqb07RO0bv0Kw4cPxmw2qx2iyAfsLgyKotjMw2A0GtHpdIwePZodO3bkaHBCiJyTbEyn77rjXIlLxVGrYcEbtTFeOU7fvm+Rnp6OyZR5nnVRNNldGNLT061DbXt5eXH48GEgY95mmdpBiPxJURQ+/ekcV+MzisJivzqk3ThNnz7dMZlMtGnTngULlsqThQLIRh9D8+bNOXDgAM2aNcPPz48JEybQp08fzp49i4+PT27EKIR4BOlmC2ODw9h7MRZHrYZl3epivHmWHj38SE1N5ZVXXmXp0pUybL6weuQZ3A4ePMju3bspX748vXv3xsXFJadiyzMyg5v9JAcZCkIevvj1Ihv+uAHAuBbVeUJ7m86d25OQcIdmzZrz7bebKFasWLb3XxBykNsKQg5ydQa3f2ratClNmzZ91N0IIXLB2tDr1qLwScsadKjtwQ8/HCMl5S6NGjVhzZoNj1QUROFkd2FYunRppiExNBoNpUuXpnr16jRuLBN3CKG2dIvCrN0X2HIyEoB3GlemQ20PAFq3bsumTduoU6cuxYsXVzNMkU/Z3fns4ODAsmXLuHz5Mi4uLpQuXZpLly4xe/ZsNm/eTJ8+fUhISMiNWIUQWbRg3yVrUWj7lDttKmm4fv2atf2555pRokRJtcIT+ZzdVwzHjx9nyZIlPPvss9ZlvXr14pdffiEwMJBhw4bx2WefMXPmzBwNVAjx3xRF4cs9l6y3jwY2fZzWVRzo2LENFouZwMBgvLyqqRylyO/svmK4fPmyTVG45+WXX+bSpUvUq1ePM2fO5EhwQoisu51s5MOtp61FoW/jyrSv5kyXLh24du0Kjo6O0p8gssTuK4aUlBQMBgPOzs42yxMTE62jrj5sIh8hRO5IMZkZHHDKOk9zvyZV8KtZgs6d23Lx4gUqVarMli3f4+HhqXKkoiCw+4rh1Vdf5d133yU0NJT4+HiSk5P5/fff6d+/P++++y7JyclSGITIQ4qiMHXXOS7HpqBz0DC59ZN0q1WKN9/sRFjYGSpU8CAwMJhKlSr/986EIBtXDGPHjmX58uVMmzaNyMhIUlNT8fb2ZuDAgbz22mvEx8czceLE3IhVCPEA6/+4wf/O3sZBq2GRXx28S2nx8+vEqVMnKFeunPQrCLvZXRi0Wi0DBgxgwIABD2wvU6YMZcqUeeTAhBD/7VhEAvP3XgJgxIvVqPtYKWJiYkhNTaV06dJs3hxEjRpPqhylKGge+QU3IYQ6YpKNjPs+DLMCrWqWx++ZigCUK1eOrVu/58aNG9Sq9bTKUYqCyO4+BiGE+tLNFsZ9H0bsXRPe5VwY/ZIXe/b8Ym0vU6YsTz9dW8UIRUGWLwrDxYsXefrpp/nqq6+sy6KjoxkwYAAdOnSgffv2rF+/3mYbRVFYtGgRbdu2pV27dowYMYLk5OS8Dl0IVczfd5njNxIprnPgszZPMmLoQLp2fZ3Vq1eoHZooBPJFYfjss89o0qSJzVAbQ4cOpV27dgQFBbF+/Xq2bt3K3r17re0bN27kxIkTbN26le+//x4fHx/Gjx+vRvhC5Kld4dGs/+tdhYmtnuCLSR8RFLQVJycnmSxL5IhsFYbLly+zcOFCZs+ebbM8OwO1/vTTT7i5uVG3bl3rsvDwcMxmMx06dADA1dWVoUOHsnHjRus6GzduZPTo0eh0OgD69evHyZMniY+Pz84pCVEgXIy5y5SfzgHQp2Elvl86jc2bN/w1VM03vPzyqypHKAoDuwvDjh07eP/993FycmL37t3W5Tdv3qR///527Ss1NZX58+fz0Ucf2SwPCQmhYcOGNssaNGjAoUOHUBSF+Ph4bt26hbe3998notXi6+vLoUOH7D0lIQqEZGM6o4POYEi30LByKW78uIS1a1eh1WpZvHg5bdq0UztEUUjY/VTSkiVL+O6773Bzc2Pnzp3W5RUrViQuLs7ufbVv354KFSrYLI+OjqZixYo2y5ydndHr9cTGxhIXF4eHh0em/Xl4eHD9+nW7YoCMccqzs35Rfo9PcpAhr/KQMQPbWa7Fp+LuqqPsmQC+Xr4EgLlzF/L662/kbgD/Qj4LBSMH9sRmd2Ewm824ubn9dSDbI5lMpizv59q1a+zatYtt27ZlaktMTMTLyyvTcr1eT2pqKomJidZbSP9sNxgMWY7hHje3rE1ekVPbFSaSgwy5mYc/byQwdstJ/ryRiM5By9LeDQj8OqO/bdGiRbz//nu5dmx7yGeh8OTA7sJQqlQpwsPDqVmzps3yn3/+mbJly2Z5P9OmTWP48OHo9fpMbTqdDqMx88Tk98Zo0ul0DyxCBoMhWy/XxcbaP4Obm1sJu7crTCQHGXI7D1tORPLZ/85b/z3qFW8quzgyfPgYmjV7mfr1GxITk5TzB7aDfBYKRg7uxZgVdheGCRMm8OGHH/Lyyy8THx/PihUrOHnyJAcPHmTNmjVZ2se+fftITU2lVatWD2z38PDg5s2bNssMBgMpKSm4ubmhKEqmdoCoqKhMBSsrFIVs/TCzu11hIjnIkBt52H3utrUoVCzlTHPHi7T0bmA9Tr16DfNV7uWzUHhyYHfns4+PD1u2bKFq1aq0atWKa9eu8cwzz1gfGc2KiIgIbt26RceOHa3/bdiwgYCAADp37oyvry+hoaE224SGhlK7dm20Wi3u7u64uLhw4cIFa7vFYuHo0aPUq1fP3lMSIt+5HJvCpz9mPH308hPl6Kw7w4zRA+nRo0u2bpcKYQ+7rxj27t1L06ZN8fPzy/ZBe/ToQY8ePWyWLViwgPT0dD788EMURSE9PZ2goCA6dOhAcnIyCxYs4J133rGu37t3b2bNmsWCBQvQ6XSsWLGCmjVrUrmyjCApCra7pnRGB50mJc1Mg8qlqG/6kyGjhgFQp84zD7z9KkROsrswbNmyBX9/f5599llee+01mjVrhpOT06MH4uho7czWaDQsXLiQCRMmsGzZMsxmM35+frRu3dq6fp8+fYiPj6djx45otVq8vb2ZMWPGI8chhJoURWHKT+e4Epfx9NELDhcY9sFAFEXh7bf7MXnyNBnWXuQ6jZKNt9KMRiP79+9n9+7dHD58mPr169O6dWuef/75HCkSeS0mxv7O53LlSti9XWEiOciQ03n49kgE8/ZewlGr4R3PGCZ/1J+0tDS6devJ3LkL0WrzxWAFNuSzUDBycC/GrMjW6Kp6vZ4WLVrQokULLBYLoaGhrFq1io8//piQkJDs7FKIIu/o9Tt8tS9jCO02paOZMuo90tLS6NSpM3PmfJUvi4IonB5p2O2TJ0+ye/du9uzZg6urKx988EFOxSVEkaEoCjvO3GLGzxcwK9DmKXfaeRRnbbFivPzyqyxcuAwHBwe1wxRFiN2F4cCBA/z000/s37+fatWq8eqrr7JixQrKlSuXG/EJUailmy18uO00h65kjPH1RPnijGvxBM5ODvzww24qVapSIG/PioLN7sKwbt06XnnlFT766CNKly6dCyEJUXQs2H/ZWhTq6OPo7uWIs1PG1YG39xNqhiaKMLsLw6JFi3IjDiGKlFtJRhbuv8wPYdEAfPC0IzOGf8gvBiNbtgTj61tf5QhFUZalwjBlyhQ++eQTIOMx0fT09Aeu5+TkxOrVq3MsOCEKm6hEAx9tO82523etyzo/DrM+6kNsbCx16/ri7V1dxQiFyGJh6NKli/X/hw8fbjOhzv3kXqgQD2dIMzNi22nO/1UUShdz4o1qDiwd9zbR0bfw8anFxo1bKFmylMqRiqIuS4Xh/qEufH19H7reH3/88egRCVHIKIrC3guxLDxwmStxqbg4OfB117qUtCTSvv1r3LgRwRNP1CAgIIiyZd3UDlcI+8dK6t69+0PbPv/880cKRojCxqIojA46w6igM1yJS8VBA1++Xouy2lTeeKM9165doWpVLwIDgylfvrza4QoB2NH5HBUVxeXLl4mLi3vgS2w3btwgJiYmR4MToiC7cPsu43eEcSk2BQDfSqUY0syL2hVLkpqaStWqXhgMBgIDg/Hw8FQ5WiH+luXCEBoayqZNm7h9+3amJ5McHBwoXbo0n332WY4HKERBkmIyc/hqPBuP3eDI9QTr8smtn6TNU3/PVFisWDFWr17H7dvRPPZYJTVCFeKhslwY2rdvT/v27enUqRNr167NzZiEKJBiko30+e4Y0cl/TyJVpUwxJrSqQd3HSpGcnMSmTRt4553+aDQadDqdFAWRL9n9HsMXX3yRG3EIUaClmy18/H0Y0ckmSjk7Us3NhV4NK/OCd0ZnckpKCm+91Y3ffttPZORNxo+fqHLEQjyc3YXB29ubq1evEhYWlmnCELPZzBtvqDcpuRBqmb/vMsduJFJc58CK7s/weFkXa5vRaOSdd3ry22/7cXUtQZs27VSMVIj/ZndhWL9+PYsXL8bX15dDhw7RpEkTLl68SGxs7L8+sSREYRV84ibrjt4AMvoS7i8KaWlpvPvu2/z6625cXFxYty5A3moW+V62xkoKCgqidOnStGvXjnnz5gHw7bffcv369RwPUIj8bM/5GPx3hgPwdqPKNK/+92CSZrOZQYPe5ccfd6DX61mzZgNNmjyrVqhCZJnd7zFoNBrr4HmOjo7cvZvxFmevXr04ePBgjgYnRH62YN8lRm4/gyHNQqMqpXnvuao27SNGDGH79i04OTmxatW3vPDCi6rEKYS97C4MiqJYh8SoXr06+/btAzL+OhKiKFAUha8PXmFNaAQAz1V3Y1q7mjhobafcfO65Zuj1er7+ejUtWrRSI1QhssXuW0mdO3fmyJEjPPvss/Tu3Zv33nuPo0ePcuHCBZo0aZIbMQqRr6z/4wbLQq4B0LthJT59o+4Dp3R8883uNGvWHE/PiipEKUT2ZWvO5/tduHCBffv2Ub58edq0aVMgZ5qSOZ/tV1RzsDb0OvP3XQbgjbqejHqlOh7uJYmJScJiUVi+fAkdOrxOhQoeKkead4rqZ+F+BSEHuT7n8/2qV69O9eoyTLAo/Hafu20tCm1rVWDMK9XR3nf7aM6cWcyYMZUVK77ml19+w8XF5WG7EiJfy1Jh+Pjjjx86B8P9nJycmDZt2iMHJUR+cyU2hU9/PAdAp9oejG9Zw6Z90aIFzJgxFYA+ffpJURAFWpYKQ9OmTR86B8P9ZD4GURhdjUthwMYTpKSZqV+5FGNa2E65uWjRIiZOHA/A2LH+vP/+B2qEKUSOyVJhaNdO3tQURVPMXRPvbTpJfGoa5V11TGvrg+N9t4/WrfuWYcMGAzB8+EhGjBitVqhC5Bi7+xiOHz/+rzO4PfPMM48akxD5gindwtigM8TcNVHWxYmlb9bFrbjO2v7990EMH55RFAYOHMS4cZ+oFaoQOcruwjB37lxMpr9HjzQYDFy+fJmSJUvSoEEDKQyiUDh5M5HBm09iSLdQXOfA0q51qVymmM06vr718PKqRsuWr/Lpp9MBzYN3JkQBY3dhWL16daZlKSkpzJs3D3d395yISQhVHbl2h8EBJ7H89djh5NY1qVo2c2fyY49V4scfd+PtXZm4uLv59jFFIexl95vPD+Li4sK4cePYsWNHTuxOCNWcv53MyO2nsSgZcyn8POhZmlf/ex7m/fv3EhS01frvMmXKotXmyNdIiHzjkd9juMdisdjcYhKioDl5M5EBG09gtih4l3NhRfdnKK77+yty6FAIb73VFYPBQKlSpWne/CUVoxUi99hdGI4cOZKp8zk+Pp6goCB8fHxyLDAh8lLsXRNjg89gtihUKKHnqy51bIrCsWNH6dGjCykpKbz44ss0biyjpIrCy+7CsHjx4kxXBiVLlqRevXq89dZbWd7PmjVr2Lx5MxqNBpPJRO3atRk5ciQVKmTMi3vx4kUmTpxIUlISGo2GQYMG0bJlS+v2aWlpzJw5k5CQEACaNGnCmDFj0Ol0DzyeEA/z26U4Rmz7E4sCXmVdWNXT9krhzz9P0bXr6yQnJ/Hcc81YvXodzs7OKkYsRO6yuzCsWLEiRw780ksv0bVrV/R6Penp6Xz11VcMHDiQbdu2YTQaGTRoEFOmTKFRo0bcvn2bXr16UaVKFWrWrAnA/PnzMZlMBAcHAzB16lTmzp3L6NHyHLnIupM3E619Ci5ODnze4SmbonD2bDh+fh24c+cODRo0Yu3aDfJWsyj0VOs1q1y5Mnq9HsiY12Ho0KFcv36dW7duceDAAXx8fGjUqBEA5cuXp2/fvgQGBgIZ/Rnbt29n5MiRaLVatFotI0aMIDg4WIb/FlkWl2JiXPAZ0i0KtTxKsP3dRlR1+/uX/q1bt+jSpQOxsbHUqfMM69cH4OqatUHIhCjI7L5iMBgMrFy5kuPHj9vM+awoCjqdLttXFKmpqWg0GsqUKUNISAgNGza0aW/YsCFr1qwBICwsDHd3d0qWLGltd3V1pWLFipw+fZo6depkKwZRdKRbFMZ/H0Z0somqZYux0K+2zZUCgLu7Ox06dOLAgf1s2rSVUqVKqxOsEHnM7sLg7++Pt7c3Y8aMyXSf1dExew85nT9/nlmzZvHBBx+g0+mIjo6madOmNut4enpapw6Njo7G09Mz0348PDyIiIiwuzBo7Hwv6d769m5XmBT0HCw+cJkj1xMo5qRlVsdauOozf3Y1Gg3Tps0kOTmJEiVKPmAvBT8POUFyUDByYE9sdv8mv3DhArNnz7Z3sweaOXMmQUFBxMTE4OfnR+/evQFITEy03ma6R6/XYzQaURSFxMTEB3Yy6/V6UlNT7Y7DzS17tweyu11hUhBzEHA0wjr72iy/ujR8soK1LTo6mhkzZjB9+nTrZ7B8+QcXhfsVxDzkNMlB4cmB3YVBq9WSlJREiRKPnoAxY8YwZswY4uPj+eqrrxg3bhwzZ85Ep9NhNBpt1jUYDOh0OjQaDTqd7oHvTBgMhmw9LRIba/9EPW5uJezerjApqDkIPHGT6f+7AEDP+o/RpGLG5CoA8fFxdOrUljNnTnP7dhxz5iz4z/0V1DzkJMlBwcjBvRizwu7CMGrUKIYMGULv3r3x9va2GWrb0dExW8NilClThvHjx9OgQQP8/f3x8PAgMjLSZp3IyEg8PDJmxfLw8ODmzZuZ9hMVFWV93NUeikK2fpjZ3a4wKUg5OHEjgc93XwSgWbWyfNDMyxp7YmICfn6vc+bMadzdK/DBB8PsOq+ClIfcIjkoPDmwuzDodDpiY2NZsmRJpts9j9L5bDKZSEtLw2w24+vry549e+jZs6e1PTQ0FF9fXwB8fHy4evUqiYmJ1g7o5ORkLl26RK1atbJ1fFG4xd41Me77MMwWhRY1yvNZu5po/rrpmpycTPfuXThx4hhubm4EBgZTrZrMSiiKLrsLw2effcbixYupVKlStg9qMpmIi4uzXgEkJibyySef0KpVK0qXLk2rVq2YP38+hw8ftr7HsHLlSmbNmgWAs7MznTp1Yvbs2UyaNAmNRsOXX35J+/btKVas2L8dWhRB6RaFj78P43ayCa+yLnzSqoa1KKSmptK7dzdCQ3+nVKnSbNq0nSefrKlyxEKoy+7CYDKZHqkoQMYQGoMGDSIlJQW9Xo9Wq6Vdu3bWzmcXFxcWL17MpEmTSElJQVEUhgwZQt26da37GDVqFNOmTaNt27YA1K9fH39//0eKSxROC/df5o+IBOsLbC46B2vb4MEDOHBgH66uJdi4cQu1a8ujzkLYXRi8vLw4evQo9evXz/ZBK1SowJYtW/51nZo1a7Jhw4aHtuv1ej799NNsxyCKht3nbvPtkYwnkCa+VsPmBTaAAQPe5/DhQyxf/g316jVQI0Qh8h27C0P58uV55513aNCgAeXKlbNp0+l0TJ06NceCE+JRnL+dzKc/ngOgV4NKvFyjfKZ1mjRpSmjoSbkFKcR97C4MPj4+TJ48+YFtMoCdyA+uxqXw+e4LHL52B4D6lUsxuJkXkDGcysSJ4+natQdPP10bQIqCEP9gd2Ho0qVLbsQhRI6ISzExaPNJopMz3nPxKuvCtLY+OGo1KIrC2LEfsXr1CgIDN3L48AkZ+0iIB7C7MBw/fjzTfAz3ODk5yZzPQjUpJjMjt50mOtlESWdHJrR6kmbeZdFqMorChAkfs3r1CjQaDVOmzJCiIMRD2F0Y5s6da/PWscFg4PLly5QsWZIGDRpIYRCq2H4qkqm7zgNQzEnLsm51qeZW3No+Y8YUli5dCMCcOV/xxhtvqhKnEAWB3YVh9erVmZalpKQwb968bL31LMSj2n8x1loUAKa0qWlTFObMmcWcORnje02fPpsePbI+oZQQRVGOzMfg4uLCuHHj2LFjR07sTogsi7iTyoQfwgF4sbobIcOfp3n1v5+W27JlM9OnTwFg4sSp9Os3QJU4hShIsjdO9gNYLJYHDmwnRG6JvZvR0ZxsNFPbsySftfPB0cH2b52WLVvz3HPNeO65ZgwePFSlSIUoWOwuDEeOHMnU+RwfH09QUBA+Pj45FpgQ/yYy0UC/9ce5nWyirIsTM9r74OSQ+QLY1dWVjRu32gz2KIT4d3YXhsWLF2e6MihZsiT16tXjrbfk3q3IfTvP3GLiD2eBjI7muZ2fxr3E3wM6btsWyJUrlxk+fCQg79cIYS+7C0N2R08V4lGlWxQ2HbvBvL2XAChdzInFfnWoXv7vjuadO7/n/ff7YzabqVGjJm3atFMrXCEKLLs7n9etW/fQtm+//faRghHi38z4+Txz9lzCosAL3m789H4Tm6Lwyy//4913+2A2m/Hz68Zrr7VRMVohCi67C8O/DWwXEBDwSMEI8TDbT0Wy/VQUAO1qVWBy6yfR3jeJ7f79e3n77Z6kpaXRocPrzJu3CK02Rx66E6LIsftW0j+n3LyfxWJ5pGCEeJBTNxP5fHfGdJyDnq/KO42r2LT//vsh3nqrKwaDgVatWrNo0TIcHXPsgTshihy7/6QqU6YMFy5cyLQ8PDxcvowixx26Ekff9ccxmRWaVStLn0aVbdpjYmLo0aMLKSkpNG/+EsuWfSOdzUI8IrsLw9ixYxk8eDA///wzd+7cITY2lu3btzNo0CDGjBmTGzGKIupGQirjd2S8vFa1bDEmt65pc/sIoFy5cvj7T+K555rxzTfrcXZ2ViNUIQoVjaLYP3X1qVOnmDNnDseOHQPA19eXwYMHP9LkPWqKiUmyawJvjQbKlSth93aFSW7nwJhuof/644RHJ+NTwZXl3Z5B5/jwv2PMZjMODg4Pbc8t8lmQHEDByMG9GLMiW/d+ateuzcqVK7OzqRD/yWxRGBt8hvDoZEoXc+LzDk/ZFIXLly8xYcI45s1bRNmybgCqFAUhCit5bEPkKxZF4aNtpzlwKQ6tBqa2rYlHyb9vD0VEXOeNN9rz008/MHbsRypGKkThJYVB5CsrQq7x2+U4APxb1qDx42WsbVFRkXTu3I6IiOt4e1dnypSZaoUpRKEmjxGJfGPLiZt8HXIVgEmvPUnbWhWsbbdv36ZLlw5cuXKZKlWqEhgYTIUKFR62KyHEI5DCIFRnURQm/XCWH8KiAXijrqdNUYiPj+PNNztx7txZKlZ8jMDAICpWfEytcIUo9LJ1K2ncuHHW/5c5GMSjWnHomrUotHyyPCNe9LZpHzZsMKdPn8LdvQKBgUE8/nhVFaIUoujI0hVDu3btqFChAh4eHnh6enLixAlCQ0OpXLkyy5cvp23btrkdpyikDl6OY9nBjNtHo16uzpu+FTOtM2nSVCIjb7JgwRK8vZ/I6xCFKHKyVBgCAgKIjIwkKiqKW7duYTQa2bJlC9euXePChQu0b9+eqlWr4uXlxYgRI3I7ZlFI3EhI5ZOd4ShA5zqeDywKANWqebNr1x40/3i5TQiRO7JUGLRaLV5eXnh5eQGwZcsWpk+fDkCnTp1Yv349V65c4dq1a7kXqShUDGlmxgaFkWhI5ymPEnz00t+3j0wmEwMH9qVHj168+uprAFIUhMhDWSoMnTp1wmAwULp0aTw9Pbl27RrffPMNlStXxmQy4eTkhI+Pj8zgJrIkxWRm5PbT1hfYZrb3sb7Alp6ezoAB77BzZzD79+/l6NFTlCpVWt2AhShislQYdu7cCcCdO3eIiopi1KhRKIrC/v37uXXrFh06dMBsNlOlShWWLVuWqwGLgi3NbGFwwEn+jExCq4Fp973AZjab+eCDAezcGYxer2fFijVSFIRQgV2Pq5YuXZrSpUtTuXJl3n77bQCOHTvGtm3bSEtLIyIiIjdiFIWEMd3CuxuOE3YrGYBpbX1o9NcLbBaLhREjhrBlSwCOjo6sWLGG5s1fUjNcIYqsbL3HsGjRIuv/+/v7A+Dk5GTtgxDin+JTTIzafsZaFL7sVItm3hnjHCmKwrhxI1m//lu0Wi1Ll66iZcvWaoYrRJH2yC+4NWjQINvb7t27l5UrVxIXF4eiKNSrV49x48ZRrFgxAC5evMjEiRNJSkpCo9EwaNAgWrZsad0+LS2NmTNnEhISAkCTJk0YM2aMjMefz0QmGnhr7R8kGNLRamD+G7VthrrYujWAVauWo9FoWLBgCe3bd1QxWiGEqmMlubi48PnnnxMcHMy2bdu4e/cu8+fPBzJmihs0aBBDhw5l+/btLFu2jC+++ILw8HDr9vPnz8dkMhEcHExwcDCKojB37lyVzkY8iDHdwpigMyQY0nHQapjR/imbogDQsWNnevbszRdfzMfPr5tKkQoh7lG1MDRs2NA63o2joyP9+/fnwIEDABw4cAAfHx8aNWoEQPny5enbty+BgYFAxj3p7du3M3LkSLRaLVqtlhEjRhAcHIzZbFbnhISNFJOZMUEZt49KOTuytV9DXnqinLX93lQgDg4OfPnlAnr16qNWqEKI++Sr0VXv3LmDXq8HICQkhIYNG9q0N2zYkIMHDwIQFhaGu7s7JUuWtLa7urpSsWJFTp8+nXdBiweKSjTw+orD/HY5Dg0Zw2d73jd89tKlCxk8eADp6emAvKcgRH6SrwbR27BhA506dQIgOjqapk2b2rR7enpy/fp1a7unp2emfXh4eBAREUGdOnWyfFx7fyfdW78o/y77txxEJRp4d8MJ4lLSAPBvVYNnvcpa27/5ZiWffJIx3larVq3p1KlzrsebW+SzIDmAgpEDe2LLN4Vh//79hIeHM2vWLAASExOtVw/36PV6jEYjiqKQmJj4wE5mvV5PamqqXcd2c8vadHc5tV1h8s8cJKSmMXjVEaKSjJR2cSL4g+epXNbF2r5mzRpGjfoQgNGjR9OvX+9CcbUgnwXJARSeHOSLwhAZGcmECROYP3++9Ze9TqfDaDTarGcwGNDpdGg0GnQ6HSaTKdO+DAaD3RPCx8baP+ezm1sJu7crTB6Ug3tXCpGJRorrHFjsV5tiFjMxMUkAbN0ayHvv9UNRFPr3H8jIkeOJjU1W8SwenXwWJAdQMHJwL8asUL0wpKSkMHjwYIYNG0bt2rWtyz08PIiMjLRZNzIyEg8PD2v7zZs3M+0vKirK7glcFIVs/TCzu11hci8HpnQLY4LCiEw0onPQsOTNOlQv52rNzw8/7GDQoHexWCz06tWHqVNnAppCkz/5LEgOoPDkQNXOZ7PZzIgRI2jWrJm1b+EeX19fDh8+bLMsNDQUX19fAHx8fLh69SqJiYnW9uTkZC5dukStWrVyPXZh68s9FzkdlURxnQOrevhSs8Lff5nEx8dZO5q7dOnKrFlz0Wrz1XMPQoj7qPrtnDZtGnq9nuHDh2dqa9WqFSdPnrQWh9u3b7Ny5Up69uwJgLOzM506dWL27NlYLBYUReHLL7+kffv21hfkRO6zKAoL9l0m8EQkGmBaOx9quLvarFOmTFlWrFiDn1835s9fjIODgzrBCiGyRKMo6lz4JCQk0KhRI7y8vGw6mTUaDcuXL6dcuXKEh4czadIkUlJSUBSFAQMG0L59e+u6RqORadOmERoaCkD9+vXx9/e3u48hJsb+PoZy5UrYvV1hci8Hn39/mkUHrgAwoOnjvPvs49Z1LBZLob8ykM+C5AAKRg7uxZilddUqDPmJFAb7aTRwJs5An5WHUYAe9R9jWPNqaP96wujEiWMMGfIeq1Z9W6hnXZPPguQACkYO7CkMhfvPOZFrIhMMDNtwDAXoWNuDD1/0thaFM2dO8+abnQgPD+Ozz6aoG6gQwm5SGITd7o1/FJ+Shk8FV0a9XN3aduHCebp06UB8fDz16zdg3ryFKkYqhMgOKQzCbl/8eoEzt5Ip7eLEzA5Pof9r9rUrVy7zxhvtiYm5Te3addmwYQuuroXjhR8hihIpDMIuQaei2HoyCg0wr5svFUtldPTfuBHBG2+0JzLyJk8+WZNNm7bJ7GtCFFBSGESWhd9KYubu8wAMfO5xmtcob22bOHE8169fo1o1bwICgnBzc1MrTCHEI1L9zWdRMMQkGxmx7TQms8Lz1crSt0kVm/bZs+ei0WiYNGkqFSp4qBSlECInSGEQ/+n+GdgeK+XM5NZPotVoSEtLs65TunQZli1brV6QQogcI7eSxL+6fwa24joHZnV8ipLOTiQlJdKsWTOWLl303zsRQhQocsUg/tWsXy5YZ2D79q16eJR05u7du3Tv7sfvv//O+fPneeONrtKnIEQhIlcM4qG2nYxk+6mMJ5CmtfXBo6QzBoOB3r278/vvIZQqVYpNm7ZJURCikJErBvFAf0Tc4fNfLgDw/vNVaVy1DCaTiX793mL//j0UL+7Kjz/+SPXqtfLtEABCiOyRKwaRSciVOAZuPEmaWaFZtbL0aVSZ9PR0Bg7sy//+9xPFihXju+820aRJE7VDFULkAikMwsaNhFT8d4QD4F3Ohcmta6LVaPjppx/YsSMInU7H6tXreO6551WOVAiRW+RWkrAypJkZGxRGoiGdWh4l+LprXXR/DXfRtm17Jk6cSvXqT/DSS6+oHKkQIjdJYRAApJktjA46Q3h0MqWLOTGjvQ9ODhqbObQHDx6qcpRCiLwgt5IEd1LS6PPdMUKuxKPVwLS2NalQQs/kyZ/g59eRpKTE/96JEKLQkMJQxBnTLQzdcorzt+8CMPG1J2n0eBk+//wzFi2az++/h7Bnzy8qRymEyEtyK6mIu/cCm4NWw8IutalfuTTz53/JF1/MBGDq1Bm0b99J3SCFEHlKrhiKsG+PRFhfYJv3+tPUr1yar79exNSpkwDw95/EgAGDVI1RCJH35IqhiFr9+zUWHrgC/P0C25o1q/D3HwvARx+NYejQESpGKIRQi1wxFEF7zsew6K+i0OHpCvRpVJmkpERmzJgKwODBwxg9+mMVIxRCqEmuGIqYnWduMfGHswC8UdeTsS2eAKBEiZJs3bqDrVsDGDNmPBqNRs0whRAqkiuGImTryUhrUahfuRQjXvQmOTnJ2v7kkzUZO9ZfioIQRZwUhiLiuyMRfPa/jGk5m3u7scivDr/t/5X69Z/mwIF9KkcnhMhPpDAUAUF/RjF37yUAXnqiHLM6PsWhkN94++0exMfH8913a1SOUAiRn0gfQyEXfiuJmT9nXCm0ecod/5Y1OHLkMD17vklqaiotWrRk3jyZhU0I8Te5YijEElLTGBN0BpNZ4flqZZn42pOEnT5J9+5duHs3mWbNXmTlym/R6XRqhyqEyEekMBRSFkVhwg/h3Ew08lgpZya3fpKz4WG8+WYnEhMTaNz4WdasWW8dIE8IIe6RwlBIrQi5xsHL8egdtczs8BQlnZ34+utFxMXF4etbj3XrNlO8eHG1wxRC5EPSx1AI/XYpjmUhVwEY1+IJnnR3BeDzz+dQvrw777//ASVKlFQzRCFEPpYvrhgCAgJ4+umniYiIsFl+8eJFevXqRceOHenUqRO7du2yaU9LS2Pq1Km0bduWtm3bMmXKFEwmU16Gnu/cSEhlwg/hKGS8wNa0ohPKX5MyOzk58fHHEyhTpqy6QQoh8jXVC8PcuXP58ccfKVWqFGaz2brcaDQyaNAghg4dyvbt21m2bBlffPEF4eHh1nXmz5+PyWQiODiY4OBgFEVh7ty5KpxF/mBIMzPmvhnYej3lSuvWrzB69AgsFova4QkhCghVC4PFYqF8+fIsXboUvV5v03bgwAF8fHxo1KgRAOXLl6dv374EBgZat92+fTsjR45Eq9Wi1WoZMWIEwcHBNgWmqFAUhZm7L3D2rxnYxjznTo9unbh06SK//PI/YmNj1Q5RCFFAqFoYtFotPXv2xMHBIVNbSEgIDRs2tFnWsGFDDh48CEBYWBju7u6ULPn3vXJXV1cqVqzI6dOnczfwfGjrqSi+P30LrQbGNfNkcN+uhIeH4eHhSWBgMOXLl1c7RCFEAZFvO5+jo6Np2rSpzTJPT0+uX79ubff09My0nYeHBxEREdSpUyfLx7J3aKB76+eXIYVORyYx+5cLAPStV55Zo/vy558nKV++PFu2BOPl5ZXjx8xvOVCL5EFyAAUjB/bElm8LQ2JiYqbbS3q9HqPRiKIoJCYmPvDFLL1eT2pqql3HcnMrka0Ys7tdToq7a2Lc94dJMyu87F2KXQtGcvToEcqWLcvu3bupXbt2rh4/P+QgP5A8SA6g8OQg3xYGnU6H0Wi0WWYwGNDpdGg0GnQ63QOfQDIYDHa/tBUbm8RfD+5kiUaT8QGwd7ucZrYoDAk8xc0EA1XKFOOVkjGsOXiQEiVKsmnTVjw9qxITk/TfO8qG/JIDtUkeJAdQMHJwL8asyLeFwcPDg8jISJtlkZGReHh4WNtv3ryZabuoqCgqVKhg17EUhWz9MLO7XU5Z+tsVDl+9g/NfL7FVL1ecFSvWUq5ceerU8c2T2NTOQX4heZAcQOHJgeqPqz6Mr68vhw8ftlkWGhqKr68vAD4+Ply9epXExERre3JyMpcuXaJWrVp5Gqsa/nf2Nit/v45iMTOkoRvVy2W8xdymTTsaNWqscnRCiIIs3xaGVq1acfLkSWtxuH37NitXrqRnz54AODs706lTJ2bPno3FYkFRFL788kvat29PsWLF1Aw91wWdiuLj78NQLGZcQ5Ywa1hXrl+/pnZYQohCIt/cSnJycsLR8e9wXFxcWLx4MZMmTSIlJQVFURgyZAh169a1rjNq1CimTZtG27ZtAahfvz7+/v55Hnte+jMykek/n0dRLDgeXM6Z337A0dGRs2fDqFy5itrhCSEKAY2iFIY7Yo8mJsb+zudy5UrYvd2jik8x0WvtH9xKMuJyZA3hv2xGq9WydOlKOnbsnHeBoF4O8hvJg+QACkYO7sWYFfnmikH8u3SLwsc7wrmVZMTy+7eE790MwLx5i/K8KAghCjcpDAVAZKKBoYGnuBKXyt2Q9cTu3wjArFlz6dq1h8rRCSEKGykM+dzNBAPvrDtGXEoaFlMq+mu/AzBlynT69OmrcnRCiMJICkM+9uv5GEYHnQGghN6ROd2aULH/Lnbt+oGePXurHJ0QorCSwpBPrTsawZw9lwBwSY1hxduv4eXmAiBFQQiRq/LtewxF2YFLsdai4BF1mHML+3HwxwCVoxJCFBVSGPKZX87d5sOtGcOG17z7J6Frp2I2mzl16oTKkQkhigq5lZSPhN1K4pOdGTPUucf9ye5V/lgsFrp378Vnn81SOTohRFEhVwz5xJ3UNMYEncFkVqiUfJ6TayaSnp5O585d+PLLBWi18qMSQuQNuWLIB1LTzIzafprIRCMl4s9xdM14jEYjbdq0Z8GCpQ+c4U4IIXKLFAaVhd9KYsDGE6SmWdA7aqmvucafhlReeeVVli5diZOTk9ohCiGKGCkMKrp3+yg1zQLAp21q8lL156j3VA06dXoj0wx2QgiRF6QwqCTdovDRttPcTDTiZo7j2wEtKFfaFYDu3XupHJ0QoiiTHk0VGNMt9F13jJM3E9Ek3OTKqo94/92epKSkqB2aEEJIYVDDrN0XCLuVTNqdKJK2TCQu9jYxMTGYTMb/3lgIIXKZFIY8tu1kJNv/jMKcGIM5eDKx0ZHUqPEkmzZto3TpMmqHJ4QQ0seQl7acuMn0ny9gTo7HEDSJmJvXqVrVi4CAIMqXL692eEIIAUhhyDO/XY5jxs8XMKckkLx1InduXqFSpcps2fI9Hh6eaocnhBBWcispD9xISGXCznAUoF7pNMxJMVSo4EFgYDCVKlVWOzwhhLAhVwy57Gx0MgM2nCAlzUwtjxIs6vo8YS94UayYC15e1dQOTwghMpHCkIvupKYxevtpklPu4mKIZca7r6Nz1FK3rq/aoQkhxENJYcglZovCJzvDuRGXRELQdO7cOk9EO288GjRSOzQhhPhX0seQS5aHXCXkYjSxQTNJvHAUs9mM2WxROywhhPhPcsWQCw5cimXZwcvEBH9ByvnfcXZ25ttvN9K4cRO1QxNCiP8kVww5LOJOKp/sCCN251xSzh7AycmJ1au/4/nnX1A7NCGEyBIpDDkoyZDOkICTXA2ax93Tv+Lg4MCyZd/w8suvqh2aEEJkmdxKyiF3UtJ469s/iIxPgqRotFotixcvp02bdmqHJoQQdpHCkAPuPYEUlWREp9ezcd0m7l47zYsvvqx2aEIIYTcpDDng65Cr7Nm/j5JedfimZz2qly8O1SqoHZYQQmSL9DE8ov0XY5kzbw631o/jiUtb8S7nonZIQgjxSAr8FcOmTZtYu3YtGo0Gd3d3pk2bRoUKefPX+vX4VAZ/Oos7e1YB8Ey1img0mjw5thBC5JYCfcWwf/9+Nm7cyLp16wgKCqJz58588MEHeXLsVJOZnuNnEfXDIgCGDhvJ8OEj8+TYQgiRmwp0Ydi4cSNDhw6lRIkSALRp0wYHBwfCwsJy9biKotD5o88J2/wFAH36vc/4jz/J1WMKIUReKdCFISQkhIYNG9osa9iwIb/99luuHnf8V9/w40J/QKHNG734/LMZcgtJCFFoFNg+hrt37+Lo6IiLi21nr4eHB+fOnbNrX/b8Tjekmdl+5AIoCg1bdGTVoq/QaoteUbiXs6JeDyUPkgMoGDmwJ7YCWxiSkpLQ6XSZluv1egwGg137cnMrkeV1LRaFt9/px50XG7Dww+44OhbYFOYIe3JXmEkeJAdQeHJQYH+r6XQ6TCZTpuVGoxG9Xm/XvmJjk1CUrK8/+iUv3LrUsXu7wkSjyfgSFOUcgOQBJAdQMHJwL8asKLCFoUyZMhgMBu7evUvx4sWtyyMjI/Hw8LBrX4pCtn6Y2d2uMJEcZJA8SA6g8OSgwHY+azQa6tSpw5EjR2yWh4aG4usrM6QJIUR2FdjCANC7d2/mzZtHcnIyADt37iQlJYXGjRurHJkQQhRcBfZWEsCrr75KZGQkXbt2RavVUq5cORYtWoRWW6DrnRBCqKpAFwbIuGro3bu32mEIIUShIX9aCyGEsCGFQQghhA0pDEIIIWxIYRBCCGFDCoMQQggbUhiEEELYkMIghBDCRoF/jyEn2DtUbkEYYje3SQ4ySB4kB1AwcmBPbBpFKQxDPgkhhMgpcitJCCGEDSkMQgghbEhhEEIIYUMKgxBCCBtSGIQQQtiQwiCEEMKGFAYhhBA2pDAIIYSwIW8+Z8OmTZtYu3YtGo0Gd3d3pk2bRoUKFdQOK8fs3buXlStXEhcXh6Io1KtXj3HjxlGsWDEALl68yMSJE0lKSkKj0TBo0CBatmxp3T4tLY2ZM2cSEhICQJMmTRgzZgw6nU6V83kUFy9epGPHjrz33nt88MEHAERHR+Pv709UVBSKotCjRw+6d+9u3UZRFBYvXsyOHTvQaDTUqFGDTz/9FFdXV7VOI1sMBgNff/01e/bswWKxYDQamTBhAs8++yxQNPKQnJzM3LlzOXz4MBqNhmLFijF06FCaNm0KFOLvgiLssm/fPqVz585KYmKioiiKsmPHDqVLly4qR5WzDh8+rERFRSmKoihpaWnKiBEjlBkzZiiKoigGg0Fp2bKl8vvvvyuKoijR0dFKy5YtlbCwMOv2s2fPVj755BPFbDYrZrNZmTx5sjJz5sy8P5Ec0LdvX6Vfv37Kl19+aV3WtWtXZfv27YqiKEpSUpLi5+en7Nmzx9q+fv16ZcCAAYrRaFQURVG+/vprZejQoXkb+CNKS0tTevXqpSxYsMB6HhaLRUlLS7OuUxTy0LdvX2XJkiWK2WxWFEVRTp06pTz33HNKREREof4uSGGw0+DBg20+/IqS8QU5c+aMShHlvjNnzijt2rVTFEVRfv75Z2XYsGE27Rs2bFCmTp2qKIqimM1mpVmzZkpCQoK1PSkpSXn++eeV9PT0PIs5J/z444/KqFGjlPnz51sLQ1hYWKY/BPbv36+8//771n936tRJuXDhgvXfZrNZefHFF5W4uLi8CTwHBAQEKO+9995D24tKHmrVqmX9I/CeAQMGKD/99FOh/i5IH4OdQkJCaNiwoc2yhg0b8ttvv6kUUe67c+cOer0eePj5Hzx4EICwsDDc3d0pWbKktd3V1ZWKFSty+vTpvAv6EaWmpjJ//nw++ugjm+UPOv8GDRpw6NAhFEUhPj6eW7du4e3tbW3XarX4+vpy6NChPIk9J+zcuZNu3bo9tL2o5KFu3bqsWrXK+u/Q0FCOHTtGnTp1CvV3QfoY7HD37l0cHR1xcXGxWe7h4cG5c+dUiir3bdiwgU6dOgEZ95Xv3V+9x9PTk+vXr1vbPT09M+3Dw8ODiIgI6tSpk+vx5oQlS5bQvn37TH1H0dHRVKxY0WaZs7Mzer2e2NhY4uLi8PDwyLQ/Dw8Pa44KgvDwcPR6PUOGDOHKlSuUKVOG/v3788ILLwBFJw8zZszg3Xff5cSJE3h5eREcHMysWbPw8PAo1N8FuWKwQ1JS0gM7jfR6PQaDQYWIct/+/fsJDw/nzTffBCAxMdF69XCPXq/HaDSiKAqJiYkPzVFqamqexPyorl27xq5du3jnnXcytT3o/OHv8/u38y9In5E7d+6wePFihg8fTnBwMB9//DETJkzg999/B4pOHh577DF69uxJSEgIa9eu5fnnn6d27dpA4f4uSGGwg06nw2QyZVpuNBof+CUp6CIjI5kwYQKzZ8+2fsB1Oh1Go9FmPYPBgE6nQ6PRPDRHBoMBZ2fnPIn7UU2bNo3hw4c/8Gf6oPOHv8+vMJw/gEajoX///tZbQTVr1uTtt98mMDAQKDp5GDVqFNu2bWPVqlX8/PPPODo60qFDB6Kiogr1d0FuJdmhTJkyGAwG7t69S/Hixa3LIyMjH3jZXJClpKQwePBghg0bZv0LCTIugyMjI23Wvf/8PTw8uHnzZqb9RUVFFYhHevft20dqaiqtWrV6YPuDzs9gMJCSkoKbmxuKojz0/GvWrJkrMecGNzc3qlatarOsSpUqHDhwACgaebh69Sp79+7l119/pUSJEgDMnDmTsWPHsm7dukL9XZArBjtoNBrq1KnDkSNHbJaHhobi6+urUlQ5z2w2M2LECJo1a2btW7jH19eXw4cP2yy7//x9fHy4evUqiYmJ1vbk5GQuXbpErVq1cj32RxUREcGtW7fo2LGj9b8NGzYQEBBA586d8fX1JTQ01Gab0NBQateujVarxd3dHRcXFy5cuGBtt1gsHD16lHr16uX16WRb7dq1M/WbXb16lccffxygSOQhOTkZd3d3a1G4p0aNGiQkJBTu74Kaj0QVRLt27VJef/11JSkpSVGUjPcY2rVrZ33OuTCYPHmyMnToUMVisWRqu3v3rvLiiy/aPLv96quvKsePH7euM2XKFOuz2xaLRZk8ebIyadKkPIs/p93/uKrFYlE6duyY6fn9nTt3WtdftWpVpuf333333bwP/BH89ttvStu2bZXo6GhFURTlwoULyksvvWR9/LQo5CE9PV3p0qWLsmrVKuv3++rVq0qrVq2UI0eOFOrvgkztmQ1r1qxh48aNaLVaypUrx6effkrlypXVDitHJCQk0KhRI7y8vGzusWs0GpYvX065cuUIDw9n0qRJpKSkoCgKAwYMoH379tZ1jUYj06ZNs/5FWb9+ffz9/fP9fdWHWbx4MWaz2frm840bN5gwYQLR0dGYzWb8/PxsOqoVRWHu3Lns2rULrVaLt7c3kyZNomzZsmqdQrZs3ryZ5cuXo9FocHFxYfjw4dankqBo5CEuLo65c+dy7NgxHBwccHZ25t133+WVV14BKLTfBSkMQgghbEgfgxBCCBtSGIQQQtiQwiCEEMKGFAYhhBA2pDAIIYSwIYVBCCGEDSkMQgghbEhhEOIvrVu3xmg0kpqaStu2bdUOx25Hjx7F399f7TBEISCFQYi/pKamotfriYyMxN3dXe1w7Jaenk56erraYYhCQAqDEEB8fDxlypQB4NatW4VutFwh7CHDbguBbTGIiop64Mxb9yxZsgRHR0fCwsK4cOECkyZNwtfXlx07drBo0SIURaFkyZJMmDCBp556yrrdli1bWLZsGRqNBo1Gw4wZM6hduzYxMTFMnz7dOh6Pt7c3H3/8MVWqVPnX423ZsoXVq1ejKAqlSpXitddesx7LZDIxfvx4wsLC0Ol01K1bl4kTJ+ZS9kRhI4VBFHnvv/8+ERERJCQk8NZbb3Hjxg10Oh03btxg5syZmdZPS0sjICCAefPmWYdPPnXqFCtWrGDt2rWULVuWP/74g+HDh7Njxw6cnJxYvXo1+/fvZ926ddYrk/uP/9JLLzF79mw0Gg3BwcH07duXnTt3otPpHni8EydOMHfuXDZs2EDFihW5cOECb7/9Ns8//zwAwcHBlCxZku+//x7IGNBOiKySW0miyFu8eDGdO3emb9++rF27lsaNGzNp0qQHFoV7qlatajOm/urVqxk6dKh15NB69epRuXJljh07hsFgYMmSJUyfPj1TUQgJCcFoNDJo0CA0Gg0A7du354knnrD+Un/Q8YKCgujVq5d13uXq1avj5+dnbbdYLDb9Dff2LURWSGEQgoxJaCpVqmT9/3sT0jzMvSkv77l48SIzZ860meDn8uXLJCYmcu7cOcqVK/fADu1z585Rv379TMvr16/P2bNnH3q8GzduUL16dZtl9xeODh06kJqayptvvsnevXv/9VyE+Ce5lSSKtCNHjjBlyhSuX7/OoUOHWLBgARcvXuS9996jf//+NmPr3++f4+kbDAY+//xz6tSpk2ndkydPPvRpIQcHhwcuVxQFrfbvv9v+eTytVpvp9tD9/9br9Xz++efW+QL27dvHJ5988sBjCfFPcsUgirQGDRqwfft23N3dCQoKYv369VStWpXt27c/tCg8SNWqVTl16tQD25544gni4uKIiIjI1PbUU09x9OjRTMv/+OMPm47rBx3v/PnzNssetJ+aNWuyatUqtm3bRlxc3H+dhhCAFAYhMJlMWCwWdDodV65c+c/bSA/SpUsXli5dajPH8b1CUKxYMfr168fYsWOJj4+32a5evXq4uLjw1VdfWf/i37ZtG5cuXbJ5yuif3nzzTTZt2mSdbP7EiRP89NNP1vbExETr/q5du4ZWq800d7EQDyO3kkSRFxERYZ2a9dq1a3h5ef3r+jqdzuY2D0CLFi0wGAx89NFHmM1mHB0dadq0KaNHjwZg4MCBFCtWjO7du6PT6TCbzUyfPp06deqwePFiZsyYQYsWLdBqtTz55JOsXLkSJyenhx6vatWqjB49mv79++Pk5ES5cuUYNWoUBw8eBGDDhg2sX78eV1dXnJycmDNnjnV/QvwXmdpTCCGEDbmVJIQQwoYUBiGEEDakMAghhLAhhUEIIYQNKQxCCCFsSGEQQghhQwqDEEIIG1IYhBBC2JDCIIQQwoYUBiGEEDakMAghhLDxfz2k8gwqbt8lAAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# convert y_valid to a categorical data type\n",
    "y_valid = y_valid.astype('category')\n",
    "\n",
    "df = pd.DataFrame({'actual':1-y_valid.cat.codes, 'prob':predProb_valid[:, 0]})\n",
    "df = df.sort_values(by = ['prob'], ascending = False).reset_index(drop = True)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "fig.set_size_inches(4,4)\n",
    "gainsChart(df.actual, ax = ax)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "outputs": [
    {
     "data": {
      "text/plain": "array([[0.49708304, 0.12572597, 0.00759929, 0.08144331, 0.03668576,\n        0.03348031, 0.01959004, 0.072658  , 0.00439384, 0.02885022,\n        0.02564477, 0.0147225 , 0.00130711, 0.02125212, 0.00213815,\n        0.02742557],\n       [0.48845264, 0.12787337, 0.01029317, 0.06663368, 0.05095632,\n        0.04850673, 0.02548061, 0.05389582, 0.00735366, 0.04017813,\n        0.01225284, 0.02842011, 0.00098473, 0.0279302 , 0.00196457,\n        0.00882341]])"
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.exp(delays_nb.feature_log_prob_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "---------------------------------------"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 08. Tree & Logistic Regression"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor\n",
    "from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier\n",
    "from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV\n",
    "import matplotlib.pylab as plt\n",
    "import dmba\n",
    "from dmba import plotDecisionTree, classificationSummary, regressionSummary"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 분류 회귀 트리\n",
    "* 가장 명료하며 결과에 대한 해석 용이\n",
    "* 관측치들을 하위 그룹으로 분리해 나가는 절차의 반복\n",
    "* 하위 그룹으로의 분할 작업-> 예측 변수 기반\n",
    "* 재귀적 분할 (Recursive Partitioning) & 가지치기(pruning)\n",
    "* 모델 생성 이후 대량의 표본을 처리하는 계산 용이함"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes: Nonowner,Owner\n"
     ]
    },
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mower_df = pd.read_csv('https://raw.githubusercontent.com/reisanar/datasets/master/RidingMowers.csv')\n",
    "\n",
    "classTree = DecisionTreeClassifier(random_state = 0, max_depth = 1)\n",
    "classTree.fit(mower_df.drop(columns = ['Ownership']), mower_df['Ownership'])\n",
    "\n",
    "print(\"Classes: {}\".format(','.join(classTree.classes_)))\n",
    "plotDecisionTree(classTree, feature_names = mower_df.columns[:2], class_names = classTree.classes_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pydotplus\n",
      "  Using cached pydotplus-2.0.2.tar.gz (278 kB)\n",
      "  Preparing metadata (setup.py): started\n",
      "  Preparing metadata (setup.py): finished with status 'error'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "  error: subprocess-exited-with-error\n",
      "  \n",
      "  python setup.py egg_info did not run successfully.\n",
      "  exit code: 1\n",
      "  \n",
      "  [20 lines of output]\n",
      "  Traceback (most recent call last):\n",
      "    File \"<string>\", line 2, in <module>\n",
      "    File \"<pip-setuptools-caller>\", line 34, in <module>\n",
      "    File \"C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-67ukf6u0\\pydotplus_791cc04f5a7a4392a3c1f7178ff01c29\\setup.py\", line 55, in <module>\n",
      "      setup(\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\__init__.py\", line 87, in setup\n",
      "      return distutils.core.setup(**attrs)\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\_distutils\\core.py\", line 147, in setup\n",
      "      _setup_distribution = dist = klass(attrs)\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\dist.py\", line 475, in __init__\n",
      "      _Distribution.__init__(\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\_distutils\\dist.py\", line 283, in __init__\n",
      "      self.finalize_options()\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\dist.py\", line 898, in finalize_options\n",
      "      for ep in sorted(loaded, key=by_order):\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\dist.py\", line 897, in <lambda>\n",
      "      loaded = map(lambda e: e.load(), filtered)\n",
      "    File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\setuptools\\_vendor\\importlib_metadata\\__init__.py\", line 196, in load\n",
      "      return functools.reduce(getattr, attrs, module)\n",
      "  AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'\n",
      "  [end of output]\n",
      "  \n",
      "  note: This error originates from a subprocess, and is likely not a problem with pip.\n",
      "error: metadata-generation-failed\n",
      "\n",
      "Encountered error while generating package metadata.\n",
      "\n",
      "See above for output.\n",
      "\n",
      "note: This is an issue with the package mentioned above, not pip.\n",
      "hint: See above for details.\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "! pip install pydotplus"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'pydotplus'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mModuleNotFoundError\u001B[0m                       Traceback (most recent call last)",
      "Input \u001B[1;32mIn [9]\u001B[0m, in \u001B[0;36m<cell line: 1>\u001B[1;34m()\u001B[0m\n\u001B[1;32m----> 1\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mpydotplus\u001B[39;00m\n\u001B[0;32m      3\u001B[0m classTree \u001B[38;5;241m=\u001B[39m DecisionTreeClassifier(random_state \u001B[38;5;241m=\u001B[39m \u001B[38;5;241m0\u001B[39m)\n\u001B[0;32m      4\u001B[0m classTree\u001B[38;5;241m.\u001B[39mfit(mower_df\u001B[38;5;241m.\u001B[39mdrop(columns \u001B[38;5;241m=\u001B[39m [\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mOwnership\u001B[39m\u001B[38;5;124m'\u001B[39m]), mower_df[\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mOwnership\u001B[39m\u001B[38;5;124m'\u001B[39m])\n",
      "\u001B[1;31mModuleNotFoundError\u001B[0m: No module named 'pydotplus'"
     ]
    }
   ],
   "source": [
    "\n",
    "classTree = DecisionTreeClassifier(random_state = 0)\n",
    "classTree.fit(mower_df.drop(columns = ['Ownership']), mower_df['Ownership'])\n",
    "\n",
    "print('Classes: {}'.format(',', feature_names = mower_df.columns[:2], class_names = classTree.classes_))\n",
    "plotDecisionTree(classTree, feature_names = mower_df.columns[:2], class_names = classTree.classes_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 트리 구조\n",
    "### 기본 구조\n",
    "* 결정 노드 (Decision Node) : 자식 노드를 가짐, 에측 변수 값만 알고 있는 새로운 관측치를 분류하기 위해 트리 모델을 사용할 경우 관측치를 트리에 떨어져서 적합한 값을 갖는 가지를 통해 트리 아래로 내려 보냄\n",
    "* 단말 노드 (Terminal Node) : 자식이 없는 노드, 예측 변수에 의한 데이터의 분할"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 의사결정 규칙\n",
    "* 트리가 적절하다면 모델이 제공하는 결정 기준이 이해하기 쉬움\n",
    "* ex) 은행 고객들의 대출 신청 승인 모델\n",
    "* 0은 대출 신청 거부, 1은 승인\n",
    "\n",
    "* 분할 노드의 상단 조건 : 분할을 위해 예측값 & 분할값 제공(ex) 상위 노드의 소득 <= 110.5)\n",
    "* Sample 해당 노드의 레코드 수\n",
    "* Values 해당 노드에 있는 클래스 개수\n",
    "* 분류 규칙 살펴보기\n",
    "* ex) 트리 모델에서 결정 노드 'Family'의 아래 왼쪽 노드?\n",
    "* --> IF 소득 > 110.5 & 교육 수준 < 1.5 & 가족 수 < 2.5 Then Class = 0 (거부)\n",
    "\n",
    "\n",
    "### 새로운 관측치의 분류\n",
    "* 관측치를 트리 모델의 위에서 아래로 할당하는 과정\n",
    "* 새로운 관측치의 클래스는 투표를 통해 결정\n",
    "\n",
    "\n",
    "### 재귀적 분할\n",
    "* 종속 변수 : Y, 예측 변수 : X1, X2, ,,, Xn\n",
    "* 분류 문제 -> 모델의 결과 변수가 범주형 변수\n",
    "* 예측 변수들의 p차원 공간을 서로 겹치지 않는 다차원의 직사각형으로 분할\n",
    "1. 하나의 변수 Xi 가 선택되고 p차원의 공간을 두 부분으로 분할하게끔 Xi의 변구 값인 Si가 선택\n",
    "2. 이 중 한 부분은 Xi<=S인 모든 점 포함\n",
    "3. 나누어진 두 부분 중 한 부분은 마찬가지로 동일한 과정에 의해 다시 한 번 변수를 선정\n",
    "4. 위의 과정 반복"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [
    {
     "data": {
      "text/plain": "   Income  Lot_Size Ownership  Number\n0    60.0      18.4     Owner       1\n1    85.5      16.8     Owner       2\n2    64.8      21.6     Owner       3\n3    61.5      20.8     Owner       4\n4    87.0      23.6     Owner       5",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Income</th>\n      <th>Lot_Size</th>\n      <th>Ownership</th>\n      <th>Number</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>60.0</td>\n      <td>18.4</td>\n      <td>Owner</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>85.5</td>\n      <td>16.8</td>\n      <td>Owner</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>64.8</td>\n      <td>21.6</td>\n      <td>Owner</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>61.5</td>\n      <td>20.8</td>\n      <td>Owner</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>87.0</td>\n      <td>23.6</td>\n      <td>Owner</td>\n      <td>5</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 잔디깎이 기계 데이터를 로드 및 분할하고 산포도를 그리기 위한 코드\n",
    "\n",
    "mower_df['Number'] = mower_df.index + 1\n",
    "mower_df.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Income  Lot_Size Ownership  Number\n",
      "5    110.1      19.2     Owner       6\n",
      "3     61.5      20.8     Owner       4\n",
      "7     82.8      22.4     Owner       8\n",
      "20    47.4      16.4  Nonowner      21\n",
      "18    59.4      16.0  Nonowner      19\n",
      "2     64.8      21.6     Owner       3\n",
      "4     87.0      23.6     Owner       5\n",
      "0     60.0      18.4     Owner       1\n",
      "13    52.8      20.8  Nonowner      14\n",
      "22    51.0      14.0  Nonowner      23\n",
      "1     85.5      16.8     Owner       2\n",
      "16    84.0      17.6  Nonowner      17\n",
      "6    108.0      17.6     Owner       7\n",
      "21    33.0      18.8  Nonowner      22     Income  Lot_Size Ownership  Number\n",
      "12    75.0      19.6  Nonowner      13\n",
      "14    64.8      17.2  Nonowner      15\n",
      "11    81.0      20.0     Owner      12\n",
      "8     69.0      20.0     Owner       9\n",
      "15    43.2      20.4  Nonowner      16\n",
      "10    51.0      22.0     Owner      11\n",
      "17    49.2      17.6  Nonowner      18\n",
      "19    66.0      18.4  Nonowner      20\n",
      "23    63.0      14.8  Nonowner      24\n",
      "9     93.0      20.8     Owner      10\n"
     ]
    },
    {
     "data": {
      "text/plain": "   Income  Lot_Size\n0      60        20",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Income</th>\n      <th>Lot_Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>60</td>\n      <td>20</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainData, validData = train_test_split(mower_df, test_size=0.4, random_state=26)\n",
    "print(trainData, validData)\n",
    "newHousehold = pd.DataFrame([{'Income':60, 'Lot_Size':20}])\n",
    "newHousehold"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "cannot unpack non-iterable AxesSubplot object",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "Input \u001B[1;32mIn [16]\u001B[0m, in \u001B[0;36m<cell line: 4>\u001B[1;34m()\u001B[0m\n\u001B[0;32m      1\u001B[0m \u001B[38;5;66;03m# 잔디깎이 기계 데이터를 로드 및 분할하고 산포도를 그리기 위한 코드\u001B[39;00m\n\u001B[0;32m      2\u001B[0m \n\u001B[0;32m      3\u001B[0m \u001B[38;5;66;03m# ScatterPlot\u001B[39;00m\n\u001B[1;32m----> 4\u001B[0m fig, ax \u001B[38;5;241m=\u001B[39m plt\u001B[38;5;241m.\u001B[39msubplot()\n\u001B[0;32m      6\u001B[0m subset \u001B[38;5;241m=\u001B[39m TrainData\u001B[38;5;241m.\u001B[39mloc[trainData[\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mOwnership\u001B[39m\u001B[38;5;124m'\u001B[39m]\u001B[38;5;241m==\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mOwner\u001B[39m\u001B[38;5;124m'\u001B[39m]\n\u001B[0;32m      7\u001B[0m ax\u001B[38;5;241m.\u001B[39mscatter(subset\u001B[38;5;241m.\u001B[39mIncome, subset\u001B[38;5;241m.\u001B[39mLot_Size, marker \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124mO\u001B[39m\u001B[38;5;124m'\u001B[39m, label \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124mOwner\u001B[39m\u001B[38;5;124m'\u001B[39m, color \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124mC1\u001B[39m\u001B[38;5;124m'\u001B[39m)\n",
      "\u001B[1;31mTypeError\u001B[0m: cannot unpack non-iterable AxesSubplot object"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error in callback <function install_repl_displayhook.<locals>.post_execute at 0x000001FD69746160> (for post_execute):\n",
      "Unexpected exception formatting exception. Falling back to standard exception\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\events.py\", line 89, in trigger\n",
      "    func(*args, **kwargs)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\pyplot.py\", line 137, in post_execute\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\_pylab_helpers.py\", line 141, in draw_all\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py\", line 2060, in draw_idle\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 400, in draw\n",
      "    self.figure.draw(self.renderer)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 74, in draw_wrapper\n",
      "    if artist.get_agg_filter() is not None:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 51, in draw_wrapper\n",
      "    renderer.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\figure.py\", line 2845, in draw\n",
      "    wspace : float, default: :rc:`figure.constrained_layout.wspace`\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\image.py\", line 132, in _draw_list_compositing_images\n",
      "    else:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 51, in draw_wrapper\n",
      "    renderer.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axes\\_base.py\", line 3091, in draw\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\image.py\", line 132, in _draw_list_compositing_images\n",
      "    else:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 51, in draw_wrapper\n",
      "    renderer.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1159, in draw\n",
      "    # attribute, and the derived code below will check for that\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1085, in _get_tick_bboxes\n",
      "    child.set_clip_path(clippath, transform)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1085, in <listcomp>\n",
      "    child.set_clip_path(clippath, transform)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\text.py\", line 910, in get_window_extent\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\text.py\", line 309, in _get_layout\n",
      "    bb = self.get_window_extent()\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 233, in get_text_width_height_descent\n",
      "    font = self._prepare_font(prop)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 267, in _prepare_font\n",
      "    font = get_font(_fontManager._find_fonts_by_props(font_prop))\n",
      "AttributeError: 'FontManager' object has no attribute '_find_fonts_by_props'\n",
      "\n",
      "During handling of the above exception, another exception occurred:\n",
      "\n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 1993, in showtraceback\n",
      "    stb = self.InteractiveTB.structured_traceback(\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 1118, in structured_traceback\n",
      "    return FormattedTB.structured_traceback(\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 1012, in structured_traceback\n",
      "    return VerboseTB.structured_traceback(\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 865, in structured_traceback\n",
      "    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 818, in format_exception_as_a_whole\n",
      "    frames.append(self.format_record(r))\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 736, in format_record\n",
      "    result += ''.join(_format_traceback_lines(frame_info.lines, Colors, self.has_colors, lvals))\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 699, in lines\n",
      "    pieces = self.included_pieces\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 647, in included_pieces\n",
      "    pos = scope_pieces.index(self.executing_piece)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 626, in executing_piece\n",
      "    return only(\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\executing\\executing.py\", line 164, in only\n",
      "    raise NotOneValueFound('Expected one value, found 0')\n",
      "executing.executing.NotOneValueFound: Expected one value, found 0\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "print_png() got an unexpected keyword argument 'orientation'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\formatters.py:339\u001B[0m, in \u001B[0;36mBaseFormatter.__call__\u001B[1;34m(self, obj)\u001B[0m\n\u001B[0;32m    337\u001B[0m     \u001B[38;5;28;01mpass\u001B[39;00m\n\u001B[0;32m    338\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[1;32m--> 339\u001B[0m     \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mprinter\u001B[49m\u001B[43m(\u001B[49m\u001B[43mobj\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    340\u001B[0m \u001B[38;5;66;03m# Finally look for special method names\u001B[39;00m\n\u001B[0;32m    341\u001B[0m method \u001B[38;5;241m=\u001B[39m get_real_method(obj, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mprint_method)\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\pylabtools.py:151\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(fig, fmt, bbox_inches, base64, **kwargs)\u001B[0m\n\u001B[0;32m    148\u001B[0m     \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01mmatplotlib\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mbackend_bases\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m FigureCanvasBase\n\u001B[0;32m    149\u001B[0m     FigureCanvasBase(fig)\n\u001B[1;32m--> 151\u001B[0m \u001B[43mfig\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcanvas\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mprint_figure\u001B[49m\u001B[43m(\u001B[49m\u001B[43mbytes_io\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkw\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    152\u001B[0m data \u001B[38;5;241m=\u001B[39m bytes_io\u001B[38;5;241m.\u001B[39mgetvalue()\n\u001B[0;32m    153\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m fmt \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124msvg\u001B[39m\u001B[38;5;124m'\u001B[39m:\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:2289\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)\u001B[0m\n\u001B[0;32m   2241\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mprint_figure\u001B[39m(\n\u001B[0;32m   2242\u001B[0m         \u001B[38;5;28mself\u001B[39m, filename, dpi\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, facecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, edgecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2243\u001B[0m         orientation\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mportrait\u001B[39m\u001B[38;5;124m'\u001B[39m, \u001B[38;5;28mformat\u001B[39m\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m,\n\u001B[0;32m   2244\u001B[0m         bbox_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, pad_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, bbox_extra_artists\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2245\u001B[0m         backend\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs):\n\u001B[0;32m   2246\u001B[0m     \u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m   2247\u001B[0m \u001B[38;5;124;03m    Render the figure to hardcopy. Set the figure patch face and edge\u001B[39;00m\n\u001B[0;32m   2248\u001B[0m \u001B[38;5;124;03m    colors.  This is useful because some of the GUIs have a gray figure\u001B[39;00m\n\u001B[0;32m   2249\u001B[0m \u001B[38;5;124;03m    face color background and you'll probably want to override this on\u001B[39;00m\n\u001B[0;32m   2250\u001B[0m \u001B[38;5;124;03m    hardcopy.\u001B[39;00m\n\u001B[0;32m   2251\u001B[0m \n\u001B[0;32m   2252\u001B[0m \u001B[38;5;124;03m    Parameters\u001B[39;00m\n\u001B[0;32m   2253\u001B[0m \u001B[38;5;124;03m    ----------\u001B[39;00m\n\u001B[0;32m   2254\u001B[0m \u001B[38;5;124;03m    filename : str or path-like or file-like\u001B[39;00m\n\u001B[0;32m   2255\u001B[0m \u001B[38;5;124;03m        The file where the figure is saved.\u001B[39;00m\n\u001B[0;32m   2256\u001B[0m \n\u001B[0;32m   2257\u001B[0m \u001B[38;5;124;03m    dpi : float, default: :rc:`savefig.dpi`\u001B[39;00m\n\u001B[0;32m   2258\u001B[0m \u001B[38;5;124;03m        The dots per inch to save the figure in.\u001B[39;00m\n\u001B[0;32m   2259\u001B[0m \n\u001B[0;32m   2260\u001B[0m \u001B[38;5;124;03m    facecolor : color or 'auto', default: :rc:`savefig.facecolor`\u001B[39;00m\n\u001B[0;32m   2261\u001B[0m \u001B[38;5;124;03m        The facecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2262\u001B[0m \u001B[38;5;124;03m        facecolor.\u001B[39;00m\n\u001B[0;32m   2263\u001B[0m \n\u001B[0;32m   2264\u001B[0m \u001B[38;5;124;03m    edgecolor : color or 'auto', default: :rc:`savefig.edgecolor`\u001B[39;00m\n\u001B[0;32m   2265\u001B[0m \u001B[38;5;124;03m        The edgecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2266\u001B[0m \u001B[38;5;124;03m        edgecolor.\u001B[39;00m\n\u001B[0;32m   2267\u001B[0m \n\u001B[0;32m   2268\u001B[0m \u001B[38;5;124;03m    orientation : {'landscape', 'portrait'}, default: 'portrait'\u001B[39;00m\n\u001B[0;32m   2269\u001B[0m \u001B[38;5;124;03m        Only currently applies to PostScript printing.\u001B[39;00m\n\u001B[0;32m   2270\u001B[0m \n\u001B[0;32m   2271\u001B[0m \u001B[38;5;124;03m    format : str, optional\u001B[39;00m\n\u001B[0;32m   2272\u001B[0m \u001B[38;5;124;03m        Force a specific file format. If not given, the format is inferred\u001B[39;00m\n\u001B[0;32m   2273\u001B[0m \u001B[38;5;124;03m        from the *filename* extension, and if that fails from\u001B[39;00m\n\u001B[0;32m   2274\u001B[0m \u001B[38;5;124;03m        :rc:`savefig.format`.\u001B[39;00m\n\u001B[0;32m   2275\u001B[0m \n\u001B[0;32m   2276\u001B[0m \u001B[38;5;124;03m    bbox_inches : 'tight' or `.Bbox`, default: :rc:`savefig.bbox`\u001B[39;00m\n\u001B[0;32m   2277\u001B[0m \u001B[38;5;124;03m        Bounding box in inches: only the given portion of the figure is\u001B[39;00m\n\u001B[0;32m   2278\u001B[0m \u001B[38;5;124;03m        saved.  If 'tight', try to figure out the tight bbox of the figure.\u001B[39;00m\n\u001B[0;32m   2279\u001B[0m \n\u001B[0;32m   2280\u001B[0m \u001B[38;5;124;03m    pad_inches : float, default: :rc:`savefig.pad_inches`\u001B[39;00m\n\u001B[0;32m   2281\u001B[0m \u001B[38;5;124;03m        Amount of padding around the figure when *bbox_inches* is 'tight'.\u001B[39;00m\n\u001B[0;32m   2282\u001B[0m \n\u001B[0;32m   2283\u001B[0m \u001B[38;5;124;03m    bbox_extra_artists : list of `~matplotlib.artist.Artist`, optional\u001B[39;00m\n\u001B[0;32m   2284\u001B[0m \u001B[38;5;124;03m        A list of extra artists that will be considered when the\u001B[39;00m\n\u001B[0;32m   2285\u001B[0m \u001B[38;5;124;03m        tight bbox is calculated.\u001B[39;00m\n\u001B[0;32m   2286\u001B[0m \n\u001B[0;32m   2287\u001B[0m \u001B[38;5;124;03m    backend : str, optional\u001B[39;00m\n\u001B[0;32m   2288\u001B[0m \u001B[38;5;124;03m        Use a non-default backend to render the file, e.g. to render a\u001B[39;00m\n\u001B[1;32m-> 2289\u001B[0m \u001B[38;5;124;03m        png file with the \"cairo\" backend rather than the default \"agg\",\u001B[39;00m\n\u001B[0;32m   2290\u001B[0m \u001B[38;5;124;03m        or a pdf file with the \"pgf\" backend rather than the default\u001B[39;00m\n\u001B[0;32m   2291\u001B[0m \u001B[38;5;124;03m        \"pdf\".  Note that the default backend is normally sufficient.  See\u001B[39;00m\n\u001B[0;32m   2292\u001B[0m \u001B[38;5;124;03m        :ref:`the-builtin-backends` for a list of valid backends for each\u001B[39;00m\n\u001B[0;32m   2293\u001B[0m \u001B[38;5;124;03m        file format.  Custom backends can be referenced as \"module://...\".\u001B[39;00m\n\u001B[0;32m   2294\u001B[0m \u001B[38;5;124;03m    \"\"\"\u001B[39;00m\n\u001B[0;32m   2295\u001B[0m     \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mformat\u001B[39m \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[0;32m   2296\u001B[0m         \u001B[38;5;66;03m# get format from filename, or from backend's default filetype\u001B[39;00m\n\u001B[0;32m   2297\u001B[0m         \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(filename, os\u001B[38;5;241m.\u001B[39mPathLike):\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:1544\u001B[0m, in \u001B[0;36m_get_renderer\u001B[1;34m(figure, print_method)\u001B[0m\n\u001B[0;32m   1542\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_press_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1543\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m event\u001B[38;5;241m.\u001B[39mkey\n\u001B[1;32m-> 1544\u001B[0m \u001B[38;5;28;01melif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_release_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1545\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
      "\u001B[1;31mTypeError\u001B[0m: print_png() got an unexpected keyword argument 'orientation'"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 잔디깎이 기계 데이터를 로드 및 분할하고 산포도를 그리기 위한 코드\n",
    "\n",
    "# ScatterPlot\n",
    "fig, ax = plt.subplot()\n",
    "\n",
    "subset = TrainData.loc[trainData['Ownership']=='Owner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'O', label = 'Owner', color = 'C1')\n",
    "\n",
    "subset = TrainData.loc[trainData['Ownership']=='Nonwner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'D', label = 'Nonwner', color = 'C0')\n",
    "\n",
    "ax.scatter(newHousehold.Income, newHousehold.Lot_Size, marker = '*', label = 'New Household', color = 'black', s = 150)\n",
    "\n",
    "plt.xlabel('Income')\n",
    "plt.ylabel('Lot_Size')\n",
    "for _, fow in trainData.iterrows():\n",
    "    ax.annotate(row.Number, (row.Income + 2, row.Lot_Size))\n",
    "\n",
    "handles, labels = ax.get_legend_handles_labels()\n",
    "ax.set_xlim(40, 115)\n",
    "ax.legend(handles, labels, loc = 4)\n",
    "\n",
    "plt.show()\n",
    "! pip install --upgrade matplotlib\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "print_png() got an unexpected keyword argument 'orientation'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\formatters.py:339\u001B[0m, in \u001B[0;36mBaseFormatter.__call__\u001B[1;34m(self, obj)\u001B[0m\n\u001B[0;32m    337\u001B[0m     \u001B[38;5;28;01mpass\u001B[39;00m\n\u001B[0;32m    338\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[1;32m--> 339\u001B[0m     \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mprinter\u001B[49m\u001B[43m(\u001B[49m\u001B[43mobj\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    340\u001B[0m \u001B[38;5;66;03m# Finally look for special method names\u001B[39;00m\n\u001B[0;32m    341\u001B[0m method \u001B[38;5;241m=\u001B[39m get_real_method(obj, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mprint_method)\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\pylabtools.py:151\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(fig, fmt, bbox_inches, base64, **kwargs)\u001B[0m\n\u001B[0;32m    148\u001B[0m     \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01mmatplotlib\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mbackend_bases\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m FigureCanvasBase\n\u001B[0;32m    149\u001B[0m     FigureCanvasBase(fig)\n\u001B[1;32m--> 151\u001B[0m \u001B[43mfig\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcanvas\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mprint_figure\u001B[49m\u001B[43m(\u001B[49m\u001B[43mbytes_io\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkw\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    152\u001B[0m data \u001B[38;5;241m=\u001B[39m bytes_io\u001B[38;5;241m.\u001B[39mgetvalue()\n\u001B[0;32m    153\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m fmt \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124msvg\u001B[39m\u001B[38;5;124m'\u001B[39m:\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:2289\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)\u001B[0m\n\u001B[0;32m   2241\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mprint_figure\u001B[39m(\n\u001B[0;32m   2242\u001B[0m         \u001B[38;5;28mself\u001B[39m, filename, dpi\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, facecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, edgecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2243\u001B[0m         orientation\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mportrait\u001B[39m\u001B[38;5;124m'\u001B[39m, \u001B[38;5;28mformat\u001B[39m\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m,\n\u001B[0;32m   2244\u001B[0m         bbox_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, pad_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, bbox_extra_artists\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2245\u001B[0m         backend\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs):\n\u001B[0;32m   2246\u001B[0m     \u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m   2247\u001B[0m \u001B[38;5;124;03m    Render the figure to hardcopy. Set the figure patch face and edge\u001B[39;00m\n\u001B[0;32m   2248\u001B[0m \u001B[38;5;124;03m    colors.  This is useful because some of the GUIs have a gray figure\u001B[39;00m\n\u001B[0;32m   2249\u001B[0m \u001B[38;5;124;03m    face color background and you'll probably want to override this on\u001B[39;00m\n\u001B[0;32m   2250\u001B[0m \u001B[38;5;124;03m    hardcopy.\u001B[39;00m\n\u001B[0;32m   2251\u001B[0m \n\u001B[0;32m   2252\u001B[0m \u001B[38;5;124;03m    Parameters\u001B[39;00m\n\u001B[0;32m   2253\u001B[0m \u001B[38;5;124;03m    ----------\u001B[39;00m\n\u001B[0;32m   2254\u001B[0m \u001B[38;5;124;03m    filename : str or path-like or file-like\u001B[39;00m\n\u001B[0;32m   2255\u001B[0m \u001B[38;5;124;03m        The file where the figure is saved.\u001B[39;00m\n\u001B[0;32m   2256\u001B[0m \n\u001B[0;32m   2257\u001B[0m \u001B[38;5;124;03m    dpi : float, default: :rc:`savefig.dpi`\u001B[39;00m\n\u001B[0;32m   2258\u001B[0m \u001B[38;5;124;03m        The dots per inch to save the figure in.\u001B[39;00m\n\u001B[0;32m   2259\u001B[0m \n\u001B[0;32m   2260\u001B[0m \u001B[38;5;124;03m    facecolor : color or 'auto', default: :rc:`savefig.facecolor`\u001B[39;00m\n\u001B[0;32m   2261\u001B[0m \u001B[38;5;124;03m        The facecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2262\u001B[0m \u001B[38;5;124;03m        facecolor.\u001B[39;00m\n\u001B[0;32m   2263\u001B[0m \n\u001B[0;32m   2264\u001B[0m \u001B[38;5;124;03m    edgecolor : color or 'auto', default: :rc:`savefig.edgecolor`\u001B[39;00m\n\u001B[0;32m   2265\u001B[0m \u001B[38;5;124;03m        The edgecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2266\u001B[0m \u001B[38;5;124;03m        edgecolor.\u001B[39;00m\n\u001B[0;32m   2267\u001B[0m \n\u001B[0;32m   2268\u001B[0m \u001B[38;5;124;03m    orientation : {'landscape', 'portrait'}, default: 'portrait'\u001B[39;00m\n\u001B[0;32m   2269\u001B[0m \u001B[38;5;124;03m        Only currently applies to PostScript printing.\u001B[39;00m\n\u001B[0;32m   2270\u001B[0m \n\u001B[0;32m   2271\u001B[0m \u001B[38;5;124;03m    format : str, optional\u001B[39;00m\n\u001B[0;32m   2272\u001B[0m \u001B[38;5;124;03m        Force a specific file format. If not given, the format is inferred\u001B[39;00m\n\u001B[0;32m   2273\u001B[0m \u001B[38;5;124;03m        from the *filename* extension, and if that fails from\u001B[39;00m\n\u001B[0;32m   2274\u001B[0m \u001B[38;5;124;03m        :rc:`savefig.format`.\u001B[39;00m\n\u001B[0;32m   2275\u001B[0m \n\u001B[0;32m   2276\u001B[0m \u001B[38;5;124;03m    bbox_inches : 'tight' or `.Bbox`, default: :rc:`savefig.bbox`\u001B[39;00m\n\u001B[0;32m   2277\u001B[0m \u001B[38;5;124;03m        Bounding box in inches: only the given portion of the figure is\u001B[39;00m\n\u001B[0;32m   2278\u001B[0m \u001B[38;5;124;03m        saved.  If 'tight', try to figure out the tight bbox of the figure.\u001B[39;00m\n\u001B[0;32m   2279\u001B[0m \n\u001B[0;32m   2280\u001B[0m \u001B[38;5;124;03m    pad_inches : float, default: :rc:`savefig.pad_inches`\u001B[39;00m\n\u001B[0;32m   2281\u001B[0m \u001B[38;5;124;03m        Amount of padding around the figure when *bbox_inches* is 'tight'.\u001B[39;00m\n\u001B[0;32m   2282\u001B[0m \n\u001B[0;32m   2283\u001B[0m \u001B[38;5;124;03m    bbox_extra_artists : list of `~matplotlib.artist.Artist`, optional\u001B[39;00m\n\u001B[0;32m   2284\u001B[0m \u001B[38;5;124;03m        A list of extra artists that will be considered when the\u001B[39;00m\n\u001B[0;32m   2285\u001B[0m \u001B[38;5;124;03m        tight bbox is calculated.\u001B[39;00m\n\u001B[0;32m   2286\u001B[0m \n\u001B[0;32m   2287\u001B[0m \u001B[38;5;124;03m    backend : str, optional\u001B[39;00m\n\u001B[0;32m   2288\u001B[0m \u001B[38;5;124;03m        Use a non-default backend to render the file, e.g. to render a\u001B[39;00m\n\u001B[1;32m-> 2289\u001B[0m \u001B[38;5;124;03m        png file with the \"cairo\" backend rather than the default \"agg\",\u001B[39;00m\n\u001B[0;32m   2290\u001B[0m \u001B[38;5;124;03m        or a pdf file with the \"pgf\" backend rather than the default\u001B[39;00m\n\u001B[0;32m   2291\u001B[0m \u001B[38;5;124;03m        \"pdf\".  Note that the default backend is normally sufficient.  See\u001B[39;00m\n\u001B[0;32m   2292\u001B[0m \u001B[38;5;124;03m        :ref:`the-builtin-backends` for a list of valid backends for each\u001B[39;00m\n\u001B[0;32m   2293\u001B[0m \u001B[38;5;124;03m        file format.  Custom backends can be referenced as \"module://...\".\u001B[39;00m\n\u001B[0;32m   2294\u001B[0m \u001B[38;5;124;03m    \"\"\"\u001B[39;00m\n\u001B[0;32m   2295\u001B[0m     \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mformat\u001B[39m \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[0;32m   2296\u001B[0m         \u001B[38;5;66;03m# get format from filename, or from backend's default filetype\u001B[39;00m\n\u001B[0;32m   2297\u001B[0m         \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(filename, os\u001B[38;5;241m.\u001B[39mPathLike):\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:1544\u001B[0m, in \u001B[0;36m_get_renderer\u001B[1;34m(figure, print_method)\u001B[0m\n\u001B[0;32m   1542\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_press_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1543\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m event\u001B[38;5;241m.\u001B[39mkey\n\u001B[1;32m-> 1544\u001B[0m \u001B[38;5;28;01melif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_release_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1545\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
      "\u001B[1;31mTypeError\u001B[0m: print_png() got an unexpected keyword argument 'orientation'"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Scatter Plot\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "subset = mower_df.loc[mower_df['Ownership']=='Owner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'o', label = 'Owner', color = 'C1')\n",
    "\n",
    "subset = mower_df.loc[mower_df['Ownership']=='Nonwner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'D', label = 'Nonwner', color = 'C0')\n",
    "\n",
    "plt.xlabel('Income')\n",
    "plt.ylabel('Lot_Size')\n",
    "for _,row in mower_df.iterrows():\n",
    "    ax.annotate(row.Number, (row.Income + 2, row.Lot_Size))\n",
    "\n",
    "handles, labels = ax.get_legend_handles_labels()\n",
    "ax.set_xlim(20, 120)\n",
    "ax.legend(handles, labels, loc = 4)\n",
    "\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "print_png() got an unexpected keyword argument 'orientation'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\formatters.py:339\u001B[0m, in \u001B[0;36mBaseFormatter.__call__\u001B[1;34m(self, obj)\u001B[0m\n\u001B[0;32m    337\u001B[0m     \u001B[38;5;28;01mpass\u001B[39;00m\n\u001B[0;32m    338\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[1;32m--> 339\u001B[0m     \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mprinter\u001B[49m\u001B[43m(\u001B[49m\u001B[43mobj\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    340\u001B[0m \u001B[38;5;66;03m# Finally look for special method names\u001B[39;00m\n\u001B[0;32m    341\u001B[0m method \u001B[38;5;241m=\u001B[39m get_real_method(obj, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mprint_method)\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\pylabtools.py:151\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(fig, fmt, bbox_inches, base64, **kwargs)\u001B[0m\n\u001B[0;32m    148\u001B[0m     \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01mmatplotlib\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mbackend_bases\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m FigureCanvasBase\n\u001B[0;32m    149\u001B[0m     FigureCanvasBase(fig)\n\u001B[1;32m--> 151\u001B[0m \u001B[43mfig\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcanvas\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mprint_figure\u001B[49m\u001B[43m(\u001B[49m\u001B[43mbytes_io\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkw\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    152\u001B[0m data \u001B[38;5;241m=\u001B[39m bytes_io\u001B[38;5;241m.\u001B[39mgetvalue()\n\u001B[0;32m    153\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m fmt \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124msvg\u001B[39m\u001B[38;5;124m'\u001B[39m:\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:2289\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)\u001B[0m\n\u001B[0;32m   2241\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mprint_figure\u001B[39m(\n\u001B[0;32m   2242\u001B[0m         \u001B[38;5;28mself\u001B[39m, filename, dpi\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, facecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, edgecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2243\u001B[0m         orientation\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mportrait\u001B[39m\u001B[38;5;124m'\u001B[39m, \u001B[38;5;28mformat\u001B[39m\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m,\n\u001B[0;32m   2244\u001B[0m         bbox_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, pad_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, bbox_extra_artists\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2245\u001B[0m         backend\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs):\n\u001B[0;32m   2246\u001B[0m     \u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m   2247\u001B[0m \u001B[38;5;124;03m    Render the figure to hardcopy. Set the figure patch face and edge\u001B[39;00m\n\u001B[0;32m   2248\u001B[0m \u001B[38;5;124;03m    colors.  This is useful because some of the GUIs have a gray figure\u001B[39;00m\n\u001B[0;32m   2249\u001B[0m \u001B[38;5;124;03m    face color background and you'll probably want to override this on\u001B[39;00m\n\u001B[0;32m   2250\u001B[0m \u001B[38;5;124;03m    hardcopy.\u001B[39;00m\n\u001B[0;32m   2251\u001B[0m \n\u001B[0;32m   2252\u001B[0m \u001B[38;5;124;03m    Parameters\u001B[39;00m\n\u001B[0;32m   2253\u001B[0m \u001B[38;5;124;03m    ----------\u001B[39;00m\n\u001B[0;32m   2254\u001B[0m \u001B[38;5;124;03m    filename : str or path-like or file-like\u001B[39;00m\n\u001B[0;32m   2255\u001B[0m \u001B[38;5;124;03m        The file where the figure is saved.\u001B[39;00m\n\u001B[0;32m   2256\u001B[0m \n\u001B[0;32m   2257\u001B[0m \u001B[38;5;124;03m    dpi : float, default: :rc:`savefig.dpi`\u001B[39;00m\n\u001B[0;32m   2258\u001B[0m \u001B[38;5;124;03m        The dots per inch to save the figure in.\u001B[39;00m\n\u001B[0;32m   2259\u001B[0m \n\u001B[0;32m   2260\u001B[0m \u001B[38;5;124;03m    facecolor : color or 'auto', default: :rc:`savefig.facecolor`\u001B[39;00m\n\u001B[0;32m   2261\u001B[0m \u001B[38;5;124;03m        The facecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2262\u001B[0m \u001B[38;5;124;03m        facecolor.\u001B[39;00m\n\u001B[0;32m   2263\u001B[0m \n\u001B[0;32m   2264\u001B[0m \u001B[38;5;124;03m    edgecolor : color or 'auto', default: :rc:`savefig.edgecolor`\u001B[39;00m\n\u001B[0;32m   2265\u001B[0m \u001B[38;5;124;03m        The edgecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2266\u001B[0m \u001B[38;5;124;03m        edgecolor.\u001B[39;00m\n\u001B[0;32m   2267\u001B[0m \n\u001B[0;32m   2268\u001B[0m \u001B[38;5;124;03m    orientation : {'landscape', 'portrait'}, default: 'portrait'\u001B[39;00m\n\u001B[0;32m   2269\u001B[0m \u001B[38;5;124;03m        Only currently applies to PostScript printing.\u001B[39;00m\n\u001B[0;32m   2270\u001B[0m \n\u001B[0;32m   2271\u001B[0m \u001B[38;5;124;03m    format : str, optional\u001B[39;00m\n\u001B[0;32m   2272\u001B[0m \u001B[38;5;124;03m        Force a specific file format. If not given, the format is inferred\u001B[39;00m\n\u001B[0;32m   2273\u001B[0m \u001B[38;5;124;03m        from the *filename* extension, and if that fails from\u001B[39;00m\n\u001B[0;32m   2274\u001B[0m \u001B[38;5;124;03m        :rc:`savefig.format`.\u001B[39;00m\n\u001B[0;32m   2275\u001B[0m \n\u001B[0;32m   2276\u001B[0m \u001B[38;5;124;03m    bbox_inches : 'tight' or `.Bbox`, default: :rc:`savefig.bbox`\u001B[39;00m\n\u001B[0;32m   2277\u001B[0m \u001B[38;5;124;03m        Bounding box in inches: only the given portion of the figure is\u001B[39;00m\n\u001B[0;32m   2278\u001B[0m \u001B[38;5;124;03m        saved.  If 'tight', try to figure out the tight bbox of the figure.\u001B[39;00m\n\u001B[0;32m   2279\u001B[0m \n\u001B[0;32m   2280\u001B[0m \u001B[38;5;124;03m    pad_inches : float, default: :rc:`savefig.pad_inches`\u001B[39;00m\n\u001B[0;32m   2281\u001B[0m \u001B[38;5;124;03m        Amount of padding around the figure when *bbox_inches* is 'tight'.\u001B[39;00m\n\u001B[0;32m   2282\u001B[0m \n\u001B[0;32m   2283\u001B[0m \u001B[38;5;124;03m    bbox_extra_artists : list of `~matplotlib.artist.Artist`, optional\u001B[39;00m\n\u001B[0;32m   2284\u001B[0m \u001B[38;5;124;03m        A list of extra artists that will be considered when the\u001B[39;00m\n\u001B[0;32m   2285\u001B[0m \u001B[38;5;124;03m        tight bbox is calculated.\u001B[39;00m\n\u001B[0;32m   2286\u001B[0m \n\u001B[0;32m   2287\u001B[0m \u001B[38;5;124;03m    backend : str, optional\u001B[39;00m\n\u001B[0;32m   2288\u001B[0m \u001B[38;5;124;03m        Use a non-default backend to render the file, e.g. to render a\u001B[39;00m\n\u001B[1;32m-> 2289\u001B[0m \u001B[38;5;124;03m        png file with the \"cairo\" backend rather than the default \"agg\",\u001B[39;00m\n\u001B[0;32m   2290\u001B[0m \u001B[38;5;124;03m        or a pdf file with the \"pgf\" backend rather than the default\u001B[39;00m\n\u001B[0;32m   2291\u001B[0m \u001B[38;5;124;03m        \"pdf\".  Note that the default backend is normally sufficient.  See\u001B[39;00m\n\u001B[0;32m   2292\u001B[0m \u001B[38;5;124;03m        :ref:`the-builtin-backends` for a list of valid backends for each\u001B[39;00m\n\u001B[0;32m   2293\u001B[0m \u001B[38;5;124;03m        file format.  Custom backends can be referenced as \"module://...\".\u001B[39;00m\n\u001B[0;32m   2294\u001B[0m \u001B[38;5;124;03m    \"\"\"\u001B[39;00m\n\u001B[0;32m   2295\u001B[0m     \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mformat\u001B[39m \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[0;32m   2296\u001B[0m         \u001B[38;5;66;03m# get format from filename, or from backend's default filetype\u001B[39;00m\n\u001B[0;32m   2297\u001B[0m         \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(filename, os\u001B[38;5;241m.\u001B[39mPathLike):\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:1544\u001B[0m, in \u001B[0;36m_get_renderer\u001B[1;34m(figure, print_method)\u001B[0m\n\u001B[0;32m   1542\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_press_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1543\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m event\u001B[38;5;241m.\u001B[39mkey\n\u001B[1;32m-> 1544\u001B[0m \u001B[38;5;28;01melif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_release_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1545\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
      "\u001B[1;31mTypeError\u001B[0m: print_png() got an unexpected keyword argument 'orientation'"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 소득 60인 값을 기준으로 24개 관측치 분할\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "subset = mower_df.loc[mower_df['Ownership']=='Owner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'o', label = 'Owner', color = 'C1')\n",
    "\n",
    "subset = mower_df.loc[mower_df['Ownership']=='Nonwner']\n",
    "ax.scatter(subset.Income, subset.Lot_Size, marker = 'D', label = 'Nonwner', color = 'C0')\n",
    "\n",
    "plt.xlabel('Income')\n",
    "plt.ylabel('Lot_Size')\n",
    "for _,row in mower_df.iterrows():\n",
    "    ax.annotate(row.Number, (row.Income + 2, row.Lot_Size))\n",
    "\n",
    "handles, labels = ax.get_legend_handles_labels()\n",
    "ax.set_xlim(20, 120)\n",
    "ax.legend(handles, labels, loc = 4)\n",
    "\n",
    "ax.axvline(x = 60, color = 'black', linestyle = '--', label = 'Income = 60')\n",
    "\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 잔디깎이 기계 데이터를 로드 및 분할하고 산포도를 그리기 위한 코드\n",
    "* 분할을 통해 생성된 2개의 직사각형은 분할 전 직사가형에 비해 동질적\n",
    "* 좌측 : 대부분 비소유 & 우측 : 대부분 소유\n",
    "* How?\n",
    "1. 알고리즘은 최적의 분할을 찾기 위해, 변수와 각 변수에 대한 모든 분할 가능 값을 검토\n",
    "2. 각 변수에 대한 분할 가능 값은 예측 변수가 갖는 연속된 두 값들 사이의 중간값\n",
    "3. 소득 & 대지 크기\n",
    "4. 분할 값은 분할의 결과로 생기는 직사각형에서 얼마나 많이 분순도(이질성)를 감소시켰는가를 통해 순위 겨정\n",
    "* 순수한 직사각형 ? -> 하나의 클래스에 속한 점들로만 구성\n",
    "* 불순도 감소 -> 분할 전의 전체 불순도에서 분할 수의 3개 직사각형에 대한 불순도의 합을 뺸 값"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 불순도 측정\n",
    "* 지니 지수( Gini index) & 엔트로피 측도(Entropy measure)\n",
    "* 지니 지수 : 두 측정치에 대한 반응 변수가 m 개의 클래스를 갖고 k = 1, 2, 3, ...로 표기\n",
    "* 측정치는 0~(m-1)/m (관측치들이 m개의 클래스에 동일하게 분포)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 잔디깎이 기계 데이터를 로드 및 분할하고 산포도를 그리기 위한 코드\n",
    "* 소득 60인 값을 기준으로 24개 관측치 분할\n",
    "* 대지의 크기 21.4인 값을 기준으로 분할\n",
    "* 위의 과정 반복"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes: Nonowner,Owner\n"
     ]
    },
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 분류 트리 실행, 시각화 코드\n",
    "classTree = DecisionTreeClassifier(random_state = 0, max_depth = 1)\n",
    "classTree.fit(mower_df.drop(columns = ['Ownership']), mower_df['Ownership'])\n",
    "\n",
    "print(\"Classes: {}\".format(','.join(classTree.classes_)))\n",
    "plotDecisionTree(classTree, feature_names = mower_df.columns[:2], class_names = classTree.classes_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes: Nonowner,Owner\n"
     ]
    },
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 분류 트리 실행, 시각화 코드\n",
    "classTree = DecisionTreeClassifier(random_state = 0, max_depth = 2)\n",
    "classTree.fit(mower_df.drop(columns = ['Ownership']), mower_df['Ownership'])\n",
    "\n",
    "print(\"Classes: {}\".format(','.join(classTree.classes_)))\n",
    "plotDecisionTree(classTree, feature_names = mower_df.columns[:2], class_names = classTree.classes_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes: Nonowner,Owner\n"
     ]
    },
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 분류 트리 실행, 시각화 코드\n",
    "classTree = DecisionTreeClassifier(random_state = 0)\n",
    "classTree.fit(mower_df.drop(columns = ['Ownership']), mower_df['Ownership'])\n",
    "\n",
    "print(\"Classes: {}\".format(','.join(classTree.classes_)))\n",
    "plotDecisionTree(classTree, feature_names = mower_df.columns[:2], class_names = classTree.classes_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 성능 평가\n",
    "* 모델을 평가하고 조정하려면 학습 데이터 셋 외에 표본 데이터가 필요함\n",
    "* 트리 구조는 선택된 표본에 따라서 불안정\n",
    "* 완전 적합된 트리 모델은 과적합(over-fitting)이슈 존재"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 예제 : 개인 대출 수락\n",
    "### 분류 트리를 활용한 개인 대출 수락 여부 예측\n",
    "* UniversalBank 고객 데이터 (5000명)를 학습 (60%)&검증(40%)로 나눠 트리 모형 설계\n",
    "* 검증은 어떻게? -> CV를 통해 서로 다른 데이터 분할에 대한 성능 변동 파악\n",
    "\n",
    "\n",
    "### 과적합 방지\n",
    "* 다양한 분할 레벨에서 모델의 총 오류를 살펴보면, 과적합이 일어나는 점에 이르기까지 분할 레벨의 수는 증가 -> 총 오류는 감소\n",
    "* 학습 데이터의 총 오류는 최대 분할 레벨에서 0이 될 떄까지 줄어듦\n",
    "* 분할 레벨에 따른 과적합의 이유 ? -> 적은 수의 관측치로 분할\n",
    "\n",
    "### 트리 모델 성장 중단\n",
    "* 과적합 되기 전 성장 중단\n",
    "* 범용적 기준 : 트리 모데의 깊이, 노드의 최소 관측치 수, 불순도의 최소 감소량 등\n",
    "\n",
    "### Grid Search\n",
    "* 과적합 되기 전 성장 중단\n",
    "* 어떻게 성장을 멈출 것인가? 가장 좋은 타이밍은? -> Grid Search\n",
    "* 파라미터 최적 조합 : 최대 깊이 [1,30], 단말 노드의 최소 레코드 수 [20,100], 불순도 기준[0.001, 0.01]\n",
    "* Grid Search는 각각의 데이터 셋(학습, 검증)에 대해 과적합 이슈가 발생할 가능성\n",
    "* 학습 데이터셋에서 CV 수행 -> Best model 선정 -> 해당 모델에 대해 새로운 검증 데이터 셋으로 평가"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 회귀 트리 모델\n",
    "### 트리 모델을 활용하여 연속형 반응 변수 예측\n",
    "* 분류 문제와 본질적으로 같음\n",
    "* 분할이 시도되고 결과로 나온 트리 모델들의 모든 가지에 대해 불순도 측정\n",
    "* -> 다음 단계에서 불순도의 합이 최소가 되는 분할 선택"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial parameters:  {'max_depth': 5, 'min_impurity_decrease': 0, 'min_samples_split': 20}\n",
      "Improved parameters:  {'max_depth': 6, 'min_impurity_decrease': 0, 'min_samples_split': 16}\n"
     ]
    }
   ],
   "source": [
    "toyota_df = pd.read_csv('https://raw.githubusercontent.com/reisanar/datasets/master/ToyotaCorolla.csv').iloc[:1000]\n",
    "toyota_df = toyota_df.rename(columns = {'age_08_04':'age', 'quarterly_tax':'tax'})\n",
    "\n",
    "predictors = ['age', 'km', 'fuel_type','hp','met_color', 'automatic','cc','doors','tax','weight']\n",
    "outcome = 'price'\n",
    "\n",
    "X = pd.get_dummies(toyota_df[predictors], drop_first=True)\n",
    "y = toyota_df[outcome]\n",
    "\n",
    "train_X, valid_X, train_y, valid_y = train_test_split(X, y, test_size = 0.4, random_state = 1)\n",
    "\n",
    "# user grid search to find optimized tree\n",
    "param_grid = {'max_depth' :[5,10,15,20,25], 'min_impurity_decrease' : [0,0.001,0.005,0.01], 'min_samples_split' : [10,20,30,40,50]}\n",
    "gridSearch = GridSearchCV(DecisionTreeRegressor(), param_grid, cv = 5, n_jobs = -1)\n",
    "gridSearch.fit(train_X, train_y)\n",
    "print('Initial parameters: ', gridSearch.best_params_)\n",
    "\n",
    "param_grid = {\n",
    "    'max_depth' :[3,4,5,6,7,8,9,10,11,12],\n",
    "    'min_impurity_decrease' : [0,0.001,0.002,0.003,0.005,0.006,0.007,0.008],\n",
    "    'min_samples_split' : [14, 15, 16, 18, 20]\n",
    "}\n",
    "gridSearch = GridSearchCV(DecisionTreeRegressor(), param_grid, cv = 5, n_jobs = -1)\n",
    "gridSearch.fit(train_X, train_y)\n",
    "print('Improved parameters: ', gridSearch.best_params_)\n",
    "\n",
    "regtree = gridSearch.best_estimator_"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Regression statistics\n",
      "\n",
      "                      Mean Error (ME) : 0.0000\n",
      "       Root Mean Squared Error (RMSE) : 1058.8202\n",
      "            Mean Absolute Error (MAE) : 767.7203\n",
      "          Mean Percentage Error (MPE) : -0.8074\n",
      "Mean Absolute Percentage Error (MAPE) : 6.8325\n",
      "\n",
      "Regression statistics\n",
      "\n",
      "                      Mean Error (ME) : 60.5241\n",
      "       Root Mean Squared Error (RMSE) : 1554.9146\n",
      "            Mean Absolute Error (MAE) : 1026.3487\n",
      "          Mean Percentage Error (MPE) : -1.3082\n",
      "Mean Absolute Percentage Error (MAPE) : 9.2311\n"
     ]
    }
   ],
   "source": [
    "regressionSummary(train_y, regtree.predict(train_X))\n",
    "regressionSummary(valid_y, regtree.predict(valid_X))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "outputs": [
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plotDecisionTree(regtree, feature_names = train_X.columns)\n",
    "plotDecisionTree(regtree, feature_names = train_X.columns, rotate = True)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 트리 모델 발전\n",
    "### 앙상블(Ensemble)\n",
    "* 여러 머신러닝 모델을 조합하여 강력한 모델 디자인\n",
    "* 의사 결정 트리 -> 랜덤 포레스트 (Random Forest) -> 그래디언트 부스팅 (Gradient Boosting)\n",
    "\n",
    "### 랜덤 포레스트\n",
    "* 훈련 데이터에 과적합 되는 의사 결정 트리\n",
    "* Bagging의 특별한 경우, 다중 분류/예측 알고리즘을 조합하여 예측 성능 향상\n",
    "\n",
    "### Procedure\n",
    "* 복원 추출 방식으로 데이터로부터 여러 랜덤 표본을 생성 (Bootstrap)\n",
    "* 각 단계마다 랜덤으로 예측 변수들을 선택하여 서브셋 생성 -> 표본에  대해 분류 트리 적합\n",
    "* 예측을 향상시키기 위해서 각 트리로부터 얻은 예측/분류 결과를 결합\n",
    "* 분류 -> 투표(voting) & 예측 -> 평균화 (Averaging)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "outputs": [
    {
     "data": {
      "text/plain": "   ID  Age  Experience  Income  ZIP Code  Family  CCAvg  Education  Mortgage  \\\n0   1   25           1      49     91107       4    1.6          1         0   \n1   2   45          19      34     90089       3    1.5          1         0   \n2   3   39          15      11     94720       1    1.0          1         0   \n3   4   35           9     100     94112       1    2.7          2         0   \n4   5   35           8      45     91330       4    1.0          2         0   \n\n   PersonalLoan  SecuritiesAccount  CDAccount  Online  CreditCard  \n0             0                  1          0       0           0  \n1             0                  1          0       0           0  \n2             0                  0          0       0           0  \n3             0                  0          0       0           0  \n4             0                  0          0       0           1  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>ID</th>\n      <th>Age</th>\n      <th>Experience</th>\n      <th>Income</th>\n      <th>ZIP Code</th>\n      <th>Family</th>\n      <th>CCAvg</th>\n      <th>Education</th>\n      <th>Mortgage</th>\n      <th>PersonalLoan</th>\n      <th>SecuritiesAccount</th>\n      <th>CDAccount</th>\n      <th>Online</th>\n      <th>CreditCard</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>25</td>\n      <td>1</td>\n      <td>49</td>\n      <td>91107</td>\n      <td>4</td>\n      <td>1.6</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>45</td>\n      <td>19</td>\n      <td>34</td>\n      <td>90089</td>\n      <td>3</td>\n      <td>1.5</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>39</td>\n      <td>15</td>\n      <td>11</td>\n      <td>94720</td>\n      <td>1</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>35</td>\n      <td>9</td>\n      <td>100</td>\n      <td>94112</td>\n      <td>1</td>\n      <td>2.7</td>\n      <td>2</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5</td>\n      <td>35</td>\n      <td>8</td>\n      <td>45</td>\n      <td>91330</td>\n      <td>4</td>\n      <td>1.0</td>\n      <td>2</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## p.27\n",
    "bank_df = pd.read_csv('https://raw.githubusercontent.com/gchoi/Dataset/master/UniversalBank.csv')\n",
    "bank_df.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "outputs": [
    {
     "data": {
      "text/plain": "   Age  Experience  Income  Family  CCAvg  Education  Mortgage  PersonalLoan  \\\n0   25           1      49       4    1.6          1         0             0   \n1   45          19      34       3    1.5          1         0             0   \n2   39          15      11       1    1.0          1         0             0   \n3   35           9     100       1    2.7          2         0             0   \n4   35           8      45       4    1.0          2         0             0   \n\n   SecuritiesAccount  CDAccount  Online  CreditCard  \n0                  1          0       0           0  \n1                  1          0       0           0  \n2                  0          0       0           0  \n3                  0          0       0           0  \n4                  0          0       0           1  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Age</th>\n      <th>Experience</th>\n      <th>Income</th>\n      <th>Family</th>\n      <th>CCAvg</th>\n      <th>Education</th>\n      <th>Mortgage</th>\n      <th>PersonalLoan</th>\n      <th>SecuritiesAccount</th>\n      <th>CDAccount</th>\n      <th>Online</th>\n      <th>CreditCard</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>25</td>\n      <td>1</td>\n      <td>49</td>\n      <td>4</td>\n      <td>1.6</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>45</td>\n      <td>19</td>\n      <td>34</td>\n      <td>3</td>\n      <td>1.5</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>39</td>\n      <td>15</td>\n      <td>11</td>\n      <td>1</td>\n      <td>1.0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>35</td>\n      <td>9</td>\n      <td>100</td>\n      <td>1</td>\n      <td>2.7</td>\n      <td>2</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>8</td>\n      <td>45</td>\n      <td>4</td>\n      <td>1.0</td>\n      <td>2</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bank_df = bank_df.drop(columns = ['ID', 'ZIP Code'])\n",
    "bank_df.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "outputs": [
    {
     "data": {
      "text/plain": "480"
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(bank_df[bank_df['PersonalLoan']==1])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "outputs": [
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = bank_df.drop(columns = ['PersonalLoan'])\n",
    "y = bank_df['PersonalLoan']\n",
    "train_X, valid_X, train_y, valid_y = train_test_split(X, y, test_size = 0.4, random_state = 1)\n",
    "\n",
    "fullClassTree = DecisionTreeClassifier()\n",
    "fullClassTree.fit(train_X, train_y)\n",
    "\n",
    "plotDecisionTree(fullClassTree, feature_names = train_X.columns)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix (Accuracy 1.0000)\n",
      "\n",
      "       Prediction\n",
      "Actual    0    1\n",
      "     0 2713    0\n",
      "     1    0  287\n",
      "Confusion Matrix (Accuracy 0.9790)\n",
      "\n",
      "       Prediction\n",
      "Actual    0    1\n",
      "     0 1790   17\n",
      "     1   25  168\n"
     ]
    }
   ],
   "source": [
    "classificationSummary(train_y, fullClassTree.predict(train_X))\n",
    "classificationSummary(valid_y, fullClassTree.predict(valid_X))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy scores of each fold:  ['0.983', '0.977', '0.992', '0.985', '0.993']\n",
      "Accuracy: 0.986 (+/- 0.012)\n",
      "Accuracy: 0.986 (+/- 0.006)\n"
     ]
    }
   ],
   "source": [
    "# full- 결정 트리의 5 - fold 교차 검증\n",
    "treeClassifier = DecisionTreeClassifier()\n",
    "\n",
    "scores = cross_val_score(treeClassifier, train_X, train_y, cv = 5)\n",
    "print('Accuracy scores of each fold: ', [f'{acc:.3f}' for acc in scores])\n",
    "print(f'Accuracy: {scores.mean():.3f} (+/- {scores.std() * 2:.3f})')\n",
    "print(f'Accuracy: {scores.mean():.3f} (+/- {scores.std():.3f})')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "print_png() got an unexpected keyword argument 'orientation'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\formatters.py:339\u001B[0m, in \u001B[0;36mBaseFormatter.__call__\u001B[1;34m(self, obj)\u001B[0m\n\u001B[0;32m    337\u001B[0m     \u001B[38;5;28;01mpass\u001B[39;00m\n\u001B[0;32m    338\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[1;32m--> 339\u001B[0m     \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mprinter\u001B[49m\u001B[43m(\u001B[49m\u001B[43mobj\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    340\u001B[0m \u001B[38;5;66;03m# Finally look for special method names\u001B[39;00m\n\u001B[0;32m    341\u001B[0m method \u001B[38;5;241m=\u001B[39m get_real_method(obj, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mprint_method)\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\pylabtools.py:151\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(fig, fmt, bbox_inches, base64, **kwargs)\u001B[0m\n\u001B[0;32m    148\u001B[0m     \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01mmatplotlib\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mbackend_bases\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m FigureCanvasBase\n\u001B[0;32m    149\u001B[0m     FigureCanvasBase(fig)\n\u001B[1;32m--> 151\u001B[0m \u001B[43mfig\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcanvas\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mprint_figure\u001B[49m\u001B[43m(\u001B[49m\u001B[43mbytes_io\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkw\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    152\u001B[0m data \u001B[38;5;241m=\u001B[39m bytes_io\u001B[38;5;241m.\u001B[39mgetvalue()\n\u001B[0;32m    153\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m fmt \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124msvg\u001B[39m\u001B[38;5;124m'\u001B[39m:\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:2289\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)\u001B[0m\n\u001B[0;32m   2241\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mprint_figure\u001B[39m(\n\u001B[0;32m   2242\u001B[0m         \u001B[38;5;28mself\u001B[39m, filename, dpi\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, facecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, edgecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2243\u001B[0m         orientation\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mportrait\u001B[39m\u001B[38;5;124m'\u001B[39m, \u001B[38;5;28mformat\u001B[39m\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m,\n\u001B[0;32m   2244\u001B[0m         bbox_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, pad_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, bbox_extra_artists\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2245\u001B[0m         backend\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs):\n\u001B[0;32m   2246\u001B[0m     \u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m   2247\u001B[0m \u001B[38;5;124;03m    Render the figure to hardcopy. Set the figure patch face and edge\u001B[39;00m\n\u001B[0;32m   2248\u001B[0m \u001B[38;5;124;03m    colors.  This is useful because some of the GUIs have a gray figure\u001B[39;00m\n\u001B[0;32m   2249\u001B[0m \u001B[38;5;124;03m    face color background and you'll probably want to override this on\u001B[39;00m\n\u001B[0;32m   2250\u001B[0m \u001B[38;5;124;03m    hardcopy.\u001B[39;00m\n\u001B[0;32m   2251\u001B[0m \n\u001B[0;32m   2252\u001B[0m \u001B[38;5;124;03m    Parameters\u001B[39;00m\n\u001B[0;32m   2253\u001B[0m \u001B[38;5;124;03m    ----------\u001B[39;00m\n\u001B[0;32m   2254\u001B[0m \u001B[38;5;124;03m    filename : str or path-like or file-like\u001B[39;00m\n\u001B[0;32m   2255\u001B[0m \u001B[38;5;124;03m        The file where the figure is saved.\u001B[39;00m\n\u001B[0;32m   2256\u001B[0m \n\u001B[0;32m   2257\u001B[0m \u001B[38;5;124;03m    dpi : float, default: :rc:`savefig.dpi`\u001B[39;00m\n\u001B[0;32m   2258\u001B[0m \u001B[38;5;124;03m        The dots per inch to save the figure in.\u001B[39;00m\n\u001B[0;32m   2259\u001B[0m \n\u001B[0;32m   2260\u001B[0m \u001B[38;5;124;03m    facecolor : color or 'auto', default: :rc:`savefig.facecolor`\u001B[39;00m\n\u001B[0;32m   2261\u001B[0m \u001B[38;5;124;03m        The facecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2262\u001B[0m \u001B[38;5;124;03m        facecolor.\u001B[39;00m\n\u001B[0;32m   2263\u001B[0m \n\u001B[0;32m   2264\u001B[0m \u001B[38;5;124;03m    edgecolor : color or 'auto', default: :rc:`savefig.edgecolor`\u001B[39;00m\n\u001B[0;32m   2265\u001B[0m \u001B[38;5;124;03m        The edgecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2266\u001B[0m \u001B[38;5;124;03m        edgecolor.\u001B[39;00m\n\u001B[0;32m   2267\u001B[0m \n\u001B[0;32m   2268\u001B[0m \u001B[38;5;124;03m    orientation : {'landscape', 'portrait'}, default: 'portrait'\u001B[39;00m\n\u001B[0;32m   2269\u001B[0m \u001B[38;5;124;03m        Only currently applies to PostScript printing.\u001B[39;00m\n\u001B[0;32m   2270\u001B[0m \n\u001B[0;32m   2271\u001B[0m \u001B[38;5;124;03m    format : str, optional\u001B[39;00m\n\u001B[0;32m   2272\u001B[0m \u001B[38;5;124;03m        Force a specific file format. If not given, the format is inferred\u001B[39;00m\n\u001B[0;32m   2273\u001B[0m \u001B[38;5;124;03m        from the *filename* extension, and if that fails from\u001B[39;00m\n\u001B[0;32m   2274\u001B[0m \u001B[38;5;124;03m        :rc:`savefig.format`.\u001B[39;00m\n\u001B[0;32m   2275\u001B[0m \n\u001B[0;32m   2276\u001B[0m \u001B[38;5;124;03m    bbox_inches : 'tight' or `.Bbox`, default: :rc:`savefig.bbox`\u001B[39;00m\n\u001B[0;32m   2277\u001B[0m \u001B[38;5;124;03m        Bounding box in inches: only the given portion of the figure is\u001B[39;00m\n\u001B[0;32m   2278\u001B[0m \u001B[38;5;124;03m        saved.  If 'tight', try to figure out the tight bbox of the figure.\u001B[39;00m\n\u001B[0;32m   2279\u001B[0m \n\u001B[0;32m   2280\u001B[0m \u001B[38;5;124;03m    pad_inches : float, default: :rc:`savefig.pad_inches`\u001B[39;00m\n\u001B[0;32m   2281\u001B[0m \u001B[38;5;124;03m        Amount of padding around the figure when *bbox_inches* is 'tight'.\u001B[39;00m\n\u001B[0;32m   2282\u001B[0m \n\u001B[0;32m   2283\u001B[0m \u001B[38;5;124;03m    bbox_extra_artists : list of `~matplotlib.artist.Artist`, optional\u001B[39;00m\n\u001B[0;32m   2284\u001B[0m \u001B[38;5;124;03m        A list of extra artists that will be considered when the\u001B[39;00m\n\u001B[0;32m   2285\u001B[0m \u001B[38;5;124;03m        tight bbox is calculated.\u001B[39;00m\n\u001B[0;32m   2286\u001B[0m \n\u001B[0;32m   2287\u001B[0m \u001B[38;5;124;03m    backend : str, optional\u001B[39;00m\n\u001B[0;32m   2288\u001B[0m \u001B[38;5;124;03m        Use a non-default backend to render the file, e.g. to render a\u001B[39;00m\n\u001B[1;32m-> 2289\u001B[0m \u001B[38;5;124;03m        png file with the \"cairo\" backend rather than the default \"agg\",\u001B[39;00m\n\u001B[0;32m   2290\u001B[0m \u001B[38;5;124;03m        or a pdf file with the \"pgf\" backend rather than the default\u001B[39;00m\n\u001B[0;32m   2291\u001B[0m \u001B[38;5;124;03m        \"pdf\".  Note that the default backend is normally sufficient.  See\u001B[39;00m\n\u001B[0;32m   2292\u001B[0m \u001B[38;5;124;03m        :ref:`the-builtin-backends` for a list of valid backends for each\u001B[39;00m\n\u001B[0;32m   2293\u001B[0m \u001B[38;5;124;03m        file format.  Custom backends can be referenced as \"module://...\".\u001B[39;00m\n\u001B[0;32m   2294\u001B[0m \u001B[38;5;124;03m    \"\"\"\u001B[39;00m\n\u001B[0;32m   2295\u001B[0m     \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mformat\u001B[39m \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[0;32m   2296\u001B[0m         \u001B[38;5;66;03m# get format from filename, or from backend's default filetype\u001B[39;00m\n\u001B[0;32m   2297\u001B[0m         \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(filename, os\u001B[38;5;241m.\u001B[39mPathLike):\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:1544\u001B[0m, in \u001B[0;36m_get_renderer\u001B[1;34m(figure, print_method)\u001B[0m\n\u001B[0;32m   1542\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_press_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1543\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m event\u001B[38;5;241m.\u001B[39mkey\n\u001B[1;32m-> 1544\u001B[0m \u001B[38;5;28;01melif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_release_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1545\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
      "\u001B[1;31mTypeError\u001B[0m: print_png() got an unexpected keyword argument 'orientation'"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot accuracy asores of each fold\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(range(1,6), scores, 'o-')\n",
    "ax.set_xlabel('Fold')\n",
    "ax.set_ylabel('Accuracy')\n",
    "ax.set_title('Accuracy of DT according to 5-fold cross-validation')\n",
    "ax.set_xticks(range(1,6))\n",
    "ax.set_ylim(0.9,1)\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: matplotlib in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (3.7.1)\n",
      "Requirement already satisfied: importlib-resources>=3.2.0 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (5.9.0)\n",
      "Requirement already satisfied: numpy>=1.20 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (1.23.5)\n",
      "Requirement already satisfied: packaging>=20.0 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (23.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (1.4.4)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (3.0.9)\n",
      "Requirement already satisfied: pillow>=6.2.0 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (9.4.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (2.8.2)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (1.0.5)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (4.37.1)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from matplotlib) (0.11.0)\n",
      "Requirement already satisfied: zipp>=3.1.0 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from importlib-resources>=3.2.0->matplotlib) (3.11.0)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "! pip install --upgrade matplotlib\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 과적합 방지\n",
    "* 다양한 분할 레벨에서 모델의 총 분류를 살펴보면, 과적합이 일어나는 점에 이르기까지 분할 레벨의 수는 증가 -> 총 오류는 감소\n",
    "* 학습 데잍의 총 오류는 최대 분할 레벨에서 0이 될 때까지 줄어듧\n",
    "* 분할 레벨에 따른 과적합의 이유 ? -> 적은 수의 관측치로 분할"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "outputs": [
    {
     "data": {
      "text/plain": "'You need to install pydotplus to visualize decision trees'"
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "smallClassTree = DecisionTreeClassifier(max_depth = 30, min_samples_split = 20, min_impurity_decrease = 0.01)\n",
    "smallClassTree.fit(train_X, train_y)\n",
    "\n",
    "plotDecisionTree(smallClassTree, feature_names = train_X.columns)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "outputs": [
    {
     "data": {
      "text/plain": "RandomForestClassifier(n_estimators=500, random_state=1)",
      "text/html": "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestClassifier(n_estimators=500, random_state=1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(n_estimators=500, random_state=1)</pre></div></div></div></div></div>"
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# p.43\n",
    "ubank_df = pd.read_csv(\"https://raw.githubusercontent.com/gchoi/Dataset/master/UniversalBank.csv\")\n",
    "ubank_df = ubank_df.drop(columns = ['ID', 'ZIP Code'])\n",
    "\n",
    "X = ubank_df.drop(columns = ['PersonalLoan'])\n",
    "y = ubank_df['PersonalLoan']\n",
    "train_X, valid_X, train_y, valid_y = train_test_split(X, y, test_size = 0.4, random_state = 1)\n",
    "\n",
    "rf = RandomForestClassifier(n_estimators= 500,random_state = 1 )\n",
    "rf.fit(train_X, train_y)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              feature  importance       std\n",
      "7   SecuritiesAccount    0.003964  0.004998\n",
      "9              Online    0.006394  0.005350\n",
      "10         CreditCard    0.007678  0.007053\n",
      "6            Mortgage    0.034243  0.023469\n",
      "1          Experience    0.035539  0.016061\n",
      "0                 Age    0.036258  0.015858\n",
      "8           CDAccount    0.057917  0.043185\n",
      "3              Family    0.111375  0.053146\n",
      "4               CCAvg    0.172105  0.103011\n",
      "5           Education    0.200772  0.101002\n",
      "2              Income    0.333756  0.129227\n",
      "Unexpected exception formatting exception. Falling back to standard exception\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 3398, in run_code\n",
      "    nnode = ast.Expr(ast.Name(target.id, ast.Load()))\n",
      "  File \"C:\\Users\\User\\AppData\\Local\\Temp\\ipykernel_21956\\636905909.py\", line 11, in <cell line: 11>\n",
      "    plt.tight_layout()\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\pyplot.py\", line 2312, in tight_layout\n",
      "    # Autogenerated by boilerplate.py.  Do not edit as changes will be lost.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\figure.py\", line 3236, in tight_layout\n",
      "    fname : str or path-like or binary file-like\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\tight_layout.py\", line 320, in get_tight_layout_figure\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\tight_layout.py\", line 82, in _auto_adjust_subplotpars\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axes\\_base.py\", line 4628, in get_tightbbox\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1105, in get_tightbbox\n",
      "    raise NotImplementedError('Derived must override')\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 2083, in _update_label_position\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1881, in _get_tick_boxes_siblings\n",
      "    \"\"\"\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1085, in _get_tick_bboxes\n",
      "    child.set_clip_path(clippath, transform)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1085, in <listcomp>\n",
      "    child.set_clip_path(clippath, transform)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\text.py\", line 910, in get_window_extent\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\text.py\", line 309, in _get_layout\n",
      "    bb = self.get_window_extent()\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 233, in get_text_width_height_descent\n",
      "    font = self._prepare_font(prop)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 267, in _prepare_font\n",
      "    font = get_font(_fontManager._find_fonts_by_props(font_prop))\n",
      "AttributeError: 'FontManager' object has no attribute '_find_fonts_by_props'\n",
      "\n",
      "During handling of the above exception, another exception occurred:\n",
      "\n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 1993, in showtraceback\n",
      "    GUI frameworks like wxPython trap exceptions and call\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 1118, in structured_traceback\n",
      "    if hasattr(self, 'tb') and self.tb is not None:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 1012, in structured_traceback\n",
      "    pygments_formatter=formatter,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 865, in structured_traceback\n",
      "    current_line,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 818, in format_exception_as_a_whole\n",
      "    # 100% sure this isn't actually a bug in inspect itself,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 736, in format_record\n",
      "    call_pdb: bool = False,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 699, in lines\n",
      "    if not pieces:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 647, in included_pieces\n",
      "    return []\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 626, in executing_piece\n",
      "    by the interpreter in this frame.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\executing\\executing.py\", line 164, in only\n",
      "    raise NotOneValueFound('Expected one value, found 0')\n",
      "executing.executing.NotOneValueFound: Expected one value, found 0\n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\events.py\", line 89, in trigger\n",
      "    func(*args, **kwargs)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\pyplot.py\", line 137, in post_execute\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\_pylab_helpers.py\", line 141, in draw_all\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py\", line 2060, in draw_idle\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 400, in draw\n",
      "    self.figure.draw(self.renderer)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 74, in draw_wrapper\n",
      "    if artist.get_agg_filter() is not None:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 51, in draw_wrapper\n",
      "    renderer.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\figure.py\", line 2845, in draw\n",
      "    wspace : float, default: :rc:`figure.constrained_layout.wspace`\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\image.py\", line 132, in _draw_list_compositing_images\n",
      "    else:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 51, in draw_wrapper\n",
      "    renderer.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axes\\_base.py\", line 3091, in draw\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\image.py\", line 132, in _draw_list_compositing_images\n",
      "    else:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\artist.py\", line 51, in draw_wrapper\n",
      "    renderer.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1159, in draw\n",
      "    # attribute, and the derived code below will check for that\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1085, in _get_tick_bboxes\n",
      "    child.set_clip_path(clippath, transform)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\axis.py\", line 1085, in <listcomp>\n",
      "    child.set_clip_path(clippath, transform)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\text.py\", line 910, in get_window_extent\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\text.py\", line 309, in _get_layout\n",
      "    bb = self.get_window_extent()\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 233, in get_text_width_height_descent\n",
      "    font = self._prepare_font(prop)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py\", line 267, in _prepare_font\n",
      "    font = get_font(_fontManager._find_fonts_by_props(font_prop))\n",
      "AttributeError: 'FontManager' object has no attribute '_find_fonts_by_props'\n",
      "\n",
      "During handling of the above exception, another exception occurred:\n",
      "\n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 1993, in showtraceback\n",
      "    GUI frameworks like wxPython trap exceptions and call\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 1118, in structured_traceback\n",
      "    if hasattr(self, 'tb') and self.tb is not None:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 1012, in structured_traceback\n",
      "    pygments_formatter=formatter,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 865, in structured_traceback\n",
      "    current_line,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 818, in format_exception_as_a_whole\n",
      "    # 100% sure this isn't actually a bug in inspect itself,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\ultratb.py\", line 736, in format_record\n",
      "    call_pdb: bool = False,\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 699, in lines\n",
      "    if not pieces:\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 647, in included_pieces\n",
      "    return []\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\utils.py\", line 145, in cached_property_wrapper\n",
      "    value = obj.__dict__[self.func.__name__] = self.func(obj)\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\stack_data\\core.py\", line 626, in executing_piece\n",
      "    by the interpreter in this frame.\n",
      "  File \"C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\executing\\executing.py\", line 164, in only\n",
      "    raise NotOneValueFound('Expected one value, found 0')\n",
      "executing.executing.NotOneValueFound: Expected one value, found 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error in callback <function install_repl_displayhook.<locals>.post_execute at 0x000001FD69746160> (for post_execute):\n",
      "Unexpected exception formatting exception. Falling back to standard exception\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "print_png() got an unexpected keyword argument 'orientation'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\formatters.py:339\u001B[0m, in \u001B[0;36mBaseFormatter.__call__\u001B[1;34m(self, obj)\u001B[0m\n\u001B[0;32m    337\u001B[0m     \u001B[38;5;28;01mpass\u001B[39;00m\n\u001B[0;32m    338\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[1;32m--> 339\u001B[0m     \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mprinter\u001B[49m\u001B[43m(\u001B[49m\u001B[43mobj\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    340\u001B[0m \u001B[38;5;66;03m# Finally look for special method names\u001B[39;00m\n\u001B[0;32m    341\u001B[0m method \u001B[38;5;241m=\u001B[39m get_real_method(obj, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mprint_method)\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\IPython\\core\\pylabtools.py:151\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(fig, fmt, bbox_inches, base64, **kwargs)\u001B[0m\n\u001B[0;32m    148\u001B[0m     \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01mmatplotlib\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mbackend_bases\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m FigureCanvasBase\n\u001B[0;32m    149\u001B[0m     FigureCanvasBase(fig)\n\u001B[1;32m--> 151\u001B[0m \u001B[43mfig\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcanvas\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mprint_figure\u001B[49m\u001B[43m(\u001B[49m\u001B[43mbytes_io\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkw\u001B[49m\u001B[43m)\u001B[49m\n\u001B[0;32m    152\u001B[0m data \u001B[38;5;241m=\u001B[39m bytes_io\u001B[38;5;241m.\u001B[39mgetvalue()\n\u001B[0;32m    153\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m fmt \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124msvg\u001B[39m\u001B[38;5;124m'\u001B[39m:\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:2289\u001B[0m, in \u001B[0;36mprint_figure\u001B[1;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)\u001B[0m\n\u001B[0;32m   2241\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mprint_figure\u001B[39m(\n\u001B[0;32m   2242\u001B[0m         \u001B[38;5;28mself\u001B[39m, filename, dpi\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, facecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, edgecolor\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2243\u001B[0m         orientation\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mportrait\u001B[39m\u001B[38;5;124m'\u001B[39m, \u001B[38;5;28mformat\u001B[39m\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m,\n\u001B[0;32m   2244\u001B[0m         bbox_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, pad_inches\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, bbox_extra_artists\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m,\n\u001B[0;32m   2245\u001B[0m         backend\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mNone\u001B[39;00m, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs):\n\u001B[0;32m   2246\u001B[0m     \u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m   2247\u001B[0m \u001B[38;5;124;03m    Render the figure to hardcopy. Set the figure patch face and edge\u001B[39;00m\n\u001B[0;32m   2248\u001B[0m \u001B[38;5;124;03m    colors.  This is useful because some of the GUIs have a gray figure\u001B[39;00m\n\u001B[0;32m   2249\u001B[0m \u001B[38;5;124;03m    face color background and you'll probably want to override this on\u001B[39;00m\n\u001B[0;32m   2250\u001B[0m \u001B[38;5;124;03m    hardcopy.\u001B[39;00m\n\u001B[0;32m   2251\u001B[0m \n\u001B[0;32m   2252\u001B[0m \u001B[38;5;124;03m    Parameters\u001B[39;00m\n\u001B[0;32m   2253\u001B[0m \u001B[38;5;124;03m    ----------\u001B[39;00m\n\u001B[0;32m   2254\u001B[0m \u001B[38;5;124;03m    filename : str or path-like or file-like\u001B[39;00m\n\u001B[0;32m   2255\u001B[0m \u001B[38;5;124;03m        The file where the figure is saved.\u001B[39;00m\n\u001B[0;32m   2256\u001B[0m \n\u001B[0;32m   2257\u001B[0m \u001B[38;5;124;03m    dpi : float, default: :rc:`savefig.dpi`\u001B[39;00m\n\u001B[0;32m   2258\u001B[0m \u001B[38;5;124;03m        The dots per inch to save the figure in.\u001B[39;00m\n\u001B[0;32m   2259\u001B[0m \n\u001B[0;32m   2260\u001B[0m \u001B[38;5;124;03m    facecolor : color or 'auto', default: :rc:`savefig.facecolor`\u001B[39;00m\n\u001B[0;32m   2261\u001B[0m \u001B[38;5;124;03m        The facecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2262\u001B[0m \u001B[38;5;124;03m        facecolor.\u001B[39;00m\n\u001B[0;32m   2263\u001B[0m \n\u001B[0;32m   2264\u001B[0m \u001B[38;5;124;03m    edgecolor : color or 'auto', default: :rc:`savefig.edgecolor`\u001B[39;00m\n\u001B[0;32m   2265\u001B[0m \u001B[38;5;124;03m        The edgecolor of the figure.  If 'auto', use the current figure\u001B[39;00m\n\u001B[0;32m   2266\u001B[0m \u001B[38;5;124;03m        edgecolor.\u001B[39;00m\n\u001B[0;32m   2267\u001B[0m \n\u001B[0;32m   2268\u001B[0m \u001B[38;5;124;03m    orientation : {'landscape', 'portrait'}, default: 'portrait'\u001B[39;00m\n\u001B[0;32m   2269\u001B[0m \u001B[38;5;124;03m        Only currently applies to PostScript printing.\u001B[39;00m\n\u001B[0;32m   2270\u001B[0m \n\u001B[0;32m   2271\u001B[0m \u001B[38;5;124;03m    format : str, optional\u001B[39;00m\n\u001B[0;32m   2272\u001B[0m \u001B[38;5;124;03m        Force a specific file format. If not given, the format is inferred\u001B[39;00m\n\u001B[0;32m   2273\u001B[0m \u001B[38;5;124;03m        from the *filename* extension, and if that fails from\u001B[39;00m\n\u001B[0;32m   2274\u001B[0m \u001B[38;5;124;03m        :rc:`savefig.format`.\u001B[39;00m\n\u001B[0;32m   2275\u001B[0m \n\u001B[0;32m   2276\u001B[0m \u001B[38;5;124;03m    bbox_inches : 'tight' or `.Bbox`, default: :rc:`savefig.bbox`\u001B[39;00m\n\u001B[0;32m   2277\u001B[0m \u001B[38;5;124;03m        Bounding box in inches: only the given portion of the figure is\u001B[39;00m\n\u001B[0;32m   2278\u001B[0m \u001B[38;5;124;03m        saved.  If 'tight', try to figure out the tight bbox of the figure.\u001B[39;00m\n\u001B[0;32m   2279\u001B[0m \n\u001B[0;32m   2280\u001B[0m \u001B[38;5;124;03m    pad_inches : float, default: :rc:`savefig.pad_inches`\u001B[39;00m\n\u001B[0;32m   2281\u001B[0m \u001B[38;5;124;03m        Amount of padding around the figure when *bbox_inches* is 'tight'.\u001B[39;00m\n\u001B[0;32m   2282\u001B[0m \n\u001B[0;32m   2283\u001B[0m \u001B[38;5;124;03m    bbox_extra_artists : list of `~matplotlib.artist.Artist`, optional\u001B[39;00m\n\u001B[0;32m   2284\u001B[0m \u001B[38;5;124;03m        A list of extra artists that will be considered when the\u001B[39;00m\n\u001B[0;32m   2285\u001B[0m \u001B[38;5;124;03m        tight bbox is calculated.\u001B[39;00m\n\u001B[0;32m   2286\u001B[0m \n\u001B[0;32m   2287\u001B[0m \u001B[38;5;124;03m    backend : str, optional\u001B[39;00m\n\u001B[0;32m   2288\u001B[0m \u001B[38;5;124;03m        Use a non-default backend to render the file, e.g. to render a\u001B[39;00m\n\u001B[1;32m-> 2289\u001B[0m \u001B[38;5;124;03m        png file with the \"cairo\" backend rather than the default \"agg\",\u001B[39;00m\n\u001B[0;32m   2290\u001B[0m \u001B[38;5;124;03m        or a pdf file with the \"pgf\" backend rather than the default\u001B[39;00m\n\u001B[0;32m   2291\u001B[0m \u001B[38;5;124;03m        \"pdf\".  Note that the default backend is normally sufficient.  See\u001B[39;00m\n\u001B[0;32m   2292\u001B[0m \u001B[38;5;124;03m        :ref:`the-builtin-backends` for a list of valid backends for each\u001B[39;00m\n\u001B[0;32m   2293\u001B[0m \u001B[38;5;124;03m        file format.  Custom backends can be referenced as \"module://...\".\u001B[39;00m\n\u001B[0;32m   2294\u001B[0m \u001B[38;5;124;03m    \"\"\"\u001B[39;00m\n\u001B[0;32m   2295\u001B[0m     \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mformat\u001B[39m \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[0;32m   2296\u001B[0m         \u001B[38;5;66;03m# get format from filename, or from backend's default filetype\u001B[39;00m\n\u001B[0;32m   2297\u001B[0m         \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(filename, os\u001B[38;5;241m.\u001B[39mPathLike):\n",
      "File \u001B[1;32m~\\.conda\\envs\\condapractice\\lib\\site-packages\\matplotlib\\backend_bases.py:1544\u001B[0m, in \u001B[0;36m_get_renderer\u001B[1;34m(figure, print_method)\u001B[0m\n\u001B[0;32m   1542\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_press_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1543\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m event\u001B[38;5;241m.\u001B[39mkey\n\u001B[1;32m-> 1544\u001B[0m \u001B[38;5;28;01melif\u001B[39;00m event\u001B[38;5;241m.\u001B[39mname \u001B[38;5;241m==\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mkey_release_event\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m   1545\u001B[0m     event\u001B[38;5;241m.\u001B[39mcanvas\u001B[38;5;241m.\u001B[39m_key \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
      "\u001B[1;31mTypeError\u001B[0m: print_png() got an unexpected keyword argument 'orientation'"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "importances = rf.feature_importances_\n",
    "std = np.std([tree.feature_importances_ for tree in rf.estimators_], axis = 0)\n",
    "\n",
    "df = pd.DataFrame({'feature': train_X.columns, 'importance' : importances, 'std' : std})\n",
    "df = df.sort_values('importance')\n",
    "print(df)\n",
    "\n",
    "ax = df.plot(kind = 'barh', xerr = 'std', x = 'feature', legend = False)\n",
    "ax.set_ylabel(\"\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix (Accuracy 0.9820)\n",
      "\n",
      "       Prediction\n",
      "Actual    0    1\n",
      "     0 1803    4\n",
      "     1   32  161\n"
     ]
    }
   ],
   "source": [
    "classificationSummary(valid_y, rf.predict(valid_X))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 부스트 트리\n",
    "* 부스트 트리에서는 트리들이 순차적으로 구성 -> 각 트리는 이전 트리의 오분류 레코드 고려\n",
    "1. 단일 트리 생성\n",
    "2. 잘못 분류된 레코드들에 가장 높은 선택 확률을 제공하는 하나의 표본 추출\n",
    "3. 새로운 표본을 트리에 학습\n",
    "4. 단계 2&3 반복\n",
    "5. 레코드들 분류 위해 가중 투표 사용\n",
    "6. 가중치의 크기는 구성되는 트리들의 역순으로 부여\n",
    "\n",
    "### 로지스택 회귀\n",
    "* 선형 회귀의 개념을 반응 변수 Y가 범주형인 경우로 확장\n",
    "* Binary classification\n",
    "* Proess\n",
    "* -- 각 클래스에 속하는 확률 추정 : 이진 변수인 경우 클래스 1에 속할 확률을 추정\n",
    "* -- 각 관측치를 클래스 중 하나로 분류하기 위해 확률값에 대한 컷 오프값 적용"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "ERROR: Could not find a version that satisfies the requirement statsmodel (from versions: none)\n",
      "ERROR: No matching distribution found for statsmodel\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "ERROR: Could not find a version that satisfies the requirement LogisticIT (from versions: none)\n",
      "ERROR: No matching distribution found for LogisticIT\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -ip (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution -illow (c:\\users\\user\\.conda\\envs\\condapractice\\lib\\site-packages)\n"
     ]
    },
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'mord'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mModuleNotFoundError\u001B[0m                       Traceback (most recent call last)",
      "Input \u001B[1;32mIn [64]\u001B[0m, in \u001B[0;36m<cell line: 7>\u001B[1;34m()\u001B[0m\n\u001B[0;32m      5\u001B[0m get_ipython()\u001B[38;5;241m.\u001B[39msystem(\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mpip install LogisticIT\u001B[39m\u001B[38;5;124m'\u001B[39m)\n\u001B[0;32m      6\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mstatsmodels\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mapi\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m \u001B[38;5;21;01msm\u001B[39;00m\n\u001B[1;32m----> 7\u001B[0m \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01mmord\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m LogisticIT\n\u001B[0;32m      8\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mmatplotlib\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mpylab\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m \u001B[38;5;21;01mplt\u001B[39;00m\n\u001B[0;32m      9\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mseaborn\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m \u001B[38;5;21;01msns\u001B[39;00m\n",
      "\u001B[1;31mModuleNotFoundError\u001B[0m: No module named 'mord'"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "from sklearn.linear_model import LogisticRegression, LogisticRegressionCV\n",
    "from sklearn.model_selection import train_test_split\n",
    "!pip install statsmodel\n",
    "!pip install LogisticIT\n",
    "import statsmodels.api as sm\n",
    "from mord import LogisticIT\n",
    "import matplotlib.pylab as plt\n",
    "import seaborn as sns\n",
    "import dmba\n",
    "from dmba import classificationSummary, gainsChart, liftChart\n",
    "from dmba.metric import AIC_score"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 분류 문제 : 이산적 값 중 하나를 선택하는 모델, '분류 모델' 이라고 부름\n",
    "* 문제점\n",
    "* -f(x)의 값이 1 이상이나 0 이하로 나올 수 있음\n",
    "* -각 feature 들이 Y에 영향을 주는 것을 해석하는 문제\n",
    "* -사건의 발생 여부는 이산적인데 실제 f(x) 수식은 연속적\n",
    "\n",
    "### 로지스틱 회귀\n",
    "* 이진 분류(binary classification)문제를 표현\n",
    "* 어떤 사건이 일어날 확률을 P(X)로 나타내고 일어나지 않을 확률을 1 - P(X)로 나타냄 (0 <= P(X) < = 1)\n",
    "* 오즈비(odds ratio) : 어떤 사건이 일어날 확률과 일어나지 않을 확률의 비율   P(X)/1-P(X)\n",
    "\n",
    "### Logit Function\n",
    "* 오즈비에 상용로그를 붙인 수식 log(P(x)/1-P(x))\n",
    "* logit(p(y = 1|x))\n",
    "* 로짓함수의 역함수 => sigmoid function 시그모이드 함수 1/1+e^(-z), 그래프가 S자 커브 형태\n",
    "\n",
    "### Sigmoid Function\n",
    "* y값을 확률 p로 표현\n",
    "* z 값은 선형회귀와 같이 가중치와 feature의 선형 결합(linear combination)으로 표현 가능\n",
    "\n",
    "### Hypothesis Function\n",
    "* z는 가중치 값과 feature 값의 선형 결합\n",
    "\n",
    "### Cost Function\n",
    "* 먼저 비용함수를 정의하고 예측값과 실제값 간의 차이를 최소화하는 방향으로 학습 -> 비용 최소화\n",
    "* 실제값이 1일 때와 실제값이 0일 때 각각 다르게 비용함수를 정의\n",
    "\n",
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 예제 : 개인 대출 신청 수락\n",
    "### 로지스틱 모형\n",
    "* 로지스틱 회귀에서 Y와 b의 관계는 비선형 -> 최소 제곱법\n",
    "* Maximum likeihood : 주어진 데이터를 얻을 가능성을 최대화하는 추정치 찾음(MLE), 반복 추정 핵심\n",
    "* 예시----> 예측값이 클 수록 제안 수락\n",
    "\n",
    "### 검증 데이터셋에서 네 명의 고객에 대한 경향\n",
    "* 추정된 로지스틱 회귀식으로부터 검증셋의 각 레코드에 대해 클래스에 속할 확률 예측\n",
    "* 실제와 예측을 비교. Default cut-off : 0.5\n",
    "\n",
    "### 누적 이득 차트와 십분위 향상 차트\n",
    "* 누적 차트에서 기준선 보다 위에 위치한 향상 곡선\n",
    "* '1'로 분류될 확률이 높은 레코드를 10% 선택한 경우는 레코드를 임의로 10% 선발한 경우보다 7.6배(예시) 높음"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 예제 : 연착 항공편 예측\n",
    "* 예측 변수에 대한 평균 연착률의 막대그래프 생성\n",
    "* 항공편 연착을 탐색하기 위한 히트맵 생성 코드 -> 요일 출발 공항, 항공사에 의한 연착 항공편 비율"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "---------------------------------------"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 09. 신경망"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import dmba\n",
    "from dmba import classificationSummary"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 신경망 : 개요\n",
    "* 인간의 뇌는 치밀하게 뉴런 약 1,000억 개로 이루어져 있음\n",
    "* 뉴런과 뉴런 사이에는 시냅스라는 연결 부위가 있는데, 신경 말단에서 자극을 받으면 시냅스에서 화학 물질이 나와 전위 변화를 일으킴\n",
    "* 전위가 임계 값을 넘으면 다음 뉴런으로 신호를 전달하고, 임계 값에 미치지 못하면 아무것도 하지 않음\n",
    "* 이 메커니즘은 우리가 앞서 배운 로지스택 회귀와 많이 닮았음\n",
    "* 이 간단한 회로는 입력 값을 놓고 활성화 함수에 의해 일성한 수준을 넘으면 참을, 그렇지 않으면 거짓을 내보내는 일을 하는데 뉴런과 유사\n",
    "\n",
    "### 신경망 : XOR\n",
    "* 사각형 종이에 검은색 두 개와 흰색 점 두 개가 놓여 있을 때 네 점 사이에 직선을 하나 긋는다고 가정\n",
    "* 직선의 한쪽 편에는 검은색 점만 있고, 다른 한쪽에는 흰색 점만 있게끔 선을 그을 수 있을까?  --> NOPE\n",
    "* XOR 문제는 논리 회로에 등장하는 개념\n",
    "* 컴퓨터는 두 가지의 디지털 값, 즉 0과 1을 입력해 하나의 값을 출력하는 회로가 모여 만들어지는데 이 회로는 '게이트(gate)'라고 함\n",
    "* 그림 7-5는 AND 게이트, OR 게이트, XOR  게이트에 대한 값을 정리한 것\n",
    "* AND 게이트는 x1과 x2 둘 다 1일 때 결과값이 1로 출력\n",
    "* OR 게이트는 둘 중 하나라도 1이면 결과값이 1로 출력\n",
    "* XOR 게이트는 둘 중 하나만 1일 때 1이 출력\n",
    "\n",
    "### 신경망 : XOR\n",
    "* XOR 문제 해결 방법\n",
    "* X1과 X2 두 연산으로 보냄 -> 첫 번째에 NAND 처리 -> 동시에 두 번째에 OR 처리 -> 두 개의 연산을 통해 구한 결과 값 1 & 2를 통해 AND 처리\n",
    "\n",
    "### 신경망\n",
    "* 입력 변수 간, 입력 변수 - 결과 변수 간의 복잡한 관계를 파악하는 방법\n",
    "* 예측 변수 정보를 결합!\n",
    "* 개발자가 올바른 형태를 지정할 필요 없음. 신경망은 관계를 데이터를 통해 학습\n",
    "* 선형/로지스틱 회귀분석 -> 입력층과 출력층만 있는 형태의 신경망 (은닉층 X)\n",
    "* 다층 전방향 신경망 (Multi-layer feedforward networks)\n",
    "* --예측 변수 값을 받는 노드들이 있는 입력층 + 이전 층 들에서 입력 받는 노드들로 이루어진 층들이 연속적으로 구성\n",
    "* -- 각 층에 있는 노드들의 출력 == 다음 층에 있는 노드들의 입력\n",
    "* -- 마지막 층은 출력층이며, 입력과 출력 사이에 있는 층은 입력층\n",
    "* -- m 개의 클래스로 이뤄진 분류 문제 ? m개의 출력 노드\n",
    "\n",
    "### 신경망 : 다중 퍼셉트론\n",
    "* 입력층과 출력층 사이에 숨어 있는 은닉층을 만드는 것\n",
    "* 입력 값 (X1 & X2)은 각 값에 가중치(w)를 곱하고 편향(b)를 더해 은닉층으로 전송\n",
    "* 노드(n1, n2)를 은닉층이라고 함\n",
    "* 은닉층에 취합된 값들은 활성화 함수를 통해 다음으로 보냄\n",
    "* 식 두 개의 결과 값이 출력층의 방향으로 보내지고, 출력층으로 전달된 값은 활성화 함수를 통해 y_hat 값을 정하게 됨\n",
    "\n",
    "### 신경망 : 데이터 전처리\n",
    "* 로지스틱 활성 함수(scikit-learn에서 activation = 'logistic') -> 신경망은 예측 변수들과 결과 변수가 [0,1] 상의 값일 떄 최상으로 작동\n",
    "* -> 모든 변소 값은 신경망에 입력되기 전에 [0,1] 사이의 값으로 맞춰져야 함\n",
    "* [a,b] 상의 값을 취하는 수치 변슈 X에 대해 a를 뺴고 b-a로 나누어 측정값을 정규화 함\n",
    "* 비대칭 예측 변수들의 변환 필요\n",
    "* 활성 함수 다양화 (예> acivation{'identy', 'logistic', 'tanh', 'relu'}, default = 'relu'\n",
    "\n",
    "### 신경망 : Back Propagation\n",
    "* 파라미터 개수가 많으면 각각의 가중치(weight)와 편향(bias)을 학습시키는 것이 어려움\n",
    "* 은닉층에 포함된 가중치를 업데이트할 방법이 없음\n",
    "* 기존의 신경망은 입력/출력 값을 알고 있는 상태에서 w, b를 넣음\n",
    "* 역전파(back propagation)는 출력층에서부터 거꾸로 은닉층들로 오차가 계산\n",
    "* (예측값과 실제값의 차이를 최소화하는 방향으로 반복학습 ㄱㄱ)\n",
    "* Procedure\n",
    "1. 순방향 계산을 통해 각 가중치의 초기 값 정해짐\n",
    "2. 초기 값의 가중치로 만들어진 값과 실제 값을 비교해 출력층의 오차를 계산\n",
    "3. 목표 = 이때 계산된 출력층의 오차를 최소화시키는 것\n",
    "4. 이를 위해 오차를 편미분하면서 가중치 수정 과정을 거침\n",
    "* -> 은닉층이 깊을 떄 역전파 알고리즘을 진행할수록 Gradient Vanishing이 일어남"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 예제 : TinyData\n",
    "### 특정 방식으로 가공된 치즈에 대한 시식 정보\n",
    "* 관측/지방 점수/염분 점수/ 수용 여부\n",
    "* 가중치와 편향 값은 0 근처로 초기화 -> 예측 변수가 없는 모델과 마찬가지임(신경망에 선지식 없는 상태)\n",
    "* N3 노드의 초기 편향과 가중치가 w1, 3 = 0.05, w2, 3 = 0.01, // - 0.3\n",
    "\n",
    "* 가중치 갱신(학습)을 위해 모델의 오차를 사용하는 back propagation 방법 알고리즘\n",
    "* 출력층에서부터 거꾸로 은닉층들로 오차가 계산됨\n",
    "* 학습 중단 기준?\n",
    "* 1. 편향과 가중치들의 새로운 값이 이전 반복에서 얻은 값과 차이가 적을 떄\n",
    "* 2. 오분류율이 필요한 목표 값에 도달\n",
    "* 3. 실행 횟수(iteration)제한 도달"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercepts\n",
      "[array([0.13368043, 4.0724755 , 7.00768105]), array([14.30748672])]\n",
      "Weight\n",
      "[array([[ -1.3065648 ,  -4.20427795, -13.29587332],\n",
      "       [ -0.04399728,  -4.91606922,  -6.03356987]]), array([[ -0.27348313],\n",
      "       [ -9.01211575],\n",
      "       [-17.63504688]])]\n",
      "   Obs.  Fat  Salt Acceptance   dislike      like\n",
      "0     1  0.2   0.9       like  0.000490  0.999510\n",
      "1     2  0.1   0.1    dislike  0.999994  0.000006\n",
      "2     3  0.2   0.4    dislike  0.999741  0.000259\n",
      "3     4  0.2   0.5    dislike  0.997368  0.002632\n",
      "4     5  0.4   0.5       like  0.002133  0.997867\n",
      "5     6  0.3   0.8       like  0.000075  0.999925\n"
     ]
    }
   ],
   "source": [
    "# TinyData셋에 대한 신경망\n",
    "ex_df = pd.read_csv('https://raw.githubusercontent.com/kwartler/Harvard_DataMining_Business_Student/master/BookDataSets/tinydata.csv')\n",
    "\n",
    "predictors = ['Fat', 'Salt']\n",
    "outcome = 'Acceptance'\n",
    "\n",
    "X = ex_df[predictors]\n",
    "y = ex_df[outcome]\n",
    "classes = sorted(y.unique())\n",
    "\n",
    "# sklearn의 신경망 모듈 MLP 활용하여 다중 퍼셉트론 분류 모델. 은닉 중 3개 활성 함수 = 로지스택, lbfgs 알고리즘\n",
    "clf = MLPClassifier(hidden_layer_sizes =(3), activation = 'logistic', solver = 'lbfgs', random_state = 1)\n",
    "clf.fit(X, y)\n",
    "clf.predict(X)\n",
    "\n",
    "#신경망 구조\n",
    "print('Intercepts')\n",
    "print(clf.intercepts_)\n",
    "\n",
    "print('Weight')\n",
    "print(clf.coefs_)\n",
    "\n",
    "# 예측\n",
    "print(pd.concat([ex_df, pd.DataFrame(clf.predict_proba(X), columns = classes)\n",
    "                 ], axis = 1))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hidden later 2 => 3\n",
      " Intercepts:\n",
      "  [0.13368043 4.0724755  7.00768105]\n",
      " Weights:\n",
      "  [ -1.3065648   -4.20427795 -13.29587332]\n",
      "  [-0.04399728 -4.91606922 -6.03356987]\n",
      "\n",
      "Output layer 3 => 1\n",
      " Intercepts:\n",
      "  [14.30748672]\n",
      " Weights:\n",
      "  [-0.27348313]\n",
      "  [-9.01211575]\n",
      "  [-17.63504688]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 신경망 레이어 별 프린트\n",
    "for i, (weights, intercepts) in enumerate(zip(clf.coefs_, clf.intercepts_)): # 신경망 네트워크의 각 가중치 및 절편 집합 반복 루프. clf.coefs_ : 가중치 행렬 & clf.intercepts_ : 절편 벡터 목록\n",
    "    print('Hidden later' if i == 0 else 'Output layer', '{0[0]} => {0[1]}'.format(weights.shape))    # 현재 레이어 번호 인쇄 ==> 은닉층 간 가중치 시각화\n",
    "    print(' Intercepts:\\n ', intercepts)\n",
    "    print(' Weights:')\n",
    "    for weight in weights:     # 현재 레이어와 각 가중치 벡터 반복\n",
    "        print(\" \", weight)\n",
    "    print()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix (Accuracy 1.0000)\n",
      "\n",
      "        Prediction\n",
      " Actual dislike    like\n",
      "dislike       3       0\n",
      "   like       0       3\n"
     ]
    }
   ],
   "source": [
    "classificationSummary(y, clf.predict(X), class_names = classes)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 예제 : 사고의 심각성 분류\n",
    "#### 데이터 설명\n",
    "* alchl_I : 음주(1)/비음주(2)\n",
    "* profil_I_R : 도로 : 평평(1)/기타(0)\n",
    "* Sur_cond : 노면 상태: 건조(1)/젖음(2)/눈(3)/결빙(4)/모름(9)\n",
    "* veh_invl : 관련 차량 수\n",
    "* max_sev_ir : 상해 정도 무상해(0)/상해(1)/치명(2)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix (Accuracy 0.8731)\n",
      "\n",
      "             Prediction\n",
      "      Actual MAX_SEV_IR_0 MAX_SEV_IR_1 MAX_SEV_IR_2\n",
      "MAX_SEV_IR_0          332            0            0\n",
      "MAX_SEV_IR_1            0          172            8\n",
      "MAX_SEV_IR_2           31           37           19\n",
      "Confusion Matrix (Accuracy 0.8625)\n",
      "\n",
      "             Prediction\n",
      "      Actual MAX_SEV_IR_0 MAX_SEV_IR_1 MAX_SEV_IR_2\n",
      "MAX_SEV_IR_0          218            0            1\n",
      "MAX_SEV_IR_1            1          115            3\n",
      "MAX_SEV_IR_2           24           26           12\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:541: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "  self.n_iter_ = _check_optimize_result(\"lbfgs\", opt_res, self.max_iter)\n"
     ]
    }
   ],
   "source": [
    "accidents_df = pd.read_csv('https://raw.githubusercontent.com/kwartler/Harvard_DataMining_Business_Student/master/BookDataSets/accidentsnn.csv')\n",
    "input_vars = ['ALCHL_I', 'PROFIL_I_R', 'VEH_INVL']\n",
    "\n",
    "accidents_df.SUR_COND = accidents_df.SUR_COND.astype('category')\n",
    "accidents_df.MAX_SEV_IR = accidents_df.MAX_SEV_IR.astype('category')\n",
    "\n",
    "# 범주형 데이터를 더미 변수로 변경\n",
    "processed = pd.get_dummies(accidents_df)\n",
    "# SUR_COND_9 = 알 수 없음에 대한 열 삭제\n",
    "processed = processed.drop(columns = ['SUR_COND_9'])\n",
    "\n",
    "outcome = ['MAX_SEV_IR_0', 'MAX_SEV_IR_1', 'MAX_SEV_IR_2']\n",
    "predictors = [c for c in processed.columns if c not in outcome]\n",
    "classes = sorted(outcome)\n",
    "\n",
    "# 데이터 분할\n",
    "X = processed[predictors]\n",
    "y = processed[outcome]\n",
    "train_X, valid_X, train_y, valid_y = train_test_split(X, y, test_size = 0.4, random_state = 1)\n",
    "\n",
    "# 은닉층 노드 2개로 신경망 훈련\n",
    "clf = MLPClassifier(hidden_layer_sizes=(2), activation = 'logistic', solver = 'lbfgs')\n",
    "clf.fit(train_X, train_y)\n",
    "\n",
    "# 훈련 성능 (idmax를 사용하여 원 핫 인코딩 되돌림)\n",
    "trainPrediction = pd.DataFrame(clf.predict(train_X), columns = outcome).idxmax(axis = 1)\n",
    "classificationSummary(train_y.idxmax(axis = 1), trainPrediction, class_names = classes )\n",
    "\n",
    "# 검증 성능 파악\n",
    "validPrediction = pd.DataFrame(clf.predict(valid_X), columns = outcome).idxmax(axis = 1)\n",
    "classificationSummary(valid_y.idxmax(axis = 1), validPrediction, class_names = classes)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [],
   "source": [
    "# MLPClassifier의 편리한 점 : 레이블 데이터를 원-핫-인코딩할 필요 없음\n",
    "# but, 실제 예측 클래스 레이블을 얻으려면 원래 클래스 레이블로 변환 필요\n",
    "# idxmax 함수를 통해 각 인스턴스에 대한 예측 클래스의 인덱스(클래스 레이블)얻을 수 있음"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 예제 : 사고의 심각성 분류\n",
    "### 과적합 회피\n",
    "* 신경망의 단점 : 주어진 데이터에 쉽게 과적합됨\n",
    "* 학습 반복 횟수 제한 필요 예) max_iter인자\n",
    "* 검증 데이터나 CV 데이터 상에서 성능이 저하될 때를 탐지 -> 검증 오차 최소 시점"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score:  0.8630672268907563\n",
      "Best Parameters:  {'hidden_layer_sizes': 4}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score, train_test_split, GridSearchCV\n",
    "\n",
    "# 은닉층 노드 2개로 신경망 훈련\n",
    "clf = MLPClassifier(hidden_layer_sizes=(2), activation = 'logistic', solver = 'lbfgs', random_state = 1, max_iter = 500)\n",
    "clf.fit(train_X, train_y.values)\n",
    "param_grid = {\n",
    "    'hidden_layer_sizes':[(1), (2), (3), (4), (5)]\n",
    "}\n",
    "gridSearch = GridSearchCV(MLPClassifier(activation = 'logistic', solver = 'lbfgs', random_state = 1, max_iter = 1000),\n",
    "                          param_grid, cv = 5, n_jobs = -1, return_train_score = True)\n",
    "gridSearch.fit(train_X, train_y)\n",
    "print('Best score: ', gridSearch.best_score_)\n",
    "print('Best Parameters: ', gridSearch.best_params_)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  param_hidden_layer_sizes  mean_test_score  std_test_score\n",
      "0                        1         0.844762        0.011030\n",
      "1                        2         0.846401        0.011433\n",
      "2                        3         0.861401        0.023566\n",
      "3                        4         0.863067        0.024167\n",
      "4                        5         0.853053        0.022982\n"
     ]
    }
   ],
   "source": [
    "display = ['param_hidden_layer_sizes', 'mean_test_score', 'std_test_score']\n",
    "print(pd.DataFrame(gridSearch.cv_results_)[display])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\User\\.conda\\envs\\condapractice\\lib\\site-packages\\pandas\\core\\indexes\\base.py:6982: FutureWarning: In a future version, the Index constructor will not infer numeric dtypes when passed object-dtype sequences (matching Series behavior)\n",
      "  return Index(sequences[0], name=names)\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGxCAYAAABIjE2TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIiklEQVR4nO3deVxU5f4H8M9s7DuD4IKSCKKIgaBSrkBlipLl1jW95U3vvSEaWqb3mvZTftqi3m7209TUNCW75lKppbmUWilotl3LwJVNkWF12GaYeX5/IJMTqIyCnBk+79eLV8w5D88833km+PicM+fIhBACRERERBIib+kBEBEREf0RAwoRERFJDgMKERERSQ4DChEREUkOAwoRERFJDgMKERERSQ4DChEREUkOAwoRERFJDgMKERERSY7FAaWwsBCJiYmIiopC3759sWjRItTU1DTYdseOHXj00UcRERGBcePG4cSJE2b73333XQwcOBDh4eGYOHEizp8/f2dVEBERkU2xOKAkJyfDyckJR48exbZt23Ds2DFs2LChXruDBw/ilVdewezZs3Hy5Ek8++yzmDJliimE7Ny5E5s2bcK6deuQlpaG0NBQTJ8+HbzyPhEREVkUUC5duoT09HTMmjULjo6O8Pf3R2JiIlJTU+u13b17N4YPH46YmBgoFAo88sgjiIqKwvbt2wEAW7duxfjx4xEUFAR7e3u88MILyMvLQ1paWtNURkRERFbLooCSmZkJDw8P+Pr6mrYFBgYiLy8PZWVlZm0NBgOcnJzMn0wuN62gnD17FsHBwaZ9KpUKAQEBOHPmjMVFEBERkW2xKKCUl5fD0dHRbFvd44qKCrPtQ4YMwccff4z09HTU1NTgwIEDOHbsGKqrq2/al4ODQ71+iIiIqPVRWtLYyckJlZWVZtvqHjs7O5ttj4+PR1FREebNm4fS0lIMGjQIw4cPN7V3dHREVVWV2c9UVVXV6+d2ioquoSlPW5HJAC8v1ybvV0psvUbWZ/1svUZbrw+w/RpZ3933fTsWBZSgoCCUlJRAo9FArVYDAM6dOwc/Pz+4upo/WUFBAQYMGICJEyeato0dOxaPPPKIqa/MzEzExMQAAPR6PS5evGh22KcxjEY0eUBpjn6lxNZrZH3Wz9ZrtPX6ANuvkfXdfd+3Y9EhnoCAAERGRmLx4sXQarXIzs7GypUrMXr06HptT5w4gYkTJyI3NxfV1dXYsGEDLly4gMcffxwAMGrUKGzevBlnzpxBdXU1li1bBrVajaioKEuGRERERDbIohUUAFi+fDkWLlyIuLg4yOVyjBw5EomJiQCAiIgILFiwAAkJCRg2bBjOnz+PcePGoaKiAqGhodi4cSO8vb0BAKNHj8a1a9cwdepUFBUVISwsDKtXr4ZKpWraComIiMjqyISVX3hEo2n6c1DUatcm71dKbL1G1mf9bL1GW68PsP0aWd/d9307Fq+gEBGR9TIajTAYGr76d1OSyWo/+KDX62z2Dzjra5hCoYRcfvd30mFAISJqBYQQKCsrQmWl9p49Z1GRHEaj8Z49373G+m7O0dEFbm5ekDX2jNgGMKAQEbUCdeHExcUTdnb2d/WHo7EUChkMBhtcXriO9dUnhIBOVw2tthgA4O7ufcfPz4BCRGTjjEaDKZy4uLjds+dVKuWoqbHdFQbW1zA7O3sAgFZbDFdXzzs+3HP3B4mIiEjSDAYDgN//cBA1t7r32t2c78SAQkTUStyLwzpEQNO81xhQiIiISHIYUIiIiJpAdnZWk/RTXV2Nq1fzm6Qva8aAQkREdJe+/voIZs6c1iR9TZ06BSdPpjdJX9aMn+IhImqlhBCoasZPoSiNAjUG8/4dlHKbPBemrKwUQjTNa1lSUtwk/Vg7BhQiolZICIHJH/6In/LK7unz3t/ODe8+eX+jQ8rly3kYMyYBc+f+D9auXYXS0hLExj6M+PjH8K9/vY68vBx0794DCxa8Cnd3d2zb9h/s2LEVxcVF6Ny5C6ZPfwEhId0AABcvXsDKlW/h7NlMlJSUoF27dnjuueno12+A6XnmzHkZGzasw7VrZejWLRT//OcraNPG95ZjPHXqJJYufRV6vR4PPzwAW7bsgLu7BzZuXId9+z6HVnsNoaE9kJw8Cx06+AMAdu7chi1bNqGsrBS+vn4YPfpJjBgxEjNmTEV+/hUsXfoqzpz5BTNnzr7lc2s0BXj11RT88st/4eDggG7dQjFz5myo1WrU1NRgw4a1+OyzXdBqtQgKCkZy8iwEBQWjuroK69atxoEDX6CiogJdugRh6tTn0a1bKACgf/8ojBnzJL744nOEhvbEG2+8iRMn0rBmzQpkZ2dBrW6DP/95Eh55ZGij5vFO8BAPEVErZU3rGMePf4PU1G1Ys2YD9u37DG+++QaWLn0LH320C/n5V7Bz50fYseMjfPjhZqSkvI7duw9g2LARSE5ORFFRIQDg5ZdfQufOXbB16yfYt+8r9OkTjWXLXjN7nm+++RrvvfcBtmzZgeLiImzcuO62Y+vVKwovvvgP+Pr6Yf/+o1CrfbBmzUp8++1RvPXWSnz88ecIDQ3DjBlJqK6uRm5uDt5++19YuvQt7N37FRITn8ebby6BRqPBm2+ugK+vH1588R+3DScAsGrV/6FNmzbYtesLpKZ+hMrKCmzevAEAsHHjOuzfvxfLlr2Nzz8/hIiISMyePQMGgwFLl76GtLTjWL58FXbt+gIDB8bg+ecTceXKFVPfubk52L59D+bNW4jMzAzMmTMTEyY8gz17DmL27LlYvnwZ0tKOWTCLluEKChFRKySTyfDuk/c37yEehbzJDvH86U8T4eDggM6du8DbW42hQ+Ph49MGABAaGobLl/Nw8OAXmDhxErp0CQIADB/+GHbv/gT79n2OP/1pAt54499Qq31gNBpx5cpluLq6oaDgqtnzPPXU03B1rb2R3YMPDsDp0z9bPFYhBD7+eBv+93/fQLt27QEAzzwzGZ9+uhPHjn2NkJDQ6222Y/DgOERG9saBA0fv6IJm9vb2+OGHUzhwYB+iovpg2bK3Tf3s3bsHEyY8g/vu6wwAePrpZxEd3Q/V1dU4cGAfFi9eYlrRGTv2T9i//3Ps378XEyc+AwB45JFH4eDgAAcHB6xevQL9+w/CoEGxAICwsPsxYsTj2L59K/r2fcDicTcGAwoRUSslk8ngqFI0W/+1VyJtmnUaNzd30/dyuRyurm5mj4UQuHw5DytW/BurVr1t2ldTU2M6xFO3ClBUVIhOne6Dh4cHxB/uhOft/ful2ZVK5R3di6akpBiVlZWYN28O5PLf69fr9bh8+TIGD47D22+vRmrq+3jppRkwGo0YNmwEnntuGuztLbuYXnLyLLz//nps2bIJixb9D7p0CUJy8izcf38ECgs18PNra2qrUqnQo0cYNBoN9Ho92rXrYNZX27btceVKnumxj4+P6fsrV/Jw6tRJPProYNM2g8GI9u3N+2hKDChERCR5jVl18fHxxeTJf8NDDw0xbcvNzYGbmzs0mgLMnz8HixYtQf/+AwEAX311EIcPf9nkY3V394CdnT3+9a//Q48eYabtWVkXoVa3QXFxEQwGI159dSmMRiN+/vknvPzyS/D374hRo8Za9Fy//XYGjz02Cs8++zcUFxdjw4Z3MXfuLOzefQBt2vgiP//3QzY1NTVYufIt/OlPE2FnZ4/c3Bx06hRg2p+bm4N+/Qbc0Pvvr7mPjy+GDh2OWbP+adqm0WgANN+9iHgOChER2YSEhMexceM6XLp0EQCQlnYMEyeOxY8/nkJFRTkMBgMcHR0BABcunMd7760FULuycbfs7OxQVVWFmpoayOVyDB+egFWr3sbVq/kwGo34/PPdmDhxHHJyspCffwUzZkzFd9+dgFwuh1qtBgB4eHiY+tJqG3fX6fffX48333wd5eVauLq6wsHBEe7utf0MGzYCH3ywCVlZl1BTU4P331+PI0e+gqenF+LjE7B69Qrk5GRDr9dj69YtuHDhvFm4u9Hw4Y9h//59SE8/DqPRiOzsLCQlTcGWLZvu+rW7Ga6gEBGRTRg3bjwAgdmzZ6KwUAMfHx/MmPES+vcfBABITHweCxe+jKqqKvj4+CIh4XGsXPkWzp07C3d391t3fhvh4ZHw9PTE0KExWLXqPUydmoz169dg6tQpKC0tRbt27fG///s6goNDAAAzZ76EJUteRWFhAVxcXPH446MRG/swgNowsGbNCpw58wvmz0+55fO+9NJcLFv2GsaMeQx6vR4hId2QklJ74u/48X9GTU0NXnhhGsrKShESEoqlS5dDqVRi6tTnsX79ajz//HMoKytDYGAX/Otfb6Njx04NPk9oaA/8z/8swurVKzBv3mw4ODjioYeG4O9/T7qr1+1WZOKPB+CsjEZzDU1ZgUwGqNWuTd6vlNh6jazP+tl6jfe6Pr1eh8LCy/D2bguVyq75n/A63u3Xut1Nfbd6z9W9/2+Hh3iIiIhIcniIh4iI6BY+/HAz1q1bfdP9jzwy1Ozk0aby1VcHsWjR/9x0f8+eEVi2bHmTP69UMKAQERHdwpNPTsCTT06ot725D/EMHhyHwYPjmq1/qeMhHiIiIpIcBhQiolbCyj8TQVakKd5rDChERDZOoai9WqxOV93CI6HWou69plDc+ZkkPAeFiMjGyeUKODq6QKstBgDY2dnf0f1wLGU0ymAw2O6qDeurTwgBna4aWm0xHB1d7uj+QnUYUIiIWgE3Ny8AMIWUe0Eul9/RvWysBeu7OUdHF9N77k4xoBARtQIymQzu7t5wdfWEwVBzD54P8PR0RnFxuc1ebI/1NUyhUN7VykkdBhQiolZELpdDLm/+q8nKZICDgwNUKr3N/gFnfc2LJ8kSERGR5DCgEBERkeQwoBAREZHkMKAQERGR5DCgEBERkeQwoBAREZHkMKAQERGR5DCgEBERkeQwoBAREZHkMKAQERGR5DCgEBERkeQwoBAREZHkMKAQERGR5DCgEBFRk6rUGxC19AgC5uxBpc7Q0sMhK8WAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCRJLCi3wREcCAQkRERBLEgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJJjcUApLCxEYmIioqKi0LdvXyxatAg1NTUNtt24cSNiY2PRq1cvjBgxAvv27TPtq6qqwvz589GvXz/07t0bTz/9NM6cOXPnlRAREZHNsDigJCcnw8nJCUePHsW2bdtw7NgxbNiwoV67w4cPY/Xq1Vi7di1OnTqFpKQkJCcnIycnBwDw9ttv4+LFi9izZw+++eYbhISEICkp6a4LIiIiIutnUUC5dOkS0tPTMWvWLDg6OsLf3x+JiYlITU2t1/b8+fMQQpi+FAoFVCoVlEolAODcuXOmfQAgl8vh6OjYBCURERGRtVNa0jgzMxMeHh7w9fU1bQsMDEReXh7Kysrg5uZm2h4fH48dO3Zg2LBhUCgUkMlkWLJkCfz8/AAAf/nLXzBt2jRER0dDoVDA09MT77//vsUFyGQW/0ij+mvqfqXE1mtkfdbtxrJkMtusk3No/Wx+Dpuxvsb2aVFAKS8vr7fKUfe4oqLCLKDo9XqEhIRg0aJFCAkJwa5duzB37lwEBgaia9euMBgMGDJkCKZOnQpnZ2e88cYbSExMxKeffgp7e/tGj8nb29WSElq8Xymx9RpZn3Wq0P1+TpuXtwuc7Cz6NWVVOIfWz1bnsE5L1mfRu8bJyQmVlZVm2+oeOzs7m21PSUlBr1690LNnTwDAqFGjsHv3buzcuRMvvPACnn/+eaxZs8a0GjNv3jz07t0b33zzDWJjYxs9psLCa7h+lKhJyGS1E9LU/UqJrdfI+qxbpc5g+r6oUIsKlaIFR9M8OIfWz9bnsDnrq+v7diwKKEFBQSgpKYFGo4FarQZQey6Jn58fXF3NnywvLw89evQwfzKlEiqVChUVFSgtLYVOpzPtqzsMpFKpLBkShECzvDmaq18psfUaWZ91urEkW62xjq3Wxzm0HS1Zn0UnyQYEBCAyMhKLFy+GVqtFdnY2Vq5cidGjR9drGxsbi82bN+P06dMwGo3Yu3cv0tLSMGzYMLi7uyMyMhJLly5FYWEhqqursWTJEnh6eiIyMrLJiiMiIiLrZPHHjJcvX46amhrExcVh7NixGDBgABITEwEAERER+PTTTwEASUlJeOqppzBt2jT07t0ba9aswYoVK9CtWzdTPwEBAUhISMDAgQNx7tw5rFu3Dk5OTk1YHhEREVkji89cUqvVWL58eYP7vv/++987Vioxbdo0TJs27ab9vPHGG5Y+PREREbUCvNQ9ERERSQ4DChEREUkOAwoRERFJDgMKERERSQ4DChEREUkOAwoRERFJDgMKERERSQ4DChEREUkOAwoRERFJDgMKERERmVTqDYhaegQBc/aY3Zn6XmNAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyVG29ACIiMj6CCGgMwiU62pQXm2o/a/OAG21ASWVOrN2RHeCAYWIqBUxCoFKveF6qDCgQlcDra72+/Lq2pDxe+j4PXiUV9dvV2O8ffiYuPl7xHX1QVyQGkE+zpDJZPegSrIFDChERFagxihQYQoLtcFBaxYq/hAwbtKuQmdAU69pONsprn8p4WyvgINSgZPZJQCAS8WVWH88C+uPZ6GjpyNig9SIC1ajaxsXhhW6JQYUIqJmpKsxmgUGra4GFXoD5NlluFKohbbaUP8wSQPBo6rG2KTjUsgAF3tlbbCo+69d3eMbv6/9r8sNAeTGdo4qBeR/CBqVegMGLv8GADB/SDCOnCvEsYvFyCquxIb0bGxIz0Y7dwfEXQ8r3f1cGVaoHgYUIqI/EEKgqsbY4GGNhg5/aBtcxajdrzc07XqFvVJeb8XCFBj+EDZc7G/ezl4pvyeh4OGuPhjRww8VOgO+uVCEQxkF+Pp8EfJKq7DpZA42ncyBr6s94oLViA1SI6ydW73AQ60TAwoR2QyDsfb8Cu0fQsKNqxN1qxg3HgJpKHg04vQKizip6lYmaoOCp4s9VDKYBQuXmwWPG75XKazzw5dOdgo83NUHD3f1QZXegG8vFuPgb7VhJf9aNT74LhcffJcLHxc7xAapERusxv3t3KGQM6y0VgwoRCRZB34rgM5gvOlhjz+eb1GhNzTp88tlaPiwhylAKOByw/cNrVS42CvhqFKY/aGVyQC12hUazTW0xg+5OKgUtSEkSI0qvQFpl4pxMEODI+cKUaDV4T/f5+E/3+fBy0mFmOuHgSI6eEDJsNKqMKAQkWT8lFeGNd9eND1esC/jjvqxU8gsO/xxPYA42d1wvoW9Eg736DBIa+agUmBQFzUGdVFDV2NEelZtWDl8thBFFXps//Eytv94GR6OKgzu4o24YDWi/D2gtNKVJGo8BhQianGnckqw7lgW0rNKzLZH+rvD1V7Z6MMfLnZKONkpYKfkHy9rZKeUo39nb/Tv7A29wYiT2SU4mKHBV5kalFTq8fHPV/Dxz1fg5qDEoEBvxAX7oE8nD6s97EW3ZnFAKSwsxLx585Ceng6FQoGEhATMnj0bSmX9rjZu3IiNGzeipKQE7du3R1JSEoYMGWLa/8EHH+C9996DRqNBhw4dMHPmTMTExNxdRURkFYQQ+C67FO8eu4RTOaUAAIVchqHd2mD36XwAwL8f7wEHlaIlh0ktRKWQ44EALzwQ4IU5DwXhVHYJDmVq8GWmBkUVeuw6nY9dp/PhYq/AoEBvxAb7oG8nT9gznNoMiwNKcnIyfH19cfToUWg0Gjz33HPYsGEDJk+ebNbu8OHDWL16NTZv3ozOnTtj3759SE5Oxv79+9GhQwfs3LkTK1aswDvvvIOwsDDs2bMH06ZNw8GDB+Hr69tkBRKRtAghkHapGGuPZeHHvDIAgFIuw2Nhfni6jz88HFWmgEIE1L4/+nTyRJ9OnpgV2wU/5JbiUIYGhzI10JTrsOeXq9jzy1U42ynQv7MX4oJ98ECAJ8OtlbMooFy6dAnp6ek4cuQIHB0d4e/vj8TERCxZsqReQDl//jyEEKYvhUIBlUplWmlZv349nn/+efTs2RMAMHz4cNx3331wcXFpotKISEqEEPj2QjHWHr+E/16+BqD2XJGRYW0xsXcH+Lk5AKi9hgbRzSjkMkT6eyDS3wMvxAbip9wyHMzU4FBGAa5qddh3pgD7zhTAUSVHv/tqz1np19kLjgwrVseigJKZmQkPDw+zFY7AwEDk5eWhrKwMbm5upu3x8fHYsWMHhg0bBoVCAZlMhiVLlsDPzw+VlZXIzMyEXC7HU089hbNnz+K+++7Diy++CGdnZ4sKaOrz1+r6s+Xz4my9RtYnLUIIHDlXhLXHLuHXfC2A2mt5jLq/Lf7cuwPULvZm7W8sSyaznjotYW1zaKl7NYcKmQwR/u6I8HfHzJjOOH35Gg5maHAwowCXy6pxIKMABzIKYK+U48H7vBAXrMaAQC8429396Ze2PIfNPX+N7c+iWSovL4ejo6PZtrrHFRUVZgFFr9cjJCQEixYtQkhICHbt2oW5c+ciMDAQHh4eEEJg/fr1eOutt9CpUyds3boVU6ZMwa5du9ChQ4dGj8nb29WSElq8Xymx9RpZX8syGgX2nb6C5YfO4tfLtYdyHFUK/PmBTpg8oDN8XO0b/LkKXY3pey9vFzg1wR8TqZL6HN6plprDNj5uiOnZHilC4OfcUnz28xV89vNlZBVV4Mvr56/YKeUYGOSDYWF+iOvmC3dH1V09py3OoVT+H7ToWZ2cnFBZWWm2re7xH1c+UlJS0KtXL9MhnFGjRmH37t3YuXMn/vrXvwIAJk2ahKCgIADAhAkTsGXLFhw+fBhPPfVUo8dUWNi01xGQyWrfcE3dr5TYeo2sr2UZjAIHMwqw9lgWzhdWAKi9SNm4Xu0wPrI9PJ3sgGodNNW6Bn++Uvf7IZ6iQi0qbHBpXupzeLekMIftHBSY3Ls9no1qh4yCchzMKMCB3zTIKq7EgV/zceDXfCjlMvTt5Im4YDUGdfG2KKzY8hw29/zVvXa3Y1FACQoKQklJCTQaDdRqNQDg3Llz8PPzg6ur+ZPl5eWhR48e5k+mVEKlUsHLywve3t7Q6cx/QRkMlh97FgLN8uZorn6lxNZrZH33Vo1RYP9vV7H+eBYuFl3/h4udAk/2ao8/9Wpv+uV/uzHfuFtqNTY1W61PWnMoQ7CPC4J9XPD3BwNwrrAChzIKcDBDg/OFFfjmQhG+uVAExX4Zevt7IC5YjcFd1PBwalxYafn6mp5U5s+igBIQEIDIyEgsXrwYCxcuRHFxMVauXInRo0fXaxsbG4vNmzcjJiYG3bp1wxdffIG0tDTMnDkTAPDkk09ixYoV6NWrF4KCgvDBBx8gPz8fDz30UNNURkT3RI1RYO+v+XgvLRtZxbXBxNVeiT9FtseTEe3h6mC7h2jIushkMnRRO6OL2hl/fTAAFworcCizNqxkFpTj+KViHL9UjNcOZKLXDWHF29mupYfeKln8m2P58uVYuHAh4uLiIJfLMXLkSCQmJgIAIiIisGDBAiQkJCApKQkKhQLTpk1DaWkpOnXqhBUrVqBbt24AgKSkJLi4uCA5ORlXr15F586d8e677/IjxkRWQm8w4rNfaoNJbmkVAMDdQYmnojpgTHg7uNgzmJC03efthGe9O+HZ6E64VFSBQ5kaHMrQ4MxVLU5kleBEVgleP3AWER3cEResRkyQGj4uDZ87RU1PJoR1L0419b0sWsM9Mmy9RtbXvHQ1Ruw+fQUb0rNxuawaAODpqMLE3h0w6v52cLK7u+PVlXoDBi7/BgBwdHo/m7yWRUvPYXOz9jnMKanEl5kaHMzQ4PSVa2b77m/nhtjg2vsD9ejsY5Nz2NzzV/f+vx3+E4eIGqW6xohPfr6MjenZuKqtPX/M29kOf+7dAY/3bMvrTJDN6ODhiIm9/TGxtz8ul1WZwspPeWX48frXm1+dR7i/BwZ19kRMkBrt3R1v3zFZhAGFiG6pSm/Ajp8uY9OJHGjKa4OJj4sdnu7tj8fC/KzuX8dElmjr5oDxkR0wPrIDrl6rrg0rmRr8kFOKH7JL8EN2Cd46fAHdfF0QG6RGXLAP/D0ZVpoCAwrZFGtfWpaSCp0B23/Mw+aTOSiq0AMAfF3t8Uwff4zo4cd7nlCr08bVHuN6tce4Xu1RWKHDictafPJdNk7llOLXfC1+zddixdcXEeTjjIeCfRAbrEaAl1NLD9tqMaAQkZlyXQ0++j4Pqd/loqSyNpi0c7PHpL4dER/qyzvHEgFQO9thYnQnDO3ihcJyHb46W4hDGQU4mVWCzIJyZBaU451vLiJQ7YS4oNqw0tnbCTJbvPRsM2FAISIAwLWqGvzn+1xsOZWLsqraK0n6ezhgUt+OGNqtDZQMJkQN8nKywxM92+KJnm1RUqnHkbOFOJhZgPRLJTinqcA5zSWsOXYJAV6OiA32QVyQGkE+zgwrt8GAQtTKlVbq8eGpXHz4fS601bUXS+zk6Yi/RHfEIyFtoJTzlyhRY3k4qpAQ5oeEMD+UVelx9FwRDmYU4PilYlwsqsT641lYfzwL/h4OtWElWI2QNi4MKw1gQCFqpUoq9PjgVA62fp+H8uuXtr7P2wmTozsiLtgHCgYTorvi5qBCfKgv4kN9oa2uwdfna8PKsYvFyC6pwsb0bGxMz0Y7N3tTWAn1c2VYuY4BhaiVKarQIfVkDj76IQ+VeiMAIMjHGc9Gd0RMkBpy/nIkanIu9ko82q0NHu3WBhU6A765UIRDGQX4+nwR8sqqsflkDjafzIGvq/31TwOpEdbOrVX//8iAQtRKaLTV2HQyB9t/vIzqmtpgEtLGBZMf6IgBgd6t+hch0b3kZKfAw1198HBXH1TpDfj2YjEO/lYbVvKvVWPLqdpzwXxc7BDTRY3YYDXC27u3ulVNBhQiG5d/rRqbTmTj45+vmIJJqJ8rJj/QEf3u8+JyMlELclApEBukRmyQGlV6A9IuFeNghgZHzhWiQKvD1h/ysPWHPHg5qRBzfWUlooNHqzg3jAHlBryGBtmSK2VV2JCejU//ewV6Q+21uMPaumHKgx0R3cmTwYRIYhxUCgzqosagLmroaoxIz6oNK4fPFqKoQo/tP17G9h8vw8NRhcFdvBEXrEaUv4fNfsKOAYXIxuSWVmJDWjZ2n85HjbE2mER0cMfk6I7o3dGDwYTICtgp5ejf2Rv9O3tDbzDiZHYJDmZo8FWmBiWVenz88xV8/PMVuDkoMSjQG3HBPujTycOmrlPEgEJkI7KLK/FeWhY++yUf1xdMENXRA5OjOyLS36NFx0ZEd06lkOOBAC88EOCFOQ8F4VR2CQ5lavBlpgZFFXrsOp2PXafz4WKvwMBAb8QG+SA6wNPqr/bMgEJk5S4WVeC9tCzs/fUqri+YIDrAE5OjO+L+9u4tOzgialJKuQx9OnmiTydPzIrtgh9yS3EoQ4NDmRpoynX47Jer+OyXq3BSKTAg0AuxwT54MMDTKk9ZYEAhslLnNOVYdywL+38rQN3d3vt39sKz0R3Ro61bi46NiJqfQi5DpL8HIv098EJsIH7KLcPBTA0OZRTgqlaHfWcKsO9MARyUcvTvXBtW+t3nBSc76wgrDChEViazQIv5+zLw2c9XTNsGBXrj2Qc6opuvawuOjIhailwmQ3gHd4R3cMeMwZ1x+vI1HMzQ4FBmAS6XVeNAhgYHMjSwV8rx4H1eiAtSo19nL7jYSzcGSHdkRGTmTP41rDueha/OFpq2xQap8ZfojujaxqUFR0ZEUiKXyRDWzg1h7dzw/KD78Gu+FgczNDiYUYDc0ip8ef38FTuFDNEBXogLVmNAZ2+4OkgrEkhrNERUz+nLZVh7PAtfny8CAMgAxPdsiwkR7RCodm7ZwRGRpMlkMnT3c0V3P1ckDQhARkE5DmUU4ECGBlnFlThyrhBHzhVCKZehbydPxAar0beTR0sPGwADCpFk/ZhbinXHs3DsYjEAQC4DhoS0wV+iO6J3V19oNNcgxG06ISK6TiaToWsbF3Rt44K/9wvAucIKHMoowMEMDc4XVuCbC0X45kKR2RVrRQv+kmFAIZKYUzklWHssCyeySgAAChkwtLsvJvXtiI6ejuBlTIjobslkMnRRO6OL2hl/fTAAFworcCizNqxkFpSb2v2YV4boAK8WGSMDCpEECCFwMrs2mJzKKQVQe4b+8FBfPNPHHx08HFt4hERky+7zdsKz3p3wbHQnZFzV4qlNpwDU3q+rpTCgELUgIQTSLhVj7bEs/JhXBqD2OgePhfnh6T7+aOvm0MIjJKLWxt/z938QteT1UxhQiFqAEALfXijG2uOX8N/L1wAAdgoZRoa1xcTeHeDHYEJErRwDCtE9JITAkXNFWHf8En7N1wIA7JVyjLq/LSZGdYDaxb6FR0hEJA0MKET3gFEIfJWpwdrjWaYT0ByUcowJb4enojrA29muhUdIRCQtDChEzchgFDiYUYB1x7NwvrACAOCkUmBsRDuMj2wPTycGEyKihjCgEDWDGqPA/t+uYv3xLFwsqgQAONsp8GSv9vhTr/Zwd1S18AiJiKSNAYWoCdUYjNh75ireS8tGVnFtMHG1V+JPke3xZER7yV1KmohIqvjbkqgJ6A1GfPZLPt5Ly0ZuaRUAwN1BiaeiOmBMeDtJ35CLiEiK+FuT6C7oaozYdfoKNqRl48q1agCAp6MKE3t3wKj721nNbc2JiKSGAYXoDlTXGPHJz5exMT0bV7U6AIC3sx3+3LsDHu/ZFo4teHEjIiJbwIBCZIEqvQE7frqMTSdyoCmvDSY+LnZ4urc/Hgvza9GrLhIR2RIGFKJGqNAZsP3HPGw+mYOiCj0AwNfVHs/08ceIHn6wV8pbeIRERLaFAYXoFsp1Nfjo+zykfpeLksraYNLOzR6T+nZEfKgvVAoGEyKi5sCAQtSAa1U1+M/3udhyKhdlVTUAAH8PB0zq2xFDu7WBksGEiKhZMaAQ3aC0Uo8PT+Xiw+9zoa02AAA6eTriL9Ed8UhIGyjlshYeIRFR68CAQgSgpEKPD07lYOv3eSjX1QaT+7ydMDm6I+KCfaBgMCEiuqcYUKhVK6rQIfVkDj76IQ+VeiMAIMjHGc9Gd0RMkBpyGYMJEVFLYEChVkmjrcamkznY/uNlVNfUBpOQNi54NrojBnbxZjAhImphDCjUquRfq8amE9nY+dNl6AwCANDdzxVTHuiIfvd5QcZgQkQkCQwo1CpcKavChvRsfPrfK9BfDyZhbd0w5cGOiO7kyWBCRCQxDChk03JLK7EhLRu7T+ejxlgbTCI6uGNydEf07ujBYEJEJFEMKGSzXj2QiX2/XsX1BRNEdfTA5OiOiPT3aNFxERHR7TGgkCQJIVBdY0S5zoBynQEVuhqU6wzQVhtQfv37Ct3172/Ydq26xtTHZ79cBQBEB3hicnRH3N/evaXKISIiCzGgUJOqMdSGigq9wRQctHVhorrmeuCoMQWPujYVdY9v2Ge4fkjmTj0Q4Im/PtgJPdq6NVF1RER0rzCgEIQQqNQbf1+N0JuHibrwoG0oTFSbh4q6j+w2JWc7BZztFHCyU8DZTln72F5p2u5ct91eAaVchv/9IhMA8EZCd95dmIjISjGgWDFdjdF8NeJ6YKi4YSWiodWLCp0BVQaBskodtNfb391aRX32SjmcVAo42/8eKpyuhwkXe2XDgaOB9k52CouuSVKpN5gCChERWS8GlHvMYBSo1Bugrf7DeRR1hztusnpRXn3DoZLr++o+LttU5DL8IUzUrkq43HL1om6beRve5ZeIiO4GA0oj/PGEzd8PbdwYLuoHjoZWL+oup96UHFVyONndEBpushrhbKeEi33t9nZqV9RU6a6vctS2cVDK+bFbIiKSBAaUm3g69XtU6n9fvWjixQoo5DK43GQ14sbViz+eY+Fsp4STXe2qhrOdEo52CovvsCuTAWq1KzSaaxBNfWyHiIioCTCg3MT5wop622SA6TyKusBw4wpEQ2GiLnC43BA4nOyUsFPIuFpBRER0EwwoN/HmyFB4Oql+P3Rir4CjyrITNomIiOjOMKDcRFRHD35ElYiIqIXwoxZEREQkOQwoREREJDkMKERERCQ5DChEREQkOQwoREREJDkMKERERCQ5DChEREQkOQwoREREJDkMKERERCQ5DChEREQkOQwoREREJDkWB5TCwkIkJiYiKioKffv2xaJFi1BTU9Ng240bNyI2Nha9evXCiBEjsG/fvgbbffTRR+jataulQyEiIiIbZXFASU5OhpOTE44ePYpt27bh2LFj2LBhQ712hw8fxurVq7F27VqcOnUKSUlJSE5ORk5Ojlm7zMxMLF68+I4LICIiIttjUUC5dOkS0tPTMWvWLDg6OsLf3x+JiYlITU2t1/b8+fMQQpi+FAoFVCoVlMrfb6BcWVmJmTNn4s9//vPdV0JEREQ2Q3n7Jr/LzMyEh4cHfH19TdsCAwORl5eHsrIyuLm5mbbHx8djx44dGDZsGBQKBWQyGZYsWQI/Pz9Tm4ULF2Lw4MF48MEHsWrVqjsqQCa7ox9ruK8/9NuUfUtJXV22WF9rmENbnj+Ac2gLOIfWrbnnr7H9WRRQysvL4ejoaLat7nFFRYVZQNHr9QgJCcGiRYsQEhKCXbt2Ye7cuQgMDETXrl3xySef4Ny5c0hJScF3331nyTDMeHu73vHP/lGF7vdzaby8XeBkZ9HLY3Wa8rWTitY0h7Y4fwDn0BZwDq2bVObPomd1cnJCZWWl2ba6x87OzmbbU1JS0KtXL/Ts2RMAMGrUKOzevRs7d+7E2LFjsWzZMqSmppod8rkThYXXIMRddWFSqTOYvi8q1KJCpWiajiVGJqv9n6opXzupaA1zaMvzB3AObQHn0Lo19/zVvXa3Y1E6CAoKQklJCTQaDdRqNQDg3Llz8PPzg6ur+ZPl5eWhR48e5k+mVEKlUmHfvn0oKyvD448/DgAwGGpfjKioKLzyyisYMWJEo8ckBJrszXFjN03Zr1TZYo2taQ5ttT7OofXjHFo3qcyfRSfJBgQEIDIyEosXL4ZWq0V2djZWrlyJ0aNH12sbGxuLzZs34/Tp0zAajdi7dy/S0tIwbNgwPPfcc/jhhx9w8uRJnDx50nT+ycmTJy0KJ0RERGSbLD6+snz5cixcuBBxcXGQy+UYOXIkEhMTAQARERFYsGABEhISkJSUBIVCgWnTpqG0tBSdOnXCihUr0K1btyYvgoiIpMNRpcDJFwdCrXaFRmN7h0Do3rA4oKjVaixfvrzBfd9///3vHSuVmDZtGqZNm3bbPvv27YvffvvN0qEQERGRjeKl7omIiEhyGFCIiIhIcmz3w+lEZJV4/gIRAVxBISIiIgliQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyWFAISIiIslhQCEiIiLJYUAhIiIiyeGVZMmm8CqkRES2gSsoREREJDkMKERERCQ5DChEREQkOQwoREREJDk8SfYGPMGSiIhIGriCQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDgEJERESSw4BCREREksOAQkRERJLDuxkTERGRiaNKgZMvDoRa7QqN5hqEaJlxcAWFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCSHAYWIiIgkhwGFiIiIJIcBhYiIiCTH4oBSWFiIxMREREVFoW/fvli0aBFqamoabLtx40bExsaiV69eGDFiBPbt22faV11djUWLFmHgwIGIjIzEmDFjcPz48TuvhIiIiGyGxQElOTkZTk5OOHr0KLZt24Zjx45hw4YN9dodPnwYq1evxtq1a3Hq1CkkJSUhOTkZOTk5AIClS5fi1KlT+M9//oP09HSMGTMGf//735GXl3fXRREREZF1syigXLp0Cenp6Zg1axYcHR3h7++PxMREpKam1mt7/vx5CCFMXwqFAiqVCkqlEkDtCsr06dPRtm1bKBQKjB07FnZ2djh9+nTTVEZERERWS2lJ48zMTHh4eMDX19e0LTAwEHl5eSgrK4Obm5tpe3x8PHbs2IFhw4ZBoVBAJpNhyZIl8PPzAwAsXLjQrO9jx47h2rVrCAkJsagAmcyi5o3ur6n7lRJbr5H1WT9br9HW6wNsv0bWd/d9345FAaW8vByOjo5m2+oeV1RUmAUUvV6PkJAQLFq0CCEhIdi1axfmzp2LwMBAdO3a1ayPH374AcnJyUhKSoK/v78lQ4K3t6tF7Vu6Xymx9RpZn/Wz9RptvT7A9mtkfc3HooDi5OSEyspKs211j52dnc22p6SkoFevXujZsycAYNSoUdi9ezd27tyJOXPmmNp99NFHWLx4MaZPn45JkyZZXEBh4TUIYfGP3ZRMVjshTd2vlNh6jazP+tl6jbZeH2D7NbK+u+/7diwKKEFBQSgpKYFGo4FarQYAnDt3Dn5+fnB1NX+yvLw89OjRw/zJlEqoVCoAgMFgwIIFC/DFF19gxYoVePDBBy0ZiokQaJY3R3P1KyW2XiPrs362XqOt1wfYfo2sr/lYdJJsQEAAIiMjsXjxYmi1WmRnZ2PlypUYPXp0vbaxsbHYvHkzTp8+DaPRiL179yItLQ3Dhg0DALz66qs4cuQItm/ffsfhhIiIiGyTRSsoALB8+XIsXLgQcXFxkMvlGDlyJBITEwEAERERWLBgARISEpCUlASFQoFp06ahtLQUnTp1wooVK9CtWzcUFRUhNTUVCoUCw4cPN+u/7ueJiIio9ZIJYd2LUxpN05+Dola7Nnm/UmLrNbI+62frNdp6fYDt18j67r7v2+Gl7omIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyLA4ohYWFSExMRFRUFPr27YtFixahpqamwbYbN25EbGwsevXqhREjRmDfvn1m+999910MHDgQ4eHhmDhxIs6fP39nVRAREZFNsTigJCcnw8nJCUePHsW2bdtw7NgxbNiwoV67w4cPY/Xq1Vi7di1OnTqFpKQkJCcnIycnBwCwc+dObNq0CevWrUNaWhpCQ0Mxffp0CCHuuigiIiKybhYFlEuXLiE9PR2zZs2Co6Mj/P39kZiYiNTU1Hptz58/DyGE6UuhUEClUkGpVAIAtm7divHjxyMoKAj29vZ44YUXkJeXh7S0tKapjIiIiKyW0pLGmZmZ8PDwgK+vr2lbYGAg8vLyUFZWBjc3N9P2+Ph47NixA8OGDYNCoYBMJsOSJUvg5+cHADh79iymTJliaq9SqRAQEIAzZ84gOjq60WOSy4GmXHSRyZqnXymx9RpZn/Wz9RptvT7A9mtkfXff9+1YFFDKy8vh6Ohotq3ucUVFhVlA0ev1CAkJwaJFixASEoJdu3Zh7ty5CAwMRNeuXRvsy8HBARUVFZYMCV5erha1b+l+pcTWa2R91s/Wa7T1+gDbr5H1NR+LDvE4OTmhsrLSbFvdY2dnZ7PtKSkpCAoKQs+ePWFnZ4dRo0YhPDwcO3fuBFAbbKqqqsx+pqqqql4/RERE1PpYFFCCgoJQUlICjUZj2nbu3Dn4+fnB1dU8ZeXl5UGn05ltUyqVUKlUpr4yMzNN+/R6PS5evIjg4GCLiyAiIiLbYlFACQgIQGRkJBYvXgytVovs7GysXLkSo0ePrtc2NjYWmzdvxunTp2E0GrF3716kpaVh2LBhAIBRo0Zh8+bNOHPmDKqrq7Fs2TKo1WpERUU1TWVERERktWTCws/1ajQaLFy4EGlpaZDL5Rg5ciRefPFFKBQKREREYMGCBUhISEBNTQ3eeecd7Ny5E6WlpejUqRNmzJiBAQMGAACEEHjvvfeQmpqKoqIihIWFYcGCBbjvvvuapVAiIiKyHhYHFCIiIqLmxkvdExERkeQwoBAREZHkMKAQERGR5DCgEBERkeS06oBSVFSEhx9++Jb3/zl8+DBGjBiB8PBwDB06FF9++eU9HOHda0yNkydPRlhYGCIiIkxfR44cuYejtNyZM2cwadIk9OnTB/369cNLL72EoqKiBtta4xxaUp81zh8AHDt2DGPGjEGvXr3Qr18/pKSk1Lt4Yx1rnENL6rPWOQQAg8GAiRMnYs6cOTdtY43zV6cx9Vnr/H322Wfo3r272bhnzZrVYNsWmUPRSp08eVI89NBDIjg4WBw/frzBNhcuXBBhYWFi//79Qq/Xiz179oiePXuKK1eu3OPR3pnG1CiEEH379hVpaWn3cGR3p7KyUvTr10+89dZborq6WhQVFYkpU6aIv/3tb/XaWuMcWlKfENY3f0IIUVhYKMLCwsT27duFwWAQ+fn5Yvjw4eKtt96q19Ya59CS+oSwzjms8+9//1uEhISI2bNnN7jfGufvRrerTwjrnb/XXntNzJkz57btWmoOW+UKys6dO/Hiiy9ixowZt20XFRWFhx56CEqlEsOGDUPv3r3xn//85x6N9M41tsbs7GyUlpaie/fu92hkdy8vLw8hISGYOnUq7Ozs4OnpiXHjxuHEiRP12lrjHFpSnzXOHwB4eXnh22+/xRNPPAGZTIaSkhJUV1fDy8urXltrnENL6rPWOQRqV4m++OILPPLIIzdtY43zV6cx9Vnz/P3888/o0aPHbdu11By2yoDSv39/7N+/33RV25s5e/ZsvUvvd+nSBWfOnGnO4TWJxtb4888/w9nZGTNmzEB0dDSGDx+Obdu23aNR3pnOnTtj7dq1UCgUpm379u1DaGhovbbWOIeW1GeN81fHxcUFADBo0CCMGDECPj4+eOKJJ+q1s8Y5BBpfn7XOYWFhIebOnYtly5bVu/Hrjax1/hpbn7XOn9FoxOnTp/HVV18hJiYGAwcOxLx581BaWlqvbUvNYasMKD4+PlAqb38j56a643JLaGyNOp0O4eHhmDFjBo4ePYo5c+Zg0aJF+Pzzz+/BKO+eEAJvvvkmvvzyS8ydO7fefmueQ+D29Vn7/AHAF198gSNHjkAul2P69On19lv7HN6uPmucQ6PRiFmzZmHSpEkICQm5ZVtrnD9L6rPG+QNqz0/s3r07hgwZgs8++wwffvghLl682OA5KC01h7f/C9aKtYY7Lo8cORIjR440Pe7fvz9GjhyJzz//HEOHDm25gTWCVqvFP/7xD5w+fRqbN29G165d67Wx5jlsTH3WPH91HBwc4ODggFmzZmHMmDEoLS2Fu7u7ab81zyFw+/qscQ5Xr14NOzs7TJw48bZtrXH+LKnPGucPANRqNVJTU02PHR0dMWvWLIwdOxZarda0Ali3ryXmsFWuoDRWcHCw2R2XgdqlrqCgoBYaUdPbtm1bvaSv0+lgb2/fQiNqnKysLIwaNQparRbbtm1r8I83YL1z2Nj6rHX+Tp06hUcffdTsjuc6nQ4qlarev9SscQ4tqc8a5/CTTz5Beno6oqKiEBUVhd27d2P37t0N3uzVGufPkvqscf6A2k8KLl26FOKGu93odDrI5XLY2dmZtW2xOWzWU3CtwK0+4XL27FkRFhYm9uzZYzpzOSwsTJw/f/4ej/Lu3KrG9957TzzwwAPi9OnTwmAwiC+//FL07NlTnDhx4h6PsvFKSkrE4MGDxZw5c4TBYLhlW2ucQ0vqs8b5E0IIrVYrBg0aJBYvXiyqq6tFTk6OGD16tHjllVfqtbXGObSkPmudwxvNnj37pp9yscb5+6Nb1Wet83f58mURHh4u1qxZI/R6vcjNzRVjx44V//znP+u1bak5ZED5wx/v8PBw8cknn5geHzlyRCQkJIjw8HARHx8vvvrqq5YY5l25VY1Go1GsWLFCxMTEiJ49e4r4+Hjx+eeft9RQG2X9+vUiODhY3H///SI8PNzsSwjrn0NL6rPG+auTmZkpJk2aJKKiokRMTIz417/+Jaqrq4UQ1j+HQjS+Pmuewzp//ANuC/N3o1vVZ83zl5aWJsaNGyciIiJEdHS0SElJEVVVVUIIacwh72ZMREREksNzUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCIiIhIchhQiIiISHIYUIiIiEhyGFCICACQk5ODrl27Iicnp8H9n376KeLj42/683PmzMGcOXNuuj82NhY7duy463Hezttvv92oe6hIyfz58zF//vyWHgaRpPBmgUTUKAkJCUhISGjpYdikhQsXtvQQiCSHKyhETaxuJWLTpk3o168fIiMjMWvWLGi1Wuh0Orz++usYOnQoIiIi8MADDyAlJcV0w66JEydizpw5iImJweDBg6HVanHo0CE8+eSTeOCBB3D//fdjwoQJuHjxIgBgx44dGD9+PF5//XX06dMH0dHR2LRpE7Zu3YqYmBhERkZa/C/zXbt2YejQoQgPD8czzzyD/Px803PFxsaa2h08eBDx8fEIDw/H3/72NxQXF5v2CSGwatUq9O/fH1FRUXj99ddhMBhM+3U6Hd566y3ExcWhT58+mDJlCi5dumTaX/f6DRkyBBEREXjyySfx22+/WTwXQgisWbMGI0aMQFRUFHr37o0XXngBVVVVyM/PR/fu3XHq1ClTe41Gg9DQUGRlZUEIgffffx9DhgxBVFQUxo8fj//+97+mtrGxsZg/fz769euHkSNHwmg03nIs+fn5mDx5Mvr06YOBAwciKSkJV69eBWC++pSQkICIiAjTV48ePdC9e3fodLrbjmnfvn2Ij49HZGQkhg4dipUrV1r8mhFJRrNfTJ+olcnOzhbBwcFiwoQJorCwUFy9elWMGTNGvPjii2LNmjUiPj5e5OfnCyGEOHXqlOjevbv49ttvhRBCTJgwQQwYMEBcuXJFlJaWisuXL4sePXqIgwcPCiGEKCoqEuPHjxcvvviiEEKI7du3i+DgYPHee+8Jg8EgUlNTRbdu3cTMmTNFRUWF+Omnn0S3bt1Eenp6o8c9depUUVZWJkpKSsTIkSPFvHnzTM8VExMjhBDi3LlzIjQ0VHzyySdCr9eL/fv3i27dupnuV/LRRx+J6Oho8d///ldUV1eLZcuWieDgYLF9+3YhhBCvvfaaGDlypMjKyhJVVVXi7bffFrGxsab7gAQHB4tx48aJq1evirKyMvHMM8+Iv/zlL416/ZcvXy4mTJgghBBiz549ol+/fuLChQtCiNqbnvXp00ds3bpVCCHE5MmTxfz5800/u27dOvHUU08JIYTYvHmzGDx4sPj111+FTqcTH330kYiKihIFBQVCCCFiYmLEY489JkpLS0Vpaeltx/XSSy+JuXPnCp1OJ65duyYmTZokUlJShBA3vxlddna26Nevn9i8efNtx1RZWSnCwsJM9906ffq0CA8PFz/++GOjXjciqeEKClEz+cc//gEvLy/4+Phg+vTp2Lt3Lx5//HFs2LABPj4+uHr1KqqqquDs7GxapQCAgQMHwtfXF25ubvDy8sKePXsQGxsLrVaLK1euwNPT06y9k5MTnn76acjlcvTv3x8GgwHPPvssHB0dERYWhjZt2iA3N7fR4/773/8OV1dXuLu7Y8CAAcjKyqrX5rPPPkOPHj2QkJAApVKJhx56CDExMab9n3zyCcaOHYvQ0FDY2dnh+eefh6enJ4DaVY0PP/wQM2fOhL+/P+zt7TF16lTo9Xp89dVXpj4mTpwIHx8fuLq6YujQoaZVI0sMHDgQ27ZtQ0BAAIqKilBcXAwPDw/T6zdq1Cjs3bsXOp0OALBz506MGjUKAJCamoq//e1vCAkJgUqlwujRoxEYGIhPP/3U1P+QIUPg5uYGNze3247F3t4e3333Hfbs2YPy8nKsXbsWL7/88k3bFxcXY/LkyRg+fDieeuqpRo3JwcEB27Ztw7FjxxAYGIjvvvsOPXv2tPh1I5ICnoNC1Ew6depk+r5t27bQ6XTQ6/VISUnBiRMn4Ofnh+7du0MIYXZ4oE2bNqbvVSoVdu/ejQ8//BAymQzBwcHQarVQKn//X9fDwwMymQwAIJfX/pvjxj+Ycrn8tocfbuTh4WH2/DcemqmTn5+Pdu3amW3r2LGj6TDP1atX0bZtW9M+hUJhal9UVISKigo8//zzpvECgF6vNwtSarXa9L1SqTQdBrOEEAJvvvkmvvzyS3h5eaFbt27Q6/WmvmJjY/HKK6/g8OHDaNeuHXJzczFkyBAAQG5uLl5//XUsXbrU1F9NTQ169OhhenzjXN3Oyy+/jNWrV2PdunWYM2cOQkJC8PLLLyMqKqpe26qqKjz33HMICgrCSy+9ZNp+qzE5ODhgy5YtWLlyJV544QVotVoMGTIEL7/8Mtzd3Rv/ohFJBAMKUTPJz89H586dAdSel+Lo6Ih58+bB3d0dX3/9Nezt7WE0GtG7d2+zn6sLGwDw+eefY/PmzdiyZYsp8KSkpCAjI6PB9veKn5+f2WoHAFy5cgX29vam/dnZ2aZ9QgjT+Raenp6wt7fH+vXrER4ebmpz/vx5+Pr6Nuk4ly5diry8PBw6dAguLi4AgBEjRpj229nZYcSIEdizZw/atWuHoUOHwsnJyVTD9OnTzT65lJWVZRbgLHntf/nlF4wbNw7Tpk1DUVERVqxYgaSkJBw/ftysndFoxAsvvACj0YglS5aYhbhbjUmr1eLq1atYtmwZAODXX3/FzJkzsWrVKsyePbvR4ySSCh7iIWomy5Ytg1arRX5+PpYvX47HHnsMWq0W9vb2kMvl0Gq1eOONN6DVaqHX6xvs49q1a5DL5XBwcIAQAkeOHMHHH3980/b3SkJCAjIyMrB161bU1NTg66+/xv79+037x4wZg61bt+L777+HXq/HO++8g4KCAgC1KzqjR4/GsmXLcOXKFRiNRuzcuRPDhw83O1G2KdS93gqFAtXV1Vi/fj0yMjLMXr/Ro0fj6NGj2L9/P5544gnT9rFjx+Kdd97BuXPnAABHjx5FfHw8Tpw4cUdjWbVqFVJSUqDVauHm5gZHR0fTYa8bpaSk4OzZs1i1ahUcHBzM9t1qTOXl5ZgyZQp27doFIQTatGkDuVze4HMQWQOuoBA1k44dO2L48OGorKzEiBEjMGvWLGRmZmL+/Pno06cPnJ2dMXjwYAwYMMBsReRGjz/+OL777jvEx8dDoVCgc+fOePrpp5Gammo6b6Il+Pv7Y9WqVXjttdewaNEihIaG4uGHHzbtHz58OIqLizFjxgyUlpbi0UcfRdeuXU37Z8+ejbfffhvjx49HSUkJ/P39sXz5cnTv3r1Jx5mcnIx//OMfePDBB+Hk5ITIyEg89thjZq93SEgIOnbsiIqKCkRGRpq2P/PMMxBCIDExEVevXoWvry/mz5+PuLi4OxrLwoULsWDBAsTFxUGn06FHjx546623zNrk5eXhgw8+gIuLC4YMGYKamhrTvnffffe2Y1q+fDn+/e9/Y/78+XBwcMCwYcPwzDPP3NF4iVqaTNzJgV0iuqmcnBzExcXh4MGD6NChQ0sPhxohKSkJPXv2xF//+teWHgoRXcdDPETUamVnZ2P//v349ttvzQ7vEFHL4yEeolaib9++tzwsVHeiqJTt27fvlpfTj4yMxNq1axvd3//93//h4MGD+Oc//2n2qaGWHhcR8RAPERERSRAP8RAREZHkMKAQERGR5DCgEBERkeQwoBAREZHkMKAQERGR5DCgEBERkeQwoBAREZHkMKAQERGR5DCgEBERkeT8P87MX6uKdpWhAAAAAElFTkSuQmCC"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(gridSearch.cv_results_)[display].plot(x = 'param_hidden_layer_sizes',\n",
    "                                                   y = 'mean_test_score', yerr = 'std_test_score', ylim = (0.8, 0.9))\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}