Lab02_part1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deep Information Retrieval and Neural Word Embeddings (Part 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task:** Classify text data into different emotions using word embedding and other deep information retrieval approaches.\n",
    "\n",
    "**Github Repo:** [LINK](https://github.com/EvanYu800112/dm_2018Fall_Lab2)\n",
    "\n",
    "**Dataset: ** [SemEval 2017 Task](https://competitions.codalab.org/competitions/16380)\n",
    "  \n",
    "---\n",
    "\n",
    "### Outline:\n",
    "1. Data preparation\n",
    "2. Feature engineering\n",
    "3. Model\n",
    "4. Results evaluation\n",
    "5. Your show time\n",
    "6. Other things you could try\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### Task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Classify text into one of 4 emotions **\n",
    "  \n",
    "![Imgur](https://i.imgur.com/k6tFirz.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 1. Data Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1-1 Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "### training data\n",
    "anger_train = pd.read_csv(\"data/semeval/train/anger-ratings-0to1.train.txt\",\n",
    "                         sep=\"\\t\", header=None,names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "sadness_train = pd.read_csv(\"data/semeval/train/sadness-ratings-0to1.train.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "fear_train = pd.read_csv(\"data/semeval/train/fear-ratings-0to1.train.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "joy_train = pd.read_csv(\"data/semeval/train/joy-ratings-0to1.train.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# combine 4 sub-dataset\n",
    "train_df = pd.concat([anger_train, fear_train, joy_train, sadness_train], ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "### testing data\n",
    "anger_test = pd.read_csv(\"data/semeval/dev/anger-ratings-0to1.dev.gold.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "sadness_test = pd.read_csv(\"data/semeval/dev/sadness-ratings-0to1.dev.gold.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "fear_test = pd.read_csv(\"data/semeval/dev/fear-ratings-0to1.dev.gold.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "joy_test = pd.read_csv(\"data/semeval/dev/joy-ratings-0to1.dev.gold.txt\",\n",
    "                         sep=\"\\t\", header=None, names=[\"id\", \"text\", \"emotion\", \"intensity\"])\n",
    "\n",
    "# combine 4 sub-dataset\n",
    "test_df = pd.concat([anger_test, fear_test, joy_test, sadness_test], ignore_index=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape of \"training data\": (3613, 4)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>emotion</th>\n",
       "      <th>intensity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10000</td>\n",
       "      <td>How the fu*k! Who the heck! moved my fridge!.....</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10001</td>\n",
       "      <td>So my Indian Uber driver just called someone t...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10002</td>\n",
       "      <td>@DPD_UK I asked for my parcel to be delivered ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10003</td>\n",
       "      <td>so ef whichever butt wipe pulled the fire alar...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10004</td>\n",
       "      <td>Don't join @BTCare they put the phone down on ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10005</td>\n",
       "      <td>My blood is boiling</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10006</td>\n",
       "      <td>When you've still got a whole season of Wentwo...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10007</td>\n",
       "      <td>@bt_uk why does tracking show my equipment del...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10008</td>\n",
       "      <td>@TeamShanny legit why i am so furious with him...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10009</td>\n",
       "      <td>How is it suppose to work if you do that? Wtf ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10010</td>\n",
       "      <td>im so mad about power rangers. im incensed. im...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10011</td>\n",
       "      <td>Wont use using @mothercareuk @Mothercarehelp a...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10012</td>\n",
       "      <td>Bitches aggravate like what inspires you to be...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10013</td>\n",
       "      <td>Why does @dapperlaughs have to come to Glasgow...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>10014</td>\n",
       "      <td>Fuking fuming 😤</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>10015</td>\n",
       "      <td>Zero help from @ups customer service. Just pus...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>10016</td>\n",
       "      <td>@ArizonaCoyotes not to mention the GRA guy sto...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>10017</td>\n",
       "      <td>I hate my lawn mower. If it had a soul, I'd co...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>10018</td>\n",
       "      <td>why are people so offended by kendall he ends ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>10019</td>\n",
       "      <td>I'm about to block everyone everywhere posting...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>10020</td>\n",
       "      <td>Making my buddy cry !! Bitch wait for revenge 🤗👌🏻</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>10021</td>\n",
       "      <td>Bes! You don't just tell a true blooded hoopju...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.891</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>10022</td>\n",
       "      <td>I got a NO RESPONSE again from an #irate custo...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>10023</td>\n",
       "      <td>Tasers immobilize, if you taser someone why th...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>10024</td>\n",
       "      <td>@LeKingCoq @MistaAggy @GiovanniiDC The way my ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>10025</td>\n",
       "      <td>Actually fuming I have nothing to wear Saturday</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>10026</td>\n",
       "      <td>.@DIVAmagazine than straight people. Even the ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>10027</td>\n",
       "      <td>Bloody parking ticket 😒💸 #fuming</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>10028</td>\n",
       "      <td>You ever just find that the people around you ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>10029</td>\n",
       "      <td>I get so angry at people that don't know that ...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3583</th>\n",
       "      <td>40756</td>\n",
       "      <td>Guys! Read my #fanfiction if you like #lost an...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3584</th>\n",
       "      <td>40757</td>\n",
       "      <td>@m_giacchino will this be aired on radio or fi...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3585</th>\n",
       "      <td>40758</td>\n",
       "      <td>Very long day. Thank goodness for Bake Off to ...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3586</th>\n",
       "      <td>40759</td>\n",
       "      <td>TONIGHT - Fulford Arms, York \\nTOMORROW - Bank...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3587</th>\n",
       "      <td>40760</td>\n",
       "      <td>I'm so proud he understands this, he shows lov...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3588</th>\n",
       "      <td>40761</td>\n",
       "      <td>blues bar</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3589</th>\n",
       "      <td>40762</td>\n",
       "      <td>@GOP she's ahead in the polls &amp;amp; after Trum...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3590</th>\n",
       "      <td>40763</td>\n",
       "      <td>@urbaneprofessor roast them. Then risotto with...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3591</th>\n",
       "      <td>40764</td>\n",
       "      <td>Happiness is always there. You just have to ch...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3592</th>\n",
       "      <td>40765</td>\n",
       "      <td>Good morning chirpy #SpringEquinox and your pe...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3593</th>\n",
       "      <td>40766</td>\n",
       "      <td>also who has amazon prime that would like to h...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3594</th>\n",
       "      <td>40767</td>\n",
       "      <td>I added Paul Walker on Xbox but he just spends...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3595</th>\n",
       "      <td>40768</td>\n",
       "      <td>Nice to see Balotelli back to his best, good p...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3596</th>\n",
       "      <td>40769</td>\n",
       "      <td>@KatieBurnett ha freshers blues would love to ...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3597</th>\n",
       "      <td>40770</td>\n",
       "      <td>What do you get when you cross an #apple tree ...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3598</th>\n",
       "      <td>40771</td>\n",
       "      <td>Please bear with me, I'm not Twitter savvy😝 in...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3599</th>\n",
       "      <td>40772</td>\n",
       "      <td>@RiRiMathew even when we're fighting I'm laugh...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3600</th>\n",
       "      <td>40773</td>\n",
       "      <td>@SquidWasTaken Pfft- Yeye, i'm a sucker for br...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3601</th>\n",
       "      <td>40774</td>\n",
       "      <td>Never a dull moment when talking to Nell 😂😂😂😂😋</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3602</th>\n",
       "      <td>40775</td>\n",
       "      <td>Never dull moment here</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3603</th>\n",
       "      <td>40776</td>\n",
       "      <td>#picoftheday : How...why... Really... !!\\n    ...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3604</th>\n",
       "      <td>40777</td>\n",
       "      <td>@EBled2 Great to have you as our tournament ch...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3605</th>\n",
       "      <td>40778</td>\n",
       "      <td>@DarbyHogle the red one would look super prett...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3606</th>\n",
       "      <td>40779</td>\n",
       "      <td>@PottzGaming hmu need players for lans next ye...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3607</th>\n",
       "      <td>40780</td>\n",
       "      <td>#FF @ElizaNealsRocks Love &amp;amp; support, alway...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3608</th>\n",
       "      <td>40781</td>\n",
       "      <td>@VivienLloyd Thank you so much! Just home - st...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3609</th>\n",
       "      <td>40782</td>\n",
       "      <td>Just put the winter duvet on ☃️❄️🌬☔️</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3610</th>\n",
       "      <td>40783</td>\n",
       "      <td>@SilkInSide @TommyJoeRatliff that's so pretty!...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3611</th>\n",
       "      <td>40784</td>\n",
       "      <td>@BluesfestByron second artist announcement loo...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3612</th>\n",
       "      <td>40785</td>\n",
       "      <td>I can literally eat creamy pesto pasta topped ...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.083</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3613 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                               text  emotion  \\\n",
       "0     10000  How the fu*k! Who the heck! moved my fridge!.....    anger   \n",
       "1     10001  So my Indian Uber driver just called someone t...    anger   \n",
       "2     10002  @DPD_UK I asked for my parcel to be delivered ...    anger   \n",
       "3     10003  so ef whichever butt wipe pulled the fire alar...    anger   \n",
       "4     10004  Don't join @BTCare they put the phone down on ...    anger   \n",
       "5     10005                                My blood is boiling    anger   \n",
       "6     10006  When you've still got a whole season of Wentwo...    anger   \n",
       "7     10007  @bt_uk why does tracking show my equipment del...    anger   \n",
       "8     10008  @TeamShanny legit why i am so furious with him...    anger   \n",
       "9     10009  How is it suppose to work if you do that? Wtf ...    anger   \n",
       "10    10010  im so mad about power rangers. im incensed. im...    anger   \n",
       "11    10011  Wont use using @mothercareuk @Mothercarehelp a...    anger   \n",
       "12    10012  Bitches aggravate like what inspires you to be...    anger   \n",
       "13    10013  Why does @dapperlaughs have to come to Glasgow...    anger   \n",
       "14    10014                                    Fuking fuming 😤    anger   \n",
       "15    10015  Zero help from @ups customer service. Just pus...    anger   \n",
       "16    10016  @ArizonaCoyotes not to mention the GRA guy sto...    anger   \n",
       "17    10017  I hate my lawn mower. If it had a soul, I'd co...    anger   \n",
       "18    10018  why are people so offended by kendall he ends ...    anger   \n",
       "19    10019  I'm about to block everyone everywhere posting...    anger   \n",
       "20    10020  Making my buddy cry !! Bitch wait for revenge 🤗👌🏻    anger   \n",
       "21    10021  Bes! You don't just tell a true blooded hoopju...    anger   \n",
       "22    10022  I got a NO RESPONSE again from an #irate custo...    anger   \n",
       "23    10023  Tasers immobilize, if you taser someone why th...    anger   \n",
       "24    10024  @LeKingCoq @MistaAggy @GiovanniiDC The way my ...    anger   \n",
       "25    10025    Actually fuming I have nothing to wear Saturday    anger   \n",
       "26    10026  .@DIVAmagazine than straight people. Even the ...    anger   \n",
       "27    10027                   Bloody parking ticket 😒💸 #fuming    anger   \n",
       "28    10028  You ever just find that the people around you ...    anger   \n",
       "29    10029  I get so angry at people that don't know that ...    anger   \n",
       "...     ...                                                ...      ...   \n",
       "3583  40756  Guys! Read my #fanfiction if you like #lost an...  sadness   \n",
       "3584  40757  @m_giacchino will this be aired on radio or fi...  sadness   \n",
       "3585  40758  Very long day. Thank goodness for Bake Off to ...  sadness   \n",
       "3586  40759  TONIGHT - Fulford Arms, York \\nTOMORROW - Bank...  sadness   \n",
       "3587  40760  I'm so proud he understands this, he shows lov...  sadness   \n",
       "3588  40761                                          blues bar  sadness   \n",
       "3589  40762  @GOP she's ahead in the polls &amp; after Trum...  sadness   \n",
       "3590  40763  @urbaneprofessor roast them. Then risotto with...  sadness   \n",
       "3591  40764  Happiness is always there. You just have to ch...  sadness   \n",
       "3592  40765  Good morning chirpy #SpringEquinox and your pe...  sadness   \n",
       "3593  40766  also who has amazon prime that would like to h...  sadness   \n",
       "3594  40767  I added Paul Walker on Xbox but he just spends...  sadness   \n",
       "3595  40768  Nice to see Balotelli back to his best, good p...  sadness   \n",
       "3596  40769  @KatieBurnett ha freshers blues would love to ...  sadness   \n",
       "3597  40770  What do you get when you cross an #apple tree ...  sadness   \n",
       "3598  40771  Please bear with me, I'm not Twitter savvy😝 in...  sadness   \n",
       "3599  40772  @RiRiMathew even when we're fighting I'm laugh...  sadness   \n",
       "3600  40773  @SquidWasTaken Pfft- Yeye, i'm a sucker for br...  sadness   \n",
       "3601  40774     Never a dull moment when talking to Nell 😂😂😂😂😋  sadness   \n",
       "3602  40775                             Never dull moment here  sadness   \n",
       "3603  40776  #picoftheday : How...why... Really... !!\\n    ...  sadness   \n",
       "3604  40777  @EBled2 Great to have you as our tournament ch...  sadness   \n",
       "3605  40778  @DarbyHogle the red one would look super prett...  sadness   \n",
       "3606  40779  @PottzGaming hmu need players for lans next ye...  sadness   \n",
       "3607  40780  #FF @ElizaNealsRocks Love &amp; support, alway...  sadness   \n",
       "3608  40781  @VivienLloyd Thank you so much! Just home - st...  sadness   \n",
       "3609  40782              Just put the winter duvet on ☃️❄️🌬☔️   sadness   \n",
       "3610  40783  @SilkInSide @TommyJoeRatliff that's so pretty!...  sadness   \n",
       "3611  40784  @BluesfestByron second artist announcement loo...  sadness   \n",
       "3612  40785  I can literally eat creamy pesto pasta topped ...  sadness   \n",
       "\n",
       "      intensity  \n",
       "0         0.938  \n",
       "1         0.896  \n",
       "2         0.896  \n",
       "3         0.896  \n",
       "4         0.896  \n",
       "5         0.875  \n",
       "6         0.875  \n",
       "7         0.875  \n",
       "8         0.875  \n",
       "9         0.875  \n",
       "10        0.667  \n",
       "11        0.854  \n",
       "12        0.854  \n",
       "13        0.938  \n",
       "14        0.854  \n",
       "15        0.854  \n",
       "16        0.854  \n",
       "17        0.833  \n",
       "18        0.833  \n",
       "19        0.812  \n",
       "20        0.812  \n",
       "21        0.891  \n",
       "22        0.812  \n",
       "23        0.812  \n",
       "24        0.792  \n",
       "25        0.792  \n",
       "26        0.792  \n",
       "27        0.792  \n",
       "28        0.792  \n",
       "29        0.792  \n",
       "...         ...  \n",
       "3583      0.188  \n",
       "3584      0.188  \n",
       "3585      0.188  \n",
       "3586      0.188  \n",
       "3587      0.188  \n",
       "3588      0.188  \n",
       "3589      0.174  \n",
       "3590      0.167  \n",
       "3591      0.167  \n",
       "3592      0.167  \n",
       "3593      0.167  \n",
       "3594      0.167  \n",
       "3595      0.167  \n",
       "3596      0.167  \n",
       "3597      0.146  \n",
       "3598      0.146  \n",
       "3599      0.146  \n",
       "3600      0.146  \n",
       "3601      0.146  \n",
       "3602      0.146  \n",
       "3603      0.146  \n",
       "3604      0.132  \n",
       "3605      0.125  \n",
       "3606      0.125  \n",
       "3607      0.125  \n",
       "3608      0.104  \n",
       "3609      0.104  \n",
       "3610      0.088  \n",
       "3611      0.083  \n",
       "3612      0.083  \n",
       "\n",
       "[3613 rows x 4 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('shape of \"training data\":', train_df.shape)\n",
    "\n",
    "train_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# shuffle dataset\n",
    "train_df = train_df.sample(frac=1)\n",
    "test_df = test_df.sample(frac=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>emotion</th>\n",
       "      <th>intensity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1704</th>\n",
       "      <td>20847</td>\n",
       "      <td>@realDonaldTrump @KellyannePolls New campaign ...</td>\n",
       "      <td>fear</td>\n",
       "      <td>0.354</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1152</th>\n",
       "      <td>20295</td>\n",
       "      <td>We in our own country are so divided in our ap...</td>\n",
       "      <td>fear</td>\n",
       "      <td>0.625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1038</th>\n",
       "      <td>20181</td>\n",
       "      <td>@twlldun @bbcquestiontime @JacobReesMogg @leic...</td>\n",
       "      <td>fear</td>\n",
       "      <td>0.696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3497</th>\n",
       "      <td>40670</td>\n",
       "      <td>#PeopleLikeMeBecause of some unknown reason bu...</td>\n",
       "      <td>sadness</td>\n",
       "      <td>0.278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1940</th>\n",
       "      <td>21083</td>\n",
       "      <td>@BigDaddyx47 awe I'm sorry! I have 2 dogs 3 ca...</td>\n",
       "      <td>fear</td>\n",
       "      <td>0.188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>217</th>\n",
       "      <td>10217</td>\n",
       "      <td>@MannyM83 @DareToReagan oh yeah. I HATE the ai...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.468</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2420</th>\n",
       "      <td>30416</td>\n",
       "      <td>@ashleynwinters I don't like the statement but...</td>\n",
       "      <td>joy</td>\n",
       "      <td>0.480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2155</th>\n",
       "      <td>30151</td>\n",
       "      <td>#ContentwiththeLordsPortion:O satisfy us early...</td>\n",
       "      <td>joy</td>\n",
       "      <td>0.688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>318</th>\n",
       "      <td>10318</td>\n",
       "      <td>I'm not saying he might be like big foot but i...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>394</th>\n",
       "      <td>10394</td>\n",
       "      <td>🔥Anger is the acid that can do more harm to th...</td>\n",
       "      <td>anger</td>\n",
       "      <td>0.500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                               text  emotion  \\\n",
       "1704  20847  @realDonaldTrump @KellyannePolls New campaign ...     fear   \n",
       "1152  20295  We in our own country are so divided in our ap...     fear   \n",
       "1038  20181  @twlldun @bbcquestiontime @JacobReesMogg @leic...     fear   \n",
       "3497  40670  #PeopleLikeMeBecause of some unknown reason bu...  sadness   \n",
       "1940  21083  @BigDaddyx47 awe I'm sorry! I have 2 dogs 3 ca...     fear   \n",
       "217   10217  @MannyM83 @DareToReagan oh yeah. I HATE the ai...    anger   \n",
       "2420  30416  @ashleynwinters I don't like the statement but...      joy   \n",
       "2155  30151  #ContentwiththeLordsPortion:O satisfy us early...      joy   \n",
       "318   10318  I'm not saying he might be like big foot but i...    anger   \n",
       "394   10394  🔥Anger is the acid that can do more harm to th...    anger   \n",
       "\n",
       "      intensity  \n",
       "1704      0.354  \n",
       "1152      0.625  \n",
       "1038      0.696  \n",
       "3497      0.278  \n",
       "1940      0.188  \n",
       "217       0.468  \n",
       "2420      0.480  \n",
       "2155      0.688  \n",
       "318       0.542  \n",
       "394       0.500  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "![Imgur](https://i.imgur.com/lXBIX1Y.png)\n",
    "Plot word frequency for Top 30 words in both train and test dataset. (Hint: refer to DM lab 1)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 1-2 Save data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Imgur](https://i.imgur.com/DMeWZY5.png)\n",
    "The pickle module implements binary protocols for serializing and de-serializing a Python object structure.   \n",
    "  \n",
    "Some advantages for using pickle structure:  \n",
    "* Because it stores the attribute type, it's more convenient for cross-platform use.  \n",
    "* When your data is huge, it could use less space to store also consume less loading time.   \n",
    "\n",
    " e.g.\n",
    "![Imgur](https://i.imgur.com/jXNE6e7.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "## save to pickle file\n",
    "train_df.to_pickle(\"train_df.pkl\") \n",
    "test_df.to_pickle(\"test_df.pkl\")\n",
    "\n",
    "## load a pickle file\n",
    "train_df = pd.read_pickle(\"train_df.pkl\")\n",
    "test_df = pd.read_pickle(\"test_df.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more information: https://reurl.cc/0Dzqx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 1-3 Exploratory data analysis (EDA)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x10a15bb38>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEhpJREFUeJzt3X2wXVV9xvHvUyJQcSZBuGVsgg3VtIq1HfEW8aWWiqWgtmHqG4ytAZlmbLFW6VRj7Yh9V3GKdaq2UaihUvBdMpaKmSi1LwZNUEkgKndQTDK8XBVSkVJFf/3jrNRjJNzknpt7Etb3M3Pm7L32Omuvs7Pvec5ee++TVBWSpP782Lg7IEkaDwNAkjplAEhSpwwASeqUASBJnTIAJKlTMwZAkkuS3JFky1DZhUm+mOT6JB9Osmho2WuSTCX5UpJfGyo/rZVNJVk1929FkrQv9uYI4N3AabuVrQN+rqp+Hvgy8BqAJMcDZwKPa695e5JDkhwCvA04HTgeOKvVlSSNyYwBUFWfAr65W9nHq+q+NrsBWNKmlwNXVNX/VtVXgCngxPaYqqqbq+o7wBWtriRpTBbMQRsvAd7bphczCIRdtrcygG27lT9ppoaPPvroWrp06Rx0UZL6sWnTpq9X1cRM9UYKgCSvBe4DLhulnd3aXAmsBHjkIx/Jxo0b56ppSepCklv2pt6srwJKcjbwHOBF9YMfFNoBHDtUbUkr21P5j6iq1VU1WVWTExMzBpgkaZZmFQBJTgNeBfxGVd0ztGgtcGaSw5IcBywDPgN8FliW5LgkhzI4Ubx2tK5LkkYx4xBQksuBk4Gjk2wHLmBw1c9hwLokABuq6qVVdUOS9wE3MhgaOq+qvtfaeRlwNXAIcElV3bAf3o8kaS/lQP456MnJyfIcgCTtmySbqmpypnreCSxJnTIAJKlTBoAkdcoAkKROGQCS1Km5+CmIg8rSVf8y7i7sla++4dnj7oKkBzmPACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqdmDIAklyS5I8mWobKHJ1mX5Kb2fGQrT5K3JplKcn2SE4Zes6LVvynJiv3zdiRJe2tvjgDeDZy2W9kqYH1VLQPWt3mA04Fl7bESeAcMAgO4AHgScCJwwa7QkCSNx4wBUFWfAr65W/FyYE2bXgOcMVR+aQ1sABYleQTwa8C6qvpmVd0JrONHQ0WSNI9mew7gmKq6tU3fBhzTphcD24bqbW9leyr/EUlWJtmYZOP09PQsuydJmsnIJ4GrqoCag77sam91VU1W1eTExMRcNStJ2s1sA+D2NrRDe76jle8Ajh2qt6SV7alckjQmsw2AtcCuK3lWAFcOlb+4XQ10ErCzDRVdDZya5Mh28vfUViZJGpMFM1VIcjlwMnB0ku0MruZ5A/C+JOcCtwAvaNWvAp4FTAH3AOcAVNU3k/w58NlW78+qavcTy5KkeTRjAFTVWXtYdMr91C3gvD20cwlwyT71TpK033gnsCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOjXjfwovPaDXLxx3D/bO63eOuwfSAccjAEnqlAEgSZ0yACSpUwaAJHXKAJCkTnkVkHQAefyax4+7C3tl84rN4+6C5sBIRwBJXpnkhiRbklye5PAkxyW5NslUkvcmObTVPazNT7XlS+fiDUiSZmfWAZBkMfByYLKqfg44BDgTeCNwUVU9GrgTOLe95FzgzlZ+UasnSRqTUc8BLAB+PMkC4KHArcAzgA+05WuAM9r08jZPW35Kkoy4fknSLM06AKpqB/Bm4GsMPvh3ApuAu6rqvlZtO7C4TS8GtrXX3tfqH7V7u0lWJtmYZOP09PRsuydJmsEoQ0BHMvhWfxzwk8ARwGmjdqiqVlfVZFVNTkxMjNqcJGkPRhkCeibwlaqarqrvAh8CngosakNCAEuAHW16B3AsQFu+EPjGCOuXJI1glMtAvwaclOShwP8ApwAbgU8CzwOuAFYAV7b6a9v8p9vyT1RVjbB+SdqjrY957Li7sFce+8WtY1v3KOcArmVwMvc6YHNrazXwauD8JFMMxvgvbi+5GDiqlZ8PrBqh35KkEY10I1hVXQBcsFvxzcCJ91P3XuD5o6xPkjR3/CkISeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTIwVAkkVJPpDki0m2JnlykocnWZfkpvZ8ZKubJG9NMpXk+iQnzM1bkCTNxqhHAH8LfKyqHgP8ArAVWAWsr6plwPo2D3A6sKw9VgLvGHHdkqQRzDoAkiwEng5cDFBV36mqu4DlwJpWbQ1wRpteDlxaAxuARUkeMeueS5JGMsoRwHHANPCPST6X5F1JjgCOqapbW53bgGPa9GJg29Drt7eyH5JkZZKNSTZOT0+P0D1J0gMZJQAWACcA76iqJwDf5gfDPQBUVQG1L41W1eqqmqyqyYmJiRG6J0l6IKMEwHZge1Vd2+Y/wCAQbt81tNOe72jLdwDHDr1+SSuTJI3BrAOgqm4DtiX52VZ0CnAjsBZY0cpWAFe26bXAi9vVQCcBO4eGiiRJ82zBiK//feCyJIcCNwPnMAiV9yU5F7gFeEGrexXwLGAKuKfVlSSNyUgBUFWfBybvZ9Ep91O3gPNGWZ8kae54J7AkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1KmRAyDJIUk+l+Sjbf64JNcmmUry3iSHtvLD2vxUW7501HVLkmZvLo4A/gDYOjT/RuCiqno0cCdwbis/F7izlV/U6kmSxmSkAEiyBHg28K42H+AZwAdalTXAGW16eZunLT+l1ZckjcGoRwBvAV4FfL/NHwXcVVX3tfntwOI2vRjYBtCW72z1JUljMOsASPIc4I6q2jSH/SHJyiQbk2ycnp6ey6YlSUNGOQJ4KvAbSb4KXMFg6OdvgUVJFrQ6S4AdbXoHcCxAW74Q+MbujVbV6qqarKrJiYmJEbonSXogsw6AqnpNVS2pqqXAmcAnqupFwCeB57VqK4Ar2/TaNk9b/omqqtmuX5I0mv1xH8CrgfOTTDEY47+4lV8MHNXKzwdW7Yd1S5L20oKZq8ysqq4BrmnTNwMn3k+de4Hnz8X6JEmj805gSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTsw6AJMcm+WSSG5PckOQPWvnDk6xLclN7PrKVJ8lbk0wluT7JCXP1JiRJ+26UI4D7gD+squOBk4DzkhwPrALWV9UyYH2bBzgdWNYeK4F3jLBuSdKIZh0AVXVrVV3Xpr8FbAUWA8uBNa3aGuCMNr0cuLQGNgCLkjxi1j2XJI1kTs4BJFkKPAG4Fjimqm5ti24DjmnTi4FtQy/b3sokSWMwcgAkeRjwQeAVVfXfw8uqqoDax/ZWJtmYZOP09PSo3ZMk7cFIAZDkIQw+/C+rqg+14tt3De205zta+Q7g2KGXL2llP6SqVlfVZFVNTkxMjNI9SdIDGOUqoAAXA1ur6m+GFq0FVrTpFcCVQ+UvblcDnQTsHBoqkiTNswUjvPapwG8Dm5N8vpX9MfAG4H1JzgVuAV7Qll0FPAuYAu4Bzhlh3ZKkEc06AKrqP4DsYfEp91O/gPNmuz5J0tzyTmBJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVPzHgBJTkvypSRTSVbN9/olSQPzGgBJDgHeBpwOHA+cleT4+eyDJGlgvo8ATgSmqurmqvoOcAWwfJ77IEli/gNgMbBtaH57K5MkzbMF4+7A7pKsBFa22buTfGmc/dlLRwNfn8sG88a5bO2gM+fbkz/NnDZ3kJn7/fPsbrfn3O+b2S/b8qf2ptJ8B8AO4Nih+SWt7P9V1Wpg9Xx2alRJNlbV5Lj78WDh9pxbbs+582DblvM9BPRZYFmS45IcCpwJrJ3nPkiSmOcjgKq6L8nLgKuBQ4BLquqG+eyDJGlg3s8BVNVVwFXzvd797KAasjoIuD3nlttz7jyotmWqatx9kCSNgT8FIUmdMgD2QpKXJ9ma5LJx90XakyT/Ne4+PJgkWZpky7j7sT8dcPcBHKB+D3hmVW2fbQNJFlTVfXPYJw1JEgZDmt8fd1/GpaqeMu4+6ODiEcAMkvw98NPAvyZ5bZJLknwmyeeSLG91lib59yTXtcdTWvnJrXwtcOMY38bYJPlIkk1Jbmg3+ZHk7iR/meQLSTYkOaaVP6rNb07yF0nuHmrnj5J8Nsn1Sf60lS1tPyx4KbCFH77HpDttuybJhUm2tO34wrbs0iRnDNW9bNf++2CX5Igk/9L2ty1JXpjkdW1/2pJkdfsCQZIntnpfAM4bauPsJB9K8rEkNyV509CyU5N8uv3tvz/Jw1r5G5Lc2PbZN7ey57d1fiHJp+Z5U/yoqvIxwwP4KoM7AP8K+K1Wtgj4MnAE8FDg8Fa+DNjYpk8Gvg0cN+73MMZt9/D2/OMMPqSPAgr49Vb+JuBP2vRHgbPa9EuBu9v0qQyuvgiDLy0fBZ4OLAW+D5w07vd5IDyAu4HnAusYXGZ9DPA14BHALwMfafUWAl8BFoy7z/O0XZ4LvHNofuGu/bLN/9PQ/ng98PQ2fSGwpU2fDdzcXns4cAuDLxxHA58Cjmj1Xg28ru3nX+IHF9osas+bgcXDZeN8eASwb04FViX5PHANgx3hkcBDgHcm2Qy8n8Evne7ymar6ynx39ADy8vZtagODP5hlwHcYfIgDbGLwQQ7wZAbbD+Cfh9o4tT0+B1wHPKa1A3BLVW3YX50/CD0NuLyqvldVtwP/BvxiVf0bg5swJ4CzgA9WP0OSm4FfTfLGJL9UVTuBX0lybfubfQbwuCSLGHwo7/pm/k+7tbO+qnZW1b0Mjuh/CjiJwd/7f7bPhRWtfCdwL3Bxkt8E7mlt/Cfw7iS/wyCkx8pzAPsmwHOr6od+nyjJ64HbgV9g8A313qHF35633h1gkpwMPBN4clXdk+QaBqH53WpfgYDvMfN+GOCvq+ofdmt/KR1v31m4FPgtBnfgnzPmvsybqvpykhOAZwF/kWQ9g+Gdyara1v5+D9+Lpv53aHrXfhtgXVWdtXvlJCcCpwDPA14GPKOqXprkScCzgU1JnlhV3xjh7Y3EI4B9czXw+0PjhU9o5QuBW2twAvK3OQCS/QCxELizffg/hsG3pQeygcHhOgw+pHa5GnjJ0Njq4iQ/Mee9fXD4d+CFSQ5p3/afDnymLXs38AqAqurmnFSSnwTuqar3MBjWOaEt+nrbp54HUFV3AXcleVpb/qK9aH4D8NQkj27rOiLJz7R2F9bgxtdXMvhySJJHVdW1VfU6YJoxn7fyCGDf/DnwFuD6JD/GYBz1OcDbgQ8meTHwMfxWusvHgJcm2cpgPHSmoZpXAO9J8tr22p0AVfXxJI8FPt2y924G32S/t786fpAq4MMMhtK+0OZfVVW3AVTV7e3f4iPj6+JYPB64MMn3ge8CvwucweCc1G0MfqNsl3OAS5IU8PGZGq6q6SRnA5cnOawV/wnwLeDKJIczOEo4vy27MMmyVraewb/T2HgnsA4YSR4K/E9VVZIzGZwQ7uJKlVElOQq4rqr2+DPAbftuBk5o4+DqnEcAOpA8Efi7NsR2F/CSMffnoNCGOK4B3vwAdZ4JXAxc5Ie/dvEIQJI65UlgSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1Kn/A7n8FOhOypxlAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x109f87908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "train_df['emotion'].value_counts().plot(kind=\"bar\", rot=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "![Imgur](https://i.imgur.com/lXBIX1Y.png)\n",
    "Is this considered an imbalanced dataset? Try to perform some more data exploration and statistical inference as done in lab session 1. \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x108bb4588>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAD2NJREFUeJzt3XuwXWV9xvHvI5EiOE24nGYwWEM1LdI6jphSFEspUMZbS6ZShfESkGnG1rudKq2Ol6ltoThVO21to1CDUqqiFUYtyqSi1kr0BIQEopJBkTBcjq2kIrWK/PrHftMeQ0JO9t4nO3n5fmb27LXedfudddZ+zpt377WTqkKS1K9HTLoASdL8MuglqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnVsw6QIADjvssFq6dOmky5Ckfcr69eu/U1VTu1pvrwj6pUuXMj09PekyJGmfkuTWuazn0I0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHVur7gzdj4sPfeTky5hTr513nMmXYKkztmjl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0md22XQJ7koyd1JNs5qOyTJVUlubs8Ht/Yk+askm5PckOSY+SxekrRrc+nRvx945nZt5wJrq2oZsLbNAzwLWNYeq4D3jKdMSdKwdhn0VfV54D+3az4NWNOm1wArZrVfXAPXAIuSHD6uYiVJu2/YMfrFVXVHm74TWNymlwC3zVpvS2uTJE3IyG/GVlUBtbvbJVmVZDrJ9MzMzKhlSJJ2Ytigv2vbkEx7vru13w48dtZ6R7S2B6mq1VW1vKqWT01NDVmGJGlXhg36K4CVbXolcPms9pe0T98cB2ydNcQjSZqABbtaIcmlwInAYUm2AG8BzgM+nOQc4Fbg+W31TwHPBjYD9wFnz0PNkqTdsMugr6ozd7Lo5B2sW8DLRy1KkjQ+3hkrSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ1bMOkCtI9468JJVzA3b9066QqkvY49eknqnEEvSZ0z6CWpcyMFfZLXJrkxycYklyY5IMmRSdYl2ZzkQ0n2H1exkqTdN3TQJ1kCvApYXlW/BOwHnAGcD7yzqp4AfBc4ZxyFSpKGM+rQzQLgUUkWAAcCdwAnAZe15WuAFSMeQ5I0gqGDvqpuB94BfJtBwG8F1gP3VNX9bbUtwJIdbZ9kVZLpJNMzMzPDliFJ2oVRhm4OBk4DjgQeAxwEPHOu21fV6qpaXlXLp6amhi1DkrQLo9wwdQrwzaqaAUjyMeB4YFGSBa1XfwRw++hlSv140ponTbqEOdmwcsOkS9CYjDJG/23guCQHJglwMnAT8Fng9LbOSuDy0UqUJI1ilDH6dQzedL0W2ND2tRp4A/C6JJuBQ4ELx1CnJGlII33XTVW9BXjLds23AMeOsl9J0vh4Z6wkdc6gl6TOGfSS1DmDXpI6Z9BLUuf8H6Yk7dM2HfXESZcwJ0/82qaJHdsevSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6txIQZ9kUZLLknwtyaYkT0tySJKrktzcng8eV7GSpN03ao/+3cCVVXUU8GRgE3AusLaqlgFr27wkaUKGDvokC4ETgAsBquqHVXUPcBqwpq22BlgxapGSpOGN0qM/EpgB/iHJdUnel+QgYHFV3dHWuRNYPGqRkqThjRL0C4BjgPdU1VOA77PdME1VFVA72jjJqiTTSaZnZmZGKEOS9FBGCfotwJaqWtfmL2MQ/HclORygPd+9o42ranVVLa+q5VNTUyOUIUl6KEMHfVXdCdyW5Bda08nATcAVwMrWthK4fKQKJUkjWTDi9q8ELkmyP3ALcDaDPx4fTnIOcCvw/BGPIUkawUhBX1VfBZbvYNHJo+xXkjQ+3hkrSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktS5kYM+yX5JrkvyiTZ/ZJJ1STYn+VCS/UcvU5I0rHH06F8NbJo1fz7wzqp6AvBd4JwxHEOSNKSRgj7JEcBzgPe1+QAnAZe1VdYAK0Y5hiRpNKP26N8FvB54oM0fCtxTVfe3+S3Akh1tmGRVkukk0zMzMyOWIUnamaGDPslzgburav0w21fV6qpaXlXLp6amhi1DkrQLC0bY9njgt5I8GzgA+Gng3cCiJAtar/4I4PbRy5QkDWvoHn1V/VFVHVFVS4EzgH+tqhcCnwVOb6utBC4fuUpJ0tDm43P0bwBel2QzgzH7C+fhGJKkORpl6Ob/VNXVwNVt+hbg2HHsV5I0Ou+MlaTOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUueGDvokj03y2SQ3Jbkxyatb+yFJrkpyc3s+eHzlSpJ21yg9+vuBP6iqo4HjgJcnORo4F1hbVcuAtW1ekjQhQwd9Vd1RVde26e8Bm4AlwGnAmrbaGmDFqEVKkoY3ljH6JEuBpwDrgMVVdUdbdCeweBzHkCQNZ+SgT/Jo4KPAa6rqv2Yvq6oCaifbrUoynWR6ZmZm1DIkSTsxUtAneSSDkL+kqj7Wmu9Kcnhbfjhw9462rarVVbW8qpZPTU2NUoYk6SGM8qmbABcCm6rqL2ctugJY2aZXApcPX54kaVQLRtj2eODFwIYkX21tfwycB3w4yTnArcDzRytRkjSKoYO+qv4NyE4WnzzsfiVJ4+WdsZLUOYNekjpn0EtS5wx6SeqcQS9JnTPoJalzBr0kdc6gl6TOGfSS1DmDXpI6Z9BLUucMeknqnEEvSZ0z6CWpcwa9JHXOoJekzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0mdM+glqXMGvSR1zqCXpM4Z9JLUOYNekjpn0EtS5wx6SeqcQS9JnZuXoE/yzCRfT7I5ybnzcQxJ0tyMPeiT7Af8DfAs4GjgzCRHj/s4kqS5mY8e/bHA5qq6pap+CPwTcNo8HEeSNAfzEfRLgNtmzW9pbZKkCVgwqQMnWQWsarP3Jvn6pGrZDYcB3xnnDnP+OPe2zxn7+eRtGevu9iHjvzbPetieS5iPazPzcj4fN5eV5iPobwceO2v+iNb2E6pqNbB6Ho4/b5JMV9XySdfRC8/n+Hgux6u38zkfQzdfAZYlOTLJ/sAZwBXzcBxJ0hyMvUdfVfcneQXwaWA/4KKqunHcx5Ekzc28jNFX1aeAT83Hvidsnxpq2gd4PsfHczleXZ3PVNWka5AkzSO/AkGSOmfQN0lelWRTkksmXYv0UJL8+6Rr6EWSpUk2TrqO+Taxz9HvhX4fOKWqtgy7gyQLqur+Mdak7SQJgyHHByZdy6RU1dMnXYP2LfbogSR/B/wc8C9J3pjkoiRfTnJdktPaOkuTfCHJte3x9NZ+Ymu/Arhpgj/GRCX5eJL1SW5sN8OR5N4kf5rk+iTXJFnc2h/f5jckeXuSe2ft5w+TfCXJDUne1tqWti/JuxjYyE/ep/Gw085rklyQZGM7jy9oyy5OsmLWupdsu4Z7luSgJJ9s19rGJC9I8uZ2LW1Msrp1Ekjy1Lbe9cDLZ+3jrCQfS3JlkpuT/MWsZacm+VJ77X8kyaNb+3lJbmrX6zta2++0Y16f5PN7+FTsWFX5GLwh/S0Gd8P9GfCi1rYI+AZwEHAgcEBrXwZMt+kTge8DR076Z5jw+TukPT+KQRgfChTwm639L4A3telPAGe26ZcB97bpUxl82iEMOiGfAE4AlgIPAMdN+ufcGx7AvcDzgKsYfIR5MfBt4HDg14CPt/UWAt8EFky65j1wTp4HvHfW/MJt12Sb/8Csa/EG4IQ2fQGwsU2fBdzStj0AuJVBp+Iw4PPAQW29NwBvbtf41/n/D7Usas8bgCWz2yb9sEf/YKcC5yb5KnA1g1/4zwKPBN6bZAPwEQbfzLnNl6vqm3u60L3Mq1oP6RoGL45lwA8ZhDXAegaBDfA0BucQ4B9n7ePU9rgOuBY4qu0H4Naquma+it8HPQO4tKp+XFV3AZ8DfrmqPsfghsUp4Ezgo/XwGE7cAPxGkvOT/GpVbQV+Pcm69po9CfjFJIsYhO+2nvYHttvP2qraWlU/YPAv9McBxzF4vX+x5cLK1r4V+AFwYZLfBu5r+/gi8P4kv8vgD/HEOUb/YAGeV1U/8d07Sd4K3AU8mUFv8wezFn9/j1W3F0pyInAK8LSqui/J1Qz+QP6oWrcG+DG7vt4C/HlV/f12+1/Kw/wc76aLgRcxuCv97AnXskdU1TeSHAM8G3h7krUMhmWWV9Vt7fV7wBx29T+zprddswGuqqozt185ybHAycDpwCuAk6rqZUl+BXgOsD7JU6vqP0b48UZmj/7BPg28ctZ43lNa+0Lgjhq8Cfhi9pK/1HuJhcB3W8gfxaAH9FCuYfBPbRiE0TafBl46a/xzSZKfGXu1ffgC8IIk+7Xe+wnAl9uy9wOvAaiqh8X7RkkeA9xXVR9kMBxzTFv0nXY9nQ5QVfcA9yR5Rlv+wjns/hrg+CRPaMc6KMnPt/0urMENoq9l0AkkyeOral1VvRmYYS94T8ke/YP9CfAu4IYkj2Awxvlc4G+BjyZ5CXAl9jBnuxJ4WZJNDMYsdzXE8hrgg0ne2LbdClBVn0nyROBL7e/svQx6pj+er8L3UQX8M4MhsOvb/Our6k6Aqrqr/S4+PrkS97gnARckeQD4EfB7wAoG7xfdyeA7uLY5G7goSQGf2dWOq2omyVnApUl+qjW/CfgecHmSAxj0+l/Xll2QZFlrW8vgdzRR3hmrPS7JgcB/V1UlOYPBG7PdfzJkHJIcClxbVTv9etp2fjcAx7Sxaj3M2aPXJDwV+Os2PHYP8NIJ17NPaMMTVwPveIh1TgEuBN5pyGsbe/SS1DnfjJWkzhn0ktQ5g16SOmfQS1LnDHpJ6pxBL0md+18cDOnC/iyA3gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108bc6cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_df['emotion'].value_counts().plot(kind=\"bar\", rot=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Feature engineering\n",
    "### Using Bag of Words\n",
    "Using scikit-learn ```CountVectorizer``` perform word frequency and use these as features to train a model.  \n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# build analyzers (bag-of-words)\n",
    "BOW_vectorizer = CountVectorizer() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Learn a vocabulary dictionary of all tokens in the raw documents.\n",
    "BOW_vectorizer.fit(train_df['text'])\n",
    "\n",
    "# 2. Transform documents to document-term matrix.\n",
    "train_data_BOW_features = BOW_vectorizer.transform(train_df['text'])\n",
    "test_data_BOW_features = BOW_vectorizer.transform(test_df['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<3613x10115 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 51467 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check the result\n",
    "train_data_BOW_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "scipy.sparse.csr.csr_matrix"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(train_data_BOW_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       ...,\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add .toarray() to show\n",
    "train_data_BOW_features.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3613, 10115)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check the dimension\n",
    "train_data_BOW_features.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['2k17', '2much', '2nd', '30', '300', '301', '30am', '30pm', '30s', '31']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# observe some feature names\n",
    "feature_names = BOW_vectorizer.get_feature_names()\n",
    "feature_names[100:110]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basically, we could put this into classification model now.  \n",
    "\n",
    "However, here are some problem....\n",
    "\n",
    "* curse of dimensionality  (we have 10,115 dimension now)\n",
    "* some important features are ignored"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"😂\" in feature_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, let us try other parameter setting..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3613, 500)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "\n",
    "# build analyzers (bag-of-words)\n",
    "BOW_500 = CountVectorizer(max_features=500, tokenizer=nltk.word_tokenize) \n",
    "\n",
    "# apply analyzer to training data\n",
    "BOW_500.fit(train_df['text'])\n",
    "\n",
    "train_data_BOW_features_500 = BOW_500.transform(train_df['text'])\n",
    "\n",
    "## check dimension\n",
    "train_data_BOW_features_500.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 8, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       ...,\n",
       "       [1, 2, 0, ..., 0, 0, 0],\n",
       "       [0, 1, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data_BOW_features_500.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['cheerful',\n",
       " 'cheering',\n",
       " 'cheery',\n",
       " 'come',\n",
       " 'comes',\n",
       " 'could',\n",
       " 'country',\n",
       " 'cry',\n",
       " 'crying',\n",
       " 'customer']"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# observe some feature names\n",
    "feature_names_500 = BOW_500.get_feature_names()\n",
    "feature_names_500[100:110]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"😂\" in feature_names_500"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 3. Model\n",
    "### Using Decision Trees\n",
    "Using scikit-learn ```DecisionTreeClassifier``` perform word frequency and use these as features to train a model.  \n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train.shape:  (3613, 500)\n",
      "y_train.shape:  (3613,)\n",
      "X_test.shape:  (347, 500)\n",
      "y_test.shape:  (347,)\n"
     ]
    }
   ],
   "source": [
    "# for a classificaiton problem, you need to provide both training & testing data\n",
    "X_train = BOW_500.transform(train_df['text'])\n",
    "y_train = train_df['emotion']\n",
    "\n",
    "X_test = BOW_500.transform(test_df['text'])\n",
    "y_test = test_df['emotion']\n",
    "\n",
    "## take a look at data dimension is a good habbit  :)\n",
    "print('X_train.shape: ', X_train.shape)\n",
    "print('y_train.shape: ', y_train.shape)\n",
    "print('X_test.shape: ', X_test.shape)\n",
    "print('y_test.shape: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['sadness', 'sadness', 'fear', 'sadness', 'sadness', 'anger', 'joy',\n",
       "       'anger', 'fear', 'joy'], dtype=object)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## build DecisionTree model\n",
    "DT_model = DecisionTreeClassifier(random_state=0)\n",
    "\n",
    "## training!\n",
    "DT_model = DT_model.fit(X_train, y_train)\n",
    "\n",
    "## predict!\n",
    "y_train_pred = DT_model.predict(X_train)\n",
    "y_test_pred = DT_model.predict(X_test)\n",
    "\n",
    "\n",
    "## so we get the pred result\n",
    "y_test_pred[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 4. Results Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training accuracy: 0.99\n",
      "testing accuracy: 0.66\n"
     ]
    }
   ],
   "source": [
    "## How to evaluate the pred result?\n",
    "\n",
    "# check statistical results\n",
    "\n",
    "## accuracy\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "acc_train = accuracy_score(y_true=y_train, y_pred=y_train_pred)\n",
    "acc_test = accuracy_score(y_true=y_test, y_pred=y_test_pred)\n",
    "\n",
    "print('training accuracy: {}'.format(round(acc_train, 2)))\n",
    "print('testing accuracy: {}'.format(round(acc_test, 2)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "       anger       0.70      0.68      0.69        84\n",
      "        fear       0.68      0.68      0.68       110\n",
      "         joy       0.62      0.67      0.65        79\n",
      "     sadness       0.63      0.59      0.61        74\n",
      "\n",
      "   micro avg       0.66      0.66      0.66       347\n",
      "   macro avg       0.66      0.66      0.66       347\n",
      "weighted avg       0.66      0.66      0.66       347\n",
      "\n"
     ]
    }
   ],
   "source": [
    "## precision, recall, f1-score,\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "print(classification_report(y_true=y_test, y_pred=y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[57 11  7  9]\n",
      " [11 75 14 10]\n",
      " [ 6 13 53  7]\n",
      " [ 7 12 11 44]]\n"
     ]
    }
   ],
   "source": [
    "## check by confusion matrix\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "cm = confusion_matrix(y_true=y_test, y_pred=y_test_pred) \n",
    "print(cm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Funciton for visualizing confusion_matrix\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import itertools\n",
    "\n",
    "def plot_confusion_matrix(cm, classes, normalize=False, title='Confusion matrix',\n",
    "                          cmap=sns.cubehelix_palette(as_cmap=True)):\n",
    "    \"\"\"\n",
    "    This function is modified from: \n",
    "    http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html\n",
    "    \"\"\"\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "        print(\"Normalized confusion matrix\")\n",
    "    else:\n",
    "        print('Confusion matrix, without normalization')\n",
    "    classes.sort()\n",
    "    tick_marks = np.arange(len(classes))    \n",
    "    plt.figure(figsize=(4, 4),dpi=115)\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    plt.xticks(tick_marks, classes, rotation=45)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "    fmt = '.2f' if normalize else 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, format(cm[i, j], fmt), horizontalalignment=\"center\", color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')\n",
    "    plt.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion matrix, without normalization\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcEAAAGsCAYAAACl0RxsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAARrwAAEa8B9/1LhAAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8FHX+x/HXJzQR6YqAymEB7pSiZwFsKBbs5exn9+yn3p139l7u7Aq2+6GeHVFUkCqKNAsg2KhKR6QjoSM1n98fMwmbZAPJ7mYnm30/fewj2ZnvzHx2DfvZbx1zd0RERLJRTtQBiIiIREVJUEREspaSoIiIZC0lQRERyVpKgiIikrWUBEVEJGspCYqISNZSEhQRkaylJCgiIllLSVBERLKWkqCIiGQtJUEREclaVaMOQEREKgYz2wWom4JTrXT3pSk4T7kz3UVCRETMbJccq7Ikz7ek4nQrgRaZkAhVExQREYC6eb6FPRu1oVqVHRI+yaYt65m9ZGJdghqlkqCIiGSO6lVrUqNazYSPt02WwmjKnwbGiIhI1lISFBGRrKXmUBERKWBmmCVePzLLrOZQJUERESmQg5FD4oksmWOjoOZQERHJWkqCIiKStdQcKiIiW5kl16+XYX2CqgmKiEjWUk1QREQK5FgOOUmMDk3m2CgoCYqISAFLsjk006ZIZFbKFhERSSElQRERyVpqDhURkQIGWBIT3jOrMVRJUEREYuSQ5MCYDGtgzKxoRUREUkhJUEREspaaQ0VEZCutGCMiIpIdlARFRKRAjlnSj7IysxFm5iU8BseUq29mr5jZr2a21sw+M7M2ybxeNYeKiEjUrgfqFNnWEXga6AdgQRttf6A5cCOwHLgDGG5m+7v7vEQurCQoIiIFLLytbjLHl5W7Tyl2HrOrgI3Au+Gm04DDgM7uPjwsMxqYDdwK3JRIvGoOFRGRCsXMdgTOAfq7e264+TRgQX4CBHD3lQS1w9MTvZaSoIiIFDDbuoh2Yo+UhHEmUBt4I2bbfsCkOGUnA83MbKdELqTmUBERKZDo4JbY40PN4ky1yI2p2W3LJcAS4OOYbQ2AOXHK5p+vPrCm1IGGVBMUKcIC/zKzn8xsfThC7YxyvF7z8Bqvl9c1Kju9hxXSUGB6kcd2++3MrClwLNDD3TeXa4SoJigVhJkdBlwDHA40AfKAucBw4BV3/y6N4VwIPAF8SzA6bSPwUxqvn5XCBHYpsKe7z4k2GkmBYwj+DccqTS3wIoIK2htFti8nqO0V1SBmf5kpCUqkzKwK8BxwHbCB4Ntjb4Ik2IrgQ/FaMzvD3fulKawTw5+nuPuiNFxvPvAHYGUarlVZ6T1MGUvqLhIx95GY6+4zEjjBpcB4dx9fZPtk4Pg45fcNr1XmplBQEpToPUqQAMcC57h7oW+OZtYAuI/43wDLSxOANCVA3H0TqmkmRe9h5WBmBxEktZvj7O4HXG5mndx9ZFi+DnAq8E6i11SfoETGzFoS/LEvBU4umgAB3D3X3f/G1rlC+cd2MrNPzGy5mf1mZpPM7FYzq1ak3FFhX9H9ZtbBzIab2RozyzWzHma2S0zZy8zMgaPD5/krVsyJ3W9ml8V5LXH3mdmxZjbEzBaZ2QYzWxiucnF6TJkS+7PK43VuT3ieEWa2u5m9F55jpZm9b2a7hmUOC6+x2syWmtkzcWKqa2a3m9nn4evfaGZzzay7mTUuUnYOQQ0AYHbMe/960ffIzNqY2YDwPfGS3kMzuzzcVuwD0sweDPf9u7TvS7bIsZykH0m4BNgM9Iizrx8wGnjbzM43sy7hNgMeT/SCSoISpUsJ/ga7u/uv2yro7hvyfzez84FhBCtKfEDQnArwGNDb4gxJAw4h6F9cDfwfMBX4M9AvpvwPwAPAz+HzB8JH1zK/siDOU4BPCYZ29yXoXxwMNCX49rq948vrdZZGfeCLMNbXgG+As4G+ZnZ4+LqWAS8RfIn5O3B3kXP8AbifYMTeB0A3gtra1cBoM4ut3XcF8pu/urH1vf+oyDn3Ab4CagGvUOTLUSx3fw34ELjAzC7K325mhwJ3hq/p/m29CZI+4ZeoC4DB7r6k6H53zwNOAYYALwJ9gC3A0e7+S6LXVXOoROnQ8OfwbZaKYWZ1ge7AOuAgd58Wbr8TGEjwj+QSineqnwic7e4fhuVzCP4xdQY6AKPd/QfgBzM7Cvidu9+f2MsqcAWwCdi/6D9qM2u4rQPL83WWMva2wBPufmtMTP0Iknd/4Fx3HxhuvxeYAdxoZg+HTZMAPwJNiw6JDxPSW8BfgYcB3L2rme0PtAO6bmNgzGHAXe7+n1K+jqsJXvcLZvYlQeJ+m6D/+cKYWCVkSd5FItFjw/8X22yxCP+WrggfKaGaoEQpv0lsfhmOOYNgjcGX8hMDQDiU+rbw6aVxjhuRnxjC8nnAm+HTg8pw/bLaRDC6tBB3X7ad46J+nWsoXkvqFf78Lj8BhtdYS5CY6wO7x2xfWcKcsB7AKoJh8GW1kGDkbqmE17+UYOL1W8ALwJ7AzbHvq2QvJUHJNO3CnyOL7ghrcitjysT6Ps62/ORbLzWhFfMeQbPdJDN73MxOtKAjvzSifp3T3X1dkW0Lw59FR+0B5A8iahq70cyOMbP+ZrbYzDaHfXh5BAm+SRniyTe+rLU3dx9K0BR9OHAxwVJc3RO4dlbIIcm7SCQ1sjT9lAQlSvkfnLuV4Zj8JLJ4G+eMl2hWxdmWPxG3ShmuX2ru/h5wFkEf4z+BQcCvZvahmTXbzuFRv85459lSimsUDI4xs/MImmKPBEYAT7G1r28lUKMM8eQr1ldUSrF9iy8meA6phNQnKFEaBRxFMBpzWCmPyf8A3rWE/bsS/0M6FfLCn/H+3cSt4bl7b4JBLPWBIwg6/s8H9jazA9zdS7hWlK8zVe4F1gMHxs4XCwfo3FriUdtW0vtVIjOrBbwK/BZu6ha+90VrupKFVBOUKL1JkFiuLsVAkfxaww/hzyPjlGlL0OT3Q9F9KbIi/Nk0zr4DtnWguy93937ufgHBggDtgG3VBqN8namyNzAlzoTpA4Caccrn1zRTXTPvCrQA/gXcArQEnknxNSoNS8F/mURJUCLj7lMJ+moaAf3NbI+iZcysnpk9DZwXbupLUAO62sz2iSlXhWDqAGwdCJJq3xLURM6LScqY2SEES60Vjf2IMK7YbVXZuszThqLHxIjydabKXKCFmTXK3xD2iXYroXz+IJqyNI9vkwVrvl4JDHL3F939BYJFma82s+1OU8lGZpbUHMFkRpZGQc2hErU7CAaPXAdMN7MhBEPrneAb+7Hh/tMhGHFoZtcSDHP/1szeJaihnQS0BgZQTsnB3eeb2fvAucC4MNbdwtgGEozojPUc0Dgcmj+HoIZzbBjnO9takSbK15lCzxMkvO/M7EOCPsATgQXho6jhBLW1l8ysN8H0kPHu3j+Ri5tZE+BlgnmMsUPqrwAmAq+YWZt4c9Ike6gmKJFy983ufj3ByL1eBB/wNxKsNv8HguH0B8Z+ELp7T4LFeccQ1BD/RvC3fDvwp230s6XC5QQDKxoD1xMMtz+ToOZW1CPA5wTNf9cTzOtbB1xL/OkNhUT8OlPhOeAGghrtNcDJBBOcjyeYOlKIuw8imMRejaDZ8iGCgUVlFvY7vgbsDFzp7gUDjMIvH1cStEC8msj5K7P8eYLJPDKJVfx/RyIiUt7CZvfpR7Y8mVo1aid8nrUbVvP5tIEALRJcQDut1BwqIiIFUnhT3Yyg5lAREclaqgmKiEiBZKc5aIqEiIhIhlASFBGRrKXmUBERKRDVrZSioiQoIiIFLMnRoUqCUq7CxYDbE9zWRjcEFREIFhhoAnwd3t9RSklJMPO0J1iAWUSkqGMo/R1ZBCXBTLQQ4My2Z1Fnh9Len7VyuuamzlGHELmqO8W7GUP2yduY3Y0ivyxaxDk3/QO23vg4YUZy0xwyqzFUSTATbQKos0Md6u1YP+pYIrXn7im72UDGqlanVtQhVAhbNmyMOoSKIulvA/l3g0jm+EySWdGKiIikkGqCIiJSwEhyikSGNYiqJigiIllLSVBERLKWmkNFRKRAtt1KSUlQRERiJHcXiUybJKHmUBERyVpKgiIikrXUHCoiIgXUJygiIlnLLLk7QWRYDlRzqIiIZC/VBEVEpEC2NYeqJigiIllLSVBERLKWmkNFRKSAJTlZPtMW0FYSFBGRApZkn2AyI0ujoOZQERHJWkqCIiKStdQcKiIiBcySvKluhjWHKglKMfu025sbul5fbPtva37jjlPvBuDPt53PISccHPf4xXOX8Milj5VrjOmwYMlSXujZk/FTpzF55kzWb9jA2J7vsEeTxoXK/eflVxg/dSoTp01n+apVdL3tVs478YSIoi5/p111HaO+/S7uvs4dO9DrhW5pjig6X377HY+99AoTfprKDjVqcOxhHbnvxr/SqEGDqENLWA5JzhPUwBipLD58tg9zf5pb8DxvS17B75+8NYSv+o0qVL5B4wZceu/FTBo1OW0xlqc58+fTb8RI2rZsQfu2bRg57pu45V7t3Yf99tmHYzt24P1PPk1zlOn3xB23sHrN2kLbxk2YxD1Pd+WETkdEFFX6jflhPOf97WaOan8I/3vkYZavXMmjL73C2Tf+nSGvvUKN6tWjDjHjmNlJwO3AH4E8YBpwq7sPC/fXB54AzgBqAqOBf7j7xESvqSQoJVr882J+/nFu3H3LFixj2YJlhba1OqglAOM+GVfusaVDh3ZtmdjnQwB6DBhYYhKcNrA/OTk5zJ43PyuSYKu99iq27a0+falerRpndjkugoii8eT/XmP3xo1547H/ULVq8FHaonlzulxxFe/0H8jlZ50ZcYSZxcyuAZ4PHw8RjFnZH9gx3G9Af6A5cCOwHLgDGG5m+7v7vESuq4ExkjIHH38Qc6f+wqI5i6MOJSVyckr3z6O05Sqrdb+tp+9nQ+ly5OHUr1s36nDS5tvJU+h0yEEFCRBg/z/8ngZ16zJo5OcRRpYcS8F/Zb6mWXOgK3CLu//D3Ye4+yfu/pi7DwiLnQYcBlzs7j3dfXC4LQe4NdHXq5qglOjiuy6kVt1a/LbmN376Zir9XxrIiiUr4pbds3Vzdtl9Fz58tk+ao5SoDRw+gjVr13HeKSdHHUpaVcnJoVq1asW2V69WjZ9mzYogotQwS279zwQPvYKg+fP/tlHmNGCBuw/P3+DuK82sP3A6cFMiF1YSlGJ+W7ueYe+NYOb4maxft57d99mNYy88hr8/fxNPXv00a1asKXbMwccfxOZNm/lu6PcRRCxR6jVgELs0qM+xh3WMOpS02rtZM76dVLj/+5eFi1i8bBnVquqjFWgWZ6Rorrvnxil7OPATcL6Z3QP8DpgDPOPuL4Rl9gMmxTl2MnCJme3k7sU/nLYju9txJK75M+bT7//6M3n0FGaOn8XID7+g+20vU7vBThz5p8OLla9arSr7H9WOyaOnsHbV2jhnlMpq4dKljBw7jrNOPKFQs2A2uOq8s/l+yo880v1lluYuZ/qcn7nhwYfJycnJuDspxMqfIpHMIzQUmF7kUVJtrSnQgmDQy6PA8cAQ4Hkz+1tYpgFBP2BR+Um1fiKvN7v+aiVh86bPZ+kvS2n2+2bF9rU+bD92rL0j4z6NP3BEKq/3Bw4mLy+P8089KepQ0u7sLsczY85c/vtOT7q+/iZmxunHduaYjh0yujk0hY4Bio6si1cLhKBCVhu4zN17h9uGhX2Fd5jZs+USIUqCFYKZ1XD3DVHHURruXmzbwV0OYs2KNUwZ82MEEUmU3hswkNYtW9C6ZcuoQ4nE7ddcyY2XXMjP8xewc4P6NGrQgMPPv4j2bdtGHVpFMNfdZ5Sy7DKCmuCQIts/BU4AmhDUAuPV9vInZcarJW5XpWsONbN9zOwtM5ttZr+Z2Swz+284vyS23OtmNs/MDjCzL8xsnZlNN7Nr45zzWDP73szWm9kMM7syPH5OkXI7mtlj4bU3hj/vMrOcmDJHmZmb2Z/M7GUzWwpU+OGUe7TcnUZ7NCo0bxBgp/o78fuDW/Ht0O8KzSOUyu/7KT8yddbsrBsQU1StmjXZd5+9adSgAcNGf830n3/m0jNPjzqshOXfVDeZRwK2N7k4LyyzX5x9+xIk3DL3B0LlrAk2BX4B/k7wzWAv4E5gEFC0574O8A7B0NwHgcuB/5rZ1PwRSGa2LzAQGAucD1QH7gHqEvyPISxXFfiE4H/IQ8BEoENYtgHwzyLXfg74GLgY2CHeCzGzBmz9lpOveHtkil1014XkLlzGvOnz+W3Nb+wWDoxZ+etKPv/wy0JlDzr2j1SpUoVxn1TOptABI0YCMGHaNACGjf2ahnXr0aBePQ7dvx0Ao34YT+6KFSzJDVp6xk+dSq2aNQE45ahOEUSdHr0GDKJq1SqcfVKXqEOJxMSp0xg6egxtW7UC4OsJE3ixR09uuOjPHNy2TcTRJS6iZdP6AH8BugAfxGw/AZjn7ovMrB9wuZl1cveR4bXqAKcSfI4npNIlQXf/HCiYpGNmo4AZwBdmdoC7xw5frA1cH5PwPif4n3ABkD8M925gFdDF3deF5b4AZgOLYs51AcEIp05hDABDwz+I+8zsMXdfElN+rLtfuZ2XcxNwX+leeeosmr2QP3Y+gCPOPJzqO1RnVe5qJnwxgcGvfVJs4MvBXQ5mwayFzJs+P91hpsVV9z9Q6PntzwRLgnVs147e3Z4B4MnXXmf0+PEFZV77qC+vfdQXgIUjhqUp0vTatGkzvQd/SueOHdklg5cIS0a1atUYOnoML/ToycaNG2nR/Hc8fus/uSDLa8YJGkTwmdvdzHYGZgHnEAyQuTws049ghZi3zewWtk6WN+DxRC9c6ZKgmVUH/gVcQjDMNraW1QqITYLrisw52WBm0yhc2+oADMpPgGG5hWFyjV064wTgZ2BUWCvM9ynwcHiefjHbSzOh7lng7SLbmhGMuio3n70zjM/eKd2H9xNXPlWeoUSuNEksPxlmk2rVqjJ12CdRhxGp3++1J/27vxh1GJWCu7uZnQE8AjxA0Pf3E3Chu78Tlskzs1OAJ4EXCT7bRwNHu/sviV670iVBgjfxRoLmzVHAamB3oDfFmx3jdaRuKFKuCbAkTrnFFE6CjQiS7qYS4mpY5PnCEsoVCOfTFBpNlWkrtItIZonqzvLuvgr4a/goqUwuwcT6KxK6SByVMQmeD7zp7g/nbzCznZI430KCBFfUrkWeLyNoIj23hPPMKfK8+DBLEZGIBSvGJHd8JqmMSXBHitfGLo9XsJTGACeZ2Y4xfYJNCNawi63NDQbOAta4+09JXE9ERNKkMibBwcClZjaRYEDMn4BDkzjfw8DZwCdm9iRQg2DE52JiRocCPQiS7VAzewoYTzCSdG+CNe/OiO1XFBGR6FXGJHgjwWihf4fPBxGM3BybyMncfYqZnUywnE8vYD7wGMFAmOYx5TaZWReCe2FdDewJrAVmEkyx2JjI9UVE0kl3ls9w7v4rQb9gUVak3GUlHH9UnG1DCO5rFZwo6GN8gCC5xZZbD9wfPkqKb0TRWEREJBqVLgmWBzN7jmCk6QKCyfh/IxjC2y3KuEREUi2HhFd9KTg+kygJls4OBE2guxI0a44FjnX3CZFGJSKSYmbJNWlmWGuokmBpuPtVUccgIiKpV+kW0BYRESkt1QRFRKRADpZUv576BEVEJHMlOUUi0zoF1RwqIiJZS0lQRESylppDRUSkQBJ3hy84PpOoJigiIllLNUERESkQTJZP7vhMopqgiIhkLSVBERHJWmoOFRGRAtk2MEZJUEREClj4XzLHZxIlQRER2UorxoiIiGQHJUEREclaag4VEZECOSQ3uCXTalaZFq+IiEjKqCYoIiIFtGKMiIhIllBNUERECmTbZHnVBEVEJGupJpihrrmpM3vuvlvUYUTquEvuizqEyH3c/a6oQ6gQqtWuGXUIkfK8vJSdSyvGiIhI1rIkm0OTWm0mAmoOFRGRrKUkKCIiWUvNoSIiUkDzBEVERLKEaoIiIlLAkryVkgbGiIiIZAjVBEVEpIBWjBEREckSqgmKiEgBI8nRoSmLJD1UExQRkUiZ2VFm5nEeK4qUq29mr5jZr2a21sw+M7M2yVxbNUEREakobgLGxTzfnP+LBcNO+wPNgRuB5cAdwHAz29/d5yVyQSVBEREpEPHaoT+6+5gS9p0GHAZ0dvfh4bVGA7OBWwkSaJmpOVRERAoYW+8kkdh/5eY0YEF+AgRw95UEtcPTEz2pkqCIiJSHZma2T5FHg+0c08PMtpjZMjN7x8yaxezbD5gU55jJ4bV2SiRINYeKiEiBFK4YMzTO7geA++NsXwk8BYwEVgEHAHcCo83sAHdfAjQA5sQ5Njf8WR9YU9Z4lQRFRKQ8HAPMLbItN15Bd/8e+D5m00gz+xwYS9DXd3e5RIiSoIiIxMix4JHM8aG57j4j0fO4+3dmNg04ONy0nKC2V1SDmP1lpj5BERGpyDz8OZmgX7CofQkSbpmbQkFJUEREYuT3CSbzSFEcBwGtCJpEAfoBu5lZp5gydYBTw30JUXOoiIhEysx6EMz3+w5YQTAw5g5gPvBsWKwfMBp428xuYetkeQMeT/TaSoJSzIIlS3mhZ0/GT53G5JkzWb9hA2N7vsMeTRoXKvefl19h/NSpTJw2neWrVtH1tls578QTIoo69V59txsHdzwg7r4vR3zNdZfeQtPdG/PJV73iljm0zUmsXpVQC02FsnDpUl58/30mTJ/GlNmzWb9hA6Nef509dm1c4jEv9HqPR197jYP33ZfeTz2dxmjLx4IlS3i+xzuM/2kqU2bM5LcNGxj7/rs0a9KkULn1Gzbw2Cuv8uGnn7Jq9Rr2a7EPd193LR33bxdR5BljEnABwUowOwKLgN7Afe7+K4C755nZKcCTwIvADgRJ8Wh3/yXRCysJSjFz5s+n34iRtG3ZgvZt2zBy3Ddxy73auw/77bMPx3bswPuffJrmKMvfw/c8zU471Sq0rd0f9+PWe29kxGdfFdr+8vNvFdu2ds26co8xHeYsXMCALz6nzT4tOGS//fj8u++2Wf7nhQt5tmdPdq5XL00Rlr/Z8+bTb9gI2rZqSft2bRkxdlzccjc/+jifjR7Dvddfy++aNuW13n244OZ/MaD7i7Ru0SLNUSco2SbNBI5190eAR0pRLhe4InykhJKgFNOhXVsm9vkQgB4DBpaYBKcN7E9OTg6z582vlElw1vSfi20764JT2bhhIx/3KzwFat4vC5jw/ZR0hZZW7Vu34fue7wLQc/DH202Cdz7/HGcefTQz581jy5Yt6Qix3HXcvx2T+n8EQI/+A+ImwcnTZ9B7yGc8c8dtXHDySQXHdbr4Mh5/5VXefGy7n/EVQgpHh2YEDYyRYnJySvdnUdpylcUOO9Tg+JOOYsTQUaxauTrqcNKmLP+f+wwfzqQZM7nt8pR9Ua8QSvMefPLVV1SrWpXTj+lcsK1q1aqcfmxnRowdx4aNG8szRElQdn2KiSThmBOOZKfatej3weBi+/526zV8P3MYoyYO4tlXHqFFq70iiDBaK1av5sGXunPnX/5C/dq1ow4n7abOnkOzJk3YcYcdCm1v1bw5GzdtYva8+RFFJtui5lCRUjr1T11YtjSXL0d8XbBt48ZN9Hq7L6O+GMfyZSvYc+9mXHnDxbzV+0UuOP0aZs8o3qRaWf37f6+w5267ce5xx0UdSiRWrFpF3TjJv36dOsH+1avSHVJCUrhsWkYodRI0s3PLcmJ3jz9kroIzsysIht3+Dljn7pWnd18StkujhnQ4/EB6vPZBoX6uX5cs46G7nip4/t24CXw5ciwfDXmDq2+4mDv+/nAU4abd15Mm8eHQoQx67vmM+xCUwrLtzvJlqQm+W4ayDmRcEjSzpsBLQA/gcmB9tBFJRXHKmcdTpUoV+sZpCi1q8cIlfP/NRPZr+/s0RFYx3PHss5x3fBea7LwzK9cE00K2bNnClrw8Vq5Zww7Vq1OjevWIoyxfdWvXZt6ixcW2L18V1ADr1a6T7pCkFMqSBP9QblFUHC2AKsAb7v5lOi5oZtWAze7u2y0skTnt7BP4afJ0pv04s/QHZdH/0um/zGX6L3N5e9DAYvtan3M29119DVeeeWYEkaVPqz2b8/HnX7Bu/fpC/YLT5vxM9WrV2HP33aILrgxykrypbjLHRqHUSdDdp5ZnIFEzs9eBS8OnQ8MmnTfc/TIzuxr4K8ESPmuAvsAt4ZyV/ONvAC4My+QAPwEPufvAmDLNCVZF+CvQHLgIaAw0JMHFX6X87dumFfu03JPHH3yuVOUbN23EAQe1YdinX5RzZBVHr8ceK7bt/u7d2ZKXx0PXXUfzJk0jiCq9jj/sUJ7432v0Hz6iYNGIzZs302/YMDodfFClrwlnqqQGxphZK+AIgg/x1919sZntASxz90ybKfwQ8C3BEj1/JVi+Z6mZPQr8M9x+C7Ab8DDQ2swOdff8DqLmwCsE97uqSrCe3QAzO9Hdi7ah3QWMA64mqHnGbXYNb0BZ9CaUzeKVTbUBI0YCMGHaNACGjf2ahnXr0aBePQ4NV78Y9cN4clesYElu8F1g/NSp1KpZE4BTjuoU56yZ6bSzurBp02YGfjSk2L5/3f1XcswY/91kcnNX0HyvPbjy+ovIy8vj5effiiDa8jHwiyChT5ge3BRg+LhvaFi3Lg3q1qVj27Z0bFt8RZQ6tWqxZcuWuPsyUf/hI4Dg7xxg2JivaVivHg3r1ePQA/anTcuWnH5MZ+7t9hybN2+mWZMmvP5RX+YuXMQL994TYeSyLQklwbAJ71XgzwT9oA4MARYDzxOs9n1nimJMC3efaWY/hk+nuPuYsOZ2C/CAuz+YXza8vceXBInuo/D4f8XszyG4oWRL4DqgaBJcDJxZiibQm4D7En1Nybjq/gcKPb/9mW4AdGzXjt7dngHgyddeZ/T48QVlXvuoL6991BeAhSOGpSnS8lW1ahVOPO1Yvhr5NbnLVhTbP2PabM676HROP+dEau5Yk5XLVzJ29Hf8t+vrzJmV8EpOFc61//l3oed3vfA8AB3atOH9x5+IIqS0u+qewv8Ub38q+HfQcf/96fNS6XOjAAAgAElEQVR88O+j65238+hLL/Poy/9j1Zo17Lv33rzz5OO0bdUy7fEmzrCkhrdU0ubQIh4CTgOuIkh+sePABxHUcDIqCZbgOIKmzR5mFvtefQ2sBo4kTIJmdiDBXZMPBnZh619CvGbkj0rZB/gs8HaRbc2If8fmlCpNEstPhpXZ5s1b6PTH00rc/1GvQXzUa1AaI4rGLx9vf0BQUZUtOS76cuR2y9SsUYMHbryBB268IQ0RlQ+zJEeHZlYOTDgJXgjc4+6vmlmVIvtmAXsmF1aF0Sj8WdKNIRsChE3AQ4EpBAvAzgU2E3xZiDegaGFpLh72ORa6E7OGn4uIpE6iSXAXglW/S7LDNvZlkmXhz+OJP3Alf/8JQF3gXHefl7/TzHYs4bzZM2xQRKQCSzQJ/kzQ7BevzewgYHrCEVUsQ4A8oJm7Fx8VsVV+stuUv8HMWgKHAfPiHiEiUgEFC2gnM0UihcGkQaJJ8G3gLjObAfQPt7mZdQRuBv6TiuCiFg6WeQx4PhwJO5JgJOceBP2Fr7j7cOAzgubPN83sKaAJQf/gXLQ+q4hkEC2bVjqPAH8E3ieYNwcwHKgN9AG6Jh9axeDud4ajRv8aPhz4haAPcHpYZrKZXQg8SHD345nA7QTNpEdFELaIiJRCQknQ3TcDZ5rZcQQf9LsQ9I8NdvdPUhhfWrn7Z8QZ3+vubwHbnPQVrpVadKm4d4uUmRPv/CIiEo2kJsuH/WTb6isTEZEMoikSZRD2AXYkWEVlHjDG3UenIjAREZHyluiKMXWBnkAXgua9tUAtgsExHwMXuvvKlEUpIiJpkW0DYxIdudiNYLWUq4Da7l6bYFDM1QQDQSrNwBgRkWxi5E+TSOyRWSkw8SR4BnCXu7/q7msB3H2tu/8PuBuo3PdMERGRSiGZOWxTtrFdK6KIiEiFl2gS7A+cVcK+PwEDEjyviIhEKL9PMJlHJin1wBgzOzTm6XvAC2b2IcGE+cXArsC5wIHA9akMUkRE0kNTJEr2JYWbOY1g+bAzw+2xL70fwc1iRUREKqyyJMETyy0KERGRCJQ6CWbycmgiIlI6OWZJ3kUis9pDk1oxRkREKpdsmyyfcBIM75d3OdCK4jfRdXc/OZnAREREyluiy6YdCHxBMCq0GTAVaAA0AhYQ3EdPREQyTLaNDk10nuCjwECgBcGo0IvcvTFwSnjO21ITnoiISPlJNAm2A14H8sLnVQDcfRDBXeUfTzoyERGRcpZon2ANYLW755lZLsFE+XxTgLZJRyYiIumX7KovGdYemmhNcBbQNPx9MnBZzL6LgCVJxCQiIhExtvYLJvSI+gWUUaI1wY+B44B3gUeA/mGNcDPQEPhXasITEREpPwklQXe/M+b3wWZ2BHA2sCMw2N37pSg+ERGRcpOSyfLuPgYYk4pziYhIdIKb4yazYkwKg0kDrRgjIiIFsm2eYFlupfQjpb9Zrrv7fomFJCIikh5lqQmOR3eMFxGRNDCzwUAX4N/ufnfM9vrAE8AZQE1gNPAPd5+YyHXKcheJ8xO5gJSPanVrUb1+7ajDiNQn/7sn6hAi99LTI6IOoUK48d+nRx1CpKrsUCNl56oIC2ib2QUEi7IU3W5Af6A5cCOwHLgDGG5m+7v7vLJeK9F5giIiIikX1vSeAW6Os/s04DDgYnfv6e6Dw205wK2JXE9JUERECiQ1UT7JQTWhx4BJ7t4zzr7TgAXuPjx/g7uvJKgdJtQcoNGhIiKyVeqWTWsW5zy57p5b8qF2OHAJcZpCQ/sBk+JsnwxcYmY7ufuasoSrmqCIiJSHocD0Io+bSipsZtWB7sCT7j61hGINCPoBi8pPrPXLGqRqgiIiUh6Oofi9ZUusBRL06dUE/l1uEcWhJCgiIgVSOFl+rrvPKN0x1gy4C7gSqGFmscNda5hZPWA1QS0wXm2vQfgzXi1xmxJuDjWzXc3sP2b2pZlNMbN9w+3Xm9lBiZ5XRESik4ORY0k8EruPxF7ADsDbBIks/wHBDRmWA20I+v7iLcSyL0HSLVN/YPB6E2BmvwcmAtcB64BWBC+A8Pe/J3JeERHJSj8AR8d5QJAYjwZmAP2A3cysU/6BZlYHODXcV2aJNoc+CcwmmM2/BtgYs+8rgtsriYiIbJe7rwBGFN0eji792d1HhM/7EawQ87aZ3cLWyfIGPJ7ItRNNgp2Ai9x9hZlVKbJvEdAkwfOKiEiEKvIC2u6eZ2anEFTEXiRogRwNHO3uvyRyzmQGxmwpYXtD4LckzisiIoK7F0up4TzDK8JH0hIdGPMNcHEJ+85C9xYUEclI+WuHJvPIJInWBP8NDDaz/kAPgrtLHGlm1wDnsrVDU0REpMJKKAm6+2dmdi7QFTg53Pw0sAA4192/SlF8IiKSRhW5T7A8JNwn6O69zawPwZyNRsAyYKK756UqOBERkfKU1Iox7u7EX8xUREQyUeoW0M4ICSXBsCl0m9y9VyLnFhGR6Kg5tHTeLWG7x/yuJCgiIhVaoknwD3G2NQROAc4GLk04IhERkTRJdHRoSfd6GmVmWwjWFB2dcFQiIhKJZOf6Zdo8wfK4qe5w4LRyOK+IiEhKlcf9BA8iuLOEiIhkGCPJgTEpiyQ9Eh0demuczdWB1sCZwMvJBCUV15AvvqLbq28w4cefsJwc9v5dM+7/+40c2f7gqENLuQVLl/Lie70YP20aU2bNYv2GDYx56032aNy4oMy8xYu554UXmTxzJstWrKDmDjvQ6ne/4/rzzuOY9odEGH1qNG+zJ5c9dmWx7evX/Maj5z4MQJN9mnLMJcfRqHljdqxTk/Vr1rNw5gJG9hzOvJ8SWtM4I5x21XWM+va7uPs6d+xArxe6pTkiSUSiNcFH42zbAswHngEeSDgiqbBef783tz36OFeefy7/vPoveF4eE6dO47f166MOrVzMWbCA/iNH0rZFC9q3bs3Ib78tVmbtb7/RoG4dbr3sMprssjNr1q2jx6CPueTuu3n53ns56YjDI4g89Qb9tz8Lps0veJ6Xt3VNjB1q7UDuwlx++Ow7Vueupla9neh4xqFc/tiVvHrLy8yfNi+KkMvdE3fcwuo1awttGzdhEvc83ZUTOh0RUVTJC6ZIJNMnmMJg0iDRJFgzzrZNWi2m8po7fwF3PfE0D/zjJq696M8F2zsf1jHCqMpXhzZtGP9+MNPnnUEfx02CrZo356l//rPQtmPat6fDxRfz3qefVJok+OsvS5k3NX6tbvb4WcweP6vQthnfTuPWd++ibef9K20SbLXXXsW2vdWnL9WrVePMLsdFEJEkoswDY8ysOnA/0NrdN8Q8lAArsR4f9SPHjMvOOSvqUNImJyexcWNVq1ShTq1aVK1S9Fab2WPT+k1s2bSZvC3Z87Gw7rf19P1sKF2OPJz6detGHU7C8ifLJ/PIJGWuCbr7RjP7G/BxOcQjFdSY78fTYs/m9B78KU+99D9+WbiIZk2bcO1FF3Dl+dtdQKjSy8vLI8+d3JUr6TFwELPmzeeB666POqyU+dMt57JjnR1Zv3Y9M7+bzmevfcLKpSsLlTEzLMeo3aA2h5/TCYDvBo+LItxIDBw+gjVr13HeKSdvv3AFZmbkZNEUiUSbQ8cD+wKfpzAWqcAWLV3KoqW/cv8zz3L3jdfTfPfd6TtkKLc98gRbtmzhmgsviDrESD388it0/+ADAGrVrMmLd97JEX88IOKokrd+7XpGffgFcybOYcO69TTZuylHnNeJvzx1Ld1vfJ61K7f2iZ1zx/nse3hrANYsX0OPe99k6S9Lowo97XoNGMQuDepzbCXuIqiMEk2CtwJvmtkMd/8slQFVdGZ2P3BfvDseV2ae56xZu5bnH3yMU4/tDMCR7Q/mlwUL6Pq/17n6z+dn3DfAVLryT2dy+lFHsWR5Lh8M+YwbHnmE7tXu4bgOHaIOLSmLZi1k0ayFBc9/njSHnyfN4aqu19L+9I4Me3PrP/8hrw7my/c/p+4udTn4lA78+f6LefOu11gwfX68U1cqC5cuZeTYcVx9wXlUrVoeM8+kvCQ6Wf5VoB7wiZmtNrPpZjYt5lHSijKVwStA1n3Vq18v6OM4qmP7QtuP7tiBJctyWbT01yjCqjCa7rIL7Vq15LgOHeh+z9388Q9/4KGXXoo6rHKxcOYCls1fRtMWuxfavnzRchZMn8+Po6bQ4943WLtyLZ0vPjaiKNPr/YGDycvL4/xTT4o6lKSpT7B0vqXwYtlZw93nAZVzuNs2/H7vvfhmwsQS9+fkZNhffjlr27Ilr/TuHXUY5ctL/gjYsnkLi2cvovFeTdIYUHTeGzCQ1i1b0Lply6hDkTJKqCbo7ue7+wXbeqQ60IrCzO43M495XsfMnjezBWa2wcymmtk/LGwbNLPGZpY/mCjeudaZWf10voZEnNz5KACGjxpTaPvQr0bTdNdG7LrzzukPqoLKy8tj7KRJ/K5p06hDKRdNW+xGw9123ubUh2o1qtG0xW7kLsxNY2TR+H7Kj0ydNTvjB8Tky187NJlHJil1TdDMZgFnuvv4cowno5hZDjAQ+CNwLzAROBl4GtgFuNPdF5nZR8DVQLeYY6sAfwF6ufvyEs7fAGhQZHOzVL+O0jjuiMM4/OCDuPmhR1i2fAXNd9+NvkM+Y/joMTz34L1RhJQWAz4Pxn5NmD4dgGHjxtGwbl0a1q1Hx3ZteerNN1mxejUH7bcfjeo3YMnyXN79eDA/TJ3KC3fcHmXoKfGnW85hxeLlLJyxgPVr19N4ryYccW4nVi9bxdf9gjXyT7nhdH5b8xsLps9n3cq11GtUj0NO7chODWrT+8n3I34F5a/XgEFUrVqFs0/qEnUokoCyNIc2B2qUUxyZ6iTgcOByd3893PapmdUC/mlmT7v7r8CLwHAzO8LdvwjLnQzsDvzfNs5/E3Bf+YReNmbGW12f4KFnX+Cx/77EilWraLFnc7o/8hBnn3RC1OGVm2seerjQ8zuffQ6Ajm3b8sFTT9Jmnxa83Kc3fUeMYPXadexSvz777rUXfZ5+moNb7xdFyCm15OfFtOnUjkNO7Ui1GtVYs3w1P46awvC3h7JuVbBE8Pyp8/hjlwM58ISDqb5DNVYtW8X8qfPo2603S+YsjvgVlK9NmzbTe/CndO7YkV0aFP2+mpl0U10piyOBPOCdItvfJqjldQT6u/sIM5sCXAPkJ8FrgAnuPoaSPRueK1YzYGiygSeizk478cSdt/HEnbdFcflIzB/y6Tb3H39oR44/tPKOk/qy1+d82WvbM6G+H/It3w8pvppONqhWrSpTh30SdRiShLImwawcDLMNDYBcd99YZPuimP35/gs8GfYN7gScANywrZO7ey5QqFMl09rbRSSz5C96kMzxmaSsSfABMyvNWHh392y4u3wu0MDMqhdJhI1j9ud7E3gEuAyoT3C7qR7pCFJEROIraxLcH9hQinLZUmMcCdwCnEPhhHYhsBEYnb/B3VeZWQ+CZtCdgJ7uviqNsYqISBFlTYJnuPvYcokkM30MfAn8n5ntAkwmGCxzJfBIOCgm1osESRC2PSBGRCQSGhgjpebueWZ2MvAf4DagITAHuBnoGqf8BDObBqxy9/h34xQRiVCyc/0qe5+gQB1gWf6TsEnzBrYzyAXAzFoBLYCryi06EZFkJLv0WWblQCXB0gonrh8GnAlsa1pDvGN3B/YBHgAWUnxKhYiIRKDUy6a5e06W9wceCbxL0Nz59zIeeyUwDNgV+LO7/5ba0EREJBGqCZaSu38E1Erw2PuB+1MZj4hIeci2PsFEb6UkIiKS8VQTFBGRAtk2RUI1QRERyVpKgiIikrXUHCoiIjGya6KgkqCIiBQI+gSTGR2awmDSQM2hIiISKTPrYmbDzGyRmW0ws3lm1svM9i1Sbg8z+8DMVprZKjPrbWbNkrm2aoIiIhK1BsC3BDcZWEpw8/DbgTFm1sbdfzazHQkWHdkAXEpwt6KHgeFm1tbd1yZyYSVBEREpEMUUCXfvCfQsfB4bC/wEnA08RbDm8l5AK3efEZaZAEwnuDvP04nEq+ZQEREpYDmW9CNF8m9UsDn8eRowJj8BArj7bOAr4PREL6IkKCIiFYKZVTGz6mbWAugOLGJrDXE/YFKcwyYD+8bZXipqDhURkQIpbA5tFmeUaa67527j8K+BA8PfZwCd3X1J+LwBsDzOMblA/YSCRTVBEREpH0MJ+utiHzdt55iLgQ7An4FVwBAza15+IaomKCIi5eMYYG6RbduqBeLuP4a/fm1mHxPcuu524FqCWmC8Gl9JNcRSURIUEZGtkryVUkx76NzYQSxl5e4rzGwGwQ3JIej72y9O0X2BKYleR82hIiJSwNjaL5jQI1VxmO0K/B6YGW7qB3Qws71iyjQHDgv3JUQ1QRERiZSZ9QG+AyYQ9AW2BP5BMD3iqbDYy8ANQF8zu5tgsvxDwC8EI0kTopqgiIhEbQxwBvAGMBC4GRgJ7O/u0wDCFWE6A9OAt4AewGyCEaRrEr2waoIiIlLAkuwTTORYd38MeKwU5eYCZyUQVomUBEVEpIDuLC8iIpIlVBPMUL4lD9+SF3UYkbIq+g537W3HRh1ChdC/68ioQ4jU4lW/Rh1CxlISFBGRAlH0CUZJSVBERLbKIbmOsgxroMmwcEVERFJHNUERESmQbc2hqgmKiEjWUhIUEZGspeZQEREpkL+AdjLHZxIlQRER2Sp1t1LKCGoOFRGRrKUkKCIiWUvNoSIiUiDbFtBWEhQRka2yLAuqOVRERLKWaoIiIlLAcsByklgxJsOqVhkWroiISOooCYqISNZSc6iIiBTIsnExSoIiIrKVkeRdJDJs4TQ1h4qISNZSEhQRkayl5lARESmgPkGROE676jpGfftd3H2dO3ag1wvd0hxR+VuwdCkv9HyXCdOmMXnmLNZv2MDX77zNHo0bF5QZP3Uqbw8YyJgJE5m/ZAkN6tahfZs23HbF5TRr0iTC6FNnwZKlvNCzJ+OnTmPyzJms37CBsT3fYY8mjQuV+8/LrzB+6lQmTpvO8lWr6HrbrZx34gkRRV2+jvvn2ezWdk/G9xvN9x9+GbdMx0uPo1Xn/Zk5agpfdB+Y5giTkGVZUElQSuWJO25h9Zq1hbaNmzCJe57uygmdjogoqvI1Z/58+o8cSdsWLWnfpjUjv/m2WJmPho1g6pyf+cufzqBV8+Ys/PVXur7VgxOvu55PX+rObo0aRRB5as2ZP59+I0bStmUL2rdtw8hx38Qt92rvPuy3zz4c27ED73/yaZqjTJ89O/ye+s122WaZRi12Y69D92Xjug1pikoSpSQopdJqr72KbXurT1+qV6vGmV2OiyCi8tehbVsmfPgBAD0GDoqbBG+44Dwa1qtXaNsh+7Wm/YUX0WPgIG69/LJ0hFquOrRry8Q+HwLQY8DAEpPgtIH9ycnJYfa8+ZU2CVbfsQaHXNCZsT2H0em6U+OWsSo5dLzsOCb0H0Oro9qlOUIpKw2MkYSs+209fT8bSpcjD6d+3bpRh1MucnK2/8+jaAIE2L3xrjSsV5dFv/5aHmGlXWneh7KUy2QHnteJ5fN/ZfaYn0os0/rEg7GcHCZ9PC6NkaWO5VjSj0xS+f9qpVwMHD6CNWvXcd4pJ0cdSoUz/eef+XX5Clo0axZ1KJJCjVrsxj6H7seYN4eUWKZ2o3q0O60jY94Ygm/JS2N0kig1h0pCeg0YxC4N6nPsYR2jDqVC2bxlC7c9042G9epxwUknRh2OpEhOlRw6Xn48kwaPY9Wi5SWW63jpcfz87XQW/fRLGqNLsSTHxWTYXPnsqQma2etmNifqOCqDhUuXMnLsOM468QSqVtX3qFh3Pfsc30yezHN33E692rWjDkdSpPXJh1C1WlUm9BtTYpm9Dt2XnfdszLiew9MYWTnIHx2azCOD6BNMyuz9gYPJy8vj/FNPijqUCuXfL73M2wMG0u22Wznq4IOiDkdSpFaD2rQ9tQOjXv2EKtWqUKValYJ9VapWofqONXB3Dr7gKCYOGkve5i1U37FGUCDHyKmSQ/Uda7BpwyY1kVZASoJSZu8NGEjrli1o3bJl1KFUGN3e7sEL777HwzfewNnHV87RstmqdqN6VK1ejSOvPaXYvtYnHULrkw6h3z1vULNOLQ4850gOPOfIQmV2aliHPdv/nmHd+jD3uxnpCltKqUIlQTNrCTwGHAbUAZYAXwMXEMT6CHAc0BxYA4wDbnH3n4qc5xjgCWBfYD7weJxrNQdmA9cCuwFXATWBL4Dr3H1ekfJXA38FWoXX7hteOzemzN/C8zUH1gMzgX+7e59wfxfgPmA/oEoYWw93f7BMb1SEvp/yI1Nnzeahm/8edSgVxiu9+/DYq69x+1+u4Iozz4g6HEmx3LlLGPzIu8W2n3DH+cz8ajLTP5/IqsXL45bpdN2pLJ+3lAn9x7B8XmaMFjaSnCufskjSo0IlQWAgsBy4DviVIDmdRNB3WQOoDTwMLAQaANcDo83sD+6+CMDM/gAMAr4Bzg+Pux/YCdgS55p3AKOAK4BGwFPA28BR+QXM7FHgn8CzwC1hXA8Drc3sUHffYmYXhsc+SJBIawJtwzgxs72AfsAHYZmNQAug+AS8CqzXgEFUrVqFs0/qEnUoaTFg5OcATJw2DYBhX4+lYb16NKxXl47t2vHRsOHc98KLHH3IwRx2wP58O2VKwbG1d6xFy+a/iyTuVBswYiQAE/Lfh7Ff07BuPRrUq8eh+wdz4Ub9MJ7cFStYkht8Lxw/dSq1atYE4JSjOkUQdWpsXLehxIEua5atKtgXr8yWTZv5bdW6jBook+w0h0ybIlFhkqCZ7QzsA5zu7v1idr0T/twIXBlTvgrwCbCYoKb4TLjrbmA1cLy7rw3LjiKolS2Ic+k57v7nmPPuAjxhZk3dfUFYY7wFeCC2xmZm04AvgVOBj4COwIQitbpBMb//EahOUMtcFW4btp33pAFhEo0R2bj7TZs203vwp3Tu2JFdGhQNq3K6+oHClfQ7uj0LQMd2bfnwmacZMW4c7s7wseMYPrbwvLD8MpXBVfc/UOj57c8Ey+R1bNeO3t2Cf3pPvvY6o8ePLyjz2kd9ee2jvgAsHLHNP3WRyFSYJAgsA2YBj5rZrsAId58eW8DMziWokbUCYmdot4r5vSMwKD8BArj7L2b2FbBnnOsOKvJ8YvizGUHSPI6gJtrDzGLfr68Jku2RBElwHHC9mT1H0FQ6yt3XxZT/AdgEvGtmrwKfu/uSuO/EVjcRNJ9WCNWqVWXqsE+iDiOtFgz7bJv7u952K11vuzVN0USnNEksPxlmi9cvfWK7ZT7410tpiESSUWGmSLi7EyScbwj6/qaZ2Swzuw7AzE4F3gN+BP4MtAcOBpYCO8ScqglB7bCoeNsAcos8z1/sL/+c+Ys/ziBIYrGP2kDDcP+bBM247QlqqLlm1jusSeLuM4AuBO/5W8AiMxtjZttqJ3qWoMk09nHMNsqLiCTFzJJ+ZJKKVBPE3WcBl1jwLrYDbgBeDOf3nQ/McPfL8subWTWKNxcuBHaNc/p420pjWfjzeIL+yrj7wyTeHehuZvXD8k8RJO72YZnhwHAzq0Ew+OdBYKCZNXf3Yr3m4aCbQkk60/7AREQqsgqVBPOFCeUHM7sZ+AvQGtgR2Fyk6MUEoyxjjQZOMrNaMX2CexAknXh9gtszBMgDmrl7yeslFY5/OfCembUHromzfwMwzMx2Img63ZNgIJCISLSM5IZ4Ztj39AqTBM2sLdCNoOY0gyC5XUaQ+IYBq4AzzOwZYABwEHAjsKLIqR4GzgE+NbMnCAaj3E/JzaHb5O4zzewx4HkzawWMJJj+sAdB8+0r7j7czF4i6CMcTTC1oyVBkv40fH3XEvQfDgJ+AXYmGJm6AJiUSGwiIpWBmZ1NMMDxIIIuqLlAb+A/7r46plx9gulvZxCMwB8N/MPdJxY7aSlVmCQILCJ44TcDuxMkmonAKe7+rZl9T5B4riCoXY0jGJnZJ/Yk7v6jmZ1E8Ea9RzAX7zGCATNHJRKYu99pZj8SzBP8K+AEiWwokD945yvgcoLEV5cgub3N1oEt44ETCfo7GxE0c34JXOjuvyUSl4hIqgUrnyUxRSKxQ/9F8Pl/JzAPOICg8nJ0OA0tL+wm608wD/tGgu6pOwi6mPYvOre7tCpMEgxHSl66jf15BNMf7i6yq3mcsp8RvImxuhcpM4c4FXd3H1HC9rcIBrSUFN8bwBvb2D8aOL2k/SIiWexUd18a83ykmeUSfKYeRdAaeBpBt1bncHwFZjaaYNGTWwlG05dZhRkdKiIi0YtidGiRBJgvf+LtbuHP04AF+QkwPG4lQe0w4QpGhakJiohIBWAkVz3amgObxUmIubFLTW5H/vSxH8Of+xF//MRkglkFO7n7mjJECqgmKCIi5SN/zETso1RNlma2G8EUss/c/ZtwcwPiT1PLT6r1EwlSNUERESkPxxAMdom13VpgzNSxzQSDDcuVkqCIiBRIdtWXmGPnhitlleXYmgR9fHsBnYqM+FxO/Npeg5j9ZabmUBERiVy4AtgHBHMFT4oz928yQb9gUfsSJNwy9weCkqCIiMRKdmRoArVIM8sBegCdgTPcfUycYv2A3WLXWzazOgTzxfvFKV8qag4VEZGovUCw0te/gbVm1iFm37ywWbQfwQoxb5vZLWydLG/EuXF6aakmKCIiW1kKHmV3YvjzLoJEF/u4EgoWTDmFYD3nFwlWC9sCHO3uCd+1WDVBERGJlLs3L2W5XIKlM69I1bWVBEVEpIDlGJaTxOjQJI6NgppDRUQkaykJiohI1lJzqIiIbJXgNIdCx2cQJUERESlgJJkDUxZJeqg5VEREspZqgiIiUiCFa4dmBNUERUQka6kmKCIiW+UAycz1y7CqVYaFKyIikjqqCYqISAH1CYqIiGQJJUEREclaag4VEZGtEr8d0tbjM4iSYOapBvDLwoVRxxG5TV1mT0kAABo8SURBVKvXRh1C5IIbcsviVb9GHUKkfl2zPP/XasmeK9v6BJUEM08TgLOuuzHqOESk4mkC/JjMCcySvJWSkqCUs6+BY4CFwKaIYmgGDA3jmBtRDFHTexDQ+xCI+n2oRpAAv47g2hlNSTDDuPtaYFiUMcR805vr7jOijCUqeg8Ceh8CFeR9SKoGmK2UBEVEZKssu5WSetVFRCRrqSYoIiJbJTk6VDVByQa5wAPhz2yl9yCg9yGg9yFDqSYoZebuucD9UccRJb0HAb0PAb0PmUtJUEREttKKMSIikq0sJ8nJ8sncizAC6hMUEZGspSQoIiJZS82hIiJSWIZNc0iGkqCIiBTItrtIqDlURFLCzKrE/J5Zn4SStZQEJS3MbGcz+1vUcZS32ESQbdx9i5nVMrN93d2VCMvGKsrNIXMs+UcGqRhvulRqZrYDMAS4wsx2jzqe8mJm5u5bwt8Hm9nlUceUTmHSexn40MxqurtHHVOmMLMq7p4X/r5X1PFkEyVBKVfhB+MxwFLgfGBBtBGVj/BDzMPf/wW0AZZUmG/3aRC+/h+BusCeUIFqNxVY+LeT/+XpGaCbmZ0ecVhZQ3+gUm7CGuCnwGVArrv/6O55lbGZLOZDrC2wL/AM8Gn+t/vKLibZPQ04cBNAtrz+RBVpPegFnAb0BSZEGFPSj0zy/+2dd7hU1dWH3x8gCopdI2qMRomfJbHEhkbAkthjicqj2JXE2LDEKEEiFmL3MxpLVNSIqGCNnSiKoICAqF8U0YCAiKJBsAFXQNb3x9pzGca5wK1zZ856n2eee++Zvc+sOXfmrL3qDiUYNCZrAW2B3wAbSlodqi2GikNSH+BNYE9gvJktqESFDyBpiczytLhpDXwLPAjsIWnzkghXRuR5D64CdgSOAu4zs8mSWktaucmFUgM8yohQgkGjYWbTcSvwEWBn4EBJK5RUqMblDuAlYCPgIEntKlXhm9lCSatI+pOkfdOx+Wa2ELdkNgN+UVIhy4Sk6LYBHjSz0WZWJWkL4H7gRUmPSdqotFJWLqEEgwYlZ/kkqwBgInABrhxuxBVh2denFlp4ya01HegGDAeOBPaTtGIp5GsiDsZ3Thgg6SlJR0taw8yGAf2Bsyo5EaquFImTGvBjYGNJP5HUExgHtAfG44uJK5pWyuwQSjBoMNKK9i+SHsZvjEcBrczsA6A7MBboR5krwoIkGKX3IgAzmwF0BSbhccGDKkURFip+MxsAbA38FlgPuBJ4U9Jvge+ANixOkMls6Ug+BVmg20lqb2ZzgUuBQ/HvyPHApWa2G35tn8bDCU1yv85aTLBsb0RB80JSO/wL/DUwG19gDQD2lXSFmU2Q9DvgVuB2oIekQbmkgHKhIJOvN54FugEwRNItZjbDzGZIyiU43AiYpKfNrKp0ktcPSa2SC7QlsAYe651hZu8D70t6DNgDzwA+H///b4InyAwvt/9zY1Dw2fk77jJ+QNJdwCBgNLA+8JmZjU/T2gE/ACbQVNE21bPWr8yUYFiCQb1JllA/4BPgaDP7pZnthcc0jsGtBMxsMnAaMA04rtxujJJa5N3EHgROwUs+3gVOBQbm3H/JIjwYmIxfh31KInQDkG7eC9NC5wFgGH7DfkXSnskFusjMhphZd9wl3BvPcNxd0o7pPOV1d2xACrJAH8LLhvoDz6ZrN9/M3jezoTkFKKkDcDWwPXB9uX1fyoVQgkFDsApuET2FxwBJrtCuQC8zG5pzCSbX6EHAASWStc7kubGuBX4OdDWzs4G3gdXx0ohHJW2Qxs8AjsBjhOOLnrQZUsTt+V1ydY/Gk34GAfcArYHHgKMkrZI3frSZ3Y//n9sA+6fjFZkktDzkuc/PB3YCjgXuN7PpktrJOypVJ7/Ia0374cryV2b2XinkbkokbSjpJkkjJc2VZJI2LjJuJUnXSPpE0rw0vlNdXzeUYFAvUpxiAzywn6sDPBZ3hfY2syvTDfQuSbsCmNnHaVzZff5S1t5PgJ5mNkrSBcA1wHHAtcAOuEWYU4QfA/uY2X9KJXMdaAfV8c6cQrwYL384xsz6mNmFwK+AwXjSxg5pTsvcXDObhidE7VYpcdEGYFvgNTMbCSyQtC3wDPAq8Lqkw9K4BcBrwC/N7K2mFLCEMcHN8ISy2fjCsSb64TkGfwYOxD1Qg9O1rDVldxMKmge5m2Oyjt4DPsAtgsNwK6G3mV2Zhm8DbApsnH+OMi2k/g9+Yx8qaT/gXOAUM3vQzK7Cb2Y7p+dzrtGycWNJ2gaYIWlTS6SntgC+BKbkxprZZ8CZwFQ8IWqF3HtNvUNXxBdIc4By/F83GHmLiS+Bn8pb6vXFFd1s3GX+FnCVpNXwuHkvM5vY9MLicb06P+r8ysPM7Admtj/wUFHR/PN5NHCOmd1hZkNwxfkhnlxUa0IJBrVGUhvgIUlbpUOt8FjRgcDDwIVm1jcZEpvjltLnwMCSCFxHilmqqQ7uBjObCXTBleJzeUPmAGPw91uONZGrAReZ2aTcjVte7rI60DK9/+prY2af4n1hfwSsnTtJen5/PB7cx8wWNOm7KDEqyIbNW0z8E5iOJ0ztDvzRzH5tZpcAI4F5wHwzm2dm85tS5lKznIviX+NWcvW9JH0mHwT2qYvHIbJDg1qRbm4HA3sDm0jqamYTJd2NxwU7Azsk1+fuwGHAikDnFFuqzpBrzhRk8m2Ixz2nAXOTldMSjwGumRQBktbGb2I3ACPN7MvSSF93Uo3fMHnLu5clXWRmz0vqD9wm6fdmdmvBDasK7w1blXeeRZLGAjsnl3BmKPjsHIIvDlYCHjGzwZJG4xmfVWY2JY1bEw8pTKLExkk9XZr5czcqcp5ZZjarzieHrYDJqawkn3fwGPVm6fflJpRgsNyk7MDr8dX9V8B2wMOSjjaz8ZLOBU7CE2KeBd4H3gBOS9mFrXKWRHMlt4IvSGXvjCeEzAJulvSomb2nlNouqR8wCm+X1hE4sxwVYAFb4tbf45L2Bu4GDgGuS/G+W9KqexM8yen/gC/yT5BigplCS2YQD8I/D62AVYFzJN0K3GZmE/Lm7IhnF+8P/MLM5jS95I3CkCLHLsEbLNSVNXH3cSGz8p6vFaEEg+VCUltgBB6EvhM4G/g9Xhc2MFmE4yVdCVyOr2o/wle71twVYEreaZPcnLljA/BuHVcCn+Gxvr5AR0mnAK/gX+jzcev4v3gSzEdNK33DY2bj0nu8Eo+B7oInI9wO/E1SNzzOtwawEDgp/Z+V8SzQXAbxTfg1Ow6YamYfSHoT6IXX/D2VxvXEM0A3wL0ltbJimjl74bG6fOpjBTYKoQSD5eVg/It6KjAqrXb/IGkiXhN2v6TDc4F8SZPybghq5gqwHZ7Y84Sk7smVty+eyn46MNi8GfYQ4A94HPAr8+L3vskVvBbwsZl9XqK30WDkFJmZDU836atwS3cX/HNwHJ7iPxMYClxSLpZ+U5Dc5zsD1wEjzOxbSWvh3oRHWdJCmgTMx12lU5pa1qLUd2PcxXM/bITEntl4/LmQnAVYayUbSjBYXtrjcbHRKbbX2rzA97aU/NIDT5bpat5FpNoaaM6WgbzQfxCu2M7Mi3WtjbsD304KcAvc8nsETxypSm6s8SnmVTFxr3yLzsyGyctAcopwTzO7S1L//GSXFAfLvAJMrIK3k5uVFGAHvMbyefwzNk/S8cCrZjao+S0e6tv6rFF7IrwDHCqpbUFccEt8MVFrpRvZocFSyUvtHpd+HgO+Y4AWN8m+A894Wwl3la3bnBVfAe3xFfqTZjZX0t3ygub1ge/MbEq6ib2K38ROTDexbsB5uKKsOHKKMP0+DG+CPg54SVLntDBokZcl2uyTnZqYBUALSevjZRD/Ak5On7Ed8G5DW0N1dmOwfDyJZ10fkTuQFrJd8f07v63tCUMJBkslT5lNw1dhJ0vqkp6bnxJJtsP30XsA3xNtqyKnaq58gsfyuid354F4jdJzwLopkWEE8AJ+E/tG0rp4ofiKeElERVJEEfbE+8M+K2kn83Zfma3/KyyDyJGSXh4DbsY9DE8AJ5jZ1ymD+Pf4Z2d0U8laLkg6XNLheEcm8J1YDpfUGcDM3sDLI26QdIqkvfDyiE3whg61JtyhwfeQ1wF2BVbGa97eNq8b+yNe55SLg92Px81Ow1e7l+PxwY54MkWzJherlBcu/xu3Ck/DYxmLJF2Px0Cn4r1OqyRtClyE9wLdw8y+qOn8lUCBa/Tl5Bq9Frf496uEGGhdKCiDOAJXarPN7Ok05Fbcm7AHvgtEu2QBngrshyfBNE8Xeq7ovT7z605hkfwt6efLeF0uQK7RwOW4J+YtYF8zG0cdCCUYLEFKEhmBJ3qsh1tJAyX9OdWLHYxnDN4K/B34Bu8Y0xPYPI2fVArZa0uelXsKXvIxD38f7+Cr9Nvxm9vpwJNp5S+8FmkfM3u3yYUuAQWK8BVJL+EJMpndHilPAQ7Ea2ZXBNpKug1vqfeafLf4b3HL5TO8hOQboIuZ/bs0ki8btRCqR2JMfeaa2TInm9k8vFPTuXV+oTxCCQaFXIu7CE/BXaBX4/Vhq0k6OynCSbiC3AHPqnwmWU5/woPTI0ojep25HbgX75l5N57p2i3dyC7C09mPwsMHY/H44eSSSVsC8l2j+AJpJTzulSkKLMDueIOII3Bl1xG4DFhT0unmLb2GpDrLNXGPwgdm9t/SSB8UI5RgAFTXyR2A7wd4m5m9lo53xxXjwbgfvof5ThAfkJSdpEOTH39fYC8rsyJpM5ua+13SSbgiHJCSX0abd/l4PsvxL1iiH+h44CYzK1a0XLEkazinAHcFOuDuu2HJrT4O94Tchu8hea6ZfWJmL5RO6mBZRGJMkMsA7YsHmE/G98BD0krJ9XAOHgvcE7heUmFG5A54SUEnM3uzyQRvHMbiMYcqfL+3neRdQKprHkspXKlJ2Xc3Nmd3XkMiqa2kXQqO5RolHIsXwi+EajfdA3jiy8F4M+wNm1jk+lOv5tn1jCeWgFCCQS42Nggv5F0NODQdr8rVA+KK8DG8UPp0WKKJci98b72y73aRrkVOEc7Bkxq2L3g+02TFIk6f79vwXqp75/3v7wX+hvf/PFBS+9yc9F0ZAPwOLye6uKYs0qB5EO7QjJOLcZjZCElVeKynp6SZZnZjrh4w/Twf30rnr1DdJLlFSpWvmCzJ5PYbi2eKXkfxXoVBhZM+30/i24DdJe8mNDjvu9AKV3YjJd2Vy5RNNZQD8XZyb5RbDaUbc/VpoN2AwjQBoQQzSlqdtsLdmNOhul/kxXhPyEslkacIV0yusOtz85PyrEirIK36R0ra07w9WpAh8rJhH5I0F2+Xd6ekE8xsiHknmB54VmjfNCdfEc7HXaPlR2lLJJqccIdmEEmr4MkfI4E3Jd0s6ScAZvY6nuE2DLhE0hnp+BKdGMptdVtXQgFmlvx746d4GVBb4N68ZhEL8Lq//rgiPCEVwwdlRFiCGSMpwDHADHwH+C9xhbiOpGvMbIyZjZF0Kd7x/kZJM8zs4ZIJHQRNSEEW6KPAD/ESiFF4oft9kk40s+eT6/NU4Dt88+j5km6uVA9JJRJKMEPIe332x+v/jjazmZIexIt4D8Drm3omRThW0jX4noCPl07qIGhacgkwknrje0kegHdN+kaLe8beLen45BpdIOl0PKP4hXJXgFI9i+XDHRo0Yzrg8b6rkwIcCOyWHscAnYALJe0MYGYjzOyCVAMVC6YgM6RSmC3wdnpjgbkAZjYAuALfJLd/vmvUzM6qiC5CUSIRVDCT8JTvl9LKdSfgqPTFfQnfHfxQ4CZJ/5M/0aLTfZAhkjXYFljHzBamTNEV0nMPAf8A1gGeSR1hgjIllGCGSEkeL6R4Rydc6b2anvsCzxK9A98s9T+lkjMISkmu/hVfGK6fiuNzpQ85j4jwraWex9uhBWVKuLgyRqqBa4mvctdMj88lbYOXS1xuZmNgyT6JQVCJFPuM58X0HsE7KJ0naa6Z3Z9CA2sAa+A9Zwea2TdNK3XjMm3GjHq5NKfNmNGA0jQ+igYY2US+K/qrwLO4m7QTngHXKRRfkAUKmmH3wHcH2RDPln7NzD5NYYGn8CYSY4AJ+J6ZOwE/N7OK8ZhIWgf3AK3WAKf7EuhQDs3CQwlmGEm74+7PlsDbwJHJ5RMWYFDRFPSDfQDYBXgD94xsie+Veb2ZfShpYzwjtAu+x+ZU4KxK7J+aFGGDKMFyUIAQSjDzpLrBFYFZyVXaKpJggqwg6SZgf6CbmY2SdBreF3Q23vHlCjObnmKBhiuIKjObWzKhgwYlYoIZJ8UzvoHq1XEowCATSOqIuzbPSgqwJ3ApcDieJX1aGneFmU1P02aVRNig0QhLMAiCTFDo5pfUBt+d/FZ8m7DbgR5m1j+VQ7wFtMFbCF5gZuWV8REsF1EiEQRBxVOQBHOupMPT/n9Xmdks4CDgBRZ3RxLeAaY13kwiqFDCHRoEQUWT3Pw5BXg/vgn0a5JeNLNZKd63OR7r+zpNWx/4AHeNfl0uSR5B7QklGARBRZOXBXoP0BHfGHp8UoAtUu3fc0B3SZfhu8Z3w8sg5oQCrGwiJhgEQcUjqRPe6uwMM3s6HVsHd4N+i+8CcQSwN95f9wvgEDN7qzQSB01FWIJBEGSB1YF2wOTU8aUznhDzHe76fBG4GLgaWAt408w+LpGsQRMSlmAQBBVP2jT6LbwgfhGwNXAf8FdgFeB1vFawPHeDD+pMZIcGQVDxmNn7wD7AV3i7wJPN7IzU9mw+3i4sagAzSLhDgyDIBGY2TNKrBbWC7YEzceuw4tqgBcsmlGAQBJmhQAEejxfJHwjsETHAbBJKMAiCzJFapp2JtwzsZGbvlFikoEREYkwQBJkjbZzbAW8cH3WAGSaUYBAEQZBZIjs0CIIgyCyhBIMgCILMEkowCIIgyCyhBIMgCILMEkowCIIgyCyhBIMgCILMEkowCIIgyCyhBIOKQ9IJkizv8bWktySdkXYRb+zX7yPJCo6ZpD61PM/Zkg5rUOH8vFPSBrNLG9Mlydyljue/r67yFTnfUElDG+p8QZBPtE0LKpkjgI+AVdPvNwHrAn8ugSwdkyy14Wx8l/NHG16cIAgglGBQ2bxpZhPT7/+StBnQgxqUoCQBK5jZ/IYWxMxGNfQ5gyCoP+EODbLEGGBVSevCYredpJMkTcD3lTsgPddW0lWSJkuan372Sj0nq5G0naThkqokTZfUG1DhCxdzh0raRtJjkj6XNE/Se5J65mQDfgR0y3Pr3lMw9wlJs9PcVyXtXuR1e6T3WSVpbLExy4ukX0l6RtInkuZKelvSeZJa1jC+u6SJ6bXHSdqjyJjOkoYkl/UcSYMlbV1XGYOgtoQlGGSJTYDv8J0DcuwBbAtcAnwGTElxw8HAlsBl+D5zuwC9gTWB8wAkrQ28CMwAjge+Bc4HNlqWIJJ2AoYCE4FzcFdpB+BnacihwDP4buh90rH/prnbA8PxXdK7A3OBU4EXJO1qZq+ncScDNwD3AAOBzYAHgHbLkq8GfgwMwd3KVcAOSbZ1gAsLxnYBfg70wq/LBcCzkrYxs/eSfAcA/wSeBo5J8y4Ahkv6mZlNq6OcQbD8mFk84lFRD+AEwIDN8YXeGsDvcAX4eN64KbgCWa9g/rFpfqeC471wa3Hd9Hff9PcP88asDMz0r9YScw3ok/f3MGAa0HYp72MKcF+R40OAd4HWecdapmOPp79bpPM/VzC3a5LlnmVcwy5pXJcanle6tr2A2UCLArkLr0s7fOf2/nnHJgJDCs67arp+N+QdGwoMLfXnKh6V+Qh3aFDJTAAW4DffW4ABwEkFY0aZ2YyCY/sCU4ERklrlHsC/gBVwqxA82WWU5VksZjYHeHJpQklqC+wGDDCzubV5Q5LaAJ2Bh4BFebIJeAHolIZumB6DCk7xCLCwNq+Z99rtJf1d0lRcyS0ALgdWxxOO8im8Ll/jFl/HdK4OwKbAgIJrPBcYmfc+gqBRCXdoUMkcirsZvwammllVkTGfFDm2Lh6PW1DDeddKP9sDbxd5/tNlyLUGbqnVNlsU3B3bEnfN9i42IMUt2xeTxcwWSvq8ti+azvkEsD7uAp0AzAMOwa3BlQqmFLsGnwIbpN9zSrNfehTyYW1lDIK6EEowqGTetsXZoTVRbEPNz4HJwJE1zJmSfn4C/KDI88WO5TMbWMRihVAbvkhzbwbuLTbAzBZJyin3JWRJ1tZa35+1TDbFY4DHmll1DaCkg2oYX9N1mZ5+zyninrgFW0iDZ+gGQTFCCQbB93kO+A3wjZlNWMq4kcD5kn6Yc/1JWhmoSTEAYGZzJb0CHCPpUjObV8PQb4E2BXPnSBoObAOMM7NFNcz9CI8JHgnclXf8N9Tte982/ay2jiWtAHSrYfwuBdelHZ55+3R6/j18MbGVmV1ZB3mCoEEIJRgE32cAcCIwRNJ1eIZma9wa+jVwSIrl/S9wGl6D2IfF2aE1KbV8/gC8DIxMr/ERnn25rZmdmcaMB3aXdCCegTrTzKYA5+KJNYMl9cMt0rWB7YGWZnZhsgYvAe6UdDfwIJ4deiHwVR2uybt4nLSvpO9wZXjOUsZ/ypLX5QI8aegy8KwhSacD/5TUGo9dzsStxV2BD83s+jrIGQS1IhJjgqAAM1sA7APcAfwWL1UYgJdBjCC56sxsJrAXfvP+B+6ifI4lLa+aXmMMnhwzDS85eAZXoPlxwp64xTQIr3Hsk+aOA3bEXYo34gk7fwV+iivH3Gv0w7vO7ImXIpwIHIW7Y2uFeQOBQ3BlfG96r8OAmqy4l4HrgL/g5RkrAfuZ2ft553wGT4BZGbgTL0u5GlgPt7KDoNGRWbGQSBAEQRBUPmEJBkEQBJkllGAQBEGQWUIJBkEQBJkllGAQBEGQWUIJBkEQBJkllGAQBEGQWUIJBkEQBJkllGAQBEGQWUIJBkEQBJkllGAQBEGQWUIJBkEQBJkllGAQBEGQWUIJBkEQBJnl/wH36ENNmiyrfAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10df94080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot your confusion matrix\n",
    "my_tags = ['anger', 'fear', 'joy', 'sadness']\n",
    "plot_confusion_matrix(cm, classes=my_tags, title='Confusion matrix')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### 5. Your show time\n",
    "\n",
    "#### Check what you are assigned to do here:  \n",
    "* Google sheet  \n",
    "https://docs.google.com/spreadsheets/d/1FMjT0LasiVIERRQH8Co8FhKnl2qwODRxOA87ivRSKMA/edit#gid=531101419\n",
    "\n",
    "#### Share what you find in our in-class practice here:\n",
    "* iLMS forum (title: Lab2 in-class discussion)  \n",
    "http://lms.nthu.edu.tw/course.php?courseID=36071&f=forumlist\n",
    "\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**To complete these tasks, you could refer to the official documents below**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Feature engineering methods:\n",
    "    * Bag of words  \n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html\n",
    "  \n",
    "    * TF-IDF  \n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Models:\n",
    "    * Decision tree  \n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier\n",
    "  \n",
    "    * Naive bayes  \n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "**During doing your experiments, please download this data at the same time:**  \n",
    "https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The data size is around 1.5GB before compress, but you **don't** **don't** **don't** need to decompress it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Other things you can try\n",
    "![Imgur](https://i.imgur.com/n7w94Dj.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}