Probabilisticclustering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Probabilistic clustering with Gaussian Mixture Models\n",
    "=======================================================\n",
    "In KMeans, we assume that the variance of the clusters is equal. This leads to a subdivi-\n",
    "sion of space that determines how the clusters are assigned; but, what about a situation\n",
    "where the variances are not equal and each cluster point has some probabilistic associa-\n",
    "tion with it?\n",
    "#### Getting ready\n",
    "There's a more probabilistic way of looking at KMeans clustering. Hard KMeans clus-\n",
    "tering is the same as applying a Gaussian Mixture Model with a covariance matrix, S,\n",
    "which can be factored to the error times of the identity matrix. This is the same covari-\n",
    "ance structure for each cluster. It leads to spherical clusters. However, if we allow S to\n",
    "vary, a GMM can be estimated and used for prediction. We'll look at how this works in\n",
    "a univariate sense, and then expand to more dimensions.\n",
    "#### Implementation\n",
    "First, we need to create some data. For example, let's simulate heights of both women\n",
    "and men. We'll use this example throughout this recipe. It's a simple example, but\n",
    "hopefully, will illustrate what we're trying to accomplish in an N dimensional space,\n",
    "which is a little easier to visualize:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "N = 1000\n",
    "in_m = 72\n",
    "in_w = 66\n",
    "s_m = 2\n",
    "s_w = s_m\n",
    "m = np.random.normal(in_m, s_m, N)\n",
    "w = np.random.normal(in_w, s_w, N)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.\n",
      "  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.\n",
      "  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f8f14afefd0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbIAAAFCCAYAAACU4VgFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHwlJREFUeJzt3XuUXXV99/H3NxeEQMgFSkKAycRwtU8fQSQEAsnQulKj\nKFgfECFC0Lq6CjxtbF0VtIYkSwr0QbSPla4WC0QFAugqctEafXAkESxaJ2AIhKBMJuQyBkhCbuZC\nfs8fZ2c4GeZyZubM5Ufer7Vm5Zzf3ue7f2dnZz75/fY++0RKCUmScjWovzsgSVJPGGSSpKwZZJKk\nrBlkkqSsGWSSpKwZZJKkrBlkykpELIuIqf3dj/4UER+JiKaIeD0i3t0L9f8lIr5Q4bp3RsT8avdB\n6gqDTANGRLwUEX/cqu2KiFi873lK6X+klB7vpM74iNgbEW/X4/v/AFellA5PKT3demHx3t/Zqu36\niPhWJcVTSn+ZUrqhGh1tqy9Stb1d/6Hr7aWrn9qP4jXRC30hIgb3Rt0uGA8s72B5e/urP+5+4B0X\n1OsMMmWlfNQWEWdExC8iYnNErIuIW4rVflr8uamYfjszSv4+IhojYn1E3BURh5fVvbxYtqFYr3w7\n10fEAxHxrYjYBFxRbPuJiNgYEWsi4msRMaSs3t6I+MuIeKHo3/yIeGdE/CwiNkXEwvL1W73Htvo6\nPCIOiogtlP7dPhMRK9vbTRXsx5MjYlFEvBoRz0XERWXL9psujIi/i4i1EfFyRHyqjVHW6Ih4pNjX\nT0bEhOJ1Py368kyx7KKIOCIiHi7226vFOlKPGGQa6Dr6pfxPwFdTSiOAicD9Rfu+c2iHF9Nv/wVc\nCVwOTAPeCQwH/hkgIt4FfB34OHA0MAIY12pbHwbuTymNBO4G9gCzgdHAWcAfA1e1es104DRgMvB3\nwL8ClwLHAX9UbK8tbfX16ymlXSml4cU++aOU0gkd7Jt2RcQwYBHwbeBI4BLgtog4uY1131+8zz8G\njgfqeOso62PA9cBI4DfADQAppWnF8j8q/h4eAP4WWA0cARwFfL4770EqZ5BpoHkwIl7b90MpYNqz\nCzg+Io5IKW1PKT3Vanl5CF4K3JpSWpVS2g5cB3ysOI/2UeChlNKTKaU9wJw2tvVkSulhgJTSzpRS\nQ0rpqVTSBPwbpeApd3NKaVtK6TlgGbCo2P4W4AeUQq4tbfX1klbn/Dobdf2qbD9uBD5Xtux84KWU\n0jeL/j8NfBe4qI06FwF3ppSeTyn9Hpjbxjr/kVL675TSXkohf2qr5eV93U3pPwsTUkpvpJR+1sn7\nkDplkGmguSClNHrfD28d5ZT7FHAS8HxE/FdEfLCDdccBq8qerwKGAGOKZav3LUgp7QBebfX61eVP\nIuKEYopsXTHdeAOl0U2535U93gE0t3p+WDf6WqnTyvbjKODmsmXjgcmtgu7Sdurvt2+Kx61DdH3Z\n4+20/74A/pHSqG1RRLwYEZ/rYF2pIgaZBpqKL9BIKf0mpXRpSukPKP2C/E5EHELbFxispfQLfJ/x\nlKYHm4F1wLEtHSjVOKL15lo9/xfgOWBiMd34ha70vRNt9XU3+wdhZzrqy2qgvjzoiqm/a9pYd799\nA9TQgws4ihHqZ1NKEylN1/5NRJzX3XoSGGTKWERcFhH7RkGbKf2C3QtsKP6cWLb6vcBnIqI2Ig6j\nNIJaWEyHfQf4UERMjoihtD191tpw4PWU0vbi3NJfVuVNdd7XangEODEiZkbEkIgYGhHvjYiT2lj3\nfuDK4uKQYcDfd3Fb6ymd5wMgIj4YEfv+XrZQ+s9Etd6XDlAGmQaSSv6nX77O+4FnI+J14CvAx4rz\nVzso/fL/WTF1Ngm4A/gW8Dilqa3twF8BpJSWA/8buI/SaOh1StOCOzvox2eBy4pt/yuwsJP30pVR\nTLt9rbBWh8tTSlspXYhyCaX3uxa4CXhHG+v+J/B/gZ8ALwBPFos62jfl5gLfLP4e/hdwAvDj4urL\nn1G6iMUrF9Uj0dkXa0bEscA3Kc2f7wX+LaX0tYi4Hvg0b54H+Hxx0BMR1wGfpPS/rb9OKS3qpf5L\nVRcRhwKbgONTSqs6W/9AUow+fw28o4ojRKlHKgmyscDYlNLSYprjv4ELKF1yuyWldGur9U8B7gHO\noDS3/mPghORXUWsAi4jzgf9HaZbiy8AZKaXT+7dXA0NEXAh8HzgUuAvYk1L6aL92SirT6dRiSml9\nSmlp8XgrpRPcxxSL2zqhfAGl+fw9KaVGYCUwqTrdlXrNBZSm2F6mdG7tkv7tzoDyF5RmXlZSuuik\noytJpT7X5p0F2hMRtZQ+I/JfwDnANRHxCeCXwN+mlDZTCrkny162hjeDTxqQUkqfpjRVrlZSSjP6\nuw9SRyq+2KOYVvwOpXNeW4HbgHemlE6ldGXSl3uni5Ikta+iEVlxT7jvAN9KKX0PIKW0oWyV24GH\ni8drKN2CZ59ji7bWNT1nJkl6i5RSlz6TWemI7A5geUrpn/Y1FBeB7PNnlG7BA/AQpdvpHFTcPPR4\noPWtg/Z11p8u/lx//fX93odcf9x37jf328D/6Y5OR2QRMQW4DPh1RDRQ+ozK54FLI+JUSpfkN1I6\nIUxKaXlE3E/payZ2U/reJEdfys6cG+fQ1NxU9bo1Y2qYf53fRSlVS6dBlko39Wzr+5f+s4PX3Ajc\n2IN+Sf2uqbmJ2gtrq1638cHGqteUDmTe2SMzdXV1/d2FbLnvusf91j3ut75jkGXGfxzd577rHvdb\n97jf+k6XPkcmSQeC2tpaVq3y7mS9afz48TQ2NlallkEmSa2sWrWq21fQqTIR1frWI6cWJUmZM8gk\nSVkzyCRJWfMcmbLXWx9cbnimoVc+RyapugwyZa+3Pri85KklVa+pfM2Z81Wamjb1Wv2ampHMnz+7\n0/Vqa2tZv349a9euZfTo0S3tp512Gk8//TSNjY3U1NT0Wj8HIoNMkirQ1LSJ2tq5vVa/sbGy2hHB\nhAkTuPfee7n66qsBWLZsGTt27KjqlYA58RyZJGXmE5/4BAsWLGh5vmDBAq644oqW57t27eKzn/0s\n48eP5+ijj+aqq65i586dAPz0pz/luOOO49Zbb2XMmDEcc8wx3HXXXX39FqrKIJOkzEyePJktW7aw\nYsUK9u7dy3333cfMmTNbPvv2uc99jhdffJFnnnmGF198kTVr1jB//ps3ql6/fj1btmxh7dq1fOMb\n3+Dqq69m8+bN/fV2eswgk6QM7RuV/ehHP+KUU05h3LhxRAR79+7l9ttv5ytf+QojRozg0EMP5dpr\nr+Xee+9tee1BBx3EF7/4RQYPHsyMGTM47LDDWLFiRT++m57xHJkkZWjmzJlMnTqVl156icsvvxwo\nfcfjK6+8wvbt2zn99NNb1t27d+9+dyo54ogjGDTozXHMsGHD2Lp1a991vsoMMknKUE1NDRMmTOAH\nP/gBd9xxB1C6EOTII49k2LBhPPvssxx99NH93Mu+4dSiJGXqjjvu4LHHHuOQQw4BSiOyQYMG8elP\nf5rZs2ezYcMGANasWcOiRYv6s6u9yiCTpIyUX2I/YcIE3vOe97xl2U033cTxxx/P5MmTGTlyJNOn\nT+eFF16oqGaOnFqUpArU1Iys+LNe3a1fid/+9rdttg8ePJg33nij5fkNN9zADTfc8Jb1pk2bRlPT\n/nfCaa9mLgwySapAJXfdUP9walGSlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNa9alDSgVPN7vyr9ji/l\nzSCTNKBU83u/evNzXxo4nFqUJGXNEZkkVWDOjXNoam7qfMVuqhlTw/zr5ne63k033cTjjz/O97//\n/Za2E044gRNPPJFHH320pe3EE0/kS1/6EhdffHGv9HcgMcgkqQJNzU3UXljba/UbH2ysaL2pU6dy\n8803k1IiIli/fj179uyhoaFhv7bf/OY3TJ06tdf6O5A4tShJGTnjjDPYtWsXS5cuBWDx4sWcd955\nnHTSSfu1TZw4kbFjx/LEE08wadIkRo0axZlnnsmTTz7ZUuu8887ji1/8IlOmTGH48OFccMEFvPba\na8ycOZMRI0Zw5pln7ndfxueff57p06dzxBFHcMopp/DAAw+0LLvyyiu55pprOP/88zn88MM566yz\neOmll/pknxhkkpSRoUOHcuaZZ/L4448D8PjjjzN16lTOOeect7Rt3LiR888/n9mzZ/Pqq6/ymc98\nhg9+8INs3Lixpd59993H3Xffzdq1a3nxxRc5++yz+dSnPsXGjRs5+eSTmTdvHgDbt29n+vTpzJw5\nk1deeYWFCxdy1VVX8fzzz+9Xa968eWzatImJEyfyhS98oU/2iUEmSZmZNm1aS2gtXryYc889d78g\nW7x4MdOmTePRRx/lxBNP5NJLL2XQoEFccsklnHzyyTz88MMtta688kpqa2sZPnw4M2bMYOLEiZx3\n3nkMGjSIiy66iIaGBgAeeeQRJkyYwOWXX05E8O53v5uPfvSj+43KPvKRj3D66aczaNAgLrvsspYR\nYm8zyCQpM1OnTmXJkiVs3LiRV155hYkTJ3L22WfzxBNPsHHjRpYtW8bUqVNZu3Yt48eP3++148eP\nZ82aNS3Px4wZ0/L4kEMOecvzrVu3ArBq1Sp+/vOfM3r0aEaPHs2oUaO45557aG5ubll/7NixLY+H\nDRvW8tre5sUekpSZs846i02bNnH77bczZcoUAIYPH864ceO4/fbbOeaYYxg/fjzjxo3ju9/97n6v\nbWpqYsaMGV3e5nHHHUddXR0//OEPq/IeqskRmSRl5uCDD+a9730vt956K+eee25L+5QpU7j11ltb\nrlb8wAc+wMqVK1m4cCFvvPEG9913H8899xwf+tCHurzN888/nxdeeIFvf/vb7Nmzh927d/PLX/6S\nFStWVO19dZdBJkkZmjZtGhs2bOCcc85paTv33HPZsGED06ZNA2D06NE88sgj3HLLLRx55JHccsst\nPProo4waNQqAiKh4e4cddhiLFi1i4cKFjBs3jnHjxnHttdeyc+fO6r6xboiUUv9sOCL117b19jJr\n9qxe+XzPtz//bWb+w8yq1218sJG7vnpX1eu+XcyaNbeqt6i6666u14oIWv9+GigfiH67aGsfl7VX\nnrB4jkySKnIghUxunFqUJGXNIJMkZc0gkyRlzSCTJGXNIJMkZc2rFiWplfHjx3fpM1bquta3zuoJ\ng0xSj8yZ81WamjZVrV5Dw3Jqa6tWrlsaGxv7twPqEoNMUo80NW2q2geYAZYsubBqtXRg8ByZJClr\nnQZZRBwbEY9FxLMR8euI+KuifVRELIqIFRHxw4gYUfaa6yJiZUQ8FxHTe/MNSJIObJWMyPYAf5NS\n+kPgLODqiDgZuBb4cUrpJOAx4DqAiHgXcDFwCjADuC08aypJ6iWdBllKaX1KaWnxeCvwHHAscAGw\noFhtAbBvYvvDwMKU0p6UUiOwEphU5X5LkgR08RxZRNQCpwI/B8aklJqhFHbAUcVqxwCry162pmiT\nJKnqKg6yiDgM+A7w18XIrPX99/1OFklSn6vo8vuIGEIpxL6VUvpe0dwcEWNSSs0RMRb4XdG+Bjiu\n7OXHFm1vMXfu3JbHdXV11NXVdanzkqS81dfXU19f36MalX6O7A5geUrpn8raHgJmATcDVwDfK2u/\nOyK+QmlK8XjgqbaKlgeZJOnA03oQM2/evC7X6DTIImIKcBnw64hooDSF+HlKAXZ/RHwSWEXpSkVS\nSssj4n5gObAbuMqvgpYk9ZZOgyyl9DNgcDuL39fOa24EbuxBvyRJqoh39pAkZc0gkyRlzSCTJGXN\nIJMkZc0gkyRlzSCTJGXNIJMkZc1viJb0ttXQ8DSzZs2tSq2ampHMnz+7KrVUXQaZpLetbdsStbVz\nq1KrsbE6dVR9Ti1KkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJ\nkrJmkEmSsmaQSZKy5k2DpT7WsLSBWbNnVb1uzZga5l83v+p1pYHOIJP62LZd26i9sLbqdRsfbKx6\nTSkHTi1KkrJmkEmSsmaQSZKy5jky9Zk5N86hqbmp6nUbnmnolXNOkvJgkKnPNDU39UrgLHlqSdVr\nSsqHU4uSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKk\nrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKx1GmQR8e8R0RwRz5S1XR8R\nL0fEr4qf95ctuy4iVkbEcxExvbc6LkkSVDYiuxP40zbab00pvaf4+U+AiDgFuBg4BZgB3BYRUbXe\nSpLUSqdBllJaAmxsY1FbAXUBsDCltCel1AisBCb1qIeSJHWgJ+fIromIpRHxjYgYUbQdA6wuW2dN\n0SZJUq8Y0s3X3QbMTymliPgS8GXgz7taZO7cuS2P6+rqqKur62Z3JEk5qq+vp76+vkc1uhVkKaUN\nZU9vBx4uHq8BjitbdmzR1qbyIJMkHXhaD2LmzZvX5RqVTi0GZefEImJs2bI/A5YVjx8CLomIgyJi\nAnA88FSXeyVJUoU6HZFFxD1AHXBERDQB1wPnRcSpwF6gEfgLgJTS8oi4H1gO7AauSiml3um6JEkV\nBFlK6dI2mu/sYP0bgRt70ilJvWvOnK/S1LSpKrUaGpZTW1uVUlK3dPdiD0kZa2raRG3t3KrUWrLk\nwqrUkbrLW1RJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJm\nkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJ\nkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKy\nZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQ\nSZKyZpBJkrLWaZBFxL9HRHNEPFPWNioiFkXEioj4YUSMKFt2XUSsjIjnImJ6b3VckiSobER2J/Cn\nrdquBX6cUjoJeAy4DiAi3gVcDJwCzABui4ioXnclSdpfp0GWUloCbGzVfAGwoHi8ALiwePxhYGFK\naU9KqRFYCUyqTlclSXqr7p4jOyql1AyQUloPHFW0HwOsLltvTdEmSVKvGFKlOqk7L5o7d27L47q6\nOurq6qrUHUlSDurr66mvr+9Rje4GWXNEjEkpNUfEWOB3Rfsa4Liy9Y4t2tpUHmSSpANP60HMvHnz\nulyj0qnFKH72eQiYVTy+AvheWfslEXFQREwAjgee6nKvJEmqUKcjsoi4B6gDjoiIJuB64CbggYj4\nJLCK0pWKpJSWR8T9wHJgN3BVSqlb046SJFWi0yBLKV3azqL3tbP+jcCNPemUJEmV8s4ekqSsGWSS\npKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSs\nGWSSpKwZZJKkrHX6fWSS8tCwtIFZs2dVtO6SZUtZ2thY0bojDq7hvMnzu98xqZcZZNLbxLZd26i9\nsLaidZfSyMiRla27qb6x232S+oJTi5KkrBlkkqSsObUoSRVoaHiaWbPmVq1eTc1I5s+fXbV6BzKD\nTJIqsG1borZ2btXqNTZWr9aBziCTMvCTn/yczZt/3+E6zc2v8eCD9RXVW7d+AyNHVqFj0gBgkEkZ\n2Lz594wcWdfhOkOHvtzpOvs0NS3seaekAcKLPSRJWTPIJElZM8gkSVkzyCRJWTPIJElZM8gkSVkz\nyCRJWTPIJElZM8gkSVkzyCRJWTPIJElZM8gkSVkzyCRJWTPIJElZM8gkSVkzyCRJWTPIJElZM8gk\nSVkzyCRJWTPIJElZM8gkSVkzyCRJWTPIJElZG9KTF0dEI7AZ2AvsTilNiohRwH3AeKARuDiltLmH\n/ZQkqU09HZHtBepSSqellCYVbdcCP04pnQQ8BlzXw21IktSungZZtFHjAmBB8XgBcGEPtyFJUrt6\nGmQJ+FFE/CIi/rxoG5NSagZIKa0HjurhNiRJalePzpEBU1JK6yLiD4BFEbGCUriVa/1ckqSq6VGQ\npZTWFX9uiIgHgUlAc0SMSSk1R8RY4HftvX7u3Lktj+vq6qirq+tJdyRJmamvr6e+vr5HNbodZBEx\nDBiUUtoaEYcC04F5wEPALOBm4Arge+3VKA8ySdKBp/UgZt68eV2u0ZMR2RjgPyIiFXXuTiktiohf\nAvdHxCeBVcDFPdiGJEkd6naQpZReAk5to/014H096ZQkSZXyzh6SpKwZZJKkrBlkkqSs9fRzZHqb\nmnPjHJqam6pas+GZBmovrK1qTUkyyNSmpuamqofOkqeWVLWeJIFTi5KkzBlkkqSsGWSSpKwZZJKk\nrHmxhyT1g4aGp5k1a25VatXUjGT+/NlVqZUjg0yS+sG2bYna2rlVqdXYWJ06uXJqUZKUNYNMkpQ1\ng0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNM\nkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNMkpS1If3dAUkD2/r1DTxYP6vd5c07f9Xh8vaM\nOLiG8ybP737HpIJBJvWSn/zk52ze/Pu3tDc3v8aDD9Z3qda69RsYObJKHeuiXWxjZF1tu8uHjh7G\nyP/Z/vL2bKpv7HafpHIGmdRLNm/+PSNH1r2lfejQl9ts70hT08LqdEp6G/IcmSQpawaZJClrBpkk\nKWsGmSQpawaZJClrBpkkKWtefi9JmWtoeJpZs+ZWpVZNzUjmz59dlVp9xSDL3Jwb59DU3FT1ug3P\nNFB7YW3V60qqvm3bErW1c6tSq7GxOnX6kkGWuabmpl4JnCVPLal6zYHu5ZfX8uqrm1uev75lK08/\n/Vy36+3avasa3ZLUCYNMKrywcg2vv34UQwa/A4Df7xjK6tWjulVr+45X2bHjrbenklR9BplUZtiw\nIzlo6KEADBnyDg47bGy36rzxxi527KhmzyS1x6sWJUlZc0QmqV+09/Uw3f1aGPCrYQ5UvRZkEfF+\n4KuURn3/nlK6ube2JSk/7X09THe/Fgb8apgDVa9MLUbEIOCfgT8F/hD4eESc3BvbOtDU19f3dxey\n1bi0sb+7kKXd2z3Z1x2NjfX93YUDRm+NyCYBK1NKqwAiYiFwAfB8L21vQFv8s8Xc+8i9Van1i8W/\n4IxzzwDg0Hccys6dO6tS90DQuLSR2lNr+7sb2dljkHVLY2M9tbV1/d2NA0JvBdkxwOqy5y9TCrcD\n0rrmdew8didjj+/eFXDlDl59MKPqSpeEr3tiHbt37+5xTUnKmRd79IEhg4ewY/UO1m9c3+NaW9du\nZf0vSnX2btvb43p605DBwebNL7BjUOmfxe49W9j8+q+7VWv37h1ERDW7J6kdkVKqftGIycDclNL7\ni+fXAqn8go+IqP6GJUnZSyl16X+BvRVkg4EVwJ8A64CngI+nlLp/vx9JktrQK1OLKaU3IuIaYBFv\nXn5viEmSqq5XRmSSJPWVPrtFVUQ0RsTTEdEQEU8VbddHxMsR8avi5/191Z9cRMSIiHggIp6LiGcj\n4syIGBURiyJiRUT8MCJG9Hc/B5p29pvHWwci4sTi3+evij83R8Rfebx1rIP95vHWiYj4TEQsi4hn\nIuLuiDioO8dbn43IIuK3wOkppY1lbdcDW1JKt/ZJJzIUEXcBP00p3RkRQ4BDgc8Dr6aU/jEiPgeM\nSild25/9HGja2W+z8XirSHFTg5eBM4Fr8HirSKv99kk83toVEeOAJcDJKaVdEXEf8H3gXXTxeOvL\nmwZHO9vzGuV2RMThwLkppTsBUkp7UkqbKX24fEGx2gLgwn7q4oDUwX4Dj7dKvQ/4TUppNR5vXVG+\n38DjrTODgUOL/2weAqyhG8dbXwZZAn4UEb+IiE+XtV8TEUsj4htOWbzFBOCViLizmJr4t4gYBoxJ\nKTUDpJTWA0f1ay8Hnvb2G3i8VepjwD3FY4+3yn0MKL+Nj8dbO1JKa4EvA02UAmxzSunHdON468sg\nm5JSeg/wAeDqiDgHuA14Z0rpVGA94BB8f0OA9wBfL/bdNuBaSv8pKOcVO/trvd+2U9pvHm8ViIih\nwIeBB4omj7cKtLHfPN46EBEjKY2+xgPjKI3MLqMbx1ufBVlKaV3x5wbgP4BJKaUN6c2TdLcDZ/RV\nfzLxMrA6pfTL4vl3Kf2Cbo6IMQARMRb4XT/1b6Bqvd++A5zm8VaxGcB/p5ReKZ57vFVm337bAKXf\ndR5vHXof8NuU0msppTco5cLZdON465Mgi4hhEXFY8fhQYDqwrOjkPn8GLOuL/uSiGF6vjogTi6Y/\nAZ4FHgJmFW1XAN/r+94NXO3st+UebxX7OPtPj3m8VWa//ebx1qkmYHJEHByl+7n9CbCcbhxvfXLV\nYkRMoJS2idK0z90ppZsi4pvAqcBeoBH4i31zoyqJiHcD3wCGAr8FrqR0gvR+4DhgFXBxSmlTv3Vy\nAGpnv30Nj7cOFecSV1GaEttStI3G461D7ew3f791orhy/RJgN9AA/DkwnC4eb34gWpKUtb682EOS\npKozyCRJWTPIJElZM8gkSVkzyCRJWTPIJElZM8gkSVkzyCRJWfv/Zf8HYQKf52wAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8f30082b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "ax.set_title(\"Histogram of Heights\")\n",
    "ax.hist(m, alpha=.5, label=\"Men\");\n",
    "ax.hist(w, alpha=.5, label=\"Women\");\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is the output: Next, we might be interested in subsampling the group,\n",
    "fitting the distribution, and then predicting the remaining groups:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "random_sample = np.random.choice([True, False], size=m.size)\n",
    "m_test = m[random_sample]\n",
    "m_train = m[~random_sample]\n",
    "w_test = w[random_sample]\n",
    "w_train = w[~random_sample]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to get the empirical distribution of the heights of both men and women\n",
    "based on the training set:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from scipy import stats\n",
    "m_pdf = stats.norm(m_train.mean(), m_train.std())\n",
    "w_pdf = stats.norm(w_train.mean(), w_train.std())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "For the test set, we will calculate based on the likelihood that the data point was gen-\n",
    "erated from either distribution, and the most likely distribution will get the appropriate\n",
    "label assigned. We will, of course, look at how accurate we were:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.143355432914\n",
      "0.0169482868237\n"
     ]
    }
   ],
   "source": [
    "print( m_pdf.pdf(m[0]) )\n",
    "print( w_pdf.pdf(m[0]) )\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the difference in likelihoods. Assume that we guess situations when the men's\n",
    "probability is higher, but we overwrite them if the women's probability is higher:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "guesses_m = np.ones_like(m_test)\n",
    "guesses_m[m_pdf.pdf(m_test) < w_pdf.pdf(m_test)] = 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obviously, the question is how accurate we are. Since guesses m will be 1 if we are\n",
    "correct, and 0 if we aren't, we take the mean of the vector and get the accuracy:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.92706333973128596"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guesses_m.mean()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "Not too bad! Now, to see how well we did with for the women's group, use the\n",
    "following commands:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.91554702495201534"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guesses_w = np.ones_like(w_test)\n",
    "guesses_w[m_pdf.pdf(w_test) > w_pdf.pdf(w_test)] = 0\n",
    "guesses_w.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's allow the variance to di\u000b",
    "er between groups. First, create some new data:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s_m = 1\n",
    "s_w = 4\n",
    "m = np.random.normal(in_m, s_m, N)\n",
    "w = np.random.normal(in_w, s_w, N)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f8f049c4c88>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbIAAAFCCAYAAACU4VgFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH/pJREFUeJzt3Xt8nNV95/HPz8YUDMayyWIwQZJjru023BIwN1suLA2E\nBLJZCAGCDSmvvgKUmjRtIGnAdkuA7Na026bdhgTiJgQDyRYIJItDQYADlFJkiLmYqyxjI3GzjTEB\n387+MWMxFiPrMqPLGT7v10svz3OeZ85zzhxZX50zzzyKlBKSJOVqxFA3QJKkShhkkqSsGWSSpKwZ\nZJKkrBlkkqSsGWSSpKwZZMpKRCyJiKlD3Y6hFBGfi4i2iHgrIg4cgPr/KSK+2ctjr4+IudVug9QX\nBpmGjYh4KSL+oEvZjIh4YMt2Sum/ppTu76GehojYHBG1+v39P4HzU0q7pJQe77qz2PePdSm7PCJ+\n1JvKU0pfSSldUY2GlmuLVG21+h9dtaWvn9qP4nNiANpCRIwciHr7oAF4ahv7u3u9huLuB95xQQPO\nIFNWSmdtEfHJiPiPiFgTEa9ExP8qHnZf8d/VxeW3w6PgLyOiNSLaI+KHEbFLSb1nF/e9Vjyu9DyX\nR8QtEfGjiFgNzCie+8GIWBURKyLi7yNiu5L6NkfEVyLi2WL75kbExyLi1xGxOiIWlB7fpY/l2jom\nIraPiLUU/t8+ERHPdfcy9eJ13D8iFkbEGxHxdEScWrJvq+XCiPiLiFgZES9HxJfLzLLGR8Qdxdf6\noYiYVHzefcW2PFHcd2pE7BoRPy++bm8Uj5EqYpBpuNvWD+W/A/42pTQWmAzcXCzf8h7aLsXlt38H\nzgHOBqYBHwPGAP8AEBG/C3wX+CKwBzAWmNjlXJ8Fbk4p1QE3ABuBWcB44AjgD4DzuzzneOBgYArw\nF8A/A2cAewG/XzxfOeXa+t2U0vqU0pjia/L7KaV9tvHadCsiRgMLgR8DHwFOB/4xIvYvc+yniv38\nA2BvoIkPzrK+AFwO1AEvAFcApJSmFff/fnEcbgH+DFgO7ArsBnyjP32QShlkGm5ujYg3t3xRCJju\nrAf2johdU0rvpJQe6bK/NATPAOallJallN4BLgW+UHwf7fPA7Smlh1JKG4HLypzroZTSzwFSSu+l\nlFpSSo+kgjbgexSCp9TVKaV1KaWngSXAwuL51wK/pBBy5ZRr6+ld3vPradb1WMnruAr4esm+k4CX\nUkr/Umz/48DPgFPL1HMqcH1K6ZmU0rvA7DLH/GtK6T9TSpsphPxBXfaXtnUDhV8WJqWUNqWUft1D\nP6QeGWQabk5OKY3f8sUHZzmlvgzsBzwTEf8eEZ/exrETgWUl28uA7YAJxX3Lt+xIKf0WeKPL85eX\nbkTEPsUlsleKy41XUJjdlHq15PFvgY4u2zv3o629dXDJ6zgOuLpkXwMwpUvQndFN/Vu9NsXHXUO0\nveTxO3TfL4DvUJi1LYyI5yPi69s4VuoVg0zDTa8v0EgpvZBSOiOl9F8o/ID8aUTsSPkLDFZS+AG+\nRQOF5cEO4BXgo50NKNSxa9fTddn+J+BpYHJxufGbfWl7D8q1dQNbB2FPttWW5UBzadAVl/4uLHPs\nVq8NUE8FF3AUZ6hfSylNprBc+9WImN7f+iQwyJSxiDgzIrbMgtZQ+AG7GXit+O/kksNvBC6OiMaI\n2JnCDGpBcTnsp8BnImJKRIyi/PJZV2OAt1JK7xTfW/pKVTrVc1ur4Q5g34g4KyK2i4hREfGJiNiv\nzLE3A+cULw4ZDfxlH8/VTuF9PgAi4tMRsWVc1lL4ZaJa/dKHlEGm4aQ3v+mXHvMp4MmIeAu4BvhC\n8f2r31L44f/r4tLZYcB1wI+A+yksbb0DXASQUnoK+BPgJgqzobcoLAu+t412fA04s3jufwYW9NCX\nvsxium1rL+va5v6U0tsULkQ5nUJ/VwJXAb9T5tj/B/xv4F7gWeCh4q5tvTalZgP/UhyH/wHsA9xd\nvPry1xQuYvHKRVUkevrDmhHxAwpvDneklD5eLPsO8BkK38wvAOeklN4q7rsUOJfCb1p/mlJaOHDN\nl6ovInYCVgN7p5SW9XT8h0lx9vkb4HeqOEOUKtKbGdn1wB92KVsI/F5K6SDgOQpXVW25jPk04ADg\nBAqX9A7Ih1KlaoqIkyJix2KI/Q3whCFWEBGnFD/DtuWikdsNMQ0nPQZZSmkRsKpL2d0l38gP8/6b\nwZ+lsJa/MaXUSiHkDqtec6UBczKFJbaXKby3dvrQNmdY+WMKS63PUbjoZFtXkkqDruydBfroXApv\nTgPsyftr6AArimXSsJZSOg84b6jbMRyllE4Y6jZI21LRxR5RuEP2hpTSjT0eLEnSAOj3jCwiZgIn\nUrh1zRYrKNx+Z4uPFsvKPd+biUqSPiCl1KdrK3o7IwtKPmBZvP/anwOfTSmVXoZ7O4Vb6WxfvHHo\n3kDX2waVNrZmvy6//PIhb4P9s3/V7t+MGZdz+eWp7NeMGcPjNXPs8v7qjx5nZBHxEwo3Ct01Itoo\n3Bz0G8D2wK+KFyU+nFI6P6X0VETcTOFPTGyg8DeTnHlp2Lnsysto62jrdv/ihxfTurq16uetn1DP\n3Ev9O5RSNfUYZCmlM8oUX7+N468ErqykUdJAa+too/GUxm73t65u3eb+/mq9tbXqdUofdt7ZY4A0\nNTUNdRMGVK33r/GgxqFuwoCq5fGr5b5B7fevPwyyAVLr32y13j+DLF+13Deo/f71RzU+RyZJNaWx\nsZFly7yxy0BqaGigtbW1KnUZZJLUxbJly/p9BZ16p5p3L3RpUZKUNYNMkpQ1g0ySlDXfI5NUFS0t\njzNz5uyy++rr65g7d9bgNkgfGgaZpKpYty7R2Di77L7W1vLlObnssr+lrW31gNXf27BvbGykvb2d\nlStXMn78+M7ygw8+mMcff5zW1lbq6+sHrJ3DkUEmSb3Q1ra626Cuht6GfUQwadIkbrzxRi644AIA\nlixZwm9/+9uqXgmYE98jk6TMfOlLX2L+/Pmd2/Pnz2fGjBmd2+vXr+drX/saDQ0N7LHHHpx//vm8\n917h/u733Xcfe+21F/PmzWPChAnsueee/PCHPxzsLlSVQSZJmZkyZQpr165l6dKlbN68mZtuuomz\nzjqr87NvX//613n++ed54okneP7551mxYgVz575/s+r29nbWrl3LypUr+f73v88FF1zAmjVrhqo7\nFTPIJClDW2Zlv/rVrzjggAOYOHEiEcHmzZu59tprueaaaxg7diw77bQTl1xyCTfe+P7fP95+++35\n1re+xciRIznhhBPYeeedWbp06RD2pjK+RyZJGTrrrLOYOnUqL730EmeffTZQ+BuPr7/+Ou+88w6H\nHnpo57GbN2/e6k4lu+66KyNGvD+PGT16NG+//fbgNb7KDDJJylB9fT2TJk3il7/8Jddddx1QuBDk\nIx/5CKNHj+bJJ59kjz32GOJWDg6XFiUpU9dddx333HMPO+64I1CYkY0YMYLzzjuPWbNm8dprrwGw\nYsUKFi5cOJRNHVAGmSRlpPQS+0mTJnHIIYd8YN9VV13F3nvvzZQpU6irq+P444/n2Wef7VWdOXJp\nUZJ6ob6+bkA/2F1fX9er41588cWy5SNHjmTTpk2d21dccQVXXHHFB46bNm0abW1tvaozFwaZJPWC\nt9gavlxalCRlzSCTJGXNIJMkZc0gkyRlzSCTJGXNIJMkZc0gkyRlzSCTJGXND0RLUi9cduVltHW0\n9XxgP9VPqGfupXN7PO6qq67i/vvv5xe/+EVn2T777MO+++7LnXfe2Vm277778td//decdtppA9Le\n4cQgk6ReaOtoo/GUxgGrv/XW1l4dN3XqVK6++mpSSkQE7e3tbNy4kZaWlq3KXnjhBaZOnTpg7R1O\nXFqUpIx88pOfZP369SxevBiABx54gOnTp7PffvttVTZ58mR23313HnzwQQ477DDGjRvH4YcfzkMP\nPdRZ1/Tp0/nWt77FUUcdxZgxYzj55JN58803Oeussxg7diyHH374VvdlfOaZZzj++OPZddddOeCA\nA7jllls6951zzjlceOGFnHTSSeyyyy4cccQRvPTSS4PymhhkkpSRUaNGcfjhh3P//fcDcP/99zN1\n6lSOPvroD5StWrWKk046iVmzZvHGG29w8cUX8+lPf5pVq1Z11nfTTTdxww03sHLlSp5//nmOPPJI\nvvzlL7Nq1Sr2339/5syZA8A777zD8ccfz1lnncXrr7/OggULOP/883nmmWe2qmvOnDmsXr2ayZMn\n881vfnNQXhODTJIyM23atM7QeuCBBzjmmGO2CrIHHniAadOmceedd7LvvvtyxhlnMGLECE4//XT2\n339/fv7zn3fWdc4559DY2MiYMWM44YQTmDx5MtOnT2fEiBGceuqptLS0AHDHHXcwadIkzj77bCKC\nAw88kM9//vNbzco+97nPceihhzJixAjOPPPMzhniQDPIJCkzU6dOZdGiRaxatYrXX3+dyZMnc+SR\nR/Lggw+yatUqlixZwtSpU1m5ciUNDQ1bPbehoYEVK1Z0bk+YMKHz8Y477viB7bfffhuAZcuW8fDD\nDzN+/HjGjx/PuHHj+MlPfkJHR0fn8bvvvnvn49GjR3c+d6B5sYckZeaII45g9erVXHvttRx11FEA\njBkzhokTJ3Lttdey55570tDQwMSJE/nZz3621XPb2to44YQT+nzOvfbai6amJu66666q9KGanJFJ\nUmZ22GEHPvGJTzBv3jyOOeaYzvKjjjqKefPmdV6teOKJJ/Lcc8+xYMECNm3axE033cTTTz/NZz7z\nmT6f86STTuLZZ5/lxz/+MRs3bmTDhg08+uijLF26tGr96i9nZNIgalncwsxZMwf1nL39fJLyMm3a\nNB5++GGOPvrozrJjjjmG7373u0ybNg2A8ePHc8cdd3DRRRfxla98hb333ps777yTcePGARARvT7f\nzjvvzMKFC7n44ov56le/SkqJAw88kHnz5lW3Y/1gkEmDaN36dQP6WaRyevv5JG1b/YT6AX0t6yfU\n9+n4b3/723z729/equzUU0/l1FNP3arsyCOP5NFHHy1bxz333LPV9l/91V9ttX3sscfy7LPPdm7v\ns88+3HHHHWXruv7667fanjZt2laX7g8kg0ySesFZ7fDV43tkEfGDiOiIiCdKysZFxMKIWBoRd0XE\n2JJ9l0bEcxHxdEQcP1ANlyQJenexx/XAH3YpuwS4O6W0H3APcClARPwucBpwAHAC8I/Rl0VYSZL6\nqMcgSyktAlZ1KT4ZmF98PB84pfj4s8CClNLGlFIr8BxwWHWaKknSB/X38vvdUkodACmldmC3Yvme\nwPKS41YUyyRJGhDVutgj9edJs2fP7nzc1NREU1NTlZojScpBc3Mzzc3NFdXR3yDriIgJKaWOiNgd\neLVYvgLYq+S4jxbLyioNMkkaLhoaGvr0GSv13ZZbZ3WdxGy5SXFf9DbIovi1xe3ATOBqYAZwW0n5\nDRFxDYUlxb2BR/rcKkkaQq2trUPdBPVBj0EWET8BmoBdI6INuBy4CrglIs4FllG4UpGU0lMRcTPw\nFLABOD+l1K9lR0mSeqPHIEspndHNruO6Of5K4MpKGiVJUm9502BJUtYMMklS1rzXoobcZVdeRlvH\n4NxcdIuWJ1oG/ea9kgaGQaYh19bRNuihsuiRRYN6PkkDx6VFSVLWDDJJUtYMMklS1gwySVLWDDJJ\nUtYMMklS1gwySVLWDDJJUtYMMklS1gwySVLWDDJJUtYMMklS1gwySVLWDDJJUtYMMklS1gwySVLW\nDDJJUtYMMklS1gwySVLWDDJJUtYMMklS1gwySVLWDDJJUtYMMklS1gwySVLWDDJJUtYMMklS1gwy\nSVLWDDJJUtYMMklS1gwySVLWDDJJUtYMMklS1ioKsoi4OCKWRMQTEXFDRGwfEeMiYmFELI2IuyJi\nbLUaK0lSV/0OsoiYCPwJcEhK6ePAdsAXgUuAu1NK+wH3AJdWo6GSJJVT6dLiSGCniNgO2BFYAZwM\nzC/unw+cUuE5JEnqVr+DLKW0EvgboI1CgK1JKd0NTEgpdRSPaQd2q0ZDJUkqp5KlxToKs68GYCKF\nmdmZQOpyaNdtSZKqZrsKnnsc8GJK6U2AiPhX4EigIyImpJQ6ImJ34NXuKpg9e3bn46amJpqamipo\njiQpN83NzTQ3N1dURyVB1gZMiYgdgPeAY4H/AN4GZgJXAzOA27qroDTIJEkfPl0nMXPmzOlzHf0O\nspTSIxHxU6AF2FD893vAGODmiDgXWAac1t9zSKoNLS2PM3Pm7LL76uvrmDt31uA2SDWlkhkZKaU5\nQNf4fJPCsqMkAbBuXaKxcXbZfa2t5cul3vLOHpKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsVXTVoqTh\nr2VxCzNnzaxKXYuWLGZxa2vZfR3vPcatze+fZ+wO9UyfMrcq55W2xSCTaty69etoPKWxKnUtppW6\nuvJ1jRo/mrqPv79vdXNrVc4p9cSlRUlS1gwySVLWDDJJUtYMMklS1gwySVLWvGpRUqd7732YNWve\n7Xb/K+2vUVc3iA2SesEgk9RpzZp3qatr6nZ/W9uCwWuM1EsuLUqSsmaQSZKyZpBJkrJmkEmSsmaQ\nSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmS\nsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJWUZBFxNiIuCUino6IJyPi\n8IgYFxELI2JpRNwVEWOr1VhJkrqqdEb2d8AvUkoHAAcCzwCXAHenlPYD7gEurfAckiR1q99BFhG7\nAMeklK4HSCltTCmtAU4G5hcPmw+cUnErJUnqRiUzsknA6xFxfUQ8FhHfi4jRwISUUgdASqkd2K0a\nDZUkqZztKnzuIcAFKaVHI+IaCsuKqctxXbc7zZ49u/NxU1MTTU1NFTRHkpSb5uZmmpubK6qjkiB7\nGVieUnq0uP0zCkHWERETUkodEbE78Gp3FZQGmSTpw6frJGbOnDl9rqPfS4vF5cPlEbFvsehY4Eng\ndmBmsWwGcFt/zyFJUk8qmZEBXATcEBGjgBeBc4CRwM0RcS6wDDitwnNIktStioIspfQ48Mkyu46r\npF5JknrLO3tIkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJm\nkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJkrJmkEmSsmaQSZKyZpBJ\nkrJmkEmSsrbdUDdAw8dlV15GW0fboJ+35YkWGk9pHPTzSqoNBpk6tXW0DUmgLHpk0aCfU1LtcGlR\nkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDWDTJKU\ntYqDLCJGRMRjEXF7cXtcRCyMiKURcVdEjK28mZIklVeNGdmfAk+VbF8C3J1S2g+4B7i0CueQJKms\nioIsIj4KnAh8v6T4ZGB+8fF84JRKziFJ0rZUOiO7BvhzIJWUTUgpdQCklNqB3So8hyRJ3ep3kEXE\np4GOlNJiILZxaNrGPkmSKlLJX4g+CvhsRJwI7AiMiYgfAe0RMSGl1BERuwOvdlfB7NmzOx83NTXR\n1NRUQXMkSblpbm6mubm5ojr6HWQppW8A3wCIiGnAn6WUvhQR3wFmAlcDM4DbuqujNMgkSR8+XScx\nc+bM6XMdA/E5squA/xYRS4Fji9uSJA2ISpYWO6WU7gPuKz5+EziuGvVKktQT7+whScqaQSZJyppB\nJknKmkEmScqaQSZJylpVrlqUpP5qaXmcmTNnl91XX1/H3LmzBrdByo5BJmlIrVuXaGycXXZfa2v5\ncqmUS4uSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrHnVovQhc++9D7Nmzbtl973S/hp1dYPcIKlCBpn0\nIbNmzbvU1TWV3dfWtmBwGyNVgUuLkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSS\npKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrBlkkqSs\nGWSSpKwZZJKkrBlkkqSsGWSSpKwZZJKkrPU7yCLioxFxT0Q8GRG/iYiLiuXjImJhRCyNiLsiYmz1\nmitJ0tYqmZFtBL6aUvo94AjggojYH7gEuDultB9wD3Bp5c2UJKm8fgdZSqk9pbS4+Pht4Gngo8DJ\nwPziYfOBUyptpCRJ3anKe2QR0QgcBDwMTEgpdUAh7IDdqnEOSZLK2a7SCiJiZ+CnwJ+mlN6OiNTl\nkK7bnWbPnt35uKmpiaampkqbI0nKSHNzM83NzRXVUVGQRcR2FELsRyml24rFHRExIaXUERG7A692\n9/zSIJMkffh0ncTMmTOnz3VUurR4HfBUSunvSspuB2YWH88Abuv6JEmSqqXfM7KIOAo4E/hNRLRQ\nWEL8BnA1cHNEnAssA06rRkMlSSqn30GWUvo1MLKb3cf1t15JkvrCO3tIkrJmkEmSsmaQSZKyZpBJ\nkrJmkEmSsmaQSZKyZpBJkrJW8b0WJamc9vYWbm2eCUDHe491Pu5qW/vefnsxM2e19vqc9RPqmXvp\n3L41VNkzyCQNiPWso66pEYBR40dT9/HGssdtax+rW2k8pZt9ZbTe2tqXJqpGuLQoScqaQSZJyppB\nJknKmkEmScqaQSZJyppBJknKmkEmScqanyMbpi678jLaOtoG9ZwtT7T06TM70kB7pb2dW29t7nb/\n2LE7MH36lMFrkIYlg2yYautoG/RQWfTIokE9n9STDeuhrq6p2/2rVzcPWls0fLm0KEnKmkEmScqa\nQSZJyprvkUk16N57H2bNmncB6Oh4c6sLJl5pf426uiFqmDQADDKpBq1Z827nRRKjRr281QUTbW0L\nhqZR0gBxaVGSlDWDTJKUNYNMkpQ1g0ySlDWDTJKUNYNMkpQ1g0ySlDU/RyZJFRqKv1ZRP6GeuZfO\nHdRzDlcGmSRVaCj+WkXrra2Der7hzKVFSVLWnJFJqhkti1uYOWvm4J/XP0o7pAwySTVj3fp1QxIo\n/lHaoeXSoiQpawaZJClrA7a0GBGfAv6WQlj+IKV09UCdS9KH0yvt7Vv9rbXSv702duwOTJ8+ZWga\npkE1IDOyiBgB/APwh8DvAV+MiP0H4lwDbdOmTf36+rd/+7d+P3fz5s1D3e0etS5uHeomDKjh3r+U\nEps3b+72qyerW1sHvpGDYMN6qKtr6vwaNWo8rG6krq6p8w+L1prh/r05FAZqRnYY8FxKaRlARCwA\nTgaeGaDzDYilS5dy9f+5mkTq83NbHmrh4CMO7td563auY+PGjf167mBpXdxK40GNQ92MATPc+7do\n0X/yxhvrID64b+QIWL9h/Tafv7q1lbrGxoFp3BCr5b7B8P/eHAoDFWR7AstLtl+mEG5Zeeutt0h7\nJBoOa+jzc19a9RINJ/f9eQDL71qexaxMQ2ft2+8xZpcjGTli1Af2rXlrCZs3vTwErZKGhpffb8PI\nkSNZ/+p6li9a3vPBXaxpW9Ov5wHEhjK/ZkslthsZrF37FIVV/K1t3rS27ExNqlWRUt+XzXqsNGIK\nMDul9Kni9iVAKr3gIyKqf2JJUvZSSn36VWyggmwksBQ4FngFeAT4Ykrp6aqfTJL0oTYgS4sppU0R\ncSGwkPcvvzfEJElVNyAzMkmSBsug3dkjIloj4vGIaImIR4pl4yJiYUQsjYi7ImLsYLWn2rrp3+UR\n8XJEPFb8+tRQt7M/ImJsRNwSEU9HxJMRcXiNjV25/tXK2O1b/J58rPjvmoi4qFbGbxv9q5Xxuzgi\nlkTEExFxQ0RsXytjB2X79zv9GbtBm5FFxIvAoSmlVSVlVwNvpJS+ExFfB8allC4ZlAZVWTf9uxxY\nm1KaN3Qtq1xE/BC4L6V0fURsB+wEfIPaGbsf8sH+zaIGxq5U8UYFLwOHAxdSI+O3RZf+nUvm4xcR\nE4FFwP4ppfURcRPwC+B3qYGx20b/Gunj2A3mvRajzPlOBuYXH88HThnE9lRbuf5tKc9WROwCHJNS\nuh4gpbQxpbSGGhm7bfQPMh+7Mo4DXkgpLadGxq+L0v5BbYzfSGCn4i9YOwIrqK2xK+3faAr9gz6O\n3WAGWQJ+FRH/ERF/VCybkFLqAEgptQO7DWJ7qq20f+eVlF8YEYsj4vuZLgFMAl6PiOuL0/zvRcRo\namfsuusf5D92XX0B+Enxca2MX6kvADeWbGc9fimllcDfAG0UfsCvSSndTY2MXZn+rS72D/o4doMZ\nZEellA4BTgQuiIhj4AP3fsr5ypOu/Tsa+EfgYymlg4B2IMdlju2AQ4DvFvu3DriE2hm7rv17h0L/\namHsOkXEKOCzwC3FoloZP6Bs/7Ifv4ioozD7agAmUpi5nEmNjF2Z/u0cEWfQj7EbtCBLKb1S/Pc1\n4FYKt6zqiIgJABGxO/DqYLWn2rr071+Bw1JKr6X334S8FvjkULWvAi8Dy1NKjxa3f0bhB3+tjF3X\n/v0UOLhGxq7UCcB/ppReL27XyvhtsaV/r0Hh/2ENjN9xwIsppTdTSpso/Fw5ktoZu679+7/Akf0Z\nu0EJsogYHRE7Fx/vBBwP/Aa4HZhZPGwGcNtgtKfauunfkuI32Rb/HVgyFO2rRHEJY3lE7FssOhZ4\nkhoZu27691QtjF0XX2TrZbeaGL8SW/WvRsavDZgSETtERFD83qR2xq5c/57uz9gNylWLETGJwm8T\nicJSzg0ppasiYjxwM7AXsAw4LaW0esAbVGXb6N+/AAcBm4FW4I+3rG3nJCIOBL4PjAJeBM6h8CZt\n9mMH3fbv76mBsYPCL1oUxuhjKaW1xbKa+L8H3favVv7vXQ6cDmwAWoA/AsZQO2NX2r/HgPOAH9DH\nsfMD0ZKkrA3mxR6SJFWdQSZJyppBJknKmkEmScqaQSZJyppBJknKmkEmScqaQSZJytr/B3HO19zF\n3H3xAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8f0fb079b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Then, create a training set:\n",
    "m_test = m[random_sample]\n",
    "m_train = m[~random_sample]\n",
    "w_test = w[random_sample]\n",
    "w_train = w[~random_sample]\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "ax.set_title(\"Histogram of Heights\")\n",
    "ax.hist(m_train, alpha=.5, label=\"Men\");\n",
    "ax.hist(w_train, alpha=.5, label=\"Women\");\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the difference in variances between the men and women: His-\n",
    "togram of Heights Chapter 3 109 Now we can create the same PDFs:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "m_pdf = stats.norm(m_train.mean(), m_train.std())\n",
    "w_pdf = stats.norm(w_train.mean(), w_train.std())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is the output: You can imagine this in a multidimensional space:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f8f0fa93da0>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAAE4CAYAAAADyhfEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtwXFedJ/Dvz3G/LKklO9s8kkykvG3ylCjznokcbELN\n1JAFttYRO+wEqyi8xiSEgUpgdstmXaECAxMITEYWEXFgYllFmAW2CkaONxJTzBBkHOUBiiFUkPIg\nk24W4sHetqXg3/7R3XKrdbt1n33vuf39VHXZkrrvPffKPr97zvmdc0RVQUREZIpVYReAiIjICQYu\nIiIyCgMXEREZhYGLiIiMwsBFRERGYeAiIiKj2A5cIjIiIi+JyBNV31srIgdF5OciMi4incEUk4iI\nqMRJi+s+ANfXfO92AIdU9TIADwP4pF8FIyIisiJOJiCLSDeA/62qV5W/PgrgWlV9SUReA2BSVdcH\nU1QiIiLvY1yvUtWXAEBV/w3Aq7wXiYiIqD6/kzO4fhQREQVqtcfPvyQir67qKszXe6OIMKgREdEy\nqipO3u+0xSXlV8V3AdxU/vtfAvhOow+ramxfu3btCr0MvD5eG68vfq+4X58bTtLh9wP4VwCXisiz\nIvIBAHcC2CIiPwfw9vLXREREgbHdVaiq76vzo80+lYWIiGhFXDnDJ/39/WEXIVBxvr44XxvA6zNd\n3K/PDUfzuDydSESbdS4iIjKDiEADTs4gIiIKFQMXEREZhYGLiIiMwsBFRERGYeAiIiKjMHAREZFR\nGLiIiMgoDFxERGQUBi4iIjIKAxcRERmFgYuIiIzCwEVEREZh4CIiIqMwcBERkVEYuIiIyCgMXERE\nZBQGLiIiMgoDFxERGYWBi4iIjMLARURERmHgIiIiozBwERGRURi4iIjIKAxcRERkFAYuIiIyCgMX\nEREZhYGLiIiMwsBFRERGYeAiIiKj+BK4RORWEfmpiDwhIg+ISNKP4xIREdXyHLhE5BwAHwHQp6pX\nAVgN4EavxyUiIrKy2qfjnAWgTUROA1gD4Nc+HZeIiGgJzy0uVf01gC8AeBbACwBeVtVDXo9LRERk\nxY+uwi4ANwDoBnAOgHYReZ/X4xIREVnxo6twM4BnVPW3ACAi/wjgLQD2175x9+7di3/v7+9Hf3+/\nD6cnIiJTTE5OYnJy0tMxRFW9HUDkDQBGAGwEcArAfQAOq+rf1bxPvZ6LiIjiRUSgquLkM36McU0B\neBDANIDHAQiAYa/HJSIisuK5xWX7RGxxERFRjVBaXEREQSsUCjh8+DAKhULYRaEIYOAiokgbHR1D\nd/d6bNmyHd3d6zE6OhZ2kShk7CokosgqFAro7l6PYnECwFUAnkAmswlzc0eRy+XCLh75gF2FRBQr\ns7OzSCZ7UApaAHAVEoluzM7OhlcoCh0DFxFFVk9PD+bnZwE8Uf7OE1hYmENPT094haLQMXARUWTl\ncjmMjNyDTGYTstk+ZDKbMDJyD7sJWxzHuIgo8gqFAmZnZ9HT08OgFTNuxrgYuIiIKDRMziAiothj\n4CIiIqMwcBERkVEYuIiIyCgMXEREZBQGLiIiMgoDFxF5xtXbqZkYuIjIE67eTs3GCchEEWDqyhBc\nvZ284gRkooAF0SVmcouFq7dTGBi4iGwKIsAUCgUMDu5AsTiBY8eOoFicwODgDmPGirh6O4WBgYvI\nhqACjOktFq7eTmFYHXYBiILmx/hRJcAUi8sDjJdKemmLpTRGZFqLZWBgKzZvvs7IMToyE1tcFGt+\nde8F1SUWlxZLLpfDxo0bjSs3mYlZhRRbfme8jY6OYXBwBxKJbiwszGFk5B4MDGz1raxssVArcpNV\nyK5Cii0/uveqA0qQXWK5XI4Bi8gmdhVSbHnt3rPqZmSXGFH42FVIsea2e48Ta4mag12FRDXcdu8F\nlUVIRN4xcFHsuRk/ikOaOlFccYyLyEKU0tSjuvJ6VMtF8ccxLqIGwk5Tr4zRJZOlFqCfKfhxLBeZ\nx80YFwMXUURFNUEkquUiM4W2OryIdIrIN0XkKRH5mYi80Y/jErWyqK5jGNVyUevwa4zrSwC+p6ob\nAFwN4CmfjkvUsqK68npUy0Wtw3PgEpEsgD9W1fsAQFVfUdV/91wyohYXpQQRE8pFrcPzGJeIXA1g\nGMAMSq2tnwC4RVWLNe/jGBeRC2EniNQT1XKRWUJJzhCR1wN4BMCbVfUnIvJFAMdUdVfN+xi4iIho\nibBWzngewHOq+pPy1w8CuM3qjbt37178e39/P/r7+304PRFFHVtnVDE5OYnJyUlPx/AlHV5EfgDg\ng6r6CxHZBWCNqt5W8x62uIg8MjEAcM4XNRLaPK7yONe9ABIAngHwAVU9VvMeBi4iD0wMAJzzRSvh\nBGSimDI1ABw+fBhbtmzHsWNHFr+Xzfbh0KG92LhxY4glo6gIbQIyEQXL1Em/nPNFQWDgIjKAqQGA\nc74oCOwqJDKE200xo8DEpBJqDo5xEcUcAwDFDQMXEZFP+JDQHEzOICJHuBmktdHRMXR3r8eWLdvR\n3b0eo6NjYReJqrDFRdQEUXx6r4yZrVr1Rzh9+jmjxsyCZOrUA1OxxUUUQUE9vRcKBRw8eBAHDx50\n3GIqFAq46aYPoVicwIkTj6FYnMBNN32ILS+YO/WglTBwEfmsuvutUChgcHAHisUJHDt2BMXiBAYH\nd3gOEKOjYzj33Itw/fXvxvXXfxjnnXeJo4A4PT2N+fkcqivn+fn/gOnpaU/ligNTpx60EgYuIh/V\ntq727v2q70/vhUIB27Ztx8LCagA/AvA05uf/2UVA/DWqK2fgRddlihPOPYs+jnER+cRqbCSdvhYi\nq3wdLzl48CBuuGEnTp7sAHBmKaW2tqsxMXGvraWUCoUCzj33QiwsJAD0AJhFIrGAF154hhV0WRTH\nJeOIY1xEIbIaG0kmL8CnPvVXvj29j46O4YYbtuLkyecB/ArVLabTp5+33Z2Vy+Vw//33Ip1WtLWd\nQDqtuP/+ewOvoE3KYszlcti4cSODVgSxxUXkk0bZaAA8P70vPf5TAAYBKIDXIpn8Dfbt2+s4K7CZ\nrQoTV7en4HECMlHIglyWaflK6wWsWfMm3HnnR3HjjTdGumXAFHOqJ6wdkImobGBgKzZvvi6QVszS\nbLerALwI1ZebGrTcttAq3ajF4vIkFQYucopjXEQ+8zo2Um8cKOxsNy/z0ZqZYm7SOBq5pKpNeZVO\nRUSN7N9/QDOZddrZ2aeZzDrdv//Asvfk83mdmprSfD7ftHLl83nNZNYp8LgCqsDjmsmsc1SGyrVl\ns711r82rRvcvjPtGKyvHBkfxhGNcRBER5XEgv3YyDjIZpNH9O3ToYSaGRBTT4YkixGmXVZSXGvKr\nqy/IFPN69296ejqQ1UsoPAxcRAFwMx4U5aWG7IyvhT22VO/+AYjsAwG55LRv0e0LHOOiFuFlPKgZ\n40Be1BsnsjM21wxW98+P8TkKDjjGRRQ+r+NBteNAUV96KGpjc1b3K8j5deQN53ERRcDy+VbOuvxy\nudyyCjeZ7MGpU8/gox/dgU2brkVvb29kgljU5mhV37+KIOfXUfOxxUVGiHqro5YfT/jLl3j6bwDW\nAXgRicQq3H//vZFoNThtcTXzd2nav5tW5KbFxTEuijyn4ydRma/jtRxTU1Pa2dmnQF6BpWM0wFpN\np7uWHLtZ1211Hrtjc80cC4vKuBs1BhdjXAxcFGlOB9bjVFmdufYHFOgrX3/l1attbZfq1NSUqgZ/\n3ZVgNTQ07HqCbzOTJJiQYQ4GLoqdM62OM5V2Ntu7WGFXi2NltX//AU2nuxRYU7fF5fd11wagSlDs\n6LhSgYzr8zj5XXoV5LmctGyj0vqPMgYuih0nlXIzK8ZmyufzumfPHZpIZBW4SIE1mki0L7Z2/Lzu\n2pZbpYVVuv9TClzt+jxxaHE5adnGqfUfJAYuiiW74ydRb3F5ffrO5/M6Pj6u4+Pjy8a2nF63VVms\njpNKdZVbWmo51ubk/lYCcDrd1ZR5an7PiXNyn6P+bzFKGLgotuxW+lGdwBv007eT665XFquWW0fH\nNZpKZasq4M8qkNGOjmscXUftOffsuaMplbifXXVOWrZxbf0HgYGLSKM3rhDE03e9FtNK192oLPV+\nVukurATFoaFhR/c3Lq0PtriCEWrgQmndw0cBfLfOz4O9eqKI8vvpu7r1kk53OWq9rFSWei03Lw8D\ncWp9uGnZRq31HzVuApdvE5BF5FYArweQVdV3Wfxc/ToXkUn8XBLJelLy2chkfmtrkvPyskwilboB\n09OPYMOGDYvv8XPSbtSWhPLKyf3hBOiVhTYBGcB5AB4C0A+2uIiWqX36djvG02hSstOFfNPpCxTI\naCZzZVMSJZLJTsusSGptcNHi8mtbk7sAfAIAm1TU0upt7TEwsBVzc0fxiU/8J6iexuc//y3b251U\nO7MO4kMAeuBmq46Bga04cuSHUP0dgEdQLD4R+B5Vmzdfh1WrBMD/BDCLhYV/4Z5Y5JrnRXZF5M8A\nvKSqj4lIP4C6Tb7du3cv/r2/vx/9/f1eT0+0omZ111QviFtvl93PfOYLOHnyBzh5stRlNji4CZs3\nX2e7XJV9sbZt246TJ+fhdiHf48ePI52+EKdO+bswbr17PTs7i1TqQpw8+b7KlYS6EC+FZ3JyEpOT\nk94O4rSJVvsC8BkAzwJ4BsCLAI4D+LrF+4JucRIt41cauh/LGfmZpFCZE5XJrNP29is0lcrq0NCw\no8+7yXprdB8a3Wtm2VE9CDsdHsC14BgXRYRflaWd4GcnKAVReQ8NDZcnCTvPXHOa9eY1MDHLjqww\ncBFV8aOFYzfY2H2fn5W3H4HQbpr7Sueye6+jNseOwucmcPm6kaSq/gDAD/w8JpFbXjd0LBQK+N73\nvofVq7thlQRRPTZTGXsaHNy0ZA+uIDc09GMDR6tNF92cy+69tns+oka4AzLFlt1gYqWSaLF69bn4\n/e9/iZUq5EKhgIsvvhBHjvwQx48fbxiU/Kq8vQZmP8+10r3mfCbyldMmmtsX2FVIIXHaPbW8W6zx\n+nzNSgCx4nfXY6Pz2zlXo00muUo6WUHYY1wNT8TARYawGq9pb79C9+3b53psq5HqDEE7lXttcPBj\n3MhucPH+EMBsQlqKgYvIBzMzM5pKddmqbN0mgFTvKGy1UWS98wXRegkyuMRpnUIKBgMXkUeVwJDJ\nVJZDusL3PcCW7yj8gAL2MvKCCDBB7xbMFhc14iZw+bXkE5HxCoUCBgd3oFicQLH4DIDv4fTpZ3Hk\nyA/rLl5bSUrIZDYhm+1DJrOpYQJI9Tl+//sRAJcC2AJgFqXEB6BekkUls8/NMk/1ynL48GG0t7dX\nJV7UP78bTu8PkR3MKiQqW57y3Y9U6iIcP3684eecpLgvPUcBwHMoLThzD0prVK9DJvM7y8rdzyzC\n2uWpBgf/AiMjzrMv7fBzCkArYkamBadNNLcvsKuQIs7vbq16mz3Wy1i0s7eWH1mE9a5zZmbG98nB\nnHDsTStkZIJjXETe+BUY9uy5Q9PpLssKp/YcbnYU9hIMmpUw0QqVbpBaZXyQgYvIB14CQ6WyBi5W\nYK0CBywrnDBbIs2oEJtd6caxZdcqGZluAheTM4hq5HI5bNy40dXuxJXEC+BpAJMAdgB47bIkCrfn\n8EMzEib8TiRpZHR0DN3d67Fly3ZXe5xF1dIxTSDIlVGM4zTSuX2BLa4VxfGpMa6sfldWT8hArwIP\nRLKLJ8h/b/l8vjw/7QEt7dYcTIsr7t1prbCiPthVaK4D+/frukxG+zo7dV0mowf27w+7SFRHvbGb\n5ZXohAIpTaWyTV8Jww0/z7t//wFNJjvLXaZrNJFoD6TSDbo7LQoPk1EoQ5AYuAyVz+d1XSajj5f/\n5z0O6LpMJrb/UE220hN+Jail06UJzKnU5Q2flGuD4M6dNy8Lin5UXPWOUb2Ch1+JFM1sBQV5LiaX\nNAcDl6Gmpqa0r7NTqx8be7PZ2A3CrsSEJ0s7T/h2l4yyqnRLK2lMLH6dSHR4rjzrVcBnVvDoLZ/3\ns75U/s1OKgiiOy3uXZBRwsBlKLa4zHm6tVOh2a24rcfELlFgqvz3vK60huFKwb7RnK3lQXNd+Zze\nAk0Ylb7fDz2tktEXBQxcBquMcfVmsy03xhWlp1s7FeDQ0LCmUlltb7dex9Du9azc4nqgPEZkXXna\nCfb1KuB9+/ZZBM2rykHT+/03PakgSv8m446By3AmdJUFISpPt3YCQXX3WirVpUNDww2PtVLFXfu+\nyhhXNtur6XRXOcFheeXpJTjWb3Fl6gZjN0z/92x68DUFAxcZKQpPt3bK4LScdivuRlmF9SpPJ8G+\n3jG8ruDRCkwPviZg4CJjhf10aycQhNUytLfmobsgyor5DN6LcDBwkdGivgxSFFqG1cIO9nFiSnJQ\nHLkJXFL6XPBERJt1LiI3Klt9VG/tUbsPl533BMVqewtueeFdoVBAd/f68lJdpe1iMplNmJs7ynva\nBCICVRVHn2HgIjrDTiAII1jU7p/VzIAZd4cPH8aWLdtx7NiRxe9ls304dGgvNm7cGGLJWgMDF1EM\nsUUQLN7fcLkJXFwdnmKnsiV9oVAIuyi+aOZK66by8jtvxmr55C8GLoqVOG5xwe0tGvPjdz4wsBVz\nc0dx6NBezM0dZTdsxLGrkGLD9C6fRmNnYSaFRJnpv3NiVyHFgJcuH5O71FZqNbBFYM3k3zm5x8BF\nkeGky8cqwJnapVa9c/KxY0dQLE5gcHDHsuAd5q7JUWXq75y8YeCKgLglE7hht/IGgLHRUazv7sb2\nLVuwvrsbY6OjAMwdZGerwT1Tf+fkkdMZy25f4MoZlrjzcYnd5ZTsbAFj2tI9UVuRw0Sm/c7pDLhY\nOcNzi0tEzhORh0XkZyLypIjc7DmatohCoYAdg4OYKBZx5NgxTBSL2DE42JItL7tdPrOzs+hJJqva\nJkB3IrGkdWJKl1qlpQ2ArQaPTPmdkz/86Cp8BcDHVPVyAG8G8GERWe/DcWPPTiXcKux2+fT09GB2\nfr4qvAFzCwvGjWnUjucBYPIFkU2+p8OLyLcBfFlV/0/N99Xvc5muUChgfXc3JorFciIvsCmTwdG5\nuZZ9crSznNLY6Ch2DA6iO5HA3MIC7hkZwdaBAcfHCQtTuInOcJMOv9rnAvQAuAbAj/08blzlcjnc\nMzKCTTWVcCtXXrlcbsU1Aq/bvBlH5+bqBqZKYOtJJjE7P28Z2MJUScYoFpcnY7Ty757ILt9aXCLS\nDmASwB5V/Y7Fz3XXrl2LX/f396O/v9+Xc5suyq2DqLAbjFZqxUbhXrPFRa1scnISk5OTi19/+tOf\ndtzi8itjcDWAfwJwS4P3BJKRQs3X7AwuO5mEFVNTU9rX2anV6Ym92axOTU1Fas8lJ3tpMWOO4gxh\nbSQJ4OsA/naF9wR68dQcYaTvNwpGteoFuZmZmcilnNsJSFEKtkRBCCVwAXgrgD8AeAzANIBHAbzT\n4n2B3wAKVr2gMD4+HmgAcNLiUj0TXHuz2cXganeeWJRwfhe1AjeBy3Nyhqr+C4CzvB6Hom8xfb9Y\nBFAanVlbLOLm97wHhdOnA0uCcJrEsnVgANdt3rxkLKtQKFTNEyuNK0V9aSATkziiMIZILcBppHP7\nAltcxrNq+awFNG+jFeTX+b2M9TgZV4oC01pc7NYkN+CixcVtTQiA/SflSnbfeatW4ZcnTuBrACpT\nZfuyWew9dCjS252b1iKI2nYm9e4fMyXJLTfzuNjiIscJF/l8XsfHx7UrnbY97kSNNWpNRiWrsFGL\nysQxRIoGhJVVaOtEDFyR5DTxoZpVEkQYolKxu2UVEKJ2TSt1W5rWrUnRwcBFjjlJNbcSdgVr+ur6\nVhV+ItERubEiOy0q08YQKRrcBC6OcbW4Zq+X6OcYUxzWejx8+DC2bNmOY8eOlL9TANAD4EeI0liR\n3TEs08YQKXxuxri4kWSLW0w1z2TQl81iUyaDe0ZGAMDx5pYrbYhZbwNIp8epiMPq+su3c3kIwDmI\n2qaSdlfv5/Yi1BROm2huX2BXoWfV3XJ+d9FVH89N99tKn7E7lubk3F7G56Kkuostne7SZLIzsmNF\nYXcNU/yAY1zxVV2hdyQS2plMuhrXWanicRMM7HzGzliam3PfvHOnZgC9BNAMoDfv3Gn7XkRJ9e+F\nY0XUShi4Yqq6Qs+XJ/1aVe4rBSU7rRk3yRp+BSWn564ccwLQKUAnDG1xWan9XbKlQ3HFwBVT1RX6\nFKB91ald5cr9jj17fOmqC6rFpbpy+rzTc3vNiDSFHytSMPBRVDFwxdRKLa6udNrX1oydAFNbCdqd\n02W3VWhnblijleDjUkn7MT+qXuBjMKMoYOAylJ0KpLpCby+PcVUq9zv27PF9/KhemRp1N/pVETo5\nTm2g+8jOmyM3B8oLrytS1At8Q0PDsbpPZC4GLoNUKufhoSFHWXRWWYV+ddXVO1f196KYxVcpaxT3\n3PLKa4vLKvB1dFyjqVQ2VveJzMXAZYhKALmyo0MzdRIt7LBKYffaVVddvtpgGvUxpbiul+cly9Aq\n8KVSXdrRcWXs7hOZiYHLANWtlilAr7ZItLBTgVgFFz+66hq1qqLa4qqI83p5Xn63tYGv0k0Yx/tE\n5mHgMkB1qyUP6DqHLa7KyuxBBZCVWlVRWVi3Hs6BslYb+HifKCrcBC6uVdhktevrfQ7AbgCXdXTg\n2VdeabiLcGUvrNyqVfjDiRN4uupnfu2FZWf9v6ivRxf18kUF7xNFgZu1Chm4QlAJQJVt6O+86y5c\n09e3ZJv52gqlOqC8FsBlACYBXxeXrZz3sUcfxe233rpYvkbBtFlYyRLFEzeSNIjTdPPaLrwDgK4B\n9KLyn+2JhKduu9rzDg8NRWaOj+lblxBRfWBXodkaddMBWPazNwK4C8B7AbwI962uKG8PEuWyEZF3\n3NbEcI226ajdfuTaVAqvzWSwHUAOzrb0qN02JMrbg/hVNrtbpRBR9DFwRUhPTw9m5+cXd2Z6AsDc\nwgJ6enoAAFsHBnB0bg57Dx3Cv05P4xhQ9731jI2O4tLzz8f7N23Cpeefj7HR0RXPGyY/ymZ3HzAi\nMoTTvkW3L3CMyxYn6eZOU9Pz+bx2JBK6trxQ79ry2JiTCcwr8TLfaKVxPzdli/rcM6JWB87jigcn\nlb+T946Pj+uamnljawAdHx93fCwrXpIo7GxE6aZsUV/tg6jVuQlcq8Ns7ZG1XC5nO/Gg8r7KmM9K\nn1u6KTzwWpfnrVUoFLBjcLCURFEslpIoBgdx3ebNKx7T6rPXbtuGtWefjd7e3iXlsnudFdVdjZXk\njqh0gxKROxzjMpyT8Zve3l4Ukskl40W/SSbR29vruRxOkyiqkyVqP/tjAPMnT+IT73734jVZjc3Z\nUZvUsimTwT0jI8xIJDKZ0yaa2xfYVeg7N+M3lS65q9rabI+L2emic1IWqzljlc8OA8sWHu5Kp+uO\nzdnFvad4DyiawDEu50z+z3zHnj16sYtFeu1es9Mxq+GhIc2mUnpFe3vdcarx8XHtSqeXBaZP3X67\nZlMpTVksPHxhJtNwbI5W5scuykRBYOByyOQVGfL5vHal08t2Q/YrY85pa24x86+jQ7tSKR0eGrL8\n+RXlIHSgZgWQq9vatCuV0j9KJJYtPNy2evWyAH0RA5dtcV41n8wXWuAC8E4ARwH8AsBtdd4T8OU7\nY3qadCVb7kB5hfnecgC4Y88eX44/Pj6ul7W1ad5Ga26le5nP57UzmVzy8yygM+Vuv+rvZwD9H+Vr\nuqr89Rf+5m+Wfb4zmTTmdxW2uO5TRvHgJnB5Ts4QkVUAvgLgegCXAxgQkfVejxu0IFaL8Ht1hkbH\nq2TLbUDpieHjAJLpND74oQ95Pu/Y6ChuvOEG/OHECVwGYAzW2XiV8k1PTze8l9PT08jNzy/5eQ7A\n5lQKZ2NpluO5KK2Y/6p0GrOpFL40NISPffzj2LtvHzZlMri6rQ2bMhns3bePCRY29fT0YH5+FtXT\n1RcW5phZSeZyGulqXwDeBOD7VV/fDotWF2Le4vK729HO8YLYG8uqdbSmPA5Vffzq8nWl05pNJOre\ny7GxMcsxqjvuuGP57wDQ/wVoNpXSmZmZZWUzdTyyIqxr4P5bFFUIo6sQpTVeh6u+/gsAd1u8L+DL\nd87P1SL8DIJOjudnRZjP5/Xuu+/WCyzGk8bGxhqWL1Xu1rsik1lyLw/s369d6bSmyt2CvVi+YkdX\nKqWXlINWZezLz0nCUQl4YSdIROU+EFVj4HLBj//MblZnaHTeMFZ7qATx1yWTmgH0sw0y+KzKdxWg\nn6tpKVUHuAOAdgJ6rkXrbWZmRrOplE4EMN4YlQQcJkgQWXMTuPxYOeMFAOdXfX1e+XvL7N69e/Hv\n/f396O/v9+H03nhZLaLC6eoMlY0ke5JJzM7PL9uosdmrPSxZuaJ8vjcB+AZKv8hVicSSScpW5fsl\ngM8AWCeC48ePA6gaRywf9zoAb2trw99+7Wu44IILUCgUkMvlsGHDBgzfdx/eW7W5ptdJwoVCAdPT\n09i+bRt+cPKk49U8/DY7O4tksgfF4pkRvUSie3Hlf6JWMTk5icnJSW8HcRrpal8AzkKp3uoGkATw\nGIANFu8LPHKHyW63o91uQDfdmH6u53cRoOeXu/+sMhWHh4a0M5nUi8pdfweqsgKtWlyVa+1IJOq2\ngPzqyqrcu6vb2pak3oe5TiFbXETWEEZXYem8eCeAnwN4GsDtdd4T8OWHz07F66Qb0ElF7qVLzDKY\nAvr35W69fD6/pCyVc12+Zo2mUFrtonIt69Np3bdv32KZqwNwVzq9LPHD7ykIVteyFtB8QOdzggkS\nRMuFFrhsnagFApcdfiRy1AY0P45ZPUE4A2hPOr0YAGuDYntNBmElMHy23OLq7ehYEjwr5R0bG9Or\n29psBW236rUeL7W5xFXQmCBBtBQDlyG8ZDNataz8SuaoVKozMzOLlatVUFxTDlTVgeHCNWuWrTFY\nHTwr2YW1afHNaHGty2R0fHycwYIoghi4DOLmybtepTwzM+OqxeW2a/MiQB+oOdfdd9+tvR0dlsGz\nNrtwbfmw3J7PAAAM1UlEQVQYQbWAgpjfRkTBYOCKufHx8bpdbU4ra7sbN1oFxc5kUrvS6SXnatRd\nWRv88uWuO6drDQa1wSYRhYeBK8bsdLX5tQVJbVC7eefOZUHR6lxWwbPeivBux+DCno9FRP5i4Iop\nv7vaGo2JNeqOrE0IsQqSVtmHfZ2d2plMansi4ar7zm3yCVtoRNHHwBVTfnW1VVQCwQSgU4BONOjW\ns0r0GB4a0q5Ualn2oNU5/EiScJN84qSFxtYcUXgYuGIqiC1Ybtm5UzOAXlpOYb95505b5xoeGmqY\nPVjh57JVTq/f6VqPJm9vQ2Q6N4HL87YmFLxcLod7RkbwJ8kkLgbwZgDzr7yChw8dcnW8QqGAb4yM\n4BGUZo0/AuAfRkYWl2C6Z2QEmzIZ9GWz2JTJLC6/VCgU8PFbbsGlWLoVyfmrVy/bDqZ6WSjA27JV\njcpkxcmWNUFsb0NEAXMa6dy+wBaXJ/l8acfjB+B9FQg7rSGrMZ+pqSm9sqNj2Q7FXamUZTn8Tkv3\nK/nE7XuJyH9gV2F8hdn1Vvu5z2LpDsXDQ0MNPxNG0oOToMl5X0ThcRO4pPS54ImINutccVQoFLC+\nu3vJCu6bMhkcnZtztbp4ZYX66tXYq1eoX+lz55x1Fp5dWMDnv/QlX3ZdDkKhUCh1Bfb0rHiPnLyX\niPwjIlBVcfQZBi5zuA029bitrKNQyUehDETkHQNXk4RZabLCXnk/MyIyBwNXE7DSDJffXaZEFC43\ngYvp8A5U7xR85NgxTBSL2DE4iEKhEHbRfFcoFHD48OHIXRvT14mIgcuBVqk0x0ZHsb67G9u3bMH6\n7m6MjY6GXaRFfs4PIyIzMXA50AqVZtRblU4nIxNR/KwOuwAmWaw0azL7TK80qxM+FluVxSKApa1K\nq+sMKlmk0XG3Dgzgus2bWz5JhahVscXl0NaBARydm8PeQ4dwdG7O+MSM2m7Bxx591LJV2d7evmzM\nK6guRTvHzeVy2LhxI4MWUQtiVmELqW3FPPXUU3hLby9+cOrUkgy9O++6C7ffeutiq/L9g4P4xsjI\nkkzK6zZvDiS7j1mDRK3FTVYhuwoDErX5Vl/duxcfv+UWdCeTeOGVV/D+wUHc99Wv4jXloAWc6Ra8\npq8PR+fmMDs7i/b2drzt9a8vBZJisRRIBgcx+u1vO+pStMtpVyURtR52FQYgzKw8qzT2r+7di1u2\nb8cFp07hhd//Hh8uFjH8la/gO6dO4TeAZbJJpSvu+PHjyzIpz1u1Ci+//HIgiSp2E2Cimq5PRE3g\ndHFDty+0yCK7Ya42brUhYj6f12wyuWRV+Sygl5TLd6C8YO4l5VXeaxeYtbqeNYB2pdN6886dgSxO\nu9Kit9z4kSg+wNXhw+fnKu5O1AuYn7ztNl0DaF85QB0A9HXlVd0r750ANJtK6czMjOWxK4HiIkDX\nlo9ROf7MzEwgq7/XW1We25AQxYubwMUxLp9Vd3W9FsBDAH41Px/4XC+rsaFzzjoLf//FL+JH5a+f\nANAPoAjg/du2YdPo6GICxvDICDZs2GB57K0DA1h79tm4+T3vwc9PnEBlpKk7kcDx48exceNG368n\nl8tZjmlxDIyIOMbls8pcr7clEugBsAuAnj7terdiu6zGhp5dWMAFNeNT6wB87PbbMTwy4iitv7e3\nF4XTp/Fi1fGrx56aNebUCpPAiWgFTptobl9oka5C1fC6s2rHhoaHhnwtR72xp8r3r25ra8qYEzd+\nJIoPcCPJaDh8+DC2b9mCI8eOLX6vL5vF3kOHAulWq1abhh/0Hl6FQgGXnHce/nl+frE78k+SSTz9\n/POBdt1FbboBEbnDbU0iImqTaIOs5A8ePIgPX389nq763sUA7hkfxzve8Q5fz0VE8cMJyBERtTUN\n6yU6+OXXKAXnSpB+sfHbiYg8YYsrQCZ1Z1XK2t7eXpp0bLPMhUIBF557LhILC+gBMAtgIZHAMy+8\nEPlrJqLwNX0jSRH5nIg8JSKPici3RCTr5XhxY8pCsJWVPgavvRavf93rsPXaa22v+JHL5XDv/fdD\n02mcaGuDptO49/77I3/NRGQuTy0uEdkM4GFVPS0id6KUHfLJOu9tuRaXCSzH4wB8C8B7HYzLmdS6\nJKLoaPoYl6pWT056BMB7vRyPms9yQi+ANjib2Bv0OBoRUYWfE5C3Afi+j8ejJrCc0AvgBDixl4ii\nacXAJSIPicgTVa8ny3/+edV7/hrAgqruD7S05LvFDMhMBldmMngTgGw6jfeW9+WanZ3lCuxEFCme\nswpF5CYAHwRwnaqeavA+3bVr1+LX/f396O/v93RuWsrLOFNtVuFjjz6K22+9dcnmkabv9kxE4Zuc\nnMTk5OTi15/+9KebOwFZRN4J4AsA/kRV/+8K72VyRoAqK2T4EWiiNoGaiOKr6enwAL4MoB3AQyLy\nqIjc4/F45EKhUMCOwUFMFIs4cuwYJopF7BgcdN3Ft5iwUf66egV2IqKwec0qvMSvgpB7fm/1UZ2w\nUWlxMVGDiKKC25rEgN9bfVQnbPRls9iUyYS6ZBURUTUu+RQTfq8CD3BSMREFj6vDtzgGGiIyDQMX\nEREZJYysQiIioqZi4CIiIqMwcBERkVEYuIiIyCgMXEREZBQGLiIiMgoDFxERGYWBi4iIjMLARURE\nRmHgIiIiozBwERGRURi4iIjIKAxcRERkFAYuIiIyCgMXEREZhYGLiIiMwsBFRERGYeAiIiKjMHAR\nEZFRGLiIiMgoDFxERGQUBi4iIjIKAxcRERmFgYuIiIzCwEVEREZh4CIiIqMwcBERkVF8CVwi8lci\nclpE1vlxPCIiono8By4ROQ/AFgBz3otjrsnJybCLEKg4X1+crw3g9Zku7tfnhh8trrsAfMKH4xgt\n7v+44nx9cb42gNdnurhfnxueApeIvAvAc6r6pE/lISIiamj1Sm8QkYcAvLr6WwAUwH8H8CmUugmr\nf0ZERBQYUVV3HxS5AsAhAP8PpYB1HoAXALxBVfMW73d3IiIiijVVddTocR24lh1I5FcA+lT1d74c\nkIiIyIKf87gU7CokIqKA+dbiIiIiaoZQVs6I44RlEfmciDwlIo+JyLdEJBt2mfwgIu8UkaMi8gsR\nuS3s8vhJRM4TkYdF5Gci8qSI3Bx2mfwmIqtE5FER+W7YZQmCiHSKyDfL//d+JiJvDLtMfhGRW0Xk\npyLyhIg8ICLJsMvkhYiMiMhLIvJE1ffWishBEfm5iIyLSKedYzU9cMV4wvJBAJer6jUAngbwyZDL\n45mIrALwFQDXA7gcwICIrA+3VL56BcDHVPVyAG8G8OGYXR8A3AJgJuxCBOhLAL6nqhsAXA3gqZDL\n4wsROQfAR1DKG7gKpQzwG8MtlWf3oVSXVLsdwCFVvQzAw7BZb4bR4orlhGVVPaSqp8tfPoJSlqXp\n3gDgaVWdU9UFAAcA3BBymXyjqv+mqo+V/34cpUrv3HBL5Z/yQ+KfArg37LIEodyr8ceqeh8AqOor\nqvrvIRfLT2cBaBOR1QDWAPh1yOXxRFV/CKA2ee8GAPeX/34/gP9o51hNDVwtNGF5G4Dvh10IH5wL\n4Lmqr59HjCr2aiLSA+AaAD8OtyS+qjwkxnUg+wIAvxGR+8rdocMikgm7UH5Q1V8D+AKAZ1GaZvSy\nqh4Kt1SBeJWqvgSUHiQBvMrOh3wPXCLyULlPtvJ6svznu1CasLyr+u1+nz9IDa7tz6ve89cAFlR1\nf4hFJQdEpB3AgwBuKbe8jCcifwbgpXKLUmDY/zWbVgPoA/B3qtqH0pzS28Mtkj9EpAul1kg3gHMA\ntIvI+8ItVVPYeshaceUMx2dV3WL1/fKE5R4Aj4tIZcLyERGxnLAcRfWurUJEbkKpa+a6phQoeC8A\nOL/q68ok89god8M8COAbqvqdsMvjo7cCeJeI/CmADIAOEfm6qv7XkMvlp+dR6sH5SfnrBwHEJYFo\nM4BnVPW3ACAi/wjgLQDi9kD8koi8WlVfEpHXALAVC5rWVaiqP1XV16jqhap6AUr/6HpNCVorEZF3\notQt8y5VPRV2eXxyGMDFItJdzmi6EUDcstO+BmBGVb8UdkH8pKqfUtXzVfVClH5vD8csaKHcxfSc\niFxa/tbbEZ9ElGcBvElE0uUH/bcjHoknta3/7wK4qfz3vwRg6+HR9xaXA3GbsPxlAEkAD5X+neER\nVd0RbpG8UdU/iMhOlDImVwEYUdU4/OcBAIjIWwH8FwBPisg0Sv8mP6Wq/xRuyciBmwE8ICIJAM8A\n+EDI5fGFqk6JyIMApgEslP8cDrdU3ojIfgD9AM4WkWdRGja6E8A3RWQbSpnm/9nWsTgBmYiITBLK\nBGQiIiK3GLiIiMgoDFxERGQUBi4iIjIKAxcRERmFgYuIiIzCwEVEREZh4CIiIqP8fx9tKk5NH0A+\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8f140a9668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "class_A = np.random.normal(0, 1, size=(100, 2))\n",
    "class_B = np.random.normal(4, 1.5, size=(100, 2))\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "ax.scatter(class_A[:,0], class_A[:,1], label='A', c='r')\n",
    "ax.scatter(class_B[:,0], class_B[:,1], label='B')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is the output:\n",
    "How it works... Okay, so now that we've looked at how we can classify points based\n",
    "on distribution, let's look at how we can do this in scikit-learn:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.mixture import GMM\n",
    "gmm = GMM(n_components=2)\n",
    "X = np.row_stack((class_A, class_B))\n",
    "y = np.hstack((np.ones(100), np.zeros(100)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:52: DeprecationWarning: Class GMM is deprecated; The class GMM is deprecated in 0.18 and will be  removed in 0.20. Use class GaussianMixture instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function distribute_covar_matrix_to_match_covariance_type is deprecated; The functon distribute_covar_matrix_to_match_covariance_typeis deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GMM(covariance_type='diag', init_params='wmc', min_covar=0.001,\n",
       "  n_components=2, n_init=1, n_iter=100, params='wmc', random_state=None,\n",
       "  tol=0.001, verbose=0)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "# We'll create a training set:\n",
    "train = np.random.choice([True, False], 200)\n",
    "gmm.fit(X[train])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fitting and predicting is done in the same way as \f",
    "tting is done for many of the\n",
    "other objects in scikit-learn:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function distribute_covar_matrix_to_match_covariance_type is deprecated; The functon distribute_covar_matrix_to_match_covariance_typeis deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/nbuser/anaconda3_410/lib/python3.5/site-packages/sklearn/utils/deprecation.py:70: DeprecationWarning: Function log_multivariate_normal_density is deprecated; The function log_multivariate_normal_density is deprecated in 0.18 and will be removed in 0.20.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gmm.fit(X[train])\n",
    "gmm.predict(X[train])[:5]\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are other methods worth looking at now that the model has been \f",
    "t. For\n",
    "example, using score samples, we can actually get the per-sample likelihood for each\n",
    "label.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}