Logistic_Regression_Brazilian_Census_2010.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression on 2010 Brazilian Census Data\n",
    "\n",
    "## What is Logistic Regression?\n",
    "\n",
    "Logistic regression is a statistical model used mainly in binary classification, although it can be used for multiclass classification.\n",
    "\n",
    "It's a predictive model that results in a binary dependent variable that can assume only two possible values. Logistic regression differs from linear regression by using logistic function rather than a linear function to rank the results.  There are several advantage of using this function: it brings more robustness to the results, since the independent variables don’t have to be normally distributed, or have equal variance in each group, and also bring clarity on classification, since logistic function is much more sensitive in data close to the border value and much less sensitive in values far from the border, causing it to generate binary results with greater reliability.\n",
    "\n",
    "Logistic function can be written as $$g(z) = \\frac{1}{1+e^{-z}}$$ and its graph can be shown below:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl0XOWZ5/HvUyotXuRd3neQjW0CGIRZmq2DbYyTxukkJKaHzgITknTIIZN0zyHDHDoh0zOTTndm0tOkCUloGrIQSAcwwcQmtAkJwYAM3uQF5AVbsrVYtizJ1lKqeuaPKpFCkayyXaVbVfp9zinXrXtfqR7fuvrp6r3La+6OiIjkl1DQBYiISPop3EVE8pDCXUQkDyncRUTykMJdRCQPKdxFRPKQwl1EJA8p3EVE8pDCXUQkD4WDeuMJEyb47Nmzg3p7EZGctGnTpiPuXjZQu8DCffbs2VRWVgb19iIiOcnM3kmlnbplRETykMJdRCQPKdxFRPKQwl1EJA8p3EVE8tCA4W5mD5lZg5lt72e5mdk/mVm1mW01s4vTX6aIiJyOVPbcHwZWnGL5jUB54nEH8C9nX5aIiJyNAc9zd/eXzGz2KZqsAh7x+Hh9G81sjJlNcffDaapRRPJcLOZ0dsfo6o7R2R2lsztGJBqjO+Z0JU1HojGiMac75sR6P7sTjTkxh5jH5/dMuzueeB8HYk58noPT88x7XvfoGYq0p03P9LvL8aS2yfOT9BrO9PoFk7hwxpg0rLn+peMipmnAwaTXNYl5fxTuZnYH8b17Zs6cmYa3FpGguTvH2yM0tnbSdKKLYye6OHqyi+aTEVo6IrS0R2jp6OZEZ/zR1hmlvaubk11R2iNROiMxuqKxoP8bGWf2h+mJo0pyItxT5u4PAg8CVFRUaGRukRzg7jS0drLvyAneaTrB/qaTHGpuTzw6aGzt7DeciwpCjBpWyKiSMCOKw4wsDjNtTCHDisIMLyxgWFEBJYUFFIdDFBeGKA7Hp4vCIYrDIQoLQoRDFn8uMMKh+HNByCiwxHPyw4yQGaEQ8efEtGGY9cxLvA6BAWaWeP5DO/hDGL9nXqJ9z3SP5OC25BcBSke41wIzkl5PT8wTkRwTicZ4q76VrTXH2VZ7nLfqWtld30prR/e7bcIhY8qYEqaOHsaSOeOYNKqEstJiJowsomxkMWNHFDF2eBFjhhdSUlgQ4P9maEtHuK8B7jSzx4DLgOPqbxfJDd3RGG8ebGbjniY27mti0zvH6IjE98JHlYQ5b8ooVl00lXmTSpk7YSSzxg9nyugSwgU6izrbDRjuZvZT4DpggpnVAH8LFAK4+wPAWmAlUA2cBD6dqWJF5Oyd6OzmhV0NvLCzng27Gmjp6MYMFkwexepLZ7J45hgunD6GWeOHZ00Xg5y+VM6WuWWA5Q58IW0ViUjaxWLO7/c08Ys3anhuex3tkSjjRhSxfNFkrj9vIlecM54xw4uCLlPSKLBb/opI5p3o7Obnm2r415f3sb/pJKUlYT60eCofumgaFbPHURDSnnm+UriL5KHWjgjf/+0+Hn55Hy0d3Vw0YwzfWTaPGxZN1kHOIULhLpJHOiJRfrTxHe7fUM2xkxFuWDSJO645h0tmjQ26NBlkCneRPPH7PUe458nt7DtygqvLJ/A3N8zngumZvVBGspfCXSTHHT8Z4e/W7uDxyhpmjR/OI7ct4Zp5Aw6xKXlO4S6Swza9c4w7f/IGDa2dfO7ac7jr+nKGFalPXRTuIjnJ3Xno5f38r7U7mTKmhCf/6kp1wch7KNxFckxHJMpXntjCs1sPs3zhJL5184WMHlYYdFmSZRTuIjmkpSPCZ/6tklf3HeXuG8/js9fM1VWk0ieFu0iOqG/p4JMPvcaexja+s/oiVl00LeiSJIsp3EVyQH1LBzc/8ApNbZ089KlLubpcZ8PIqSncRbLcsRNd/OUPX6WprZMf/efLWDxTFyTJwBTuIlmsrbObT/3ra+xvOsnDn75UwS4p002ZRbJUJBrjjkcq2X6ohe/+xcVcec6EoEuSHKJwF8lSf/fsTn6/p4lvfuQCli6cFHQ5kmMU7iJZ6InKgzz8+/3cftUcPnrJ9KDLkRykcBfJMlsONnPPU9u58pzxfPXG84IuR3KUwl0kixw/GeHzP9pE2chi/t8tizVWqZwxnS0jkkW+9kwV9a2d/OLzVzJ+ZHHQ5UgO026BSJZYu+0wT75Zyxfffy4XztBNwOTsKNxFskBDawf3PLmNC6aP5gt/em7Q5UgeULiLBMzd+W+/2MbJrijf/tiFFKqfXdJAW5FIwJ7bXsevdzbwNzfM59yJpUGXI3lC4S4SoJNd3fyPX+5gwZRRfOrK2UGXI3lE4S4SoO9u2MOh4x18Y9UinfYoaaWtSSQg+46c4MGX9vLhxdOomD0u6HIkzyjcRQLg7nz9mSqKwiHu1lWokgEKd5EA/K76CC/ubuSu68uZOKok6HIkDyncRQaZu/OtdbuZNmYYn7hyVtDlSJ5SuIsMsnVV9WytOc5dS8spDhcEXY7kqZTC3cxWmNluM6s2s7v7WD7TzDaY2ZtmttXMVqa/VJHcF405335+N3PLRvDhxRrgWjJnwHA3swLgfuBGYCFwi5kt7NXsvwOPu/tiYDXw3XQXKpIP1myp5a36Nr68bJ5OfZSMSmXrWgJUu/ted+8CHgNW9WrjwKjE9GjgUPpKFMkPkWiM//P82yyYMoqV508JuhzJc6mE+zTgYNLrmsS8ZF8DbjWzGmAt8MW0VCeSR57efIgDR0/ylWXzCIUs6HIkz6Xr78JbgIfdfTqwEnjUzP7oe5vZHWZWaWaVjY2NaXprkewXizkPvrSH8yaXcv2CiUGXI0NAKuFeC8xIej09MS/Z7cDjAO7+ClAC/NFQ7e7+oLtXuHtFWVnZmVUskoM27G7grfo2PnvtXMy01y6Zl0q4vw6Um9kcMysifsB0Ta82B4DrAcxsAfFw1665SML3frOXaWOG8cELpgZdigwRA4a7u3cDdwLrgJ3Ez4qpMrP7zOymRLOvAJ8xsy3AT4FPubtnqmiRXLLpnWO8tv8ot181R/dql0GT0hiq7r6W+IHS5Hn3Jk3vAP4kvaWJ5Ifv/WYPo4cV8vFLZwzcWCRNtBshkkF7Gtt4fmc9n7hiFiOKNR69DB6Fu0gGPfrKOxSGQnziitlBlyJDjMJdJENOdHbz75tqWPm+yZSVFgddjgwxCneRDHl68yFaO7v5yyt050cZfAp3kQxwdx55ZT8Lpozi4pljgy5HhiCFu0gGvHHgGLvqWvnLy2fpoiUJhMJdJAMefeUdSovDrLpIFy1JMBTuIml2pK2Ttdvq+Mgl03X6owRG4S6SZv++qYauaIxbL58ZdCkyhCncRdLI3XliUw2XzBrLuRNLgy5HhjCFu0gabT7YTHVDGzdfMj3oUmSIU7iLpNETm2ooKQzxgQs00pIES+EukiYdkSjPbDnEjedPobSkMOhyZIhTuIukybqqOlo7utUlI1lB4S6SJk9U1jB97DAunzs+6FJEFO4i6VDb3M7Le47wkYuna/BryQoKd5E0eOrNWtzho+qSkSyhcBc5S+7OU2/WcunsscwYNzzockQAhbvIWdtV18rbDW3cdNG0oEsReZfCXeQsPb35EOGQ8YH36dx2yR4Kd5GzEIs5z2w5xNXlExg3oijockTepXAXOQubDhyjtrmdVeqSkSyjcBc5C2s2H6KkMMSyhZOCLkXkPRTuImcoEo3x7LbDLF0wSfdtl6yjcBc5Q7+rPsLRE13qkpGspHAXOUPPbDnEqJIw18ybEHQpIn9E4S5yBrq6Yzy/o57liyZTHC4IuhyRP6JwFzkDL1cfobWjm5Xvmxx0KSJ9UriLnIFntx2mtCTMVeeWBV2KSJ8U7iKnqas7xvqqOpYtnERRWD9Ckp20ZYqcppf3HKGlo1u3G5CsllK4m9kKM9ttZtVmdnc/bT5mZjvMrMrMfpLeMkWyx3PbDlNaHOaqcp0lI9lrwCsvzKwAuB9YBtQAr5vZGnffkdSmHPgq8CfufszMJmaqYJEgRaIx1u+oZ+nCSTpLRrJaKnvuS4Bqd9/r7l3AY8CqXm0+A9zv7scA3L0hvWWKZIff72mi+WSEleqSkSyXSrhPAw4mva5JzEs2D5hnZi+b2UYzW9HXNzKzO8ys0swqGxsbz6xikQA9t+0wI4vDXK0uGcly6TqgGgbKgeuAW4Dvm9mY3o3c/UF3r3D3irIynUImuSUac57fUc+fnjeRkkJ1yUh2SyXca4EZSa+nJ+YlqwHWuHvE3fcBbxEPe5G8Ubn/KE0nulixSBcuSfZLJdxfB8rNbI6ZFQGrgTW92jxFfK8dM5tAvJtmbxrrFAncr6rqKAqHuG6+/uqU7DdguLt7N3AnsA7YCTzu7lVmdp+Z3ZRotg5oMrMdwAbgb9y9KVNFiww2d2d9VT3XlE/Q7X0lJ6S0lbr7WmBtr3n3Jk078OXEQyTvbK9toba5nbuWqrdRcoOuUBVJwbqqOgpCxtIFGnFJcoPCXSQFv6qqY8nscRoEW3KGwl1kANUNbVQ3tLHifJ0lI7lD4S4ygHVVdQAsX6QuGckdCneRAazfUc+F00czZfSwoEsRSZnCXeQU6o53sOVgM8t14ZLkGIW7yCk8v7MegBvUJSM5RuEucgrrq+qYO2EE55SNDLoUkdOicBfpx/H2CK/saWLZokmYWdDliJwWhbtIP17c3UB3zFm+UP3tknsU7iL9WL+jngkji1k844/uXi2S9RTuIn3o7I7ym92NLFs4iVBIXTKSexTuIn14ZU8TbZ3dunBJcpbCXaQP63fUM6KogCvPGR90KSJnROEu0kssMZzetfPLKA5rOD3JTQp3kV421zTT2NrJDboqVXKYwl2kl/VV9YRDxnXzJwZdisgZU7iL9LJ+Rx1XnDOe0cMKgy5F5Iwp3EWSVDe0sbfxBMsX6iwZyW0Kd5Ekz++I3yhsqcJdcpzCXSTJ+h11XKB7t0seULiLJDS0dPDmgWZ1yUheULiLJPTcu10Dc0g+ULiLJKyvqmf2+OGUT9S92yX3KdxFgJaOCL/fc4Tliybr3u2SFxTuIsCGXQ1Eoq6rUiVvKNxFgF9tr2Niqe7dLvlD4S5DXkckyou7G1m+SPdul/yhcJch76W3GmmPRNUlI3lF4S5D3rqqekaVhLl8ru7dLvkjpXA3sxVmttvMqs3s7lO0+4iZuZlVpK9EkcyJRGP8emc9SxdMorBA+zqSPwbcms2sALgfuBFYCNxiZgv7aFcK3AW8mu4iRTLltX1HOd4e0YVLkndS2VVZAlS7+1537wIeA1b10e4bwDeBjjTWJ5JRv9peR0lhiGvnlQVdikhapRLu04CDSa9rEvPeZWYXAzPc/dk01iaSUbGYs66qjmvnlTGsSMPpSX45605GMwsB3wa+kkLbO8ys0swqGxsbz/atRc7KpgPHaGjtZOX7pgRdikjapRLutcCMpNfTE/N6lALnAy+a2X7gcmBNXwdV3f1Bd69w94qyMv0ZLMF6duthisIh3n+ehtOT/JNKuL8OlJvZHDMrAlYDa3oWuvtxd5/g7rPdfTawEbjJ3SszUrFIGsRizq+213FNeRmlJRpOT/LPgOHu7t3AncA6YCfwuLtXmdl9ZnZTpgsUyYQ3Dx6jrqWDD1ygs2QkP4VTaeTua4G1vebd20/b686+LJHMenZrHUUFIa5foIE5JD/pqg0ZcmIx57nth7lm3gRGqUtG8pTCXYaczTXNHD7ewY3n6ywZyV8Kdxly1m49TGGBsVRjpUoeU7jLkOLuPLe9jqvLyxg9TF0ykr8U7jKkvHHgGLXN7XxAFy5JnlO4y5Dy9OZDFIdD3HC+ToGU/KZwlyEjEo3x7NbDLF04iZHFKZ0FLJKzFO4yZLxcfYSmE12sunBq0KWIZJzCXYaMNVsOMaokzLXzdV8jyX8KdxkSOiJR1m2vY+X7plAc1u19Jf8p3GVIeGFnAye6otykLhkZIhTuMiQ8vbmWiaXFXKZBsGWIULhL3ms+2cWLuxv54AVTKQhZ0OWIDAqFu+S9NVsO0RWN8ZFLpg3cWCRPKNwl7z1RWcPCKaNYNHV00KWIDBqFu+S1XXUtbKs9zs0V04MuRWRQKdwlrz1RWUNhgbHqInXJyNCicJe8FYnGeOrNWpYumMS4EUVBlyMyqBTukrf+Y1cDTSe6+Ogl6pKRoUfhLnnricoaykqLuXaebjcgQ4/CXfJSQ2sHG3Y38OHF0wgXaDOXoUdbveSln712kGjM+filM4IuRSQQCnfJO93RGD957QBXl09gbtnIoMsRCYTCXfLOC7saOHy8g1svnxV0KSKBUbhL3vnRxneYOrqE68+bGHQpIoFRuEte2dvYxm/fPsJfXDZTB1JlSNPWL3nlRxsPUFhgfEwHUmWIU7hL3jjZ1c0Tmw6y4vwpTCwtCbockUAp3CVv/HxTDa0d3XziCh1IFVG4S17ojsb4/m/3cvHMMVTMGht0OSKBU7hLXli7vY6DR9v57LXnYKbRlkRSCnczW2Fmu82s2szu7mP5l81sh5ltNbMXzEx/F8ugcXe+95s9zC0bwbIFk4IuRyQrDBjuZlYA3A/cCCwEbjGzhb2avQlUuPsFwM+Bv093oSL9ebm6iapDLXz2mrmENEaqCJDanvsSoNrd97p7F/AYsCq5gbtvcPeTiZcbAd1jVQbNA7/Zw8TSYj60WANyiPRIJdynAQeTXtck5vXnduC5vhaY2R1mVmlmlY2NjalXKdKPrTXN/K76CLddNYficEHQ5YhkjbQeUDWzW4EK4Ft9LXf3B929wt0rysp0j205e/+4/i3GDC/kLy6bGXQpIlkllXCvBZIv95uemPceZrYUuAe4yd0701OeSP9e33+U37zVyOeuPYdRJYVBlyOSVVIJ99eBcjObY2ZFwGpgTXIDM1sMfI94sDekv0yR93J3vvWr3ZSVFvPJK2YHXY5I1hkw3N29G7gTWAfsBB539yozu8/Mbko0+xYwEnjCzDab2Zp+vp1IWrz09hFe23+UL77/XIYVqa9dpLdwKo3cfS2wtte8e5Oml6a5LpF+uTv/sG4308YMY/Wl6msX6YuuUJWc8+y2w2yrPc6XlpZTFNYmLNIX/WRITjnZ1c3/fHYnC6aM4s91XrtIvxTuklPu31DNoeMd3LdqkQbjEDkF/XRIzth35ATff2kfH148jUtnjwu6HJGspnCXnODufP2ZKorCIe6+8bygyxHJegp3yQnrqup4cXcjX1pazsRRGmVJZCAKd8l6R9o6uefJ7SycMopPXjk76HJEckJK57mLBMXduefJbbR2dPOTz1xEoQ6iiqREPymS1X7xRi3rqur56xvmMX9yadDliOQMhbtkrdrmdr62pools8dx+1Vzgy5HJKco3CUrdXZH+cKP3yDmzj/cfCEFGmFJ5LSoz12yjrtz71NVbD7YzAO3XszM8cODLkkk52jPXbLOj189wM8qD/KFPz2HFedPCbockZykcJes8vr+o3z9mSqum1/Gl5fND7ockZylcJesseNQC7c//Dozxg7nO6sXq59d5Cwo3CUr7Dtygk889CojisM8cvsSRg/TsHkiZ0PhLoE71NzOrT94FXd49PbLmD5WB1BFzpbCXQK1p7GNmx94hZb2CP922xLOnTgy6JJE8oJOhZTAvHngGLc9/DohM37ymcs5f9rooEsSyRsKdwnEf+yq5ws/fpOy0mIeuW0JsyeMCLokkbyicJdBFY05//fXb/HPG6pZNHUUD33qUiaW6ha+IummcJdB09DawV0/3cwre5v4eMUMvr5qESWFBUGXJZKXFO6Sce7Ok2/W8o1f7qA9EuUfbr6Qj14yPeiyRPKawl0y6kDTSe55ahu/ffsIF88cwzc/cgHlk3TrXpFMU7hLRjS1dXL/hj38aOM7FIVDfGPVIv7TZbMI6apTkUGhcJe0amzt5NFX9vPD3+2jPRLl5ktm8F+WzWPyaB00FRlMCndJix2HWvjXl/fx9OZDdEVj3Hj+ZL6yfL4uShIJiMJdzlhjaydrthziF2/UUHWohZLCEB+7dDqf/pM5nFOmUBcJksJdUubuVDe08cKuBn69o543Dhwj5vC+aaP52z9byIcumsbYEUVBlykiKNzlFCLRGLvrWtl8sJlX9x1l494mGls7AVg0dRR3vr+cP7tgis5+EclCKYW7ma0AvgMUAD9w9//da3kx8AhwCdAEfNzd96e3VMmUWMypb+1gb+MJ3qpv5a36NnbVtbDjUAud3TEAJpYWc8Xc8Vw+dzzXzi9j2phhAVctIqcyYLibWQFwP7AMqAFeN7M17r4jqdntwDF3P9fMVgPfBD6eiYLl9HR2R2k+GeFIWyeNrfFHQ2sntc3tHG5up7a5nXeaTr4b4gCjhxUyf3Ipt14+iwumj+bC6WOYNX44ZjqNUSRXpLLnvgSodve9AGb2GLAKSA73VcDXEtM/B/7ZzMzdPY215rxYzIm6E43FH909z9EYkZgTjTqRWIxINEak2+mKRunsjtGVeHR0x+iIROmMRGmPRDnZFaW9K8qJrm5OdEZp7eimrTPC8fZuWtojHG+P0NbZ3WctY4cXMnXMMGaNH8G188qYNX4Es8ePYN6kkZSVFivIRXJcKuE+DTiY9LoGuKy/Nu7ebWbHgfHAkXQUmezx1w/y4G/3vvu6v98f3s+Lnkl3f0+bnm/jOO5Jr5PauceXx95d3jMdbxOLxb825vH5UXc8EeaxDP2aKw6HGFEcZmRxmBHFYUqLw0wbM4wFU0oZPayQ8SOKGDuiiHHDi5g4qpiykSWUlRYzrEj3dBHJZ4N6QNXM7gDuAJg5c+YZfY+xI4qY3/sAXj87mcmzk/dE7d15fbexxD+Gvdum58sNIxRKTBmEktqFzAhZfLog9Id5BWaEQkbIIByKTxeYES4IEQ4ZBSGjsMAoCIUoLDCKCkIUFoQIFxjF4QKKwiGKE4+SwgKKC0MMLwozrLBA44yKSJ9SCfdaYEbS6+mJeX21qTGzMDCa+IHV93D3B4EHASoqKs5oX3bZwkksWzjpTL5URGTISGWYvdeBcjObY2ZFwGpgTa82a4BPJqY/CvyH+ttFRIIz4J57og/9TmAd8VMhH3L3KjO7D6h09zXAD4FHzawaOEr8F4CIiAQkpT53d18LrO01796k6Q7g5vSWJiIiZyqVbhkREckxCncRkTykcBcRyUMKdxGRPKRwFxHJQxbU6ehm1gi8c4ZfPoEM3NogTbK1tmytC7K3NtV1+rK1tmytC06/tlnuXjZQo8DC/WyYWaW7VwRdR1+ytbZsrQuytzbVdfqytbZsrQsyV5u6ZURE8pDCXUQkD+VquD8YdAGnkK21ZWtdkL21qa7Tl621ZWtdkKHacrLPXURETi1X99xFROQUsjbczexmM6sys5iZVfRa9lUzqzaz3WZ2Qz9fP8fMXk20+1nidsWZqPNnZrY58dhvZpv7abffzLYl2lVmopZe7/c1M6tNqm1lP+1WJNZjtZndnem6Eu/5LTPbZWZbzexJMxvTT7tBWWcDrQMzK058ztWJbWp2pmpJes8ZZrbBzHYkfg7u6qPNdWZ2POkzvrev75Wh+k752VjcPyXW2VYzu3gQapqftC42m1mLmX2pV5tBW2dm9pCZNZjZ9qR548zseTN7O/E8tp+v/WSizdtm9sm+2gzIE0PBZdsDWADMB14EKpLmLwS2AMXAHGAPUNDH1z8OrE5MPwB8fhBq/kfg3n6W7QcmDOL6+xrw1wO0KUisv7lAUWK9LhyE2pYD4cT0N4FvBrXOUlkHwF8BDySmVwM/G4R1NAW4ODFdCrzVR13XAb8crG3qdD4bYCXwHPGBzC4HXh3k+gqAOuLnhAeyzoBrgIuB7Unz/h64OzF9d1/bPjAO2Jt4HpuYHnu675+1e+7uvtPdd/exaBXwmLt3uvs+oJr4IN7vsvh4ee8nPlg3wL8BH8pkvYn3/Bjw00y+T5q9O/i5u3cBPYOfZ5S7r3f3npG7NxIf3SsoqayDVcS3IYhvU9dbhkcQd/fD7v5GYroV2El8rOJcsQp4xOM2AmPMbMogvv/1wB53P9MLJc+au79EfHyLZMnbUn+5dAPwvLsfdfdjwPPAitN9/6wN91Poa8Du3hv9eKA5KUD6apNuVwP17v52P8sdWG9mmxJjyQ6GOxN/Ej/Uz59/qazLTLuN+B5eXwZjnaWyDt4zADzQMwD8oEh0Ay0GXu1j8RVmtsXMnjOzRYNVEwN/NkFvW6vpf0crqHUGMMndDyem64C+xgxNy7ob1AGyezOzXwOT+1h0j7s/Pdj19CfFOm/h1HvtV7l7rZlNBJ43s12J3+wZqQv4F+AbxH8Iv0G8y+i2s3m/dNXWs87M7B6gG/hxP98m7ess15jZSODfgS+5e0uvxW8Q73ZoSxxTeQooH6TSsvazSRxfuwn4ah+Lg1xn7+HubmYZO10x0HB396Vn8GWpDNjdRPzPwHBiT6uvNikbqE6LDwr+YeCSU3yP2sRzg5k9Sbw74Kx+GFJdf2b2feCXfSxKZV2ekRTW2aeADwLXe6KjsY/vkfZ11oe0DQCfbmZWSDzYf+zuv+i9PDns3X2tmX3XzCa4e8bvoZLCZ5OxbSsFNwJvuHt97wVBrrOEejOb4u6HE91UDX20qSV+bKDHdOLHHk9LLnbLrAFWJ85gmEP8t+5ryQ0SYbGB+GDdEB+8O5N/CSwFdrl7TV8LzWyEmZX2TBM/oLi9r7bp0qt/88/7eb9UBj/PRG0rgP8K3OTuJ/tpM1jrLCsHgE/06f8Q2Onu3+6nzeSevn8zW0L853kwfumk8tmsAT6ROGvmcuB4UndEpvX7V3RQ6yxJ8rbUXy6tA5ab2dhEd+ryxLzTMxhHjc/kQTyQaoBOoB5Yl7TsHuJnOOwGbkyavxaYmpieSzz0q4EngOIM1vow8Lle86YCa5Nq2ZJ4VBHvmsj0+nsU2AZsTWxQU3rXlXi9kviZGHsGo67Ee1YT71PcnHg80Lu2wVxnfa0D4D7iv3wAShLbUHVim5o7COvoKuJdaluT1tNK4HM92xpwZ2LdbCF+YPrKQfr8+vxsetVmwP2JdbqNpDPeMlzsPHKoAAAAa0lEQVTbCOJhPTppXiDrjPgvmMNAJJFltxM/VvMC8Dbwa2Bcom0F8IOkr70tsb1VA58+k/fXFaoiInkoF7tlRERkAAp3EZE8pHAXEclDCncRkTykcBcRyUMKdxGRPKRwFxHJQwp3EZE89P8BXWtVVx87CyMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "\n",
    "def sigmoid(x):\n",
    "    a = []\n",
    "    for item in x:\n",
    "        a.append(1/(1+math.exp(-item)))\n",
    "    return a\n",
    "\n",
    "x = np.arange(-10., 10., 0.2)\n",
    "sig = sigmoid(x)\n",
    "plt.plot(x,sig)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since linear regression has the formula $$h_\\theta(X) = \\theta^TX$$ where $\\theta^TX = \\theta_0 + \\theta_1 X_1 + \\theta_2 X_2 + ... + \\theta_n x_n$, we just need to generalize it in order to be used as logistic regression:\n",
    "\n",
    "$$h_\\theta(X) = g(\\theta^TX)$$\n",
    "\n",
    "Yielding:\n",
    "\n",
    "$$h_\\theta(X) = \\frac{1}{1 + e^{-\\theta^TX}}$$\n",
    "\n",
    "So, how can this model be used on machine learning? It can be used to make predictions using a set of samples that will train a logistic regression classifier. When this classifier is trained, it's use is very straightforward. So, let's see an use case from 2010 brazilian census.\n",
    "\n",
    "## Hands on!\n",
    "\n",
    "In this case, we will train a classifier in order to predict if the income over or under the median based on following characteristics: quantity of rooms, quantity of bathrooms, presence of sewer, presence of tap water, kind of garbage collection, kind of electric network, presence of washing machine, presence of refrigerator, kind of habitation and adequation of habitation. These will be used to predict the family net income.\n",
    "\n",
    "First we load data from IBGE (Brazilian Institute of Geography and Statistics) site and remove undesired fields. The data set is in fixed width format and it's dictionary can be downloaded at ftp://ftp.ibge.gov.br/Censos/Censo_Demografico_2010/Resultados_Gerais_da_Amostra/Microdados/Documentacao.zip . For this case, we will only use data from São Paulo state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "79f83e0a6b5e42e0b3b20b06bab4c113",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HTML(value='', placeholder='')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import ftplib\n",
    "import zipfile\n",
    "from IPython.display import display\n",
    "import ipywidgets as widgets\n",
    "\n",
    "out = widgets.HTML(value='',\n",
    "    placeholder='',\n",
    "    description='')\n",
    "display(out)\n",
    "\n",
    "col_names = ['a1', 'a2', 'a3', 'a4', 'a5', 'a6', 'a7', 'a8', 'a9', 'a10', \n",
    "             'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'qtyrooms', 'a18', 'a19', 'a20', \n",
    "             'qtybath', 'a22', 'sewer', 'waterpipes', 'a25', 'trash', 'electricity', 'a28', 'a29', 'a30', \n",
    "             'washmachine', 'refrigerator', 'a33', 'a34', 'a35', 'a36', 'a37', 'car', 'a39', 'a40', \n",
    "             'a41', 'a42', 'familynetincome', 'a44', 'a45', 'a46', 'kindhabitation', 'a48', 'adequation', 'a50', \n",
    "             'a51', 'a52', 'a53', 'a54', 'a55', 'a56', 'a57', 'a58', 'a59', 'a60', \n",
    "             'a61', 'a62', 'a63', 'a64', 'a65', 'a66', 'a67', 'a68', 'a69', 'a70', \n",
    "             'a71', 'a72', 'a73', 'a74', 'a75', 'a76', 'a77', 'a78']\n",
    "\n",
    "col_widths = [2, 5, 13, 8, 16, 1, 2, 3, 2, 1, 2, 2, 1, 6, 9, 1, \n",
    "              2, 3, 2, 3, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, \n",
    "              1, 1, 1, 1, 1, 1, 2, 1, 1, 7, 10, 8, 9, 1, 1, 1, 1, \n",
    "              1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, \n",
    "              1, 1, 1, 1, 1, 1, 1, 1]\n",
    "\n",
    "\n",
    "#downloading file\n",
    "out.value = \"Downloading file...\"\n",
    "ftp = ftplib.FTP(\"ftp.ibge.gov.br\")\n",
    "ftp.login(\"\", \"\") \n",
    "ftp.cwd(\"/Censos/Censo_Demografico_2010/Resultados_Gerais_da_Amostra/Microdados\")\n",
    "ftp.retrbinary(\"RETR SP1.zip\", open(\"/tmp/SP1.zip\", 'wb').write)\n",
    "\n",
    "out.value = \"Extracting file...\"\n",
    "zip_ref = zipfile.ZipFile(\"/tmp/SP1.zip\", 'r')\n",
    "zip_ref.extractall(\"/tmp\")\n",
    "zip_ref.close()\n",
    "\n",
    "out.value = \"Done.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read into dataframe\n",
    "tmp = pd.read_fwf('/tmp/SP1/Amostra_Domicilios_35_outras.txt', widths=col_widths, names=col_names)\n",
    "df = tmp[['qtyrooms', 'qtybath', 'sewer', 'waterpipes', 'trash', 'electricity', \n",
    "          'washmachine', 'refrigerator', 'familynetincome', 'kindhabitation', 'adequation']].copy()\n",
    "\n",
    "del tmp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.dropna(inplace=True)\n",
    "df['qtyrooms'] = df['qtyrooms'].astype(int)\n",
    "df['qtybath'] = df['qtybath'].astype(int)\n",
    "df['sewer'] = df['sewer'].astype(int)\n",
    "df['waterpipes'] = df['waterpipes'].astype(int)\n",
    "df['trash'] = df['trash'].astype(int)\n",
    "df['electricity'] = df['electricity'].astype(int)\n",
    "df['washmachine'] = df['washmachine'].astype(int)\n",
    "df['refrigerator'] = df['refrigerator'].astype(int)\n",
    "df['familynetincome'] = df['familynetincome'].astype(int)\n",
    "df['kindhabitation'] = df['kindhabitation'].astype(int)\n",
    "df['adequation'] = df['adequation'].astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So now we can have a glance of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>qtyrooms</th>\n",
       "      <th>qtybath</th>\n",
       "      <th>sewer</th>\n",
       "      <th>waterpipes</th>\n",
       "      <th>trash</th>\n",
       "      <th>electricity</th>\n",
       "      <th>washmachine</th>\n",
       "      <th>refrigerator</th>\n",
       "      <th>familynetincome</th>\n",
       "      <th>kindhabitation</th>\n",
       "      <th>adequation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>510</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1900</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1020</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>68</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3758</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   qtyrooms  qtybath  sewer  waterpipes  trash  electricity  washmachine  \\\n",
       "0         5        1      1           1      1            1            2   \n",
       "1         5        1      1           1      1            1            1   \n",
       "2         5        1      1           1      1            1            2   \n",
       "3         4        1      1           1      1            1            2   \n",
       "4        10        3      1           1      1            1            1   \n",
       "\n",
       "   refrigerator  familynetincome  kindhabitation  adequation  \n",
       "0             1              510               1           2  \n",
       "1             1             1900               2           2  \n",
       "2             1             1020               2           2  \n",
       "3             1               68               3           2  \n",
       "4             1             3758               2           2  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what is the value of the median of family net income. For the sake of this study, we will set all values equal or above median to 1 and below median to 0:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1703.0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "median = df.familynetincome.median()\n",
    "median"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.loc[df['familynetincome'] < median, 'familynetincome'] = 0\n",
    "df.loc[df['familynetincome'] >= median, 'familynetincome'] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's separate all independent variables from the dependent, and let's train the classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn import linear_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df[['qtyrooms', 'qtybath', 'sewer', 'waterpipes', 'trash', 'electricity', \n",
    "          'washmachine', 'refrigerator', 'kindhabitation', 'adequation']]\n",
    "y = df['familynetincome']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = linear_model.LogisticRegression(C=1e40, solver='newton-cg')\n",
    "fitted_model = clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pretty straightforward, huh?\n",
    "\n",
    "So now we can use this trained classifier to predict if the income is above or below median:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prediction_result = clf.predict([(10, 3, 1, 1, 1, 1, 1, 1, 2, 2)])\n",
    "prediction_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prediction_result = clf.predict([(4, 1, 1, 1, 1, 1, 2, 1, 3, 2)])\n",
    "prediction_result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Do you remember that, for logistic regression, we need to get $\\theta$ in order to compute it on $h_\\theta(X) = g(\\theta^TX)$, right? So, if you want to read values for $\\theta$, you just need to do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-4.40468002]\n"
     ]
    }
   ],
   "source": [
    "# For theta_0:\n",
    "\n",
    "print(fitted_model.intercept_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.2286218   0.43581317 -0.22098449 -0.11330352 -0.05717134 -0.20821814\n",
      "  -1.09090793 -0.85579353  0.69167597  2.05417498]]\n"
     ]
    }
   ],
   "source": [
    "# For theta_1 to theta_10, in this case:\n",
    "\n",
    "print(fitted_model.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, if you want to directly compute it from above $\\theta$ values, we just need to first compute the dot product between $\\theta$ and $X$ arrays and then feed the result to $g(z)$ function. Let's do it for values shown above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.50828866])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = fitted_model.intercept_ + np.dot(fitted_model.coef_,[4, 1, 1, 1, 1, 1, 2, 1, 3, 2])\n",
    "res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.37559478970025234"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 / (1 + math.exp(-res))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the result is below $0.5$, the result goes to $0$.\n",
    "\n",
    "Let's make same calculations for another $X$ array above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2.13430044])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = fitted_model.intercept_ + np.dot(fitted_model.coef_,[10, 3, 1, 1, 1, 1, 1, 1, 2, 2])\n",
    "res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8941925729153888"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 / (1 + math.exp(-res))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the value is above $0.5$, result goes to $1$.\n",
    "\n",
    "## Conclusion:\n",
    "\n",
    "Logistic Regression is a very fast and robust way to estimate in which group a individual pertains, just looking for some features. Sklearn provides a way to do it quickly over a big set of individuals.\n",
    "\n",
    "Beyound this simple application on census data, logistic regression can be used to predict churns, failures on networks, telemarketing conversion rates, and so on. Possibilities are simply infinite."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}