0026 Bayes Theorem and Conditional Probability.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:d443ffa594c6954f13f43452a7b28b7a29cf2a827e1e34e8088e0981467c7291"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline\n",
      "import sys\n",
      "from __future__ import print_function\n",
      "sys.version"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 1,
       "text": [
        "'3.4.1 |Anaconda 2.1.0 (64-bit)| (default, Sep 24 2014, 18:32:42) [MSC v.1600 64 bit (AMD64)]'"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Bayes' Theorem\n",
      "It is said that the Reverend Thomas Bayes developed his rule on inverse probability while he was trying to prove the existence of God somewhere around 1740. He came up with a method for calculating the probability of an event occurring given that another event has occurred. Starting out with the prior probability (or believe) $P(A)$, when given a likelihood) $P(B\\ |\\ A)$ and evidence $P(B)$ we arrive at the posterior probability $P(A\\ |\\ B)$. Bayes Rule proves to be a powerful tool and is widely used in diverging areas like economics, artificial intelligence, medicine, journalism, military, just to name a few. Most spam filters use Bayes Rule in one way or another. The Bayes' Theorem formula is, posterior = likelihood times prior, over evidence:\n",
      "\n",
      "$$\n",
      "P(A\\ |\\ B)=\\frac{P(B\\ |\\ A)\\cdot P(A)}{P(B)}\n",
      "$$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The practical power of Bayes Rule is that we often can't find the posterior directly, yet we do know the likelihood of the test and $P(A)$.\n",
      "\n",
      "An example; What is the chance of someone having COPD (a life-threatening lung disease) given he or she is a smoker - $P(A|B)$. This statistic is hard to figure out, but we do know from medical studies the probability of someone being a smoker given that he/she has COPD - $P(B|A)$. We also know $P(B)$ - the probability that a person is a smoker and $P(A)$ - the chance that someone has COPD. The figures below are rough estimations: \n",
      "\n",
      "$$\n",
      "P(A)=0.07\\ \\small{having\\ COPD}\\\\\n",
      "P(B)=0.18\\ \\small{smokers}\\\\\n",
      "P(B\\ |\\ A)=0.85\\ \\small{is\\ or\\ was\\ smoker\\ and\\ given\\ COPD\\ diagnosis}\n",
      "$$\n",
      "\n",
      "What is the probability of someone having COPD given the person is or was a smoker?:\n",
      "\n",
      "$$\n",
      "P(A\\ |\\ B)=\\frac{0.85\\cdot 0.07}{0.18}=0.33\n",
      "$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Calculate posterior probability\n",
      "likelihood = 0.85\n",
      "prior = 0.07\n",
      "evidence = 0.18\n",
      "\n",
      "posterior = (likelihood * prior) / evidence\n",
      "posterior"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "0.3305555555555556"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Test Accuracy Example\n",
      "Where Bayes' Rule shines is when we have evidence (for instance a test result) which is not 100% accurate. Let's say we have a test for detecting cancer. The probability of this test returning a positive result given the person has cancer is 99% (called the [sensitivity](http://en.wikipedia.org/wiki/Sensitivity_and_specificity)). The specificity is 93% and denotes the probability of having a negative test result given the person has no cancer. The cancer prevalence is 1% (proportion of people having cancer of the total population).\n",
      "\n",
      "$$\n",
      "P(C)=0.10\\ \\small{prevalence}\\\\\n",
      "P(Pos\\ |\\ C)=0.99\\ \\small{sensitivity}\\\\\n",
      "P(Neg\\ |\\ C')=0.93\\ \\small{specitivity}\\\\\n",
      "$$\n",
      "\n",
      "From these, we can compute the complements:\n",
      "\n",
      "$$\n",
      "P(C')=1-0.10=0.90\\\\\n",
      "P(Neg\\ |\\ C)=1-0.99=0.01\\\\\n",
      "P(Pos\\ |\\ C')=1-0.93=0.07\\\\\n",
      "$$\n",
      "\n",
      "Now a random person is picked, took the test, and the test is positive. What is the probability that this person has the disease or not given the test was positive?\n",
      "\n",
      "$$\n",
      "P(C\\cap Pos)=P(Pos\\ |\\ C)\\cdot P(C)=0.99\\cdot 0.10=0.099\\\\\n",
      "P(C'\\cap Pos)=P(Pos\\ |\\ C')\\cdot P(C')=0.07\\cdot 0.90=0.063\\\\\n",
      "P(Pos)=0.099+0.063=0.162\\\\\n",
      "$$\n",
      "\n",
      "So our answers are:\n",
      "\n",
      "$$\n",
      "P(C\\ |\\ Pos)=\\frac{0.099}{0.162}=0.611\\\\\n",
      "P(C'\\ |\\ Pos)=\\frac{0.063}{0.162}=0.389\\\\\n",
      "$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Calculate the probability having cancer or not \n",
      "# given a positive test result\n",
      "prevalence = 0.10\n",
      "sensitivity = 0.99\n",
      "specitivity = 0.93\n",
      "\n",
      "c_pos = sensitivity * prevalence\n",
      "no_c_pos = (1-specitivity) * (1-prevalence)\n",
      "normalizer = c_pos + no_c_pos\n",
      "\n",
      "print('Probabilities given a positive test result:')\n",
      "print('Cancer:', c_pos / normalizer)\n",
      "print('No cancer', no_c_pos / normalizer)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Probabilities given a positive test result:\n",
        "Cancer: 0.6111111111111113\n",
        "No cancer 0.3888888888888887\n"
       ]
      }
     ],
     "prompt_number": 3
    }
   ],
   "metadata": {}
  }
 ]
}