numpy-and-pandas-answers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "e2d2e96d-e50e-4d2b-9ca2-b748fec717d5",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Agenda\n",
    "* Numpy\n",
    "* Pandas\n",
    "* Lab\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "b4120c3a-6f30-49ba-ab8f-a86e3aa3adbd",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Introduction\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "485a2c1b-c45a-40da-a088-d164ede434c3",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Create a new notebook for your code-along:\n",
    "\n",
    "From our submission directory, type:\n",
    "    \n",
    "    jupyter notebook\n",
    "\n",
    "From the IPython Dashboard, open a new notebook.\n",
    "Change the title to: \"Numpy and Pandas\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "2bd62dd0-8f25-4203-8faf-9ec99b96aef2",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Introduction to Numpy\n",
    "\n",
    "* Overview\n",
    "* ndarray\n",
    "* Indexing and Slicing\n",
    "\n",
    "More info: [http://wiki.scipy.org/Tentative_NumPy_Tutorial](http://wiki.scipy.org/Tentative_NumPy_Tutorial)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "7db20dcb-cef2-4ae9-b44a-7b04744f74cf",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Numpy Overview\n",
    "\n",
    "* Why Python for Data? Numpy brings *decades* of C math into Python!\n",
    "* Numpy provides a wrapper for extensive C/C++/Fortran codebases, used for data analysis functionality\n",
    "* NDAarray allows easy vectorized math and broadcasting (i.e. functions for vector elements of different shapes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "247250c7-2583-41fa-9198-6198b83fd243",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "7b1b97a1-ccb0-44a9-a0f8-12dcc590bc66",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Creating ndarrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "d6f83896-c0f2-4185-a528-481262fe2d91",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "An array object represents a multidimensional, homogeneous array of fixed-size items. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "54f9011d-2143-4bb7-81fc-e3216ac4c866",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Creating arrays\n",
    "a = np.zeros((3))\n",
    "b = np.ones((2,3))\n",
    "c = np.random.randint(1,10,(2,3,4))\n",
    "d = np.arange(0,11,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "668f00fe-00c6-4866-87a7-9293497f3a15",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "What are these functions?\n",
    "\n",
    "    arange?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "a84a1773-b477-421f-9f17-e26eb326b408",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 0.,  0.,  0.]), array([[ 1.,  1.,  1.],\n",
       "        [ 1.,  1.,  1.]]), array([[[4, 8, 5, 2],\n",
       "         [4, 2, 3, 4],\n",
       "         [2, 8, 7, 5]],\n",
       " \n",
       "        [[8, 9, 3, 9],\n",
       "         [7, 1, 6, 8],\n",
       "         [8, 5, 8, 8]]]), array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10]))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note the way each array is printed:\n",
    "a,b,c,d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "67c9acc2-c9a7-4636-acca-038bd99def6d",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "## Arithmetic in arrays is element wise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "34461526-ff37-4e7d-8ff9-2f6887c10a1a",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[20 30 40 50] [0 1 2 3]\n"
     ]
    }
   ],
   "source": [
    "a = np.array( [20,30,40,50] )\n",
    "b = np.arange( 4 )\n",
    "b\n",
    "print a, b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "9b21c965-2734-4f09-aae0-4d344b93e681",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([20, 29, 38, 47])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = a-b\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "03828206-43b2-445f-b54d-7ebaca5585f5",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 4, 9])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "e4c29549-9a0a-420c-9f86-99a798daa288",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Indexing, Slicing and Iterating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "100e8b85-16f4-43e0-9c47-67c60e1f81ed",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# one-dimensional arrays work like lists:\n",
    "a = np.arange(10)**2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "26090d07-f5d6-4baa-aacc-979982af6f14",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "8d56b369-acfc-4e82-b325-c1aba3ee35cd",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 4,  9, 16])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a[2:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "7dbe0ac8-d44b-4698-82c6-537afff33cea",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Multidimensional arrays use tuples with commas for indexing\n",
    "# with (row,column) conventions beginning, as always in Python, from 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "27ba2fb0-9c69-438f-9136-da9c2204b100",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "b = np.random.randint(1,100,(4,4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "845e0d23-74e7-4f8b-a6ca-cc7a8984b13b",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[99, 37, 51,  6],\n",
       "       [62, 41, 13,  1],\n",
       "       [14, 27, 87, 69],\n",
       "       [29, 48,  4, 58]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "58463e8a-312c-4ff9-970f-c8204d6dbc30",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "69\n",
      "99\n"
     ]
    }
   ],
   "source": [
    "# Guess the output\n",
    "print(b[2,3])\n",
    "print(b[0,0])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "77b7d441-34a9-4d52-aab7-563e966702c9",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([37, 41, 27]), array([37, 41, 27, 48]))"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[0:3,1],b[:,1]\n",
    "#Note that for range, the last value is n-1. But for a single number, last value is n starting from 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "9b24506c-d8e4-4f1b-8fdb-b9c835b980c9",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[62, 41, 13,  1],\n",
       "       [14, 27, 87, 69]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[1:3,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "39dee2d5-b969-40d1-9bcb-9061b29c317b",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Introduction to Pandas\n",
    "\n",
    "* Object Creation\n",
    "* Viewing data\n",
    "* Selection\n",
    "* Missing data\n",
    "* Grouping\n",
    "* Reshaping\n",
    "* Time series\n",
    "* Plotting\n",
    "* i/o\n",
    " \n",
    "\n",
    "_pandas.pydata.org_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "7306e5f4-bc93-4370-9156-7ad4bb0b3662",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Pandas Overview\n",
    "\n",
    "_Source: [pandas.pydata.org](http://pandas.pydata.org/pandas-docs/stable/10min.html)_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "25de189f-b377-4fac-896f-757f71485109",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "c56e480d-bc62-4502-89ae-1b3c5f8f9c65",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2014-01-01', '2014-01-02', '2014-01-03', '2014-01-04',\n",
       "               '2014-01-05', '2014-01-06'],\n",
       "              dtype='datetime64[ns]', freq='D')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates = pd.date_range('20140101',periods=6)\n",
    "dates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "c1eaf66c-b3b0-4286-b8cf-ac023341c523",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              A    B    C    D\n",
      "2014-01-01  NaN  NaN  NaN  NaN\n",
      "2014-01-02  NaN  NaN  NaN  NaN\n",
      "2014-01-03  NaN  NaN  NaN  NaN\n",
      "2014-01-04  NaN  NaN  NaN  NaN\n",
      "2014-01-05  NaN  NaN  NaN  NaN\n",
      "2014-01-06  NaN  NaN  NaN  NaN\n",
      "                   A         B         C         D\n",
      "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
      "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
      "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938\n",
      "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
      "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
      "2014-01-06  0.183026 -0.858343  1.541798  1.509804\n"
     ]
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.randn(6,4),index=dates,columns=list('ABCD'))\n",
    "z = pd.DataFrame(index = df.index, columns = df.columns)\n",
    "df.columns\n",
    "print z \n",
    "print df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "8e7170fc-f502-45b0-b82a-bdac82dabf7e",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2014-01-01 00:00:00</th>\n",
       "      <th>2014-01-02 00:00:00</th>\n",
       "      <th>2014-01-03 00:00:00</th>\n",
       "      <th>2014-01-04 00:00:00</th>\n",
       "      <th>2014-01-05 00:00:00</th>\n",
       "      <th>2014-01-06 00:00:00</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.706283</td>\n",
       "      <td>-1.082676</td>\n",
       "      <td>0.841728</td>\n",
       "      <td>1.290401</td>\n",
       "      <td>0.183026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>-0.221338</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>-0.858343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>C</th>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>1.541798</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>D</th>\n",
       "      <td>0.965058</td>\n",
       "      <td>-0.488835</td>\n",
       "      <td>-1.291938</td>\n",
       "      <td>0.712852</td>\n",
       "      <td>-1.540861</td>\n",
       "      <td>1.509804</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   2014-01-01  2014-01-02  2014-01-03  2014-01-04  2014-01-05  2014-01-06\n",
       "A    0.579888   -0.706283   -1.082676    0.841728    1.290401    0.183026\n",
       "B   -0.221338    0.652975   -1.939761   -0.780663   -0.617174   -0.858343\n",
       "C    1.006163    0.264237    1.036881    0.327462    0.494087    1.541798\n",
       "D    0.965058   -0.488835   -1.291938    0.712852   -1.540861    1.509804"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Index, columns, underlying numpy data\n",
    "df.T\n",
    "df.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "0c3d9fa3-7522-4e18-8be5-74661547954f",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     A          B    C  D    E\n",
      "0  1.0 2013-01-02  1.0  3  foo\n",
      "1  1.0 2013-01-02  1.0  3  foo\n",
      "2  1.0 2013-01-02  1.0  3  foo\n",
      "3  1.0 2013-01-02  1.0  3  foo\n",
      "     A          B  D    E\n",
      "0  1.0 2013-01-02  3  foo\n",
      "1  1.0 2013-01-02  3  foo\n",
      "2  1.0 2013-01-02  3  foo\n",
      "3  1.0 2013-01-02  3  foo\n"
     ]
    }
   ],
   "source": [
    "df2 = pd.DataFrame({ 'A' : 1.,\n",
    "                         'B' : pd.Timestamp('20130102'),\n",
    "                         'C' : pd.Series(1,index=list(range(4)),dtype='float32'),\n",
    "                         'D' : np.array([3] * 4,dtype='int32'),\n",
    "                         'E' : 'foo' })\n",
    "    \n",
    "\n",
    "df2\n",
    "df3 = pd.DataFrame({ 'A' : 1.,\n",
    "                         'B' : pd.Timestamp('20130102'),\n",
    "                         'D' : np.array([3] * 4,dtype='int32'),\n",
    "                         'E' : 'foo' })\n",
    "    \n",
    "\n",
    "print df2\n",
    "print df3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "4c3d6e0a-6a4f-48dd-87c8-71d94577f429",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A           float64\n",
       "B    datetime64[ns]\n",
       "C           float32\n",
       "D             int32\n",
       "E            object\n",
       "dtype: object"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# With specific dtypes\n",
    "df2.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "2792318e-6f4c-4ae8-a937-1964456f91f8",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Viewing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "dee84a01-9e14-4996-a334-502fd8234ad4",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>-1.540861</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
       "2014-01-05  1.290401 -0.617174  0.494087 -1.540861"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "791189f7-053a-44bd-b13c-2c6ba2cb4df5",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>-1.540861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
       "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
       "2014-01-06  0.183026 -0.858343  1.541798  1.509804"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "fc1c8091-670a-4f81-9d6f-158958c3b413",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2014-01-01', '2014-01-02', '2014-01-03', '2014-01-04',\n",
       "               '2014-01-05', '2014-01-06'],\n",
       "              dtype='datetime64[ns]', freq='D')"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "337688a9-1ab9-4909-80c8-159cec177b2e",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.184347</td>\n",
       "      <td>-0.627384</td>\n",
       "      <td>0.778438</td>\n",
       "      <td>-0.022320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.917663</td>\n",
       "      <td>0.848936</td>\n",
       "      <td>0.500001</td>\n",
       "      <td>1.264779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-1.540861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.483955</td>\n",
       "      <td>-0.838923</td>\n",
       "      <td>0.369118</td>\n",
       "      <td>-1.091162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.381457</td>\n",
       "      <td>-0.698919</td>\n",
       "      <td>0.750125</td>\n",
       "      <td>0.112009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.776268</td>\n",
       "      <td>-0.320297</td>\n",
       "      <td>1.029202</td>\n",
       "      <td>0.902007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              A         B         C         D\n",
       "count  6.000000  6.000000  6.000000  6.000000\n",
       "mean   0.184347 -0.627384  0.778438 -0.022320\n",
       "std    0.917663  0.848936  0.500001  1.264779\n",
       "min   -1.082676 -1.939761  0.264237 -1.540861\n",
       "25%   -0.483955 -0.838923  0.369118 -1.091162\n",
       "50%    0.381457 -0.698919  0.750125  0.112009\n",
       "75%    0.776268 -0.320297  1.029202  0.902007\n",
       "max    1.290401  0.652975  1.541798  1.509804"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "29c9b7a6-9d06-4f3b-b9d5-6ebc09cd561f",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                   A         B         C         D\n",
      "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
      "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
      "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938\n",
      "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
      "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
      "2014-01-06  0.183026 -0.858343  1.541798  1.509804\n",
      "                   A         B         C         D\n",
      "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938\n",
      "2014-01-06  0.183026 -0.858343  1.541798  1.509804\n",
      "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
      "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
      "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
      "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
      "                   A         B         C         D\n",
      "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938\n",
      "2014-01-06  0.183026 -0.858343  1.541798  1.509804\n",
      "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
      "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
      "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
      "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n"
     ]
    }
   ],
   "source": [
    "print df\n",
    "print df.sort_values(by='B')\n",
    "df2 = df.sort_values(by='B')\n",
    "print df2\n",
    "# Note the df.sort_values(by='B\") sorts by column B but creates a new dataframe. The original df is unaffected"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "1440616a-c3e7-4b5c-aec3-3705d76429cd",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "7825e53c-bd40-4027-9581-42d8f5aafa93",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B\n",
       "2014-01-01  0.579888 -0.221338\n",
       "2014-01-02 -0.706283  0.652975\n",
       "2014-01-03 -1.082676 -1.939761\n",
       "2014-01-04  0.841728 -0.780663\n",
       "2014-01-05  1.290401 -0.617174\n",
       "2014-01-06  0.183026 -0.858343"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['A','B']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "817868ae-d7df-425b-9368-e1b3c86a21d1",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[0:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "dcf6444e-ae7d-4058-abb0-5046d16733d2",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    0.579888\n",
       "B   -0.221338\n",
       "C    1.006163\n",
       "D    0.965058\n",
       "Name: 2014-01-01 00:00:00, dtype: float64"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# By label\n",
    "df.loc[dates[0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "da51e123-4546-4c14-bdcc-969751f1991e",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B\n",
       "2014-01-01  0.579888 -0.221338\n",
       "2014-01-02 -0.706283  0.652975\n",
       "2014-01-03 -1.082676 -1.939761\n",
       "2014-01-04  0.841728 -0.780663\n",
       "2014-01-05  1.290401 -0.617174\n",
       "2014-01-06  0.183026 -0.858343"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# multi-axis by label\n",
    "df.loc[:,['A','B']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "e3e52e42-a7f3-4298-80ae-d4554764b9fe",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>0.652975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.939761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>-0.780663</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   B\n",
       "2014-01-02  0.652975\n",
       "2014-01-03 -1.939761\n",
       "2014-01-04 -0.780663"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Date Range\n",
    "df.loc['20140102':'20140104',['B']]\n",
    "# Note here the second date is included -vc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "6ec7d13c-1a41-4358-bbd3-742f6253b497",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.65297482836608778"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fast access to scalar\n",
    "df.at[dates[1],'B']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "42f70279-ecea-4460-872c-95949b247caf",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>-1.540861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
       "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
       "2014-01-06  0.183026 -0.858343  1.541798  1.509804"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# iloc provides integer locations similar to np style\n",
    "df.iloc[3:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "16735d8f-3712-42a2-bd0e-8cd141fbe232",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Boolean Indexing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "a0af5955-c7e0-4d83-aea2-1972b21b8974",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df.A < 0] # Basically a 'where' operation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "c22c86db-8892-4337-b70f-5e849ee63daa",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Setting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "8d65e283-2ba8-43c1-ba16-e69da80b51b9",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "df_posA = df.copy() # Without \"copy\" it would act on the dataset\n",
    "\n",
    "df_posA[df_posA.A < 0] = -1*df_posA\n",
    "# This flips the signs of all elements of rows where values of A < 0. Other rows are not touched."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "ce51651b-1c4a-4a02-8e84-6a938c5c7a4c",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>0.706283</td>\n",
       "      <td>-0.652975</td>\n",
       "      <td>-0.264237</td>\n",
       "      <td>0.488835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>1.082676</td>\n",
       "      <td>1.939761</td>\n",
       "      <td>-1.036881</td>\n",
       "      <td>1.291938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>-1.540861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058\n",
       "2014-01-02  0.706283 -0.652975 -0.264237  0.488835\n",
       "2014-01-03  1.082676  1.939761 -1.036881  1.291938\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852\n",
       "2014-01-05  1.290401 -0.617174  0.494087 -1.540861\n",
       "2014-01-06  0.183026 -0.858343  1.541798  1.509804"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_posA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "f45e9be0-2c20-4d49-b167-911f076a00b2",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#Setting new column aligns data by index\n",
    "s1 = pd.Series([1,2,3,4,5,6],index=pd.date_range('20140102',periods=6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "12df6591-5940-4198-a7f9-9f7563afc75b",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2014-01-02    1\n",
       "2014-01-03    2\n",
       "2014-01-04    3\n",
       "2014-01-05    4\n",
       "2014-01-06    5\n",
       "2014-01-07    6\n",
       "Freq: D, dtype: int64"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "7281af7b-395f-417b-aa07-c53a26591fc7",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "df['F'] = s1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "d41dfe6d-025f-4078-a33b-3e7e0ea0d1dc",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>-1.540861</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D    F\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058  NaN\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835  1.0\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938  2.0\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852  3.0\n",
       "2014-01-05  1.290401 -0.617174  0.494087 -1.540861  4.0\n",
       "2014-01-06  0.183026 -0.858343  1.541798  1.509804  5.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "203fd0e7-9ad9-4dcc-8167-72bd28e7a029",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Missing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "98b7be6a-8d81-4c29-9439-2b5c5fae1dce",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "      <th>E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D    F   E\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058  NaN NaN\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835  1.0 NaN\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938  2.0 NaN\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852  3.0 NaN"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a column with missing data\n",
    "df1 = df.reindex(index=dates[0:4],columns=list(df.columns) + ['E'])\n",
    "df1\n",
    "# Dates go from location 0 to 3, note the use of list() on df.columns to add addl. col. name 'E'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "e13067da-23ce-440f-9e0d-693e4e5565c2",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "df1.loc[dates[0]:dates[1],'E'] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "24e2f95e-75dc-42ed-93fb-8fae3e9623a6",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "      <th>E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D    F    E\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058  NaN  1.0\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835  1.0  1.0\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938  2.0  NaN\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852  3.0  NaN"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1\n",
    "# Not clear why dates[1] as endpoint is not excluded in this statement ***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "3521940b-df04-432a-9cfc-18a1353d9652",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "      <th>E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                A      B      C      D      F      E\n",
       "2014-01-01  False  False  False  False   True  False\n",
       "2014-01-02  False  False  False  False  False  False\n",
       "2014-01-03  False  False  False  False  False   True\n",
       "2014-01-04  False  False  False  False  False   True"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find where values are null\n",
    "pd.isnull(df1)\n",
    "# Creates a new dataframe with True/False values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "74c34377-4b13-4562-9ef5-35ab50ee0a52",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "98905902-a6d4-4935-ac8e-f2cb95e123e9",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.184347</td>\n",
       "      <td>-0.627384</td>\n",
       "      <td>0.778438</td>\n",
       "      <td>-0.022320</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.917663</td>\n",
       "      <td>0.848936</td>\n",
       "      <td>0.500001</td>\n",
       "      <td>1.264779</td>\n",
       "      <td>1.581139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-1.540861</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.483955</td>\n",
       "      <td>-0.838923</td>\n",
       "      <td>0.369118</td>\n",
       "      <td>-1.091162</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.381457</td>\n",
       "      <td>-0.698919</td>\n",
       "      <td>0.750125</td>\n",
       "      <td>0.112009</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.776268</td>\n",
       "      <td>-0.320297</td>\n",
       "      <td>1.029202</td>\n",
       "      <td>0.902007</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              A         B         C         D         F\n",
       "count  6.000000  6.000000  6.000000  6.000000  5.000000\n",
       "mean   0.184347 -0.627384  0.778438 -0.022320  3.000000\n",
       "std    0.917663  0.848936  0.500001  1.264779  1.581139\n",
       "min   -1.082676 -1.939761  0.264237 -1.540861  1.000000\n",
       "25%   -0.483955 -0.838923  0.369118 -1.091162  2.000000\n",
       "50%    0.381457 -0.698919  0.750125  0.112009  3.000000\n",
       "75%    0.776268 -0.320297  1.029202  0.902007  4.000000\n",
       "max    1.290401  0.652975  1.541798  1.509804  5.000000"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "372219c5-5800-4502-83de-ac91bacb6cdb",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(A    0.184347\n",
       " B   -0.627384\n",
       " C    0.778438\n",
       " D   -0.022320\n",
       " F    3.000000\n",
       " dtype: float64, 2014-01-01    0.582443\n",
       " 2014-01-02    0.144419\n",
       " 2014-01-03   -0.255499\n",
       " 2014-01-04    0.820276\n",
       " 2014-01-05    0.725291\n",
       " 2014-01-06    1.475257\n",
       " Freq: D, dtype: float64)"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.mean(),df.mean(1) # Operation on two different axes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "aefd18b0-3fc6-4b15-bd6f-dd18136547a1",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Applying functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "1b4b693d-27fe-4eb4-8aa4-4b2a4c27a1a2",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.706283</td>\n",
       "      <td>0.652975</td>\n",
       "      <td>0.264237</td>\n",
       "      <td>-0.488835</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.082676</td>\n",
       "      <td>-1.939761</td>\n",
       "      <td>1.036881</td>\n",
       "      <td>-1.291938</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>0.841728</td>\n",
       "      <td>-0.780663</td>\n",
       "      <td>0.327462</td>\n",
       "      <td>0.712852</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>1.290401</td>\n",
       "      <td>-0.617174</td>\n",
       "      <td>0.494087</td>\n",
       "      <td>-1.540861</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>0.183026</td>\n",
       "      <td>-0.858343</td>\n",
       "      <td>1.541798</td>\n",
       "      <td>1.509804</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D    F\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058  NaN\n",
       "2014-01-02 -0.706283  0.652975  0.264237 -0.488835  1.0\n",
       "2014-01-03 -1.082676 -1.939761  1.036881 -1.291938  2.0\n",
       "2014-01-04  0.841728 -0.780663  0.327462  0.712852  3.0\n",
       "2014-01-05  1.290401 -0.617174  0.494087 -1.540861  4.0\n",
       "2014-01-06  0.183026 -0.858343  1.541798  1.509804  5.0"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "acb5d0e0-b6f8-4dc3-8f42-e2ca34a6aa67",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2014-01-01</th>\n",
       "      <td>0.579888</td>\n",
       "      <td>-0.221338</td>\n",
       "      <td>1.006163</td>\n",
       "      <td>0.965058</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-02</th>\n",
       "      <td>-0.126394</td>\n",
       "      <td>0.431637</td>\n",
       "      <td>1.270400</td>\n",
       "      <td>0.476222</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-03</th>\n",
       "      <td>-1.209070</td>\n",
       "      <td>-1.508124</td>\n",
       "      <td>2.307282</td>\n",
       "      <td>-0.815715</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-04</th>\n",
       "      <td>-0.367342</td>\n",
       "      <td>-2.288787</td>\n",
       "      <td>2.634744</td>\n",
       "      <td>-0.102863</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-05</th>\n",
       "      <td>0.923058</td>\n",
       "      <td>-2.905961</td>\n",
       "      <td>3.128831</td>\n",
       "      <td>-1.643724</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014-01-06</th>\n",
       "      <td>1.106084</td>\n",
       "      <td>-3.764304</td>\n",
       "      <td>4.670629</td>\n",
       "      <td>-0.133921</td>\n",
       "      <td>15.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C         D     F\n",
       "2014-01-01  0.579888 -0.221338  1.006163  0.965058   NaN\n",
       "2014-01-02 -0.126394  0.431637  1.270400  0.476222   1.0\n",
       "2014-01-03 -1.209070 -1.508124  2.307282 -0.815715   3.0\n",
       "2014-01-04 -0.367342 -2.288787  2.634744 -0.102863   6.0\n",
       "2014-01-05  0.923058 -2.905961  3.128831 -1.643724  10.0\n",
       "2014-01-06  1.106084 -3.764304  4.670629 -0.133921  15.0"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(np.cumsum)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "76608f05-bac8-4938-b1c5-efa228fcbb7b",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    2.373077\n",
       "B    2.592735\n",
       "C    1.277561\n",
       "D    3.050665\n",
       "F    4.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(lambda x: x.max() - x.min())\n",
    "# Above does not alter df but creates a new series with the difference of max-min by COLUMN!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "802ec78f-0525-40e0-ab4e-4600eb22b1ed",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0       A\n",
      "1       B\n",
      "2       C\n",
      "3    Aaba\n",
      "4    Baca\n",
      "5     NaN\n",
      "6    CABA\n",
      "7     dog\n",
      "8     cat\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0       a\n",
       "1       b\n",
       "2       c\n",
       "3    aaba\n",
       "4    baca\n",
       "5     NaN\n",
       "6    caba\n",
       "7     dog\n",
       "8     cat\n",
       "dtype: object"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Built in string methods\n",
    "s = pd.Series(['A', 'B', 'C', 'Aaba', 'Baca', np.nan, 'CABA', 'dog', 'cat'])\n",
    "print s\n",
    "s.str.lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "ce36d58b-a2b3-4fb0-b21d-d8e40425d203",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Merge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "5a03c068-8ce7-4bc2-a25a-ed1744d56765",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.39462207, -0.35701611, -1.29920775,  0.59460181],\n",
       "       [ 1.05647765,  0.83904796,  1.01490374,  0.39531841],\n",
       "       [ 0.81759941, -0.63986786, -1.4096965 , -0.67753529],\n",
       "       [ 0.64417677,  0.42545764,  0.42673915,  1.26885913],\n",
       "       [ 0.01719219, -0.76216963,  1.00313238,  1.47111861],\n",
       "       [ 0.34304415,  1.02793888, -0.42312357, -1.01902576],\n",
       "       [ 0.31148253,  1.66334743, -0.09581272,  0.29241327],\n",
       "       [ 0.08148912,  1.06697393,  0.43033573,  1.14205797],\n",
       "       [ 1.54486785,  0.36092142, -1.92131053,  0.53001535],\n",
       "       [ 1.65267611,  0.84374708, -0.91686157,  0.2799484 ]])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.randn(10,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "54d4be63-ab84-4c0c-8c87-a4bf1affb1ec",
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.664186</td>\n",
       "      <td>0.451233</td>\n",
       "      <td>-0.333167</td>\n",
       "      <td>-0.132597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.606885</td>\n",
       "      <td>0.247958</td>\n",
       "      <td>1.047617</td>\n",
       "      <td>0.488071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-2.964315</td>\n",
       "      <td>1.046227</td>\n",
       "      <td>0.352172</td>\n",
       "      <td>-1.485879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.799703</td>\n",
       "      <td>-0.518114</td>\n",
       "      <td>-1.059849</td>\n",
       "      <td>0.591196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.090540</td>\n",
       "      <td>-0.776548</td>\n",
       "      <td>1.727447</td>\n",
       "      <td>0.097877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.610227</td>\n",
       "      <td>0.691842</td>\n",
       "      <td>-0.102435</td>\n",
       "      <td>1.364433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.293393</td>\n",
       "      <td>-0.737383</td>\n",
       "      <td>0.706023</td>\n",
       "      <td>1.400901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.096599</td>\n",
       "      <td>-0.611628</td>\n",
       "      <td>0.341076</td>\n",
       "      <td>1.634748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.639985</td>\n",
       "      <td>-2.100968</td>\n",
       "      <td>0.870013</td>\n",
       "      <td>0.823543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>-0.184826</td>\n",
       "      <td>1.812616</td>\n",
       "      <td>1.129015</td>\n",
       "      <td>-1.628226</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0         1         2         3\n",
       "0  1.664186  0.451233 -0.333167 -0.132597\n",
       "1  0.606885  0.247958  1.047617  0.488071\n",
       "2 -2.964315  1.046227  0.352172 -1.485879\n",
       "3  0.799703 -0.518114 -1.059849  0.591196\n",
       "4  1.090540 -0.776548  1.727447  0.097877\n",
       "5 -0.610227  0.691842 -0.102435  1.364433\n",
       "6  1.293393 -0.737383  0.706023  1.400901\n",
       "7 -0.096599 -0.611628  0.341076  1.634748\n",
       "8  0.639985 -2.100968  0.870013  0.823543\n",
       "9 -0.184826  1.812616  1.129015 -1.628226"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Concatenating pandas objects together\n",
    "df = pd.DataFrame(np.random.randn(10,4))\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "f39f64e5-7416-4067-867d-93d36674c19b",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[          0         1         2         3\n",
       " 0  1.664186  0.451233 -0.333167 -0.132597\n",
       " 1  0.606885  0.247958  1.047617  0.488071\n",
       " 2 -2.964315  1.046227  0.352172 -1.485879,\n",
       "           0         1         2         3\n",
       " 3  0.799703 -0.518114 -1.059849  0.591196\n",
       " 4  1.090540 -0.776548  1.727447  0.097877\n",
       " 5 -0.610227  0.691842 -0.102435  1.364433\n",
       " 6  1.293393 -0.737383  0.706023  1.400901,\n",
       "           0         1         2         3\n",
       " 7 -0.096599 -0.611628  0.341076  1.634748\n",
       " 8  0.639985 -2.100968  0.870013  0.823543\n",
       " 9 -0.184826  1.812616  1.129015 -1.628226]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Break it into pieces\n",
    "pieces = [df[:3], df[3:7],df[7:]]\n",
    "pieces\n",
    "# pieces is a list of dataframes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "1b3617e6-0a0e-4f0a-b8a1-8715c961aa8f",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.664186</td>\n",
       "      <td>0.451233</td>\n",
       "      <td>-0.333167</td>\n",
       "      <td>-0.132597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.606885</td>\n",
       "      <td>0.247958</td>\n",
       "      <td>1.047617</td>\n",
       "      <td>0.488071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-2.964315</td>\n",
       "      <td>1.046227</td>\n",
       "      <td>0.352172</td>\n",
       "      <td>-1.485879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.799703</td>\n",
       "      <td>-0.518114</td>\n",
       "      <td>-1.059849</td>\n",
       "      <td>0.591196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.090540</td>\n",
       "      <td>-0.776548</td>\n",
       "      <td>1.727447</td>\n",
       "      <td>0.097877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.610227</td>\n",
       "      <td>0.691842</td>\n",
       "      <td>-0.102435</td>\n",
       "      <td>1.364433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.293393</td>\n",
       "      <td>-0.737383</td>\n",
       "      <td>0.706023</td>\n",
       "      <td>1.400901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.096599</td>\n",
       "      <td>-0.611628</td>\n",
       "      <td>0.341076</td>\n",
       "      <td>1.634748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.639985</td>\n",
       "      <td>-2.100968</td>\n",
       "      <td>0.870013</td>\n",
       "      <td>0.823543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>-0.184826</td>\n",
       "      <td>1.812616</td>\n",
       "      <td>1.129015</td>\n",
       "      <td>-1.628226</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0         1         2         3\n",
       "0  1.664186  0.451233 -0.333167 -0.132597\n",
       "1  0.606885  0.247958  1.047617  0.488071\n",
       "2 -2.964315  1.046227  0.352172 -1.485879\n",
       "3  0.799703 -0.518114 -1.059849  0.591196\n",
       "4  1.090540 -0.776548  1.727447  0.097877\n",
       "5 -0.610227  0.691842 -0.102435  1.364433\n",
       "6  1.293393 -0.737383  0.706023  1.400901\n",
       "7 -0.096599 -0.611628  0.341076  1.634748\n",
       "8  0.639985 -2.100968  0.870013  0.823543\n",
       "9 -0.184826  1.812616  1.129015 -1.628226"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat(pieces)\n",
    "# pd.concat() concatenates by row, i.e. add verically\n",
    "# The argument is a list of dfs here. Could be a series object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "487ed85b-a5c7-436b-80ce-53b7c5e23948",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.664186</td>\n",
       "      <td>0.451233</td>\n",
       "      <td>-0.333167</td>\n",
       "      <td>-0.132597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.606885</td>\n",
       "      <td>0.247958</td>\n",
       "      <td>1.047617</td>\n",
       "      <td>0.488071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-2.964315</td>\n",
       "      <td>1.046227</td>\n",
       "      <td>0.352172</td>\n",
       "      <td>-1.485879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.799703</td>\n",
       "      <td>-0.518114</td>\n",
       "      <td>-1.059849</td>\n",
       "      <td>0.591196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.090540</td>\n",
       "      <td>-0.776548</td>\n",
       "      <td>1.727447</td>\n",
       "      <td>0.097877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.610227</td>\n",
       "      <td>0.691842</td>\n",
       "      <td>-0.102435</td>\n",
       "      <td>1.364433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.293393</td>\n",
       "      <td>-0.737383</td>\n",
       "      <td>0.706023</td>\n",
       "      <td>1.400901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.096599</td>\n",
       "      <td>-0.611628</td>\n",
       "      <td>0.341076</td>\n",
       "      <td>1.634748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.639985</td>\n",
       "      <td>-2.100968</td>\n",
       "      <td>0.870013</td>\n",
       "      <td>0.823543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>-0.184826</td>\n",
       "      <td>1.812616</td>\n",
       "      <td>1.129015</td>\n",
       "      <td>-1.628226</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0         1         2         3\n",
       "0  1.664186  0.451233 -0.333167 -0.132597\n",
       "1  0.606885  0.247958  1.047617  0.488071\n",
       "2 -2.964315  1.046227  0.352172 -1.485879\n",
       "3  0.799703 -0.518114 -1.059849  0.591196\n",
       "4  1.090540 -0.776548  1.727447  0.097877\n",
       "5 -0.610227  0.691842 -0.102435  1.364433\n",
       "6  1.293393 -0.737383  0.706023  1.400901\n",
       "7 -0.096599 -0.611628  0.341076  1.634748\n",
       "8  0.639985 -2.100968  0.870013  0.823543\n",
       "9 -0.184826  1.812616  1.129015 -1.628226"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Also can \"Join\" and \"Append\"\n",
    "df\n",
    "# Join and Append add a second df (or series) to one being considered"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "d71e2439-0018-4564-a3c9-08b21df14f49",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Grouping\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "fa9636d6-592b-4166-b440-ab7fccf1a617",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'A' : ['foo', 'bar', 'foo', 'bar',\n",
    "                       'foo', 'bar', 'foo', 'foo'],\n",
    "                       'B' : ['one', 'one', 'two', 'three',\n",
    "                             'two', 'two', 'one', 'three'],\n",
    "                       'C' : np.random.randn(8),\n",
    "                       'D' : np.random.randn(8)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "be026ca6-3b88-4e6c-935f-339089743711",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>foo</td>\n",
       "      <td>one</td>\n",
       "      <td>-0.156692</td>\n",
       "      <td>-1.283333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>bar</td>\n",
       "      <td>one</td>\n",
       "      <td>0.060946</td>\n",
       "      <td>-0.864953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>foo</td>\n",
       "      <td>two</td>\n",
       "      <td>-1.820843</td>\n",
       "      <td>-0.508642</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>bar</td>\n",
       "      <td>three</td>\n",
       "      <td>-0.689419</td>\n",
       "      <td>0.803342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>foo</td>\n",
       "      <td>two</td>\n",
       "      <td>0.455165</td>\n",
       "      <td>-1.287950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>bar</td>\n",
       "      <td>two</td>\n",
       "      <td>1.428970</td>\n",
       "      <td>-0.926617</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>foo</td>\n",
       "      <td>one</td>\n",
       "      <td>-0.521938</td>\n",
       "      <td>0.180666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>foo</td>\n",
       "      <td>three</td>\n",
       "      <td>2.031093</td>\n",
       "      <td>-0.795033</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A      B         C         D\n",
       "0  foo    one -0.156692 -1.283333\n",
       "1  bar    one  0.060946 -0.864953\n",
       "2  foo    two -1.820843 -0.508642\n",
       "3  bar  three -0.689419  0.803342\n",
       "4  foo    two  0.455165 -1.287950\n",
       "5  bar    two  1.428970 -0.926617\n",
       "6  foo    one -0.521938  0.180666\n",
       "7  foo  three  2.031093 -0.795033"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "20574670-3a23-4a2c-8b67-fbf3a9a7737e",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">bar</th>\n",
       "      <th>one</th>\n",
       "      <td>0.060946</td>\n",
       "      <td>-0.864953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>three</th>\n",
       "      <td>-0.689419</td>\n",
       "      <td>0.803342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>1.428970</td>\n",
       "      <td>-0.926617</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">foo</th>\n",
       "      <th>one</th>\n",
       "      <td>-0.678630</td>\n",
       "      <td>-1.102667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>three</th>\n",
       "      <td>2.031093</td>\n",
       "      <td>-0.795033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>-1.365678</td>\n",
       "      <td>-1.796592</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  C         D\n",
       "A   B                        \n",
       "bar one    0.060946 -0.864953\n",
       "    three -0.689419  0.803342\n",
       "    two    1.428970 -0.926617\n",
       "foo one   -0.678630 -1.102667\n",
       "    three  2.031093 -0.795033\n",
       "    two   -1.365678 -1.796592"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['A','B']).sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "fa94e6a7-1981-4d98-b43e-3090182e5836",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Reshaping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "df08ec54-d000-4c15-8f93-100cae7a72da",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# You can also stack or unstack levels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "ab096955-7280-41aa-9ca8-e1d714d6f12c",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "a = df.groupby(['A','B']).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "19be1110-9007-4818-b7df-7bfe76201f28",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"3\" halign=\"left\">C</th>\n",
       "      <th colspan=\"3\" halign=\"left\">D</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>bar</th>\n",
       "      <td>0.060946</td>\n",
       "      <td>-0.689419</td>\n",
       "      <td>1.428970</td>\n",
       "      <td>-0.864953</td>\n",
       "      <td>0.803342</td>\n",
       "      <td>-0.926617</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>foo</th>\n",
       "      <td>-0.339315</td>\n",
       "      <td>2.031093</td>\n",
       "      <td>-0.682839</td>\n",
       "      <td>-0.551334</td>\n",
       "      <td>-0.795033</td>\n",
       "      <td>-0.898296</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            C                             D                    \n",
       "B         one     three       two       one     three       two\n",
       "A                                                              \n",
       "bar  0.060946 -0.689419  1.428970 -0.864953  0.803342 -0.926617\n",
       "foo -0.339315  2.031093 -0.682839 -0.551334 -0.795033 -0.898296"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pivot Tables\n",
    "pd.pivot_table(df,values=['C','D'],index=['A'],columns=['B'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "67f5eb61-baaf-4cae-973d-808a2bf89829",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Time Series\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "09b41ef7-1b6f-4231-a3c3-b083ee337aa4"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "af09a1c8-fd55-44ad-a340-93284b578f52",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# 100 Seconds starting on January 1st\n",
    "rng = pd.date_range('1/1/2014', periods=100, freq='S')\n",
    "#rng"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "948b5dba-e17c-4e0a-9bcb-e5cfd5dbd151",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Give each second a random value\n",
    "ts = pd.Series(np.random.randint(0, 500, len(rng)), index=rng)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "ee2c76e8-2e6b-4083-97a2-ee65d8a4adde",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2014-01-01 00:00:00    445\n",
       "2014-01-01 00:00:01    371\n",
       "2014-01-01 00:00:02    382\n",
       "2014-01-01 00:00:03    432\n",
       "2014-01-01 00:00:04    339\n",
       "2014-01-01 00:00:05    495\n",
       "2014-01-01 00:00:06    238\n",
       "2014-01-01 00:00:07    300\n",
       "2014-01-01 00:00:08    336\n",
       "2014-01-01 00:00:09    194\n",
       "2014-01-01 00:00:10    189\n",
       "2014-01-01 00:00:11    155\n",
       "2014-01-01 00:00:12    255\n",
       "2014-01-01 00:00:13    209\n",
       "2014-01-01 00:00:14    337\n",
       "2014-01-01 00:00:15     29\n",
       "2014-01-01 00:00:16    461\n",
       "2014-01-01 00:00:17    197\n",
       "2014-01-01 00:00:18    480\n",
       "2014-01-01 00:00:19    206\n",
       "2014-01-01 00:00:20    485\n",
       "2014-01-01 00:00:21    417\n",
       "2014-01-01 00:00:22    154\n",
       "2014-01-01 00:00:23    258\n",
       "2014-01-01 00:00:24    237\n",
       "2014-01-01 00:00:25    157\n",
       "2014-01-01 00:00:26     61\n",
       "2014-01-01 00:00:27    386\n",
       "2014-01-01 00:00:28    256\n",
       "2014-01-01 00:00:29    433\n",
       "                      ... \n",
       "2014-01-01 00:01:10    116\n",
       "2014-01-01 00:01:11    114\n",
       "2014-01-01 00:01:12    188\n",
       "2014-01-01 00:01:13    422\n",
       "2014-01-01 00:01:14    265\n",
       "2014-01-01 00:01:15    287\n",
       "2014-01-01 00:01:16    360\n",
       "2014-01-01 00:01:17    310\n",
       "2014-01-01 00:01:18      4\n",
       "2014-01-01 00:01:19    445\n",
       "2014-01-01 00:01:20    387\n",
       "2014-01-01 00:01:21    365\n",
       "2014-01-01 00:01:22     84\n",
       "2014-01-01 00:01:23    274\n",
       "2014-01-01 00:01:24    250\n",
       "2014-01-01 00:01:25     48\n",
       "2014-01-01 00:01:26    187\n",
       "2014-01-01 00:01:27    235\n",
       "2014-01-01 00:01:28    341\n",
       "2014-01-01 00:01:29    493\n",
       "2014-01-01 00:01:30     18\n",
       "2014-01-01 00:01:31    423\n",
       "2014-01-01 00:01:32    413\n",
       "2014-01-01 00:01:33      7\n",
       "2014-01-01 00:01:34    342\n",
       "2014-01-01 00:01:35    391\n",
       "2014-01-01 00:01:36    341\n",
       "2014-01-01 00:01:37    405\n",
       "2014-01-01 00:01:38    177\n",
       "2014-01-01 00:01:39    389\n",
       "Freq: S, Length: 100, dtype: int32"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "a5d4c14a-379a-4e9a-a3cc-e6cd5476b781",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2014-01-01 00:00:00    248.25\n",
       "2014-01-01 00:01:00    245.70\n",
       "Freq: T, dtype: float64"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Built in resampling\n",
    "ts.resample('1Min').mean() # Resample secondly to 1Minutely"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "e3a2aac3-1aec-41fb-8076-ca3433d38c87",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# Many additional time series features\n",
    "#ts "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "b281a484-9cf5-41da-b898-ac5ea012053a",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Plotting\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "e0d64120-b593-447e-9088-731b31629e25",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'ts' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-2-132f3667ee95>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mts\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'ts' is not defined"
     ]
    }
   ],
   "source": [
    "ts.plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "05fb3a7f-f96d-4d2c-8944-bf02768b7eb2",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def randwalk(startdate,points):\n",
    "    ts = pd.Series(np.random.randn(points), index=pd.date_range(startdate, periods=points))\n",
    "    ts=ts.cumsum()\n",
    "    ts.plot()\n",
    "    return(ts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "0ad89701-d9a8-41d2-b93b-f0e28a76498c",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Using pandas to make a simple random walker by repeatedly running:\n",
    "a=randwalk('1/1/2012',1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true,
    "focus": false,
    "id": "d7c096a1-789c-494e-abf1-c253ad93729c",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Pandas plot function will print with labels as default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": false,
    "focus": false,
    "id": "f2951491-24b2-4552-bd06-304e1b12704d",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0xb4f7278>"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.randn(100, 4), index=ts.index,columns=['A', 'B', 'C', 'D'])\n",
    "df = df.cumsum()\n",
    "plt.figure();df.plot();plt.legend(loc='best') #"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "2f982866-332b-4e39-a784-092157257f8f",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### I/O\n",
    "I/O is straightforward with, for example, pd.read_csv or df.to_csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": false,
    "id": "a9c47dc7-ff2c-4d5f-9f77-0015da6e02aa",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### The benefits of open source:\n",
    "\n",
    "Let's look under x's in plt modules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "focus": true,
    "id": "a51cdd68-c291-46ae-9bf5-ed4f0819f161",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Next Steps\n",
    "\n",
    "**Recommended Resources**\n",
    "\n",
    "Name | Description\n",
    "--- | ---\n",
    "[Official Pandas Tutorials](http://pandas.pydata.org/pandas-docs/stable/10min.html) | Wes & Company's selection of tutorials and lectures\n",
    "[Julia Evans Pandas Cookbook](https://github.com/jvns/pandas-cookbook) | Great resource with examples from weather, bikes and 311 calls\n",
    "[Learn Pandas Tutorials](https://bitbucket.org/hrojas/learn-pandas) | A great series of Pandas tutorials from Dave Rojas\n",
    "[Research Computing Python Data PYNBs](https://github.com/ResearchComputing/Meetup-Fall-2013/tree/master/python) | A super awesome set of python notebooks from a meetup-based course exclusively devoted to pandas"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}