02-index-slice-subset_clean.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Indexing, Slicing and Subsetting DataFrames in Python\n",
    "\n",
    "## Loading our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Selecting data using Labels (Column Headings)\n",
    "We use square brackets `[]` to select a subset of an Python object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can pass a list of column names too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Extracting Range based Subsets: Slicing\n",
    "\n",
    "**REMINDER**: Python Uses 0-based Indexing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"fig/slicing-indexing.svg\">\n",
    "<img src=\"fig/slicing-slicing.svg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ## Challenge - Extracting data  \n",
    ">    ```python\n",
    ">    a = [1, 2, 3, 4, 5]\n",
    ">    ```\n",
    ">\n",
    ">\n",
    "> 1. What value does the code below return?\n",
    ">\n",
    ">    ```python\n",
    ">    a[0]\n",
    ">    ```\n",
    ">\n",
    "> 2. How about this:\n",
    ">\n",
    ">    ```python\n",
    ">    a[5]\n",
    ">    ```\n",
    ">\n",
    "> 3. In the example above, calling `a[5]` returns an error. Why is that?\n",
    ">\n",
    "> 4. What about?\n",
    ">\n",
    ">    ```python\n",
    ">    a[len(a)]\n",
    ">    ```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slicing Subsets of Rows in Python\n",
    "\n",
    "Slicing using the `[]` operator selects a set of rows and/or columns from a\n",
    "DataFrame. To slice out a set of rows, you use the following syntax:\n",
    "`data[start:stop]`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Copying Objects vs Referencing Objects in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `=`\n",
    "operator in the simple statement `y = x` does **not** create a copy of our\n",
    "DataFrame. Instead, `y = x` creates a new variable `y` that references the\n",
    "**same** object that `x` refers to. In contrast, the `copy()` method for a DataFrame creates a true copy of the\n",
    "DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slicing Subsets of Rows and Columns in Python\n",
    "We can select specific ranges of our data in both the row and column directions\n",
    "using either label or integer-based indexing.\n",
    "\n",
    "- `loc` is primarily *label* based indexing. *Integers* may be used but\n",
    "  they are interpreted as a *label*.\n",
    "- `iloc` is primarily *integer* based indexing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ## Challenge - Range\n",
    ">\n",
    "> 1. What happens when you execute:\n",
    ">\n",
    ">    - `surveys_df[0:1]`\n",
    ">    - `surveys_df[:4]`\n",
    ">    - `surveys_df[:-1]`\n",
    ">\n",
    "> 2. What happens when you call:\n",
    ">\n",
    ">    - `dat.iloc[0:4, 1:4]`\n",
    ">    - `dat.loc[0:4, 1:4]`\n",
    ">\n",
    "> - How are the two commands different?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Subsetting Data using Criteria"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Syntax Cheat Sheet\n",
    "* Equals: `==`\n",
    "* Not equals: `!=`\n",
    "* Greater than, less than: `>` or `<`\n",
    "* Greater than or equal to `>=`\n",
    "* Less than or equal to `<=`\n",
    "\n",
    "\n",
    "> ## Challenge - Queries\n",
    ">\n",
    "> 1. Select a subset of rows in the `surveys_df` DataFrame that contain data from\n",
    ">   the year 1999 and that contain weight values less than or equal to 8. How\n",
    ">   many rows did you end up with? What did your neighbor get?\n",
    ">\n",
    "> 2. You can use the `isin` command in Python to query a DataFrame based upon a\n",
    ">   list of values as follows:\n",
    ">\n",
    ">    ```python\n",
    ">    surveys_df[surveys_df['species_id'].isin([listGoesHere])]\n",
    ">    ```\n",
    ">\n",
    ">   Use the `isin` function to find all plots that contain particular species\n",
    ">   in the \"surveys\" DataFrame. How many records contain these values?\n",
    ">\n",
    "> 3. Experiment with other queries. Create a query that finds all rows with a\n",
    ">   weight value > or equal to 0.\n",
    ">\n",
    "> 4. The `~` symbol in Python can be used to return the OPPOSITE of the\n",
    ">   selection that you specify in Python. It is equivalent to **is not in**.\n",
    ">   Write a query that selects all rows with sex NOT equal to 'M' or 'F' in\n",
    ">   the \"surveys\" data.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using masks to identify a specific condition\n",
    "\n",
    "A **mask** can be useful to locate where a particular subset of values exist or\n",
    "don't exist - for example,  NaN, or \"Not a Number\" values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ## Challenge - Putting it all together\n",
    ">\n",
    "> 1. Create a new DataFrame that only contains observations with sex values that\n",
    ">   are **not** female or male. Assign each sex value in the new DataFrame to a\n",
    ">   new value of 'x'. Determine the number of null values in the subset.\n",
    ">   \n",
    "> 2. Create a new DataFrame that contains only observations that are of sex male\n",
    ">   or female and where weight values are greater than 0. Create a stacked bar\n",
    ">   plot of average weight by plot with male vs female values stacked for each\n",
    ">   plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}