Part_III_Hands_On_Session.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Part III: Hands On Session"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 1. Data Wranginling\n",
    "\n",
    "* Count number of `POST` requests logged in `DBLog.out`\n",
    "\n",
    "* Convert `DBLog.out` to csv with fileds \"Method,Path,RepsoneCode,Latency\"\n",
    "\n",
    "* Make a bash script `log2csv` that can be used as follows:  \n",
    "  `cat DBLog.out | ./log2csv > DBLog.csv`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 2. IPython toolchain\n",
    "\n",
    "* Start IPython notebook\n",
    "* Import dataset \"LogDB.csv\"\n",
    "* Make a bar chart with \"HTTP Method | Count\" like so:\n",
    "\n",
    "```\n",
    "GET  | ################\n",
    "POST | ######\n",
    "PUT  | ##\n",
    "```\n",
    "\n",
    "* Make a histogram of response times\n",
    "* Make a boxplot of the response times\n",
    "* Compute the 0.95-quantile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 3. Quantile Checking\n",
    "\n",
    "* Load the file `WebLatency.csv` containing request latencies for 6 servers in 5 minute intervals into a np.array\n",
    "\n",
    "* Plot the repsonse times of server 1 against time in a scatter plot\n",
    "\n",
    "* Calculate the averages over 15m intervals as follows:\n",
    "  \n",
    "  1. Generate a sequence of timestamps 15 minutes distance\n",
    "  2. Generate a sequence of sample windows of length 15 minutes\n",
    "  3. Count the number of samples in each window\n",
    "  4. Compute the average of the samples in each window\n",
    "\n",
    "  Alternative approach:\n",
    "  \n",
    "  1. Make a function `getWindowStart(t)` that returns the start of the last 15m window for a given time stamp t\n",
    "  2. Add a column to the np.array cointaining the window start times for each row\n",
    "  3. Use boolean indexing to select all rows in a 15m window\n",
    "  4. Compute the count and average of samples in each window\n",
    "\n",
    "\n",
    "* Calculate the `0.95`, `0.99` and `1`-quantile over the 15m intervals for each day."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 4. Peek time regression\n",
    "\n",
    "* Load the file `ReqRateTrend.csv` containg request rates over time.\n",
    "\n",
    "* Calculate the maximum value for each day\n",
    "\n",
    "* Make a linear regression of the maximum values for each day over time\n",
    "\n",
    "Variant:\n",
    "\n",
    "* Make a regression over daily .95 quantiles instead"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}