sentiment-pipeline-sklearn-3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to build a social media sentiment analysis pipeline with scikit-learn\n",
    "\n",
    "*This is Part 3 of 5 in a series on building a sentiment analysis pipeline using scikit-learn. You can find Part 4 [here](./sentiment-pipeline-sklearn-4.ipynb), and the introduction [here](./sentiment-pipeline-sklearn-1.ipynb).*\n",
    "\n",
    "*Jump to:* \n",
    "\n",
    "* *[**Part 1 - Introduction and requirements**](./sentiment-pipeline-sklearn-1.ipynb)*\n",
    "* *[**Part 2 - Building a basic pipeline**](./sentiment-pipeline-sklearn-2.ipynb)*\n",
    "* *[**Part 4 - Adding a custom feature to a pipeline with FeatureUnion**](./sentiment-pipeline-sklearn-4.ipynb)*\n",
    "* *[**Part 5 - Hyperparameter tuning in pipelines with GridSearchCV**](./sentiment-pipeline-sklearn-5.ipynb)*\n",
    "\n",
    "# Part 3 - Adding a custom function to a pipeline\n",
    "\n",
    "Now that we know how to build a basic scikit-learn pipeline, let's take it to the next level. Text data often rewards feature engineering and preprocessing, but there aren't a ton of built-in ways to do so. We're going to have to do some weird stuff to be able to add in our own functions, but once we do so, we'll be able to include any arbitrary function (to a certain extent, of course) in a pipeline.\n",
    "\n",
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "got 92 posts from page 1...\n",
      "got 88 posts from page 2...\n",
      "got 88 posts from page 3...\n",
      "got 91 posts from page 4...\n",
      "got 87 posts from page 5...\n",
      "got 89 posts from page 6...\n",
      "got 95 posts from page 7...\n",
      "got 93 posts from page 8...\n",
      "got 86 posts from page 9...\n",
      "got 90 posts from page 10...\n",
      "got all pages - 899 posts in total\n",
      "CPU times: user 1.1 s, sys: 256 ms, total: 1.36 s\n",
      "Wall time: 6.61 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from fetch_twitter_data import fetch_the_data\n",
    "import nltk\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "df = fetch_the_data()\n",
    "X, y = df.text, df.sentiment\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "\n",
    "tokenizer = nltk.casual.TweetTokenizer(preserve_case=False, reduce_len=True)\n",
    "count_vect = CountVectorizer(tokenizer=tokenizer.tokenize) \n",
    "classifier = LogisticRegression()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The function\n",
    "\n",
    "What happens when we replace all @ mentions with a generic token?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "def genericize_mentions(text):\n",
    "    return re.sub(r'@[\\w_-]+', 'thisisanatmention', text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparing a function for a scikit-learn pipeline\n",
    "\n",
    "scikit-learn's pipelines are dope, but every step has to look like a sklearn transformer. Basically this means that everything that goes into a pipeline has to implement `fit()` and `transform()` methods. The built-in [FunctionTranformer](scikit-learn.org/stable/modules/generated/sklearn.preprocessing.FunctionTransformer.html) does this handily, but items in a pipeline get passed as a full array/series/list to each step, not individual items. So, we're going to wrap our custom functions in a function that creates a list comprehension that applies our custom function to the series passed in, then wraps that in a FunctionTransformer. [Cue Inception horn](http://inception.davepedu.com/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import FunctionTransformer\n",
    "\n",
    "def pipelinize(function, active=True):\n",
    "    def list_comprehend_a_function(list_or_series, active=True):\n",
    "        if active:\n",
    "            return [function(i) for i in list_or_series]\n",
    "        else: # if it's not active, just pass it right back\n",
    "            return list_or_series\n",
    "    return FunctionTransformer(list_comprehend_a_function, validate=False, kw_args={'active':active})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adding a function to a scikit-learn pipeline\n",
    "\n",
    "Okay, so now that we have our function to wrap our function, we're going to insert it into our pipeline and train and test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "null accuracy: 45.33%\n",
      "accuracy score: 65.78%\n",
      "model is 20.44% more accurate than null accuracy\n",
      "---------------------------------------------------------------------------\n",
      "Confusion Matrix\n",
      "\n",
      "Predicted  negative  neutral  positive  __all__\n",
      "Actual                                         \n",
      "negative         28        9        12       49\n",
      "neutral          15       46        13       74\n",
      "positive         12       16        74      102\n",
      "__all__          55       71        99      225\n",
      "---------------------------------------------------------------------------\n",
      "Classification Report\n",
      "\n",
      "                precision    recall  F1_score support\n",
      "Classes                                              \n",
      "negative         0.509091  0.571429  0.538462      49\n",
      "neutral          0.647887  0.621622  0.634483      74\n",
      "positive         0.747475   0.72549  0.736318     102\n",
      "__avg / total__  0.662807  0.657778  0.659737     225\n"
     ]
    }
   ],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn_helpers import train_test_and_evaluate\n",
    "\n",
    "sentiment_pipeline = Pipeline([\n",
    "        ('genericize_mentions', pipelinize(genericize_mentions)),\n",
    "        ('vectorizer', count_vect),\n",
    "        ('classifier', classifier)\n",
    "    ])\n",
    "\n",
    "sentiment_pipeline, confusion_matrix = train_test_and_evaluate(sentiment_pipeline, X_train, y_train, X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at you, so accomplished! Now you can define whatever kind of function you like and include it in a pipeline.\n",
    "\n",
    "# What's next?\n",
    "\n",
    "We're going to do \\*nearly\\* the same thing we just did, but instead of using the output of a step as the input for the next step, we're going to take the output of a step and use it as a new feature. Click on over to [part four](./sentiment-pipeline-sklearn-4.ipynb). You know you want to do it.\n",
    "\n",
    "*This is Part 3 of 5 in a series on building a sentiment analysis pipeline using scikit-learn. You can find Part 4 [here](./sentiment-pipeline-sklearn-4.ipynb), and the introduction [here](./sentiment-pipeline-sklearn-1.ipynb).*\n",
    "\n",
    "*Jump to:* \n",
    "\n",
    "* *[**Part 1 - Introduction and requirements**](./sentiment-pipeline-sklearn-1.ipynb)*\n",
    "* *[**Part 2 - Building a basic pipeline**](./sentiment-pipeline-sklearn-2.ipynb)*\n",
    "* *[**Part 4 - Adding a custom feature to a pipeline with FeatureUnion**](./sentiment-pipeline-sklearn-4.ipynb)*\n",
    "* *[**Part 5 - Hyperparameter tuning in pipelines with GridSearchCV**](./sentiment-pipeline-sklearn-5.ipynb)*"
   ]
  }
 ],
 "metadata": {
  "_draft": {
   "nbviewer_url": "https://gist.github.com/bfeb07535373d908f1bba6842e7797d9"
  },
  "gist": {
   "data": {
    "description": "Sentiment Blog Rough Draft",
    "public": false
   },
   "id": "bfeb07535373d908f1bba6842e7797d9"
  },
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}