Intro Pytorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro to Pytorch - Tensors\n",
    "\n",
    "A deep learning research platform that provides maximum flexibility and speed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensors\n",
    "\n",
    "What is an array in numpy is called a tensor in pytorch. And tensors or numpy arrays can be changed from one another\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "1.00000e-10 *\n",
      "  0.0000  0.0000  0.0000  0.0000  0.0000  0.0000\n",
      "  0.0000  0.0000  0.0000  0.0000  0.0000  0.0000\n",
      "  0.0000  0.0000  0.0000  0.0000  0.0000  0.0000\n",
      "  0.0000  4.6566  0.0000  0.0000  0.0000  0.0000\n",
      "[torch.FloatTensor of size 4x6]\n",
      "\n",
      "\n",
      " 0.6616  0.9945  0.5736  0.7569  0.6309  0.3372\n",
      " 0.2882  0.6051  0.4260  0.0714  0.8999  0.7592\n",
      " 0.9089  0.8799  0.2501  0.4241  0.3436  0.2998\n",
      " 0.7892  0.9875  0.0565  0.3858  0.6677  0.6135\n",
      "[torch.FloatTensor of size 4x6]\n",
      "\n",
      "\n",
      " 1  1  1  1\n",
      " 1  1  1  1\n",
      "[torch.FloatTensor of size 2x4]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "# Make a Tensor\n",
    "x = torch.Tensor(4,6) \n",
    "print x \n",
    "y = torch.rand(4,6)\n",
    "print y\n",
    "a = torch.ones(2,4)\n",
    "print a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "size: torch.Size([4, 6])\n"
     ]
    }
   ],
   "source": [
    "# Get size\n",
    "print 'size:', y.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.  1.  1.  1.]\n",
      " [ 1.  1.  1.  1.]]\n",
      "[ 1.  1.  1.  1.]\n",
      "[ 1.  1.  1.  1.]\n"
     ]
    }
   ],
   "source": [
    "# Change with Numpy (Same memory so changine one changes the other)\n",
    "#   Tensor to np\n",
    "b = a.numpy()\n",
    "print b\n",
    "\n",
    "#   np to Tensor\n",
    "import numpy as np\n",
    "f = np.ones(4)\n",
    "print f\n",
    "g = torch.from_numpy(f)\n",
    "print f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indexing into tensors\n",
    "Indexing, a key aspect of numpy arrays is exactly the same for tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "row 0 \n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      "[torch.FloatTensor of size 6]\n",
      "\n",
      "col 2 \n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      "[torch.FloatTensor of size 4]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "t = torch.ones(4,6)\n",
    "\n",
    "print 'row 0', t[0,:]\n",
    "print 'col 2', t[:, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.6616  0.9945  0.5736  0.7569  0.6309  0.3372\n",
      " 0.2882  0.6051  0.4260  0.0714  0.8999  0.7592\n",
      " 0.9089  0.8799  0.2501  0.4241  0.3436  0.2998\n",
      " 0.7892  0.9875  0.0565  0.3858  0.6677  0.6135\n",
      "[torch.FloatTensor of size 4x6]\n",
      "\n",
      "\n",
      " 0.6616  0.9945  0.5736  0.7569  0.6309  0.3372\n",
      " 0.2882  0.6051  0.4260  0.0714  0.8999  0.7592\n",
      " 0.9089  0.8799  0.2501  0.4241  0.3436  0.2998\n",
      " 0.7892  0.9875  0.0565  0.3858  0.6677  0.6135\n",
      "[torch.FloatTensor of size 4x6]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Operations on Tensors (Docs: http://pytorch.org/docs/master/torch.html)\n",
    "\n",
    "print torch.add(x, y) # One of the ways to add\n",
    "\n",
    "print x.add_(y) # Another one of the ways to add"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Autograd\n",
    "\n",
    "It is a package that allows us to differentiate  \n",
    "\n",
    "Think of for example backpropgation in a neural network where we have $ Loss = out_{computed} - out_{desired}$ and where we want to change the weights so that we approach a loss of zero.  \n",
    "\n",
    "Say we want to optimize $w_{i,j}$, the weight between two neurones \n",
    "\n",
    "We will do $\\frac{\\partial loss}{\\partial w_{i,j}}$, by chaining through intermediate neurone outputs\n",
    "\n",
    "Autograd can easily differentiate loss with respect to any of the variables involved in the net, doing the chaining automatically.\n",
    "\n",
    "In the forward phase, autograd remembers all the operations it executed based on our definition of the net where we define a function that acts on inputs and a function that acts on this function in the first hidden layer etc.\n",
    "\n",
    "In the backward phase, it will replay these operations and give us the derivatives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable:  Variable containing:\n",
      " 1\n",
      "[torch.FloatTensor of size 1]\n",
      "\n",
      "output Variable:  Variable containing:\n",
      " 21  21  21\n",
      " 21  21  21\n",
      "[torch.FloatTensor of size 2x3]\n",
      "\n",
      "Tensor in output Variable:  \n",
      " 21  21  21\n",
      " 21  21  21\n",
      "[torch.FloatTensor of size 2x3]\n",
      "\n",
      "Gradient fn to be used to compute gradient:  <torch.autograd.function.AddConstantBackward object at 0x10d4c9430>\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "\n",
    "input_tensor = torch.ones(1)\n",
    "\n",
    "# Autograd has a `Variable` class which wraps around a `Tensor`\n",
    "# (requires_grad=True to build the operations map)\n",
    "inputs = Variable(input_tensor, requires_grad=True) \n",
    "print 'input Variable: ', inputs\n",
    "\n",
    "# Can manipulate Variables\n",
    "outputs = x**2 + 20\n",
    "print 'output Variable: ', outputs\n",
    "\n",
    "# Variable.data will return the `Tensor`\n",
    "print 'Tensor in output Variable: ', outputs.data\n",
    "\n",
    "# Variable.grad_fn holds history of all functions and variables \n",
    "# involved in reaching to this Variable\n",
    "print 'Gradient fn to be used to compute gradient: ', outputs.grad_fn\n",
    "\n",
    "# Backpropagation\n",
    "# Call variable.backward to move back from output\n",
    "# Backward takes in a loss etc (refer to NN section)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}