tutorial_wrap_c_function_in_python.ipynb

{
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "name": "",
  "signature": "sha256:2a3508ec05212ab548b4b5563322fb59510d1369954e6a94f4d24d6bfb1db502"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Tutorial: How to wrap a C-Function in Cython\n",
      "\n",
      "This simple tutorial is meant to explain in a simple way how to include a C-function into python using cython.\n",
      "We explain which files are needed, what their format needs to be and which cython syntax to use to include the C-function into python. This Tutorial assumes basic knowledge of C and python but nearly no knowledge of cython. Our purpose is not to provide comprehensive documentation, which is exhaustively covered in [Calling C functions](http://docs.cython.org/src/tutorial/external.html), but rather to provide a simple, quickstart pattern to match in your python code development."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Needed Files\n",
      "\n",
      "In order to wrap a C-function into python using the pattern described here, one needs 5 files:\n",
      "- .h file with the C-function declaration\n",
      "- .c file containing the C-code\n",
      "- .pxd file containing the C-function declaration in cython (basically copied-and-pasted from the .h file)\n",
      "- .pyx file containing the wrapper function calling the C-function\n",
      "- a setup.py file needed to compile the C and cython code (if you're used to C, setup.py serves as the Makefile)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Example\n",
      "\n",
      "We chose a simple example to explain the structure and the content of the files needed. Our example is a C-function that finds the index of an input array to some input value. We chose such an example to have a function expecting an array and some integer to emphasize how to pass a python array to a C-function.\n",
      "\n",
      "The example files are named:\n",
      "- minimal_to_wrap.h\n",
      "- minimal_to_wrap.c\n",
      "- cython_declaration_module.pxd\n",
      "- find_closest_element.pyx\n",
      "- setup.py\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### C-header file - minimal_to_wrap.h\n",
      "\n",
      "This is the normal C header file containing the function declaration."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "//header file with function declaration\n",
      "\n",
      "int find_closest_element_in_c(double * input_data, int size_of_data, double key);"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### C-file - minimal_to_wrap.c\n",
      "\n",
      "This is the C-code with the function definition to include in the python code. Our example function finds the closest element to a given key.\n",
      "\n",
      "The function ``find_closest_element_in_c`` takes three arguments:\n",
      "\n",
      "1. an input array ``input_data``, \n",
      "2. an integer ``size_of_data`` providing the  length of ``input_data``, and \n",
      "3. a value to match called ``key``. \n",
      "\n",
      "The function ``find_closest_element_in_c`` returns *closest,* the index of the closest element of ``input_data`` to the input ``key``. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "#include <math.h>\n",
      "\n",
      "//Returns the index of the element of input_data\n",
      "//closest to key\n",
      "int find_closest_element_in_c(double * input_data, int size_of_data, double key)\n",
      "{\n",
      "    if (size_of_data <= 0)\n",
      "        return -1;\n",
      "    //Linearly search through the array to find the closest element\n",
      "    //distance stores the current minimal distance\n",
      "    //closest stores the index\n",
      "    //Sorting the array first would certainly be much faster!\n",
      "    double distance = fabs(input_data[0] - key);\n",
      "    int closest = 0;\n",
      "    for(int i = 0; i < size_of_data; ++i)\n",
      "    {\n",
      "        if (fabs(input_data[i] - key) < distance )\n",
      "        {\n",
      "            closest = i;\n",
      "            distance = fabs(input_data[i] - key);\n",
      "        }\n",
      "    }\n",
      "    return closest;\n",
      "}"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### .pxd-file - cython_declaration_module.pxd\n",
      "This is the cython declaration of the C-function to include into the python code. Note that this declaration is almost identical to ordinary declaration syntax in C; the only difference is the boilerplate ``cdef extern from \".h-file\"``, which specifies the external file where the C-function is defined. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "\"\"\" Cython declaration of the C extension. \n",
      "\"\"\"\n",
      "\n",
      "cdef extern from \"minimal_to_wrap.h\":\n",
      "    int find_closest_element_in_c(double * input_data, int size_of_data, double key);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### .pyx-file - find_closest_element.pyx\n",
      "\n",
      "This is the cython code that calls the underlying C-function. The ``find_closest_element_wrapper`` function defined in this module is the object you will interact with at the python level.\n",
      "\n",
      "Note the use of ``cimport`` as opposed to the usual ``import`` statement. \n",
      "\n",
      "We then define our python function ``find_closest_element_wrapper`` which takes two input parameters, an input array ``input_data`` and a ``key``. Because we will be passing these objects into our C-function, we need to declare them in a way that C will understand:\n",
      "\n",
      "1. For the ``input_data`` array, we can accomplish this with boilerplate syntax ``c_np.ndarray[c_np.float64_t, ndim=1]`` that allows C to interpret a `Numpy` array; the arguments inside the braces declare the type as a `double` and the shape as a 1-d array. See [Working with Numpy](http://docs.cython.org/src/tutorial/numpy.html) for comprehensive details.\n",
      "2. For the scalar ``key``, note that we can conveniently allow the Cython compiler to interpret the type.\n",
      "\n",
      "Inside our python function we call the C-function via ``cython_declaration_module.find_closest_element_in_c(&input_data[0], len(input_data), key)``; the standard way to pass an array from cython to C is in the manner we would pass arrays in C. Namely, this means we pass the pointer to the first element of our array ``&input_data[0]``. The second argument is the length of the array, as demanded by the C definition of the function. The third argument then is just passing the key.\n",
      "\n",
      "As a last step we return the index of the closest element as the result calculated by the C-function, and return it as we normally would in pure python."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "cimport numpy as c_np \n",
      "cimport cython_declaration_module\n",
      "\n",
      "def find_closest_element_wrapper(c_np.ndarray[c_np.float64_t, ndim=1] input_data, key):\n",
      "    cdef int result\n",
      "    result = cython_declaration_module.find_closest_element_in_c(&input_data[0], len(input_data), key)\n",
      "    return result"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### setup.py file - find_closest_element.pyx\n",
      "\n",
      "This is the file that compiles our cython and C code so that ``find_closest_element_wrapper`` is callable from pure python. This file is needed to compile the C/cython code in order to create a importable library (.so file). \n",
      "\n",
      "To make the setup file run, we need to import some essential modules.\n",
      "\n",
      "``import numpy as np``  \n",
      "``from distutils.extension import Extension``\n",
      "`` from distutils.extension import Extension``  \n",
      "``from distutils.core import setup``  \n",
      "``from Cython.Build import cythonize``\n",
      "\n",
      "We then define the source files:\n",
      "``sources = ['find_closest_element.pyx', 'minimal_to_wrap.c']``\n",
      "\n",
      "**Important**: The name given to the .pyx file must match the name specified in the Extension section. \n",
      "\n",
      "The next line defining ``extension_obj_instance`` builds the extension object. **Important**: Note the use of the include_dirs keyword argument - necessary whenever our cython wrapper requires a cimport of numpy. See [the Extension API](https://docs.python.org/2/distutils/apiref.html#distutils.core.Extension) for details.\n",
      "\n",
      "In the final line of setup.py we define what to do when setup.py is called."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "import numpy as np\n",
      "from distutils.extension import Extension\n",
      "from distutils.core import setup\n",
      "from Cython.Build import cythonize\n",
      "\n",
      "sources = ['find_closest_element.pyx', 'minimal_to_wrap.c']\n",
      "\n",
      "extension_obj_instance = Extension(name=\"find_closest_element\", sources=sources,\n",
      "    include_dirs=[np.get_include()])\n",
      "\n",
      "setup(name=\"cython_wrapper\",ext_modules = cythonize([extension_obj_instance]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Inclusion into python\n",
      "\n",
      "short example program for importing and calling the newly defined C-function"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os\n",
      "os.system(\"python setup.py build_ext --inplace\")\n",
      "\n",
      "import numpy as np\n",
      "from find_closest_element import find_closest_element_wrapper\n",
      "\n",
      "data = np.random.rand(1000)*1000\n",
      "key = 34\n",
      "\n",
      "result = find_closest_element_wrapper(data, key)\n",
      "print data[result]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "33.6566030726\n"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    }
   ],
   "metadata": {}
  }
 ]
}