05_SharingAnalysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sharing your code with the world\n",
    "\n",
    "We now have a fully automated and tested workflow!!! 🤩🎉😎\n",
    "\n",
    "And we are ready to share our awesomeness with the world. \n",
    "\n",
    "Head to [https://github.com/](https://github.com/) and login to your account\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In your repository section click on the green **new repository** button\n",
    "![gh1](assets/github1.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to link your local project to your GitHub repository\n",
    "![link](assets/github2.PNG)\n",
    "\n",
    "type those command on your shell (or copy and paste). **Use your own details**\n",
    "\n",
    "Refresh your web browser and... ta dah! Your project is online"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Continuous integration \n",
    "\n",
    "Now, instead of running our tests manually every time we want for this to be tested every time we push something from our local computer to our GitHub account. \n",
    "\n",
    "Some of the advantages of doing this are:\n",
    "- check every version of your code\n",
    "- check for errors continuously\n",
    "- report the results of the tests\n",
    "- identify when things stop working\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Activate Travis CI\n",
    "\n",
    "Travis CI is a continuous integration server hosting platform. All you need is an account. Now let's go to [https://travis-ci.org/](https://travis-ci.org/) and activate CI for your project\n",
    "\n",
    "![travis](assets/travisact.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## .travis.yml\n",
    "A **.travis.yml** script tells Travis CI what steps are needed to test your project\n",
    "\n",
    "```yaml\n",
    "language: python\n",
    "\n",
    "python:\n",
    "  - 3.6\n",
    "  \n",
    "before_install:\n",
    "  # Here we download miniconda and create our conda environment\n",
    "  - export MINICONDA=$HOME/miniconda\n",
    "  - export PATH=\"$MINICONDA/bin:$PATH\"\n",
    "  - hash -r\n",
    "  - wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh\n",
    "  - bash miniconda.sh -b -f -p $MINICONDA\n",
    "  - conda config --set always_yes yes\n",
    "  - conda update conda\n",
    "  - conda info -a\n",
    "  # Create the environment from a yml file\n",
    "  # Is this familiar?\n",
    "  - conda env create -f environment.yml -v\n",
    "  - source activate testenv\n",
    "  \n",
    "script:\n",
    "  - python3 -m pytest tests/\n",
    "  - pytest --nbval notebooks/00_explore-data.ipynb\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Does Your Software Work on Your Colleague’s Computer?\n",
    "\n",
    "Complex analysis depend on a number of packages and libraries native to your OS, packages the user installs, environmental variables and so on. Manually maintaining these dependencies is a rather tedious task. \n",
    "\n",
    "That is why we use package managers, such as Conda. \n",
    "Do you remember we started this workshop by installing all the packages we needed and one of the steps was to use an **environment.yml** file? This ensures we all have the same packages.  \n",
    "\n",
    "Then at the beginning of the course we activated our **reproPython** environment and have been using the packages installed in this environment. \n",
    "\n",
    "<div class='warn'> conda is only one package manager, there are many more alternatives for you </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create our **environment.yml** file from which Travis CI will create our testing environment.\n",
    "\n",
    "```yaml\n",
    "name: testenv\n",
    "channels:\n",
    "  - conda-forge\n",
    "  - defaults\n",
    "dependencies:\n",
    "  - python>3.6\n",
    "  - pytest\n",
    "  - pandas\n",
    "  - matplotlib\n",
    "  - jinja2\n",
    "  - pip:\n",
    "    - nbval\n",
    "```\n",
    "Make sure to place it i the root directory of your repository.\n",
    "\n",
    "<div class='info'> Note we are not specifying versions of the packages so by default conda will install the latest versions available </div>.\n",
    "\n",
    "Also you can use a `requirements.txt` or specify the install steps directly in the `travis.yml` file by adding an `install` step."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, open the `00_explore-data.ipynb` and add a `#NBVAL_SKIP` comment to the top of cells 3 and 4. Save the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Commit your changes and push this to github\n",
    "```\n",
    "$ git add .\n",
    "$ git commit -m \"Add files for CI\"\n",
    "$ git push\n",
    "```\n",
    "\n",
    "This will start an automated testing of your project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 😢But it failed\n",
    "\n",
    "You will find that the test failed... since we did not add the data files. \n",
    "We will in this case add the raw data to our repository for demonstration purposes. \n",
    "\n",
    "First we need to tell git to add the interim and processed data. open your `.gitignore` file and add the following lines:\n",
    "\n",
    "```\n",
    "# Add data for test\n",
    "!data/interim/2018-05-09-winemag_priceGBP.*\n",
    "!data/processed/2018-05-09-winemag_Chile.*\n",
    "```\n",
    "\n",
    "<div class='info'> The prefix **!** negates the pattern, so these files will not be ignored.</div>\n",
    "\n",
    "Now you can commit and push to GitHub."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, sometimes when using nbval and notebooks with html table outputs the html/css does not really match. Make sure to add the following to cells 3 and 4 in your exploratory notebooks:\n",
    "\n",
    "#NBVAL_SKIP\n",
    "\n",
    "What if you want to check the notebooks are working instead of the cel by cell output?\n",
    "\n",
    "use \n",
    "\n",
    "```\n",
    "$ pytest --nbval-lax <notebook>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Making your software citable \n",
    "\n",
    "Head to [https://zenodo.org/](https://zenodo.org/) and login using your GitHub account\n",
    "\n",
    "![zenodo](assets/zenodo-login.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find your repository and toggle on Zenodo\n",
    "![toggle](assets/zenodo-toggle-on.png)\n",
    "\n",
    "We will be redirected to GitHub to create a release of the repository."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After creating the release, go back to Zenodo and refresh the page. You should now see the newly created DOI 🤩\n",
    "\n",
    "Click on the DOI and copy the markdown text, add this to your README.md and push to GitHub!\n",
    "\n",
    "![doi](assets/DOI.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a citation file\n",
    "\n",
    "```\n",
    "cff-version: 1.0.3\n",
    "message: If you use this software, please cite it as below.\n",
    "authors:\n",
    "  - family-names: Allard\n",
    "    given-names: Tania\n",
    "    orcid: https://orcid.org/0000-0003-4925-7248\n",
    "title: My reproducibel python workflow\n",
    "version: 0.1\n",
    "doi: 10.5281/zenodo.1241049\n",
    "date-released: 2018-05-09\n",
    "```\n",
    "\n",
    "Save as CITATION.cff and commit and push to GitHub\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href=\"https://fonts.googleapis.com/css?family=Didact+Gothic|Dosis:400,500,700\" rel=\"stylesheet\"><style>\n",
       "@font-face {\n",
       "  font-family: \"Computer Modern\";\n",
       "  src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "/* div.cell{\n",
       "width:800px;\n",
       "margin-left:16% !important;\n",
       "margin-right:auto;\n",
       "} */\n",
       "h1 {\n",
       "  font-family: 'Dosis', \"Helvetica Neue\", Arial, sans-serif;\n",
       "  color: #0B132B;\n",
       "}\n",
       "h2 {\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  color: #1C2541;\n",
       "}\n",
       "h3{\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  margin-top:12px;\n",
       "  margin-bottom: 3px;\n",
       "  color: #40a8a6;\n",
       "}\n",
       "h4{\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  color: #40a8a6;\n",
       "}\n",
       "h5 {\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  color: #40a8a6;\n",
       "}\n",
       "div.text_cell_render{\n",
       "  font-family: 'Didact Gothic',Computer Modern, \"Helvetica Neue\", Arial, Helvetica,\n",
       "  Geneva, sans-serif;\n",
       "  line-height: 130%;\n",
       "  font-size: 110%;\n",
       "  /* width:600px; */\n",
       "  /* margin-left:auto;\n",
       "  margin-right:auto; */\n",
       "}\n",
       "\n",
       ".text_cell_render h1 {\n",
       "  font-weight: 200;\n",
       "  font-size: 30pt;\n",
       "  /* font-size: 50pt */\n",
       "  line-height: 100%;\n",
       "  color:#0B132B;\n",
       "  margin-bottom: 0.5em;\n",
       "  margin-top: 0.5em;\n",
       "  display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h2{\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       ".text_cell_render h3{\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       "\n",
       ".warning{\n",
       "  color: rgb( 240, 20, 20 )\n",
       "}\n",
       "\n",
       "div.warn {\n",
       "  background-color: #FF5A5F;\n",
       "  border-color: #FF5A5F;\n",
       "  border-left: 5px solid #C81D25;\n",
       "  padding: 0.5em;\n",
       "\n",
       "  color: #fff;\n",
       "  opacity: 0.8;\n",
       "}\n",
       "\n",
       "div.info {\n",
       "  background-color: #087E8B;\n",
       "  border-color: #087E8B;\n",
       "  border-left: 5px solid #0B3954;\n",
       "  padding: 0.5em;\n",
       "  color: #fff;\n",
       "  opacity: 0.8;\n",
       "}\n",
       "\n",
       "</style>\n",
       "<script>\n",
       "MathJax.Hub.Config({\n",
       "  TeX: {\n",
       "    extensions: [\"AMSmath.js\"]\n",
       "    },\n",
       "    tex2jax: {\n",
       "      inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
       "      displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
       "      },\n",
       "      displayAlign: 'center', // Change this to 'center' to center equations.\n",
       "      \"HTML-CSS\": {\n",
       "        styles: {'.MathJax_Display': {\"margin\": 4}}\n",
       "      }\n",
       "      });\n",
       "      </script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML\n",
    "\n",
    "\n",
    "def css_styling():\n",
    "    styles = open(\"styles/custom.css\", \"r\").read()\n",
    "    return HTML(styles)\n",
    "css_styling()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}