dataPrepare.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\SoftWare\\Anaconda2\\envs\\python3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from data import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# data augmentation \n",
    "\n",
    "In deep learning tasks, a lot of data is need to train DNN model, when the dataset is not big enough, data augmentation should be applied.\n",
    "\n",
    "keras.preprocessing.image.ImageDataGenerator is a data generator, which can feed the DNN with data like : (data,label), it can also do data augmentation at the same time.\n",
    "\n",
    "It is very convenient for us to use keras.preprocessing.image.ImageDataGenerator to do data augmentation by implement image rotation, shift, rescale and so on... see [keras documentation](https://keras.io/preprocessing/image/) for detail.\n",
    "\n",
    "For image segmentation tasks, the image and mask must be transformed **together!!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## define your data generator\n",
    "\n",
    "If you want to visualize your data augmentation result, set save_to_dir = your path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#if you don't want to do data augmentation, set data_gen_args as an empty dict.\n",
    "#data_gen_args = dict()\n",
    "\n",
    "data_gen_args = dict(rotation_range=0.2,\n",
    "                    width_shift_range=0.05,\n",
    "                    height_shift_range=0.05,\n",
    "                    shear_range=0.05,\n",
    "                    zoom_range=0.05,\n",
    "                    horizontal_flip=True,\n",
    "                    fill_mode='nearest')\n",
    "myGenerator = trainGenerator(20,'data/membrane/train','image','label',data_gen_args,save_to_dir = \"data/membrane/train/aug\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## visualize your data augmentation result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#you will see 60 transformed images and their masks in data/membrane/train/aug\n",
    "num_batch = 3\n",
    "for i,batch in enumerate(myGenerator):\n",
    "    if(i >= num_batch):\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## create .npy data\n",
    "\n",
    "If your computer has enough memory, you can create npy files containing all your images and masks, and feed your DNN with them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_arr,mask_arr = geneTrainNpy(\"data/membrane/train/aug/\",\"data/membrane/train/aug/\")\n",
    "#np.save(\"data/image_arr.npy\",image_arr)\n",
    "#np.save(\"data/mask_arr.npy\",mask_arr)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}