pytorch/loading_and_normalizing_datasets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasets and Dataloaders\n",
    "\n",
    "Code for processing data samples can get messy and hard to maintain; we ideally want out dataset code to be decoupled from our model training code for better readability and modularity. PyTorch provides two data primitives:\n",
    "* `torch.utils.data.DataLoader`\n",
    "* `torch.utils.data.Dataset`\n",
    "\n",
    "that allo you to use pre-loaded datasets as well as you own data.\n",
    "\n",
    "`Dataset` stores the samples and their corresponding labels, and `DataLoader` wraps an iterable around the `Dataset` to enable easy access to the samples.\n",
    "\n",
    "PyTorch domain libraries provide a number of sample pre-loaded datasets (such as FashionMNIST) that subclass `torch.utils.data.Dataset` and implement functions specific to the particulart data. They can be found and used as samples for prototyping and benchmarking you model here:\n",
    "* Image Datasets\n",
    "* Text Datasets\n",
    "* Audio Datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing Environment and Importing Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading a dataset\n",
    "\n",
    "We will be loading the Fashion-MNIST dataset from `TorchVision`. `Fashion-MNIST` is a dataset of Zalando's article images consisting of 60,000 training examples and 10,000 test examples. Each example comprises a 28x28 grayscale image and an associated label from one of 10 classes.\n",
    "* Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total.\n",
    "* The 10 classes tell what type of image it is. For example, T-shirt/top, Trouser, Pullover, Dress, Bag, Ankle boot, etc.\n",
    "* The grascale are values between 0 to 255 that measure the instensity of black and white images. The intensity value increase from while to black. For example: a white color will be 0, while a black color will 255.\n",
    "\n",
    "We load the `FashionMNIST` Dataset with the following parameters:\n",
    "* `root` is the path where the train/test data is stored\n",
    "* `train` specifies training or test dataset\n",
    "* `download=True` downloads the data from the internet if it's not available at `root`\n",
    "* `transform` and `target_transform` specify the feature and label transformations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from __future__ import annotations\n",
    "import torch\n",
    "from torchvision import datasets\n",
    "from torchvision.transforms import ToTensor, Lambda\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "training_data = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=True,\n",
    "    download=True,\n",
    "    transform=ToTensor(),\n",
    ")\n",
    "\n",
    "test_data = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=False,\n",
    "    download=True,\n",
    "    transform=ToTensor(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterating and Visualizing the Dataset\n",
    "\n",
    "We can index `Datasets` manually like a list: `training_data[index]`. We use `matplotlib` to visualize some samples in our training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels_map = {\n",
    "    0: \"T-Shirt\",\n",
    "    1: \"Trouser\",\n",
    "    2: \"Pullover\",\n",
    "    3: \"Dress\",\n",
    "    4: \"Coat\",\n",
    "    5: \"Sandal\",\n",
    "    6: \"Shirt\",\n",
    "    7: \"Sneaker\",\n",
    "    8: \"Bag\",\n",
    "    9: \"Ankle Boot\",\n",
    "}\n",
    "\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "cols, rows = 3, 3\n",
    "for i in range(1, cols* rows+1):\n",
    "    sample_idx = torch.randint(len(training_data), size=(1,)).item()\n",
    "    img, label = training_data[sample_idx]\n",
    "    figure.add_subplot(rows, cols, i)\n",
    "    plt.title(labels_map[label])\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(img.squeeze(), cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing your data for training with `DataLoader`\n",
    "\n",
    "The `Dataset` retrieves our dataset's features and labels one sample at a time. While training a model, we typically want to pass samples in \"minibatches\", reshuffle the data at every epoch to reduce model overfitting, and use Python's multiprocessing to speed up data retrieval.\n",
    "\n",
    "In machine learning, you need to specify what the feature and label are in your dataset. `Features` are input and `labels` are output. We train use features and train the model to predict the label.\n",
    "* Labels are what 10 class types: T-shirt, Sandal, Dress etc\n",
    "* Feature aer the patterns in the image pixels\n",
    "\n",
    "The `DataLoader` is an iterable that abstracts this complexity for us in an easy API. We use the Dataloader, we need to set the folloiwing parameters:\n",
    "* `data` the training data that will be used to train the model; and test data to evaluate the model\n",
    "* `batch_size` the number of records to be processed in each batch\n",
    "* `shuffle` the randoms sample of the data by indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "train_dataloader = DataLoader(training_data, batch_size=64, shuffle=True)\n",
    "test_dataloader = DataLoader(test_data, batch_size=64, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterate through the DataLoader\n",
    "\n",
    "We have loaded that dataset into the `Dataloader` and can iterate through the dataset as needed. Each iteration below returns a batch of `train_features` and `train_labels`(containing `batch_size=64` features and labels respectively). Because we specified `shuffle=True`, after we iterate overall batches the data is shuffled (for finer-grained control over the data loading order.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display image and label.\n",
    "train_features, train_labels = next(iter(train_dataloader))\n",
    "print(f\"Feature batch shape: {train_features.size()}\")\n",
    "print(f\"Labels batch shape: {train_labels.size()}\")\n",
    "img = train_features[0].squeeze()\n",
    "label = train_labels[0]\n",
    "plt.imshow(img, cmap=\"gray\")\n",
    "plt.show()\n",
    "print(f\"Label: {label}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalization\n",
    "\n",
    "Normalization is a common data pre-processing technique that is applied to scale or transform the data to make sure there's an equal learning contribution from each feature. For example, each pixel in the grayscale images have a value between 0 to 255, which are features. If one pixel value is 17 and another pixel is 197. There will be an uneven distribution of the pixel importance, because the higher pixel volumes will deviate the learning. `Normalization changes the range of your data, without distorting its distintion among out feature`. This pre-processing is to avoid:\n",
    "* A reduction of the prediction accuracy\n",
    "* Difficulty for the model to learn\n",
    "* Unfavorable distribution of the feature data ranges"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transforms\n",
    "\n",
    "Data does not always come in its final processed form that is required for training machine learning algorithms. We use `transforms` to perform some manipulation of the data and make it suitable for training.\n",
    "\n",
    "All TorchVision datasets have two parameters - (`transform` to modify the features and `target_transform` to modify the labels) that accept callables containing the transformation logic. The `torchvision.transforms` module offers several commonly-used transforms out of the box.\n",
    "\n",
    "The FashionMINST features are in PIL Image format, and the labels are intergers. For training, we need the features as normalized tensors, and the labels as one-hot encoded tensors, and the labels as one-hot encoded tensors. To make these transformations, we use `ToTensor` and `Lambda`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchvision import datasets\n",
    "from torchvision.transforms import ToTensor, Lambda\n",
    "\n",
    "ds = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=True,\n",
    "    download=True,\n",
    "    transform=ToTensor(),\n",
    "    target_transform=Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, torch.tensor(y), value=1))\n",
    ")\n",
    "\n",
    "print(ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ToTensor()\n",
    "\n",
    "`ToTensor` converts a PIL image or NumPy `ndarray` into a `FloatTensor` and scales the image's pixel intensity values in the range [0., 1.]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lambda transforms\n",
    "\n",
    "Lambda transforms apply any user-defined lambda function. Here, we define a function to turn the integer into a one-hot encoded tensor. It first creates a zero tensor of size 10(the number of lables in our dataset) and calls scatter which assigns a value=1 on the index as given by the label `y`. You can also use `torch.nn.functional.one_hot` as another option to do that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_transform = Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, torch.tensor(y), value=1))\n",
    "\n",
    "print(target_transform(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn.functional as F\n",
    "target_transform2 = Lambda(lambda y: F.one_hot(torch.tensor(y), num_classes=10))\n",
    "\n",
    "print(target_transform2(3))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "diffusers",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.4"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}