Project_21C02.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Project_21C02",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5xVALNYBurlN",
        "colab_type": "text"
      },
      "source": [
        "# **SAR Target/Object Detection Using Machine Learning**\n",
        "\n",
        "The notebook represents real time implementation of data processing, model building and testing of multiple Machine Learning Algorithms for detection of the target in MSTAR dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "unN2bCKpIDq4",
        "colab_type": "text"
      },
      "source": [
        "**Necassary Packages Installation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GzaMFWkDhmEL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 241
        },
        "outputId": "b4228afa-8a66-4b24-e17c-68768f1701b5"
      },
      "source": [
        "!pip install scipy==1.2.1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting scipy==1.2.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7f/5f/c48860704092933bf1c4c1574a8de1ffd16bf4fde8bab190d747598844b2/scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8MB)\n",
            "\u001b[K     |████████████████████████████████| 24.8MB 128kB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.8.2 in /usr/local/lib/python3.6/dist-packages (from scipy==1.2.1) (1.18.5)\n",
            "\u001b[31mERROR: umap-learn 0.4.6 has requirement scipy>=1.3.1, but you'll have scipy 1.2.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: tensorflow 2.3.0 has requirement scipy==1.4.1, but you'll have scipy 1.2.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.\u001b[0m\n",
            "Installing collected packages: scipy\n",
            "  Found existing installation: scipy 1.4.1\n",
            "    Uninstalling scipy-1.4.1:\n",
            "      Successfully uninstalled scipy-1.4.1\n",
            "Successfully installed scipy-1.2.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dkm_A64Ph1sC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "from PIL import Image\n",
        "import scipy.misc as im\n",
        "\n",
        "import os\n",
        "from sklearn.preprocessing import OneHotEncoder\n",
        "from sklearn.decomposition import PCA\n",
        "from skimage.transform import resize\n",
        "from skimage import data"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wgEg-TV1h9_t",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import PIL #resizing the images"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "isTLNXTDyWwP",
        "colab_type": "text"
      },
      "source": [
        "**Data Processing**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x4uLqaMEGCCE",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://drive.google.com/uc?export=view&id=11u7TILbn_1a5OvI97Zrv4nzGke8LSC88)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DGZYIzbCHNWK",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://drive.google.com/uc?export=view&id=1LtSjZGAiqjl6UPZYBJjiJupzPyyA6YGN)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bZDCGwImiAOd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def get_mstar_data(stage, width=128, height=128, crop_size=128, aug=False):\n",
        "    data_dir = \"/content/gdrive/My Drive/PSIT Files/MSTAR_Machine_learning/MSTAR-10/train/\" if stage == \"train\" else \"/content/gdrive/My Drive/PSIT Files/MSTAR_Machine_learning/MSTAR-10/test/\" if stage == \"test\" else None\n",
        "    print(\"------ \" + stage + \" ------\")\n",
        "    sub_dir = [\"2S1\", \"BMP2\", \"BRDM_2\", \"BTR60\", \"BTR70\", \"D7\", \"T62\", \"T72\", \"ZIL131\", \"ZSU_23_4\"]\n",
        "    X = []\n",
        "    y = []\n",
        "\n",
        "    for i in range(len(sub_dir)):\n",
        "        tmp_dir = data_dir + sub_dir[i] + \"/\"\n",
        "        img_idx = [x for x in os.listdir(tmp_dir) if x.endswith(\".jpeg\")]\n",
        "        print(sub_dir[i], len(img_idx))\n",
        "        y += [i] * len(img_idx)\n",
        "        for j in range(len(img_idx)):\n",
        "            img = im.imresize(im.imread((tmp_dir + img_idx[j])), [height, width])\n",
        "            img = img[(height - crop_size) // 2 : height - (height - crop_size) // 2, \\\n",
        "                  (width - crop_size) // 2: width - (width - crop_size) // 2]\n",
        "            # img = img[16:112, 16:112]   # crop\n",
        "            X.append(img)\n",
        "\n",
        "    return np.asarray(X), np.asarray(y)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ioONgNWqiRrq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def data_shuffle(X, y, seed=0):\n",
        "    data = np.hstack([X, y[:, np.newaxis]])\n",
        "    np.random.shuffle(data)\n",
        "    return data[:, :-1], data[:, -1]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Sk_-cl3iUjV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def one_hot(y_train, y_test):\n",
        "    one_hot_trans = OneHotEncoder().fit(y_train[:, np.newaxis])\n",
        "    return one_hot_trans.transform(y_train[:, np.newaxis]).toarray(), one_hot_trans.transform(y_test[:, np.newaxis]).toarray()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "drM-FP2jiXoF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def mean_wise(X):\n",
        "    return (X.T - np.mean(X, axis=1)).T"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9unVSh6NINkP",
        "colab_type": "text"
      },
      "source": [
        "**Principal Component Analysis for Optimized Processing**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jSheEJ9fibF1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def pca(X_train, X_test, n):\n",
        "    pca_trans = PCA(n_components=n).fit(X_train)\n",
        "    return pca_trans.transform(X_train), pca_trans.transform(X_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lraJ7zHjyt2e",
        "colab_type": "text"
      },
      "source": [
        "**Analyzing Various Machine Learning Algorithms**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5tBV5voEiekV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.ensemble import GradientBoostingClassifier\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.naive_bayes import GaussianNB"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qk0nPc6Dy4af",
        "colab_type": "text"
      },
      "source": [
        "**Decision Tree Classifier**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZzNuEACjiiA2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def dt(criterion=\"entropy\", max_features=\"sqrt\"):\n",
        "    return DecisionTreeClassifier(criterion=criterion, max_features=max_features, max_depth=None, random_state=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4hazypO0zCwP",
        "colab_type": "text"
      },
      "source": [
        "**Random Forest Classification**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CMrTs8KoilXu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def rf(n_tree=100, max_features=\"sqrt\"):\n",
        "    return RandomForestClassifier(n_estimators=n_tree, max_features=max_features, min_samples_split=2, \\\n",
        "                                  max_depth=None, bootstrap=True, oob_score=False, random_state=0, n_jobs=4)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kjvr4aCJzMc8",
        "colab_type": "text"
      },
      "source": [
        "**Gradient Boosting Classification**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jSrFiKg6inw1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def gbdt(n_tree=100, max_features=\"sqrt\"):\n",
        "    return GradientBoostingClassifier(n_estimators=n_tree, learning_rate=0.005, \\\n",
        "                                      max_features=max_features, max_depth=None, random_state=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JIPB4ULRzYLG",
        "colab_type": "text"
      },
      "source": [
        "**Logistic Regression**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TObK0n1oiqwt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def logit(C=1.0):\n",
        "    return LogisticRegression(C=1.0, solver=\"lbfgs\", max_iter=1000, random_state=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BDOcVlbZ0DyH",
        "colab_type": "text"
      },
      "source": [
        "**Multi-layer Perceptron Classifier**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qNUrR6QFitGu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def mlp(hidden=(100), act=\"logistic\", batch=32):\n",
        "    return MLPClassifier(hidden_layer_sizes=hidden, activation=act, solver=\"sgd\", batch_size=batch, \\\n",
        "                         learning_rate=\"constant\", learning_rate_init=0.1, early_stopping=False, max_iter=1000, random_state=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JQ53ALJvCbLP",
        "colab_type": "text"
      },
      "source": [
        "**Support Vector Machine Classifier**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pyQsAp1V0Pcf",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://drive.google.com/uc?export=view&id=1ABtElBwMDn1dmSOenCGok6i5rhsaq4Ql)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bEAsvZ5iiwjk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def svm(C=1.0, kernel=\"rbf\"):\n",
        "    return SVC(C=C, kernel=kernel, max_iter=-1, random_state=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zXmNhiwe0WHW",
        "colab_type": "text"
      },
      "source": [
        "**K Nearest Neighbours Classification**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pyFO4JwVi15N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def knn(n_neighbors=10, weights=\"distance\"):\n",
        "    return KNeighborsClassifier(n_neighbors=n_neighbors, weights=weights, algorithm=\"auto\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q6vUw42D0h9s",
        "colab_type": "text"
      },
      "source": [
        "**Naive Bayes Classification**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rW1Em8c6i4R8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def bayes():\n",
        "    return GaussianNB()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Am6qP2aN0rd0",
        "colab_type": "text"
      },
      "source": [
        "**Trainer Fuction**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WdX5EV74i7Qd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def train(X, y, classifier):\n",
        "    return classifier.fit(X, y)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Df-y3hxS0vhm",
        "colab_type": "text"
      },
      "source": [
        "**Testing Function**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rfdBkPVKi9nk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def test(X, classifier):\n",
        "    return classifier.predict(X)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LhwEiL5m02J1",
        "colab_type": "text"
      },
      "source": [
        "**Accuracy Function**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MWUJS3YMi_wE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def acc(X, y, classifier):\n",
        "    return classifier.score(X, y)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JziS0Uol1Au8",
        "colab_type": "text"
      },
      "source": [
        "**Implementation of Algorithms real time in order to find the most optimized solution**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cTUW-zbFjCaN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import matplotlib.cm as cm"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wgISV6Bf1La1",
        "colab_type": "text"
      },
      "source": [
        "*Let's load the processed M-STAR dataset*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qbm2lAWajFQs",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 561
        },
        "outputId": "d57895d1-0867-402f-a57a-83988bbfe223"
      },
      "source": [
        "print(\"loading ... \")\n",
        "X_train, y_train = get_mstar_data(\"train\", 128, 128, 96)\n",
        "X_test, y_test = get_mstar_data(\"test\", 128, 128, 96)\n",
        "X_train = np.reshape(X_train, [X_train.shape[0], X_train.shape[1] * X_train.shape[2]])\n",
        "X_test = np.reshape(X_test, [X_test.shape[0], X_test.shape[1] * X_test.shape[2]])\n",
        "print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "loading ... \n",
            "------ train ------\n",
            "2S1 299\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:14: DeprecationWarning: `imread` is deprecated!\n",
            "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
            "Use ``imageio.imread`` instead.\n",
            "  \n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:14: DeprecationWarning: `imresize` is deprecated!\n",
            "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.3.0.\n",
            "Use Pillow instead: ``numpy.array(Image.fromarray(arr).resize())``.\n",
            "  \n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "BMP2 233\n",
            "BRDM_2 298\n",
            "BTR60 256\n",
            "BTR70 233\n",
            "D7 299\n",
            "T62 298\n",
            "T72 232\n",
            "ZIL131 299\n",
            "ZSU_23_4 299\n",
            "------ test ------\n",
            "2S1 274\n",
            "BMP2 195\n",
            "BRDM_2 274\n",
            "BTR60 195\n",
            "BTR70 196\n",
            "D7 274\n",
            "T62 273\n",
            "T72 196\n",
            "ZIL131 274\n",
            "ZSU_23_4 274\n",
            "(2746, 9216) (2746,) (2425, 9216) (2425,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qEWH9fBX1ZLM",
        "colab_type": "text"
      },
      "source": [
        "*Shuffling the loaded dataset to eliminate any bias*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DeWdKdt9jIi9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "0b09c09b-6bc7-4b45-fe18-603ca8e997e0"
      },
      "source": [
        "print(\"shuffling ... \")\n",
        "X_train, y_train = data_shuffle(X_train, y_train)\n",
        "X_test, y_test = data_shuffle(X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "shuffling ... \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mDp06Ipi1hZn",
        "colab_type": "text"
      },
      "source": [
        "*Preprocessing the dataset*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gem5U7RoxFv_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "95568cc5-599b-4f9e-fb8f-e126ecea474b"
      },
      "source": [
        "print(\"preprocessing ...\")\n",
        "X_train = X_train / 255.0\n",
        "X_test = X_test / 255.0\n",
        "X_train = mean_wise(X_train)\n",
        "X_test = mean_wise(X_test)\n",
        "X_train, X_test = pca(X_train, X_test, 80)\n",
        "# y_train, y_test = data.one_hot(y_train, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "preprocessing ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "owaJixtu10Iu",
        "colab_type": "text"
      },
      "source": [
        "*Finally Training the model*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R0yEQFMpxOTz",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "2bd97ed2-88d5-4652-ce7c-9b9bb9c2a35f"
      },
      "source": [
        "print(\"training ...\")\n",
        "# classifier = train(X_train, y_train, dt(\"entropy\", 0.8)) # 70.68%\n",
        "# classifier = train(X_train, y_train, rf(1000, \"sqrt\")) # 96.49%\n",
        "# classifier = train(X_train, y_train, gbdt(1000, \"sqrt\")) # 95.17%\n",
        "# classifier = train(X_train, y_train, logit(1.0))    # 90.14%\n",
        "# classifier = train(X_train, y_train, mlp(1000, \"logistic\"))   # 93.36%\n",
        "# classifier = train(X_train, y_train, svm(1.0, \"rbf\"))     # 96.82%\n",
        "# classifier = train(X_train, y_train, knn(10, \"uniform\"))   # 95.34%\n",
        "classifier = train(X_train, y_train, svm())   # 97.81%"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "training ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E3V5CtQS2DM2",
        "colab_type": "text"
      },
      "source": [
        "*As seen while analyzing the above algorithms, Support Vector machine was able to provide higest accuracy for determining the vehicle images from the test data, that is 97.8% accuracy.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BSZGPQRoxj4h",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "90763604-c30c-48ea-849f-081bcb2a52e7"
      },
      "source": [
        "print(\"testing ...\")\n",
        "print(acc(X_train, y_train, classifier))\n",
        "print(acc(X_test, y_test, classifier))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "testing ...\n",
            "0.9996358339402768\n",
            "0.9781443298969072\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QmjN0SNS3LMA",
        "colab_type": "text"
      },
      "source": [
        "**The END**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j0Rsh1NhIeem",
        "colab_type": "text"
      },
      "source": [
        "**Thank You**"
      ]
    }
  ]
}