Tensorflow_test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rX8mhOLljYeM"
   },
   "source": [
    "##### Copyright 2019 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "id": "BZSlp3DAjdYf",
    "tags": []
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3wF5wszaj97Y"
   },
   "source": [
    "# TensorFlow 2 quickstart for beginners"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DUNzJc4jTj6G"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/quickstart/beginner\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/beginner.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/quickstart/beginner.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/quickstart/beginner.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "04QgGZc9bF5D"
   },
   "source": [
    "This short introduction uses [Keras](https://www.tensorflow.org/guide/keras/overview) to:\n",
    "\n",
    "1. Build a neural network that classifies images.\n",
    "2. Train this neural network.\n",
    "3. And, finally, evaluate the accuracy of the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hiH7AC-NTniF"
   },
   "source": [
    "This is a [Google Colaboratory](https://colab.research.google.com/notebooks/welcome.ipynb) notebook file. Python programs are run directly in the browser—a great way to learn and use TensorFlow. To follow this tutorial, run the notebook in Google Colab by clicking the button at the top of this page.\n",
    "\n",
    "1. In Colab, connect to a Python runtime: At the top-right of the menu bar, select *CONNECT*.\n",
    "2. Run all the notebook code cells: Select *Runtime* > *Run all*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nnrWf3PCEzXL"
   },
   "source": [
    "Download and install TensorFlow 2. Import TensorFlow into your program:\n",
    "\n",
    "Note: Upgrade `pip` to install the TensorFlow 2 package. See the [install guide](https://www.tensorflow.org/install) for details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "0trJmd6DjqBZ",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-09-23 12:29:19.511678: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:19.512311: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:19.523226: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:19.523828: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:19.524393: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:19.524963: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from S"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU'),\n",
       " PhysicalDevice(name='/physical_device:GPU:1', device_type='GPU')]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "ysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.config.list_physical_devices('GPU')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show the available GPUs on the system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num GPUs Available:  2\n"
     ]
    }
   ],
   "source": [
    "print(\"Num GPUs Available: \", len(tf.config.list_physical_devices('GPU')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7NAbSZiaoJ4z"
   },
   "source": [
    "Load and prepare the [MNIST dataset](http://yann.lecun.com/exdb/mnist/). Convert the samples from integers to floating-point numbers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "7FP5258xjs-v",
    "tags": []
   },
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BPZ68wASog_I"
   },
   "source": [
    "Build the `tf.keras.Sequential` model by stacking layers. Choose an optimizer and loss function for training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "h3IKyzTCDNGo",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-09-23 12:29:37.349180: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2021-09-23 12:29:37.538379: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:37.538947: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:37.539478: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:37.540019: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:37.540532: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:37.541055: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.208625: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.209312: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.209875: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.210403: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.210916: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.211503: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14794 MB memory:  -> device: 0, name: Quadro RTX 5000, pci bus id: 0000:04:00.0, compute capability: 7.5\n",
      "2021-09-23 12:29:38.212026: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-09-23 12:29:38.212533: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14330 MB memory:  -> device: 1, name: Quadro RTX 5000, pci bus id: 0000:2b:00.0, compute capability: 7.5\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dropout(0.2),\n",
    "  tf.keras.layers.Dense(10)\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l2hiez2eIUz8"
   },
   "source": [
    "For each example the model returns a vector of \"[logits](https://developers.google.com/machine-learning/glossary#logits)\" or \"[log-odds](https://developers.google.com/machine-learning/glossary#log-odds)\" scores, one for each class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "OeOrNdnkEEcR",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.54824024, -0.03014868, -0.47142434, -0.1944471 ,  0.05341867,\n",
       "         0.14791009,  0.38350242,  0.06496009, -0.69080716,  0.6523241 ]],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions = model(x_train[:1]).numpy()\n",
    "predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tgjhDQGcIniO"
   },
   "source": [
    "The `tf.nn.softmax` function converts these logits to \"probabilities\" for each class: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "zWSRnQ0WI5eq",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.05685479, 0.095449  , 0.06139427, 0.08098741, 0.10376818,\n",
       "        0.11405158, 0.14435034, 0.10497277, 0.04930047, 0.18887112]],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.nn.softmax(predictions).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "he5u_okAYS4a"
   },
   "source": [
    "Note: It is possible to bake this `tf.nn.softmax` in as the activation function for the last layer of the network. While this can make the model output more directly interpretable, this approach is discouraged as it's impossible to\n",
    "provide an exact and numerically stable loss calculation for all models when using a softmax output. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hQyugpgRIyrA"
   },
   "source": [
    "The `losses.SparseCategoricalCrossentropy` loss takes a vector of logits and a `True` index and returns a scalar loss for each example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "RSkzdv8MD0tT",
    "tags": []
   },
   "outputs": [],
   "source": [
    "loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SfR4MsSDU880"
   },
   "source": [
    "This loss is equal to the negative log probability of the true class:\n",
    "It is zero if the model is sure of the correct class.\n",
    "\n",
    "This untrained model gives probabilities close to random (1/10 for each class), so the initial loss should be close to `-tf.math.log(1/10) ~= 2.3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "NJWqEVrrJ7ZB",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.1711044"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss_fn(y_train[:1], predictions).numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "9foNKHzTD2Vo",
    "tags": []
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss=loss_fn,\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ix4mEL65on-w"
   },
   "source": [
    "The `Model.fit` method adjusts the model parameters to minimize the loss: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "y7suUbJXVLqP",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-09-23 13:17:11.971180: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 188160000 exceeds 10% of free system memory.\n",
      "2021-09-23 13:17:12.126452: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 188160000 exceeds 10% of free system memory.\n",
      "2021-09-23 13:17:12.192937: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 2s 851us/step - loss: 0.2878 - accuracy: 0.9154\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 2s 833us/step - loss: 0.1402 - accuracy: 0.9573\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 2s 836us/step - loss: 0.1040 - accuracy: 0.9680\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 2s 834us/step - loss: 0.0867 - accuracy: 0.9729\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 2s 835us/step - loss: 0.0732 - accuracy: 0.9769\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f884c17bb50>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4mDAAPFqVVgn"
   },
   "source": [
    "The `Model.evaluate` method checks the models performance, usually on a \"[Validation-set](https://developers.google.com/machine-learning/glossary#validation-set)\" or \"[Test-set](https://developers.google.com/machine-learning/glossary#test-set)\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "F7dTAzgHDUh7",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-09-23 13:17:25.711881: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 31360000 exceeds 10% of free system memory.\n",
      "2021-09-23 13:17:25.741185: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 31360000 exceeds 10% of free system memory.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 - 0s - loss: 0.0685 - accuracy: 0.9796\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.06854204833507538, 0.9796000123023987]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(x_test,  y_test, verbose=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "T4JfEh7kvx6m"
   },
   "source": [
    "The image classifier is now trained to ~98% accuracy on this dataset. To learn more, read the [TensorFlow tutorials](https://www.tensorflow.org/tutorials/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Aj8NrlzlJqDG"
   },
   "source": [
    "If you want your model to return a probability, you can wrap the trained model, and attach the softmax to it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "rYb6DrEH0GMv",
    "tags": []
   },
   "outputs": [],
   "source": [
    "probability_model = tf.keras.Sequential([\n",
    "  model,\n",
    "  tf.keras.layers.Softmax()\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "cnqOZtUp1YR_",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5, 10), dtype=float32, numpy=\n",
       "array([[3.1136963e-08, 3.3622932e-08, 1.7763908e-06, 7.3147668e-05,\n",
       "        5.4277052e-12, 3.2857608e-07, 2.0773150e-13, 9.9992168e-01,\n",
       "        7.1082738e-08, 2.9021983e-06],\n",
       "       [3.8227927e-09, 1.5221661e-05, 9.9997282e-01, 1.1666591e-05,\n",
       "        3.6162206e-17, 1.6983063e-07, 6.9123978e-08, 1.6284683e-15,\n",
       "        1.2028497e-08, 1.6690890e-17],\n",
       "       [9.2493849e-07, 9.9919242e-01, 3.4038920e-04, 6.2446065e-06,\n",
       "        4.2024940e-06, 1.3552634e-06, 6.7675792e-06, 3.1956288e-04,\n",
       "        1.2695888e-04, 1.1590469e-06],\n",
       "       [9.9965072e-01, 3.6739158e-07, 1.8961815e-05, 6.1372680e-07,\n",
       "        6.0094260e-07, 4.7552615e-07, 2.7517290e-04, 4.2670308e-05,\n",
       "        2.3375812e-08, 1.0506049e-05],\n",
       "       [5.7881666e-06, 1.0873231e-07, 4.6472309e-07, 1.5747602e-07,\n",
       "        9.9610823e-01, 9.9019610e-07, 1.0502272e-06, 4.4568387e-05,\n",
       "        9.0217804e-07, 3.8376499e-03]], dtype=float32)>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probability_model(x_test[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run in tensorman: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (2398047115.py, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"/tmp/ipykernel_32334/2398047115.py\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    tensorman run --gpu python -- ./tensorflow_demo.py\u001b[0m\n\u001b[0m              ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "#tensorman run --gpu python -- ./tensorflow_demo.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "rX8mhOLljYeM"
   ],
   "name": "beginner.ipynb",
   "toc_visible": true
  },
  "interpreter": {
   "hash": "b442af2a33c9ecef0299ec5e034f160ddbbf1400865b329d6c1f962a4a783493"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}