|
| 1 | +{ |
| 2 | + "cells": [ |
| 3 | + { |
| 4 | + "cell_type": "markdown", |
| 5 | + "id": "31b417bd-68d5-4d05-8e14-8ce600651809", |
| 6 | + "metadata": {}, |
| 7 | + "source": [ |
| 8 | + "### How to write a Neural Network in Tensorflow from scratch (without using Keras)\n", |
| 9 | + "\n", |
| 10 | + "[Source](https://medium.com/analytics-vidhya/how-to-write-a-neural-network-in-tensorflow-from-scratch-without-using-keras-e056bb143d78)" |
| 11 | + ] |
| 12 | + }, |
| 13 | + { |
| 14 | + "cell_type": "markdown", |
| 15 | + "id": "4ecf9aed-a501-453b-a821-3a994a82eefb", |
| 16 | + "metadata": {}, |
| 17 | + "source": [ |
| 18 | + "Introduction\n", |
| 19 | + "\n", |
| 20 | + "Writing a simple feedforward neural network is probably the first step in your journey towards mastering deep learning. Today, there are countless libraries and frameworks available to develop a machine learning model. The two most famous and go-to frameworks for developing a machine learning model are Tensorflow and PyTorch. Keras has been integrated into Tensorflow2.0 which undoubtedly makes it easier to write any machine learning code with Tensorflow as the backbone. However, if you are a beginner in the field, you probably want to write your code from scratch in order to understand the ins and outs of a model. I searched the internet for sources that would help me in writing a basic code for a simple feed-forward neural network in TensorFlow but couldn’t find any. Therefore, I thought of writing one myself. Following is the way to get it done!\n", |
| 21 | + "Why not use Keras?\n", |
| 22 | + "\n", |
| 23 | + "\n", |
| 24 | + "\n", |
| 25 | + "\n", |
| 26 | + "\n", |
| 27 | + "1. Learning difference between a keras model and a basic model\n", |
| 28 | + "\n", |
| 29 | + "Keras is a very simple library that helps you write machine learning code in python without any hassle. It is quick, easy, and clean! However, under the hood, Keras does a lot of work to provide you this seamless experience. If you look at the backend code of functions like model.fit() or model.train() on Tensorflow’s github repository, you would find that they contain numerous optimizations like warm start and optimized implementation of functions on hardware level. These tricks definitely make your code run faster but that may not always mean your model is learning the way it should learn. For comparison, if you write a simple 2–3 layered neural network code both using keras and without, you would find a major difference between the loss curves of both the programs. The loss curve for a basic code written without using keras would indicate loss values falling from a high number to a low number whereas, the loss value for keras code would begin with a low value in the first place, and unlike the former coder, wouldn’t significantly fall to a lower value. This does not mean that the keras model does not learn but it just means that the magnitude of learning occurring in a basic code is more than that in a keras code. Optimizing keras code is like peeling a strand of hair.\n", |
| 30 | + "\n", |
| 31 | + "2. Better control over the execution of architecture\n", |
| 32 | + "Sometimes, your architecture may not be a simple sequential neural network. It may contain residual/skip connections or multiple sub neural networks. In such a case, you need more control over the execution of your architecture. Having a custom code definitely proves handy for such use cases.\n", |
| 33 | + "\n", |
| 34 | + "Some other sources for the comparison:\n", |
| 35 | + "1. https://www.javatpoint.com/tensorflow-vs-keras\n", |
| 36 | + "2. https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/" |
| 37 | + ] |
| 38 | + }, |
| 39 | + { |
| 40 | + "cell_type": "markdown", |
| 41 | + "id": "954a6b3f-d23d-4ce0-a410-63a2d36ff96a", |
| 42 | + "metadata": {}, |
| 43 | + "source": [ |
| 44 | + "Following is an example of a simple feed forward neural network containing 2 hidden layers that learn to predict mnist digits using gradient descent optimization.\n", |
| 45 | + "\n", |
| 46 | + "" |
| 47 | + ] |
| 48 | + }, |
| 49 | + { |
| 50 | + "cell_type": "code", |
| 51 | + "execution_count": 2, |
| 52 | + "id": "44e1bf52-adb9-4689-8cd4-7a1d95ed833e", |
| 53 | + "metadata": {}, |
| 54 | + "outputs": [], |
| 55 | + "source": [ |
| 56 | + "class Network(object):\n", |
| 57 | + " '''\n", |
| 58 | + " This class contains code for a simple feed forward neural network\n", |
| 59 | + " '''\n", |
| 60 | + " def __init__(self, n_layers):\n", |
| 61 | + " '''\n", |
| 62 | + " constructor\n", |
| 63 | + " :param n_layers: number of nodes in each layer of the network\n", |
| 64 | + " '''\n", |
| 65 | + " # store the parameters of network\n", |
| 66 | + " self.params = []\n", |
| 67 | + "\n", |
| 68 | + " # Declare layer-wise weights and biases\n", |
| 69 | + " self.W1 = tf.Variable(tf.random.normal([n_layers[0], n_layers[1]], stddev=0.1), name='W1')\n", |
| 70 | + " \n", |
| 71 | + " # self.b1 = tf.Variable(tf.random.normal([n_layers[1]], mean=0.0, stddev=0.1, dtype=tf.dtypes.float32, seed=0), name='b1')\n", |
| 72 | + " self.b1 = tf.Variable(tf.zeros([1, n_layers[1]]))\n", |
| 73 | + " self.W2 = tf.Variable(tf.random.normal([n_layers[1], n_layers[2]], stddev=0.1),name='W2')\n", |
| 74 | + " # self.b2 = tf.Variable(tf.random.normal([n_layers[2]], mean=0.0, stddev=0.1, dtype=tf.dtypes.float32, seed=0), name='b2')\n", |
| 75 | + " self.b2 = tf.Variable(tf.zeros([1, n_layers[2]]))\n", |
| 76 | + " self.W3 = tf.Variable(tf.random.normal([n_layers[2], n_layers[3]],stddev=0.1), name='W3')\n", |
| 77 | + " # self.b3 = tf.Variable(tf.random.normal([n_layers[3]], mean=0.0, stddev=0.1, dtype=tf.dtypes.float32, seed=0), name='b3')\n", |
| 78 | + " self.b3 = tf.Variable(tf.zeros([1, n_layers[3]]))\n", |
| 79 | + " # Collect all initialized weights and biases in self.params\n", |
| 80 | + " self.params = [self.W1, self.b1, self.W2, self.b2, self.W3, self.b3]" |
| 81 | + ] |
| 82 | + }, |
| 83 | + { |
| 84 | + "cell_type": "markdown", |
| 85 | + "id": "097c9c5f-24aa-4f6b-a0d4-d30e4e3d4a86", |
| 86 | + "metadata": {}, |
| 87 | + "source": [ |
| 88 | + "In the above code snippet, I create a class for a network and initialize it. n_layers contains number of nodes per layer in the network\n", |
| 89 | + "Each weight and bias matrix is initialized using tf.Variable() function\n", |
| 90 | + "We save all the parameters in self.params for further use" |
| 91 | + ] |
| 92 | + }, |
| 93 | + { |
| 94 | + "cell_type": "code", |
| 95 | + "execution_count": null, |
| 96 | + "id": "11a63ef3-266a-4e6f-a3ec-5e0ec52d82fa", |
| 97 | + "metadata": {}, |
| 98 | + "outputs": [], |
| 99 | + "source": [] |
| 100 | + }, |
| 101 | + { |
| 102 | + "cell_type": "code", |
| 103 | + "execution_count": null, |
| 104 | + "id": "6087c8e5-c139-4808-9bbb-80c10cf58c6b", |
| 105 | + "metadata": {}, |
| 106 | + "outputs": [], |
| 107 | + "source": [] |
| 108 | + }, |
| 109 | + { |
| 110 | + "cell_type": "code", |
| 111 | + "execution_count": null, |
| 112 | + "id": "7331668a-27fc-4b00-b068-4a9c2e86978e", |
| 113 | + "metadata": {}, |
| 114 | + "outputs": [], |
| 115 | + "source": [] |
| 116 | + }, |
| 117 | + { |
| 118 | + "cell_type": "code", |
| 119 | + "execution_count": null, |
| 120 | + "id": "4192ce5d-193a-4068-9b9f-abbef45acccd", |
| 121 | + "metadata": {}, |
| 122 | + "outputs": [], |
| 123 | + "source": [] |
| 124 | + } |
| 125 | + ], |
| 126 | + "metadata": { |
| 127 | + "kernelspec": { |
| 128 | + "display_name": "Python 3 (ipykernel)", |
| 129 | + "language": "python", |
| 130 | + "name": "python3" |
| 131 | + }, |
| 132 | + "language_info": { |
| 133 | + "codemirror_mode": { |
| 134 | + "name": "ipython", |
| 135 | + "version": 3 |
| 136 | + }, |
| 137 | + "file_extension": ".py", |
| 138 | + "mimetype": "text/x-python", |
| 139 | + "name": "python", |
| 140 | + "nbconvert_exporter": "python", |
| 141 | + "pygments_lexer": "ipython3", |
| 142 | + "version": "3.8.10" |
| 143 | + } |
| 144 | + }, |
| 145 | + "nbformat": 4, |
| 146 | + "nbformat_minor": 5 |
| 147 | +} |
0 commit comments