|
| 1 | +## My-NN ⋮∶• Playground for neural networks |
| 2 | + |
| 3 | +A simple and self-descriptive implementation of neural networks with support |
| 4 | +of different topology, activation functions and optimization strategies. |
| 5 | + |
| 6 | +A useful tool if you are interested to understand the intuition of neural networks through experimentation. |
| 7 | + |
| 8 | +### Activation functions |
| 9 | + |
| 10 | +Library provides abstract interface for activation functions as long it is possible to provide a forward computation |
| 11 | +and a derivative estimation. The package is shipped with 3 implementations of activation functions: |
| 12 | + |
| 13 | +* `ReLU` |
| 14 | +* `Tanh` |
| 15 | +* `Sigmoid` |
| 16 | + |
| 17 | +### Optimization strategy |
| 18 | +The optimization strategy is a core component for any optimization problem. Depending the strategy |
| 19 | +the training can be faster, more effective. For this reason it is very beneficial to understand the mechanics behind |
| 20 | +each algorithm before choosing one. Sebastian Ruder has published [a very nice blog post](http://ruder.io/optimizing-gradient-descent/index.html#gradientdescentoptimizationalgorithms) |
| 21 | +where he presents in detail different optimization algorithms. |
| 22 | + |
| 23 | +MyNN is shipped with 3 different implementation of generic optimization strategies. |
| 24 | + |
| 25 | +* `GradientDescent`: Classic gradient descent that works in batches |
| 26 | +* `GradientDescentMomentum`: It works as the classic gradient descent but takes |
| 27 | +into account the previous grads in order to keep a momentum |
| 28 | +* `AdaptiveGradientDescent` (**Default**): A gradient descent that adapts learning rate per |
| 29 | +optimized parameter. |
| 30 | + |
| 31 | + |
| 32 | +### Installation |
| 33 | +MyNN can **only** work on `python >= 3.6`. It is proposed to use `virtualenv` to perform |
| 34 | +installation. |
| 35 | + |
| 36 | +```sh |
| 37 | +$ virtualenv -ppython3.6 venv |
| 38 | +$ source venv/bin/activate |
| 39 | +``` |
| 40 | + |
| 41 | +You can install the dependencies using `requirements.txt` file. |
| 42 | +```sh |
| 43 | +$ pip install -r requirements.txt |
| 44 | +``` |
0 commit comments