Skip to content

Commit 2b3d91b

Browse files
mhjensenmhjensen
committed
update week 2
1 parent 7f09c59 commit 2b3d91b

17 files changed

+971
-180
lines changed

‎doc/pub/week2/html/week2-bs.html‎

Lines changed: 126 additions & 3 deletions
Large diffs are not rendered by default.

‎doc/pub/week2/html/week2-reveal.html‎

Lines changed: 121 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -215,8 +215,8 @@ <h2 id="overview-of-second-week-january-27-31">Overview of second week, January
215215
<h2 id="videos-on-neural-networks">Videos on Neural Networks </h2>
216216

217217
<ul>
218-
<p><li> <a href="https://www.youtube.com/watch?v=CqOfi41LfDw" target="_blank">Video on Neural Networks</a></li>
219-
<p><li> <a href="https://www.youtube.com/watch?v=Ilg3gGewQ5U" target="_blank">Video on the back propagation algorithm</a></li>
218+
<p><li> Video on Neural Networks at <a href="https://www.youtube.com/watch?v=CqOfi41LfDw" target="_blank"><tt>https://www.youtube.com/watch?v=CqOfi41LfDw</tt></a></li>
219+
<p><li> Video on the back propagation algorithm at <a href="https://www.youtube.com/watch?v=Ilg3gGewQ5U" target="_blank"><tt>https://www.youtube.com/watch?v=Ilg3gGewQ5U</tt></a></li>
220220
</ul>
221221
</section>
222222

@@ -2831,6 +2831,125 @@ <h2 id="using-tensorflow-collect-and-pre-process-data">Using TensorFlow: Collect
28312831
</div>
28322832
</section>
28332833

2834+
<section>
2835+
<h2 id="and-a-similar-code-using-pytorch">And a similar code using PyTorch </h2>
2836+
<p>See also Rashkca et al., chapter 11, jupyter-notebook sent separately, from <a href="https://github.com/rasbt/machine-learning-book" target="_blank">GitHub</a></p>
2837+
2838+
2839+
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
2840+
<div class="cell border-box-sizing code_cell rendered">
2841+
<div class="input">
2842+
<div class="inner_cell">
2843+
<div class="input_area">
2844+
<div class="highlight" style="background: #eeeedd">
2845+
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #228B22"># Simple NN code using PyTorch on the MNIST dataset (this time the 28 x 28 set)</span>
2846+
<span style="color: #228B22"># The MNIST dataset is loaded using `torchvision.datasets`. The images are transformed to tensors and normalized.</span>
2847+
<span style="color: #228B22"># A simple feedforward neural network with one hidden layer is defined using `nn.Module`.</span>
2848+
<span style="color: #228B22"># The model is trained using the Adam optimizer and CrossEntropyLoss. The training loop iterates over the dataset for a specified number of epochs.</span>
2849+
<span style="color: #228B22"># Note that we don&#39;t include additional hyperparameters and the learning rate is set to 0.001. </span>
2850+
<span style="color: #228B22"># After training, the model is evaluated on the test dataset to compute accuracy.</span>
2851+
<span style="color: #228B22"># The trained model&#39;s weights are saved to a file for later use.</span>
2852+
<span style="color: #228B22"># To do: add loops over hyperparameters and learning rates</span>
2853+
2854+
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">torch</span>
2855+
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">torch.nn</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">nn</span>
2856+
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">torch.optim</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">optim</span>
2857+
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">torch.utils.data</span> <span style="color: #8B008B; font-weight: bold">import</span> DataLoader
2858+
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">torchvision.transforms</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">transforms</span>
2859+
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">torchvision.datasets</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">datasets</span>
2860+
2861+
<span style="color: #228B22"># Hyperparameters</span>
2862+
input_size = <span style="color: #B452CD">784</span> <span style="color: #228B22"># 28x28 images</span>
2863+
hidden_size = <span style="color: #B452CD">128</span>
2864+
num_classes = <span style="color: #B452CD">10</span>
2865+
num_epochs = <span style="color: #B452CD">5</span>
2866+
batch_size = <span style="color: #B452CD">64</span>
2867+
learning_rate = <span style="color: #B452CD">0.001</span>
2868+
2869+
<span style="color: #228B22"># Device configuration</span>
2870+
device = torch.device(<span style="color: #CD5555">&#39;cuda&#39;</span> <span style="color: #8B008B; font-weight: bold">if</span> torch.cuda.is_available() <span style="color: #8B008B; font-weight: bold">else</span> <span style="color: #CD5555">&#39;cpu&#39;</span>)
2871+
2872+
<span style="color: #228B22"># MNIST dataset</span>
2873+
train_dataset = datasets.MNIST(root=<span style="color: #CD5555">&#39;./data&#39;</span>, train=<span style="color: #8B008B; font-weight: bold">True</span>, transform=transforms.ToTensor(), download=<span style="color: #8B008B; font-weight: bold">True</span>)
2874+
test_dataset = datasets.MNIST(root=<span style="color: #CD5555">&#39;./data&#39;</span>, train=<span style="color: #8B008B; font-weight: bold">False</span>, transform=transforms.ToTensor())
2875+
2876+
<span style="color: #228B22"># Data loader</span>
2877+
train_loader = DataLoader(dataset=train_dataset, batch_size=batch_size, shuffle=<span style="color: #8B008B; font-weight: bold">True</span>)
2878+
test_loader = DataLoader(dataset=test_dataset, batch_size=batch_size, shuffle=<span style="color: #8B008B; font-weight: bold">False</span>)
2879+
2880+
<span style="color: #228B22"># Fully connected neural network with one hidden layer</span>
2881+
<span style="color: #8B008B; font-weight: bold">class</span> <span style="color: #008b45; font-weight: bold">NeuralNet</span>(nn.Module):
2882+
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">__init__</span>(<span style="color: #658b00">self</span>, input_size, hidden_size, num_classes):
2883+
<span style="color: #658b00">super</span>(NeuralNet, <span style="color: #658b00">self</span>).<span style="color: #008b45">__init__</span>()
2884+
<span style="color: #658b00">self</span>.fc1 = nn.Linear(input_size, hidden_size)
2885+
<span style="color: #658b00">self</span>.relu = nn.ReLU()
2886+
<span style="color: #658b00">self</span>.fc2 = nn.Linear(hidden_size, num_classes)
2887+
2888+
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">forward</span>(<span style="color: #658b00">self</span>, x):
2889+
out = <span style="color: #658b00">self</span>.fc1(x)
2890+
out = <span style="color: #658b00">self</span>.relu(out)
2891+
out = <span style="color: #658b00">self</span>.fc2(out)
2892+
<span style="color: #8B008B; font-weight: bold">return</span> out
2893+
2894+
model = NeuralNet(input_size, hidden_size, num_classes).to(device)
2895+
2896+
<span style="color: #228B22"># Loss and optimizer</span>
2897+
criterion = nn.CrossEntropyLoss()
2898+
optimizer = optim.Adam(model.parameters(), lr=learning_rate)
2899+
2900+
<span style="color: #228B22"># Train the model</span>
2901+
total_step = <span style="color: #658b00">len</span>(train_loader)
2902+
<span style="color: #8B008B; font-weight: bold">for</span> epoch <span style="color: #8B008B">in</span> <span style="color: #658b00">range</span>(num_epochs):
2903+
<span style="color: #8B008B; font-weight: bold">for</span> i, (images, labels) <span style="color: #8B008B">in</span> <span style="color: #658b00">enumerate</span>(train_loader):
2904+
<span style="color: #228B22"># Move tensors to the configured device</span>
2905+
images = images.reshape(-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">28</span>*<span style="color: #B452CD">28</span>).to(device)
2906+
labels = labels.to(device)
2907+
2908+
<span style="color: #228B22"># Forward pass</span>
2909+
outputs = model(images)
2910+
loss = criterion(outputs, labels)
2911+
2912+
<span style="color: #228B22"># Backward and optimize</span>
2913+
optimizer.zero_grad()
2914+
loss.backward()
2915+
optimizer.step()
2916+
2917+
<span style="color: #8B008B; font-weight: bold">if</span> (i+<span style="color: #B452CD">1</span>) % <span style="color: #B452CD">100</span> == <span style="color: #B452CD">0</span>:
2918+
<span style="color: #658b00">print</span>(<span style="color: #CD5555">f&#39;Epoch [{</span>epoch+<span style="color: #B452CD">1</span><span style="color: #CD5555">}/{</span>num_epochs<span style="color: #CD5555">}], Step [{</span>i+<span style="color: #B452CD">1</span><span style="color: #CD5555">}/{</span>total_step<span style="color: #CD5555">}], Loss: {</span>loss.item()<span style="color: #CD5555">:.4f}&#39;</span>)
2919+
2920+
<span style="color: #228B22"># Test the model</span>
2921+
<span style="color: #228B22"># In test phase, we don&#39;t need to compute gradients (for memory efficiency)</span>
2922+
<span style="color: #8B008B; font-weight: bold">with</span> torch.no_grad():
2923+
correct = <span style="color: #B452CD">0</span>
2924+
total = <span style="color: #B452CD">0</span>
2925+
<span style="color: #8B008B; font-weight: bold">for</span> images, labels <span style="color: #8B008B">in</span> test_loader:
2926+
images = images.reshape(-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">28</span>*<span style="color: #B452CD">28</span>).to(device)
2927+
labels = labels.to(device)
2928+
outputs = model(images)
2929+
_, predicted = torch.max(outputs.data, <span style="color: #B452CD">1</span>)
2930+
total += labels.size(<span style="color: #B452CD">0</span>)
2931+
correct += (predicted == labels).sum().item()
2932+
2933+
<span style="color: #658b00">print</span>(<span style="color: #CD5555">f&#39;Accuracy of the network on the 10000 test images: {</span><span style="color: #B452CD">100</span> * correct / total<span style="color: #CD5555">:.2f}%&#39;</span>)
2934+
2935+
<span style="color: #228B22"># Save the model checkpoint</span>
2936+
torch.save(model.state_dict(), <span style="color: #CD5555">&#39;model.ckpt&#39;</span>)
2937+
</pre>
2938+
</div>
2939+
</div>
2940+
</div>
2941+
</div>
2942+
<div class="output_wrapper">
2943+
<div class="output">
2944+
<div class="output_area">
2945+
<div class="output_subarea output_stream output_stdout output_text">
2946+
</div>
2947+
</div>
2948+
</div>
2949+
</div>
2950+
</div>
2951+
</section>
2952+
28342953

28352954

28362955
</div> <!-- class="slides" -->

0 commit comments

Comments
 (0)