The goal of this series is absolute educational transparency. Python (PyTorch/TensorFlow) abstracts away memory management, pointer alignment, and gradient topological sorting. C++ and Rust offer high-level STL/Collections that can hide the actual algorithmic cost of allocations. By using C99, every single byte of RAM allocated for a matrix multiplication or a neural network parameter must be explicitly tracked and released, forcing the developer to deeply understand the hardware reality of deep learning.
Through rigorous structural contracts. Every module that allocates memory has a corresponding _free method that explicitly nullifies internal pointers before freeing the struct itself. Additionally, every one of the 17 repositories enforces a strict Valgrind memory-leak test gate. If a single byte is lost during an epoch of training, the CI pipeline fails.
This series is titled Transformer from Scratch. Linking a pre-compiled FORTRAN library defeats the purpose of learning how a dot product physically executes in the CPU cache. By writing the triple-nested loops manually, we expose exactly where the algorithmic bottlenecks lie and provide FUTURE tags for loop tiling and SIMD optimization.
No. This implementation is purely CPU-bound. However, the architectural design (specifically separating the Autograd engine from the mini-tensor layer) means that integrating CUDA would only require swapping out the bottom layer (Layer 1). The entire neural network and Transformer stack above it would theoretically not need to change.
Yes. Project 17 (machine-translation) implements a complete end-to-end Neural Machine Translation pipeline. If fed a sufficient parallel corpus (e.g., WMT14 EN-FR) and allowed to train, it will output functionally translated strings using Greedy Decoding.