nanoGPT in MATLAB

This is a MATLAB® adaptation of Andrej Karpathy's online lecture "Let's build GPT: from scratch, in code, spelled out".

This demo builds and trains a generative pretrained transformer (GPT) model to generate text based on a training data set of Shakespeare's sonnets. The model is built up step-by-step, starting from a completely random model and iterating until it generates recognizable Shakespearean text.

Setup

Requirements

• MATLAB® 2023b or later
• Deep Learning Toolbox™
• Text Analytics Toolbox™

Running the demo on a GPU also requires Parallel Computing Toolbox™ and supported hardware.

Clone Repository

To run this demo with a local installation of MATLAB, first clone the repository.

1. In the system command prompt, run:

git clone https://github.com/matlab-deep-learning/nanogpt-in-matlab.git

2. Open MATLAB and navigate to the directory where you cloned the repository.
3. Add the directory to the MATLAB path.

addpath("path/to/nanogpt-in-matlab");

Run Demo

To run the full demo, open the live script BuildNanoGPT.mlx in the MATLAB Editor and click the Run button in the MATLAB Toolstrip. The script builds and trains a series of models increasing in size, complexity, and performance.