-## Introduction
+## 简介
-LLAISYS (Let's Learn AI SYStem) is an educational project that aims to provide a platform for new and future AI engineers to learn how to build AI systems from scratch. LLAISYS consists of several assignments, which help students learn and build the basic modules, and projects that challenge them to add more fancy features to their systems. LLAISYS uses C++ as primary programming language for system backend, and is compiled into shared libraries exposing C language APIs. Frontend codes are written in Python which calls these APIs to provide more convenient testing and interaction with other architectures such as PyTorch.
+LLAISYS（Let's Learn AI SYStem）是一个教育项目，旨在为新手和未来的AI工程师提供一个从零开始构建AI系统的学习平台。LLAISYS包含多个作业，帮助学生学习和构建基础模块；以及一些项目挑战，让他们为系统添加更多高级功能。LLAISYS使用C++作为系统后端的主要编程语言，并编译成共享库，提供C语言API。前端代码使用Python编写，调用这些API以提供更便捷的测试和与其他架构（如PyTorch）的交互。
-### Project Structure Overview
+### 项目结构概览
-- `\include`: directory that contains of the header files which defines all the C APIs exposed by the shared library. (Functions declarations start with `__export`)
+- `\include`：包含所有定义共享库提供的C API的头文件的目录。（函数声明以`__export`开头）
-- `\src`: C++ source files.
- - `\src\llaisys` contains all the direct implementation of waht are defined in the header files and follows the same directory structure as the `\include`. This is also as far as C++ codes can go.
- - other directories contain the actual implementaion of different modules.
+- `\src`：C++源文件。
+ - `\src\llaisys`包含头文件中定义的所有直接实现，并遵循与`\include`相同的目录结构。这也是C++代码的边界。
+ - 其他目录包含不同模块的实际实现。
-- `xmake.lua`: build rules for llaisys backend. `\xmake` directory contains the sub-xmake files for different devices. You may add `nvidia.lua` in the directory in the future for instance to support CUDA.
+- `xmake.lua`：llaisys后端的构建规则。`\xmake`目录包含不同设备的子xmake文件。例如，将来可以在目录中添加`nvidia.lua`来支持CUDA。
-- `\python`: Python source files.
- - `\python\llaisys\libllaisys` contains all the ctypes wrapper functions of llaisys APIs. It basically matches the structure of C header files.
- - `\python\llaisys` contains Python warppers of the ctypes functions to make the package more Python-like.
+- `\python`：Python源文件。
+ - `\python\llaisys\libllaisys`包含llaisys API的所有ctypes封装函数。它基本上与C头文件的结构相匹配。
+ - `\python\llaisys`包含ctypes函数的Python包装器，使包更符合Python风格。
-- `\test`: Python test files that import llaisys python package.
+- `\test`：导入llaisys python包的Python测试文件。
-## Assignment #0: Getting Started
+## 作业 #0：入门
-### Task-0.1 Install Prerequisites
+### 任务-0.1 安装必备组件
-- Compile Tool: [Xmake](https://xmake.io/)
-- C++ Compiler: MSVC (Windows) or Clang or GCC
-- Python >= 3.9 (PyTorch, Transformers, etc.)
-- Clang-Format-16 (Optional): for formatting C++ codes.
+- 编译工具：[Xmake](https://xmake.io/)
+- C++编译器：MSVC（Windows）或Clang或GCC
+- Python >= 3.9（PyTorch、Transformers等）
+- Clang-Format-16（可选）：用于格式化C++代码。
-### Task-0.2 Fork and Build LLAISYS
+### 任务-0.2 Fork并构建LLAISYS
-- FORK LLAISYS Repository and Clone it to your local machine. Both Windows and Linux are supported.
+- Fork LLAISYS仓库并克隆到本地机器。支持Windows和Linux。
-- Compile and Install
+- 编译和安装
```bash
- # compile c++ codes
+ # 编译c++代码
xmake
- # install llaisys shared library
+ # 安装llaisys共享库
xmake install
- # install llaisys python package
+ # 安装llaisys python包
pip install ./python/
```
-- Github Auto Tests
+- Github自动测试
- LLAISYS uses Github Actions to run automated tests on every push and pull request. You can see testing results on your repo page. All tests should pass once you have finished all assignment tasks.
+ LLAISYS使用Github Actions在每次推送和拉取请求时运行自动化测试。你可以在仓库页面上看到测试结果。完成所有作业任务后，所有测试都应该通过。
-### Task-0.3 Run LLAISYS for the First Time
+### 任务-0.3 首次运行LLAISYS
-- Run cpu runtime tests
+- 运行cpu运行时测试
```bash
python test/test_runtime.py --device cpu
```
- You should see the test passed.
+ 你应该看到测试通过。
-### Task-0.4 Download test model
+### 任务-0.4 下载测试模型
-- The model we use for assignments is [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
+- 我们用于作业的模型是[DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)。
-- Run an inference test with the model using PyTorch
+- 使用PyTorch运行模型推理测试
```bash
python test/test_infer.py --model [dir_path/to/model]
```
- You can see that PyTorch is able to load the model and perform inference with the sample input. You can debug into `transformers` library codes to see how what is going on behind. Right now, your code cannot do anything yet, but you are going to build a system that can achieve the same functionality in the assignments.
+ 你可以看到PyTorch能够加载模型并使用示例输入执行推理。你可以调试进入`transformers`库代码来深入查看并了解其内部运作原理。现在，你的代码还无法执行任何操作，但在后续的作业中，你将构建一个能够实现相同功能的系统。
-## Assignment #1: Tensor
+## 作业 #1：张量
-Tensor is a data structure that represents multi-dimensional data. It is the basic building block of LLAISYS, and most AI frameworks such as PyTorch. In this assignment, you will learn how to implement a basic tensor class.
+张量是表示多维数据的数据结构。它是LLAISYS和大多数AI框架（如PyTorch）的基本构建单元。在这个作业中，你将学习如何实现一个基本的张量类。
-A Tensor object has the following fields:
+张量对象具有以下字段：
-- `storage`: a shared pointer to a memory block that stores the tensor's data. It can be shared by multiple tensors. Check storage class for more details.
-- `offset`: the starting index (in bytes) of the tensor in the storage.
-- `meta`: metadata that describes the tensor's shape, data type, and strides.
+- `storage`：指向存储张量数据的内存块的共享指针。它可以被多个张量共享。有关更多详细信息，请查看storage类。
+- `offset`：张量在存储中的起始索引（以字节为单位）。
+- `meta`：描述张量形状、数据类型和步长的元数据。
-Implement the following functions defined in the `src/tensor/tensor.hpp`:
+实现`src/tensor/tensor.hpp`中定义的以下函数：
-### Task-1.1
+### 任务-1.1
```c++
void load(const void *src);
```
-Load host (cpu) data to the tensor (can be on device). Check contructor to see how to get runtime apis of the current device context, and do a memcpy from host to device.
+将主机（cpu）数据加载到张量（可以在设备上）。查看构造函数了解如何获取当前设备上下文的运行时API，并执行从主机到设备的内存复制。
-### Task-1.2
+### 任务-1.2
```c++
bool isContiguous() const;
```
-Check shape and strides of the tensor, and tell wether it is contiguous in memory.
+检查张量的形状和步长，判断它在内存中是否连续。
-### Task-1.3
+### 任务-1.3
```c++
tensor_t view(const std::vector &shape) const;
```
-Create a new tensor which reshapes the original tensor to the given shape by splitting or merging the original dimensions. No data transfer is involved. For example change a tensor of shape (2, 3, 5) to (2, 15) by merging the last two dimensions.
+创建一个新张量，通过拆分或合并原始维度将原始张量重塑为给定形状。不涉及数据传输。例如，通过合并最后两个维度，将形状为(2, 3, 5)的张量更改为(2, 15)。
-This function is not as easy as simply changing the shape of the tensor, although the test will pass. It should raise an error if new view is not compatible with the original tensor. Think about a tensor of shape (2, 3, 5) and strides (30, 10, 1). Can you still reshape it to (2, 15) without data transfer?
+这个函数不是简单地改变张量的形状那么简单，尽管测试会通过。如果新视图与原始张量不兼容，它应该引发错误。想想一个形状为(2, 3, 5)、步长为(30, 10, 1)的张量。你还能在不传输数据的情况下将其重塑为(2, 15)吗？
-### Task-1.4
+### 任务-1.4
```c++
tensor_t permute(const std::vector &order) const;
```
-Create a new tensor which changes the order of the dimensions of original tensor. Transpose can be achieved by this function without moving data around.
+创建一个新张量，改变原始张量维度的顺序。转置可以通过这个函数实现，而无需移动数据。
-### Task-1.5
+### 任务-1.5
```c++
tensor_t slice(size_t dim, size_t start, size_t end) const;
```
-Create a new tensor which slices the original tensor along the given dimension,
-start (inclusive) and end (exclusive) indices.
+创建一个新张量，沿给定维度，start（包含）和end（不包含）索引对原始张量进行切片操作。
-### Task-1.6
+### 任务-1.6
-Run tensor tests.
+运行张量测试。
```bash
python test/test_tensor.py
```
-You should see all tests passed. Commit and push your changes. You should see the auto tests for assignment #1 passed.
+你应该看到所有测试都通过了。提交并推送你的更改。你应该看到作业#1的自动测试通过了。
-## Assignment #2: Operators
+## 作业 #2：算子
-In this assignment, you will implement the cpu verision the following operators:
+在这个作业中，你将实现以下算子的cpu版本：
- argmax
- embedding
@@ -152,102 +151,102 @@ In this assignment, you will implement the cpu verision the following operators:
- self_attention
- swiglu
-Read the codes in `src/ops/add/` to see how "add" operator is implemented. Make sure you understand how the operator codes are organized, compiled, linked, and exposed to Python frontend. **Your operators should at least support Float32, Float16 and BFloat16 data types**. A helper function for naive type casting is provided in `src/utils/`. All python tests are in `test/ops`, you implementation should at least pass these tests. Try running the test script for "add" operator for starting.
+阅读`src/ops/add/`中的代码，了解"add"算子是如何实现的。确保你理解算子代码是如何组织、编译、链接以及暴露给Python前端的。**你的算子应该至少支持Float32、Float16和BFloat16数据类型**。`src/utils/`中提供了一个用于简单类型转换的辅助函数。所有python测试都在`test/ops`中，你的实现应该至少通过这些测试。首先尝试运行"add"算子的测试脚本。
-### Task-2.1 argmax
+### 任务-2.1 Argmax
```c++
void argmax(tensor_t max_idx, tensor_t max_val, tensor_t vals);
```
-Get the max value and its index of tensor `vals`, and store them in `max_val` and `max_idx` respectively. You can assume that `vals` is a 1D tensor for now, and `max_idx` and `max_val` are both 1D tensors with a single element (, which means the dimension of `vals` is kept).
+获取张量`vals`的最大值及其索引，并分别存储在`max_val`和`max_idx`中。你暂时可以假设`vals`是一个1D张量，`max_idx`和`max_val`都是包含单个元素的1D张量（这意味着保留了`vals`的维度）。
-You should be able to pass the test cases in `test/ops/argmax.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/argmax.py`中的测试用例。
-### Task-2.2 embedding
+### 任务-2.2 Embedding
```c++
void embedding(tensor_t out, tensor_t index, tensor_t weight);
```
-Copy the rows in `index` (1-D) from `weight` (2-D) to `output` (2-D). `index` must be of type Int64 (the default data type for int of PyTorch).
+从`weight`（2-D）中复制`index`（1-D）中的行到`output`（2-D）。`index`必须是Int64类型（PyTorch中int的默认数据类型）。
-You should be able to pass the test cases in `test/ops/embedding.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/embedding.py`中的测试用例。
-### Task-2.3 linear
+### 任务-2.3 Linear
```c++
void linear(tensor_t out, tensor_t in, tensor_t weight, tensor_t bias);
```
-Compute the following:
+计算以下内容：
$$
Y = xW^T + b
$$
-- `out`: output $Y$ . You can assume output is a 2D contiguous tensor and no broadcasting is involved for now.
-- `input`: input $X$ . You can assume input is a 2D contiguous tensor and no broadcasting is involved for now.
-- `weight`: weight $W$ . 2D contiguous tensor. Note that weight tensor is not transposed. You need to deal with this during your calculation.
-- `bias` (optional): bias $b$ . 1D tensor. You need to support the situation where bias is not provided.
+- `out`：输出 $Y$ 。你暂时可以假设输出是一个2D连续张量，不涉及广播。
+- `input`：输入 $X$ 。你暂时可以假设输入是一个2D连续张量，不涉及广播。
+- `weight`：权重 $W$ 。2D连续张量。注意权重张量没有转置。你需要在计算过程中处理这个问题。
+- `bias`（可选）：偏置 $b$ 。1D张量。你需要支持不提供偏置的情况。
-You should be able to pass the test cases in `test/ops/linear.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/linear.py`中的测试用例。
-### Task-2.4 rms normalization
+### 任务-2.4 RMS Normalization
```c++
void rms_norm(tensor_t out, tensor_t in, tensor_t weight, float eps);
```
-Compute the following for each row:
+为每一行计算以下内容：
$$
Y_i = \frac{W_i \times X_i}{\sqrt{\frac{1}{d}(\sum_{j=1}^d X_j^2) + \epsilon}}
$$
-- `out`: output $Y$ . You can assume output is a 2D contiguous tensor and no broadcasting is involved for now.
-- `input`: input $X$ . You can assume input is a 2D contiguous tensor and no broadcasting is involved for now. The normalization is performed along the last dimension (a.k.a. each row of length $d$ ) of the input tensor.
-- `weight`: weight $W$ . 1D tensor, same length as a row of input tensor.
-- `eps`: small value $\epsilon$ to avoid division by zero.
+- `out`：输出 $Y$ 。你暂时可以假设输出是一个2D连续张量，不涉及广播。
+- `input`：输入 $X$ 。你暂时可以假设输入是一个2D连续张量，不涉及广播。标准化沿输入张量的最后一个维度（即每一行，长度为 $d$ ）执行。
+- `weight`：权重 $W$ 。1D张量，与输入张量的一行长度相同。
+- `eps`：小值 $\epsilon$ 以避免除以零。
-You should be able to pass the test cases in `test/ops/rms_norm.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/rms_norm.py`中的测试用例。
-### Task-2.5 rope
+### 任务-2.5 旋转位置编码（RoPE）
```c++
void rope(tensor_t out, tensor_t in, tensor_t pos_ids, float theta);
```
-Compute the following for each vector of input tensor `in`, corresponding to a position id in `pos_ids`:
+为输入张量`in`的每个向量（这些向量与 pos_ids 中的位置 id 相对应）计算以下内容：
-Let $\mathbf{x}_i = [\mathbf{a}_i, \mathbf{b}_i] \in \mathbb{R}^d$ be the input vector and $\mathbf{y}_i = [\mathbf{a}'_i, \mathbf{b}'_i] \in \mathbb{R}^d$ be the output vector at index $i$, where $\mathbf{a}_i, \mathbf{b}_i,\mathbf{a}'_i, \mathbf{b}'_i \in \mathbb{R}^{d/2}$ .
+设 $\mathbf{x}_i = [\mathbf{a}_i, \mathbf{b}_i] \in \mathbb{R}^d$ 为输入向量， $\mathbf{y}_i = [\mathbf{a}'_i, \mathbf{b}'_i] \in \mathbb{R}^d$ 为索引 $i$ 处的输出向量，其中 $\mathbf{a}_i, \mathbf{b}_i,\mathbf{a}'_i, \mathbf{b}'_i \in \mathbb{R}^{d/2}$ 。
-Let $\theta$ be a fixed base (e.g. $\theta = 10000$) and $j = 0, 1, \ldots, d/2 - 1$.
+设 $\theta$ 为固定基数（例如 $\theta = 10000$）， $j = 0, 1, \ldots, d/2 - 1$。
-Let $p_i \in \mathbb{N}$ is the position id for token at input index i.
+设 $p_i \in \mathbb{N}$ 是输入索引i处token的位置id。
-Then the angle for RoPE is $\phi_{i,j} = \frac{p_i}{\theta^{2j/d}}$
+那么RoPE的角度为 $\phi_{i,j} = \frac{p_i}{\theta^{2j/d}}$
-The output vector $\mathbf{y}_i = [\mathbf{a}'_i, \mathbf{b}'_i]$ is computed as follows:
+输出向量 $\mathbf{y}_i = [\mathbf{a}'_i, \mathbf{b}'_i]$ 计算如下：
$$a_{i,j}' = a_{i,j} \cos(\phi_{i,j}) - b_{i,j} \sin(\phi_{i,j})$$
$$b_{i,j}' = b_{i,j} \cos(\phi_{i,j}) + a_{i,j} \sin(\phi_{i,j})$$
-- `out`: the resulting **q** or **k** tensor. Shape should be [seqlen, nhead, d] or [seqlen, nkvhead, d]. You can assume that the tensor is contiguous for now.
-- `in`: the orignal **q** or **k** tensor. Shape should be [seqlen, nhead, d] or [seqlen, nkvhead, d]. You can assume that the tensor is contiguous for now.
-- `pos_ids`: the position id (index in the whole context) for each token in the input sequence. Shape should be [seqlen,], dtype should be int64.
-- `theta`: the base value for the frequency vector.
+- `out`：结果**q**或**k**张量。形状应该是 [seqlen, nhead, d] 或 [seqlen, nkvhead, d]。你暂时可以假设张量是连续的。
+- `in`：原始**q**或**k**张量。形状应该是 [seqlen, nhead, d] 或 [seqlen, nkvhead, d]。你暂时可以假设张量是连续的。
+- `pos_ids`：输入序列中每个token的位置id（整个上下文中的索引）。形状应该是 [seqlen,]，dtype应该是int64。
+- `theta`：频率向量的基值。
-You should be able to pass the test cases in `test/ops/rope.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/rope.py`中的测试用例。
-### Task-2.6 self-attention
+### 任务-2.6 自注意力（self-attention）
```c++
void self_attention(tensor_t attn_val, tensor_t q, tensor_t k, tensor_t v, float scale);
```
-Compute the self-attention for query tensor `q`, key tensor `k`, and value tensor `v`. You should concat kvcache tensors, if needed, before doing this calculation.
+为查询张量`q`、键张量`k`和值张量`v`计算自注意力。如果需要，你应该在进行此计算之前连接kvcache张量。
$$
A = Q K^\top * scale \\
@@ -257,108 +256,109 @@ $$
Y = \mathrm{causalsoftmax}(A) \cdot V \\
$$
-- `attn_val`: the resulting attention value tensor. Shape should be [seqlen, nhead, dv]. You can assume that the tensor is contiguous for now.
-- `q`: the query tensor. Shape should be [seqlen, nhead, d]. You can assume that the tensor is contiguous for now.
-- `k`: the key tensor. Shape should be [total_len, nkvhead, d]. You can assume that the tensor is contiguous for now.
-- `v`: the value tensor. Shape should be [total_len, nkvhead, dv]. You can assume that the tensor is contiguous for now.
-- `scale`: a scaling factor. It is set to $\frac{1}{\sqrt{d}}$ in most cases.
+- `attn_val`：结果注意力值张量。形状应该是[seqlen, nhead, dv]。你暂时可以假设张量是连续的。
+- `q`：查询张量。形状应该是 [seqlen, nhead, d]。你暂时可以假设张量是连续的。
+- `k`：键张量。形状应该是 [total_len, nkvhead, d]。你暂时可以假设张量是连续的。
+- `v`：值张量。形状应该是 [total_len, nkvhead, dv]。你暂时可以假设张量是连续的。
+- `scale`：缩放因子。在大多数情况下取值为 $\frac{1}{\sqrt{d}}$ 。
-You should be able to pass the test cases in `test/ops/self_attention.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/self_attention.py`中的测试用例。
-### Task-2.7 swiglu
+### 任务-2.7 SwiGLU
```c++
void swiglu(tensor_t out, tensor_t gate, tensor_t up);
```
-This is an element-wise function that computes the following:
+这是一个逐元素函数，计算以下内容：
$$
out_{i} = up_{i} \circ \frac { gate_{i}}{1 + e^{-gate_{i}}}
$$
-`out`, `up` and `gate` are 2D contiguous tensors with the same shape [seqlen, intermediate_size].
+`out`、`up`和`gate`是具有相同形状 [seqlen, intermediate_size] 的2D连续张量。
-You should be able to pass the test cases in `test/ops/swiglu.py` after you finish the implementation.
+完成实现后，你应该能够通过`test/ops/swiglu.py`中的测试用例。
-### Task-2.8
+### 任务-2.8
-Run operator tests.
+运行算子测试。
```bash
python test/test_ops.py
```
-You should see all tests passed. Commit and push your changes. You should see the auto tests for assignment #2 passed.
+你应该看到所有测试都通过了。提交并推送你的更改。你应该看到作业#2的自动测试通过了。
-### Task-2.9 (Optional) rearrange
+### 任务-2.9（可选）rearrange
-This is a bonus task. You may or may not need it for model inference.
+这是一个奖励任务。你在模型推理中可能需要也可能不需要它。
```c++
void rearrange(tensor_t out, tensor_t in);
```
-This operator is used to copy data from a tensor to another tensor with the same shape but different strides. With this, you can easily implement `contiguous` functionality for tensors.
+此算子用于将数据从一个张量复制到另一个具有相同形状但不同步长的张量。有了这个，你可以轻松地为张量实现`contiguous`功能。
-## Assignment #3: Large Language Model Inference
+## 作业 #3：大语言模型推理
-Finally, it is the time for you to achieve text generation with LLAISYS.
+终于，是时候用LLAISYS实现文本生成了。
-- In `test/test_infer.py`, your implementation should be able to generate the same texts as PyTorch, using argmax sampling. The model we use for this assignment is [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
+- 在`test/test_infer.py`中，你的实现应该能够使用argmax采样生成与PyTorch相同的文本。我们用于此作业的模型是[DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)。
-- The python wrapper of your implementation is in `python/llaisys/models/qwen2.py`. You are NOT allowed to implement your model infer logic here using any python based frameworks, such as PyTorch. Instead, you need to implement the model with C/C++ in LLAISYS backend. The script loads each tensor in the safetensors file, and you will need to load data from them into your model backend.
+- 你的实现的python包装器在`python/llaisys/models/qwen2.py`中。你不允许在这里使用任何基于python的框架（如PyTorch）实现你的模型推理逻辑。相反，你需要在LLAISYS后端用C/C++实现模型。脚本加载safetensors文件中的每个张量，你需要从它们加载数据到你的模型后端。
-- In `include/llaisys/models/qwen2.h`, a prototype is defined for you. Feel free to modify the codes as you want, but you should at least provide basic APIs for model creation, destruction, data loading, and infer. Implement your C APIs in `src/llaisys/` and organize your C++ codes as other modules in `src/`. Remember to define the compiling procedures in `xmake.lua`.
+- 在`include/llaisys/models/qwen2.h`中，为你定义了一个原型。你可以随意修改代码，但你应该至少提供模型创建、销毁、数据加载和推理的基本API。在`src/llaisys/`中实现你的C API，并像`src/`中的其他模块一样组织你的C++代码。记得在`xmake.lua`中定义编译过程。
-- In `python/llaisys/libllaisys/`, define the ctypes wrapper functions for your C APIs. Implement `python/llaisys/models/qwen2.py` with your wrapper functions.
+- 在`python/llaisys/libllaisys/`中，为你的C API定义ctypes包装函数。使用你的包装函数实现`python/llaisys/models/qwen2.py`。
-- You need to implement KV Cache, or your model will be too slow.
+- 你需要实现 KV-Cache 功能，否则模型推理速度会过慢。
-- Debug until your model works. Take advantage of tensor's `debug` function which prints the tensor data. It allows you to compare the data of any tensor during the model inference with PyTorch.
+- 调试直到你的模型工作。利用张量的`debug`函数打印张量数据。它允许你在模型推理期间将任何张量的数据与PyTorch进行比较。
-After you finish the implementation, you can run the following command to test your model:
+完成实现后，你可以运行以下命令来测试你的模型：
```bash
python test/test_infer.py --model [dir_path/to/model] --test
```
-Commit and push your changes. You should see the auto tests for assignment #3 passed.
+提交并推送你的更改。你应该看到作业#3的自动测试通过了。
+## 只有完成作业后，才能开始做项目。
-## You can proceed to the projects only after you finish the assignments.
+## 项目#1：优化 LLAISYS 的 CPU 推理
-## Project #1: Optimize LLAISYS for CPU
-You probably have already noticed that your model inference is very slow compared to PyTorch. This is mostly because your operators are not optimized. Run your operater test scripts with "--profile" flag to see how your operators perform. You would probably see that `linear` operation is much slower than PyTorch. This operator is mainly a matrix multiplication, and is the most time consuming operation in transformer-based models.
+你可能已经注意到，你的模型推理速度相比 PyTorch 非常慢。这主要是因为你的算子没有经过优化。运行算子测试脚本时加上 ``--profile`` 参数，看看算子的性能表现。你可能会发现 ``linear`` 操作比 PyTorch 慢很多。这个算子本质上是矩阵乘法，是 Transformer 模型里最耗时的操作。
-There are several ways to optimize your operators for CPU:
+以下是几种优化 CPU 算子的方法：
-### SIMD instructions
+### 使用 SIMD 指令
-SIMD (Single Instruction Multiple Data) instructions are instructions that can perform the same operation on multiple data elements in a single instruction. Modern CPUs have support for SIMD instructions. Look for online materials to learn about compiler intrinsics (such as AVX2, AVX-512, NEON, SVE) to vectorize your operations.
+SIMD（单指令多数据）是一类可以在单条指令中对多个数据元素同时执行相同操作的指令。现代 CPU 都支持 SIMD。你可以查阅相关资料，学习编译器内建函数（如 AVX2、AVX-512、NEON、SVE）来向量化你的算子。
-### Use OpenMP for parallelism
+### 使用 OpenMP 实现并行
-You can use multi-threading to parallelize your operators. OpenMP is a popular library for multi-threading in C/C++. Add OpenMP support for LLAISYS to parallelize your `linear` and other operators.
+你可以用多线程来并行化算子。OpenMP 是 C/C++ 中常见的多线程库。为 LLAISYS 增加 OpenMP 支持，使得 ``linear`` 等算子能够并行执行。
-### 3rd-party Libraries
+### 使用第三方库
-There are several libraries that can help you optimize your operators for CPU. Look for libraries like Eigen, OpenBLAS, MKL, etc. to optimize your linear algebra operations. Note that some libraries are supported only for certain hardware platforms. Check their documentations and use them in your codes with care. You can also try to dig out how PyTorch implement these operators and see if you can use them.
+有很多库能帮你优化 CPU 上的算子，例如 Eigen、OpenBLAS、MKL 等，它们能高效处理线性代数运算。但要注意，有些库只支持特定硬件平台，需要仔细阅读文档并小心使用。你也可以参考 PyTorch 的算子实现，看是否能复用。
-Optimize your implementation with any methods you like and report your performance improvement.
+用任何你喜欢的方法优化你的推理实现，并报告性能提升情况。
-## Project #2: Intigrate CUDA into LLAISYS
+## 项目#2：在 LLAISYS 中集成 CUDA，适配两款CUDA或类CUDA平台(以下统称CUDA)
-This project does not depend on **Project #1**. You should choose two CUDA/CUDA-ish hardware platforms from Nvidia, Iluvatar, Metax, and Moore Threads.
+这个项目不依赖 ``项目#1``。需要选择 Nvidia、天数、摩尔、沐曦中的至少两款平台。
-This camp session provides computation resources from the four platforms above, access to which is granted based on applications from the official website. You can accelerate your model with CUDA on these GPU platforms. Before doing that, let's dive deeper into LLAISYS framework.
+本次训练营提供了以上四种平台的算力，可以在官方进行申请算力，并用 CUDA 加速模型推理。在动手前，先深入理解 LLAISYS 框架。
-LLAISYS is actually a framework with homogeous hardware support. When using LLAISYS, each thread will create a thread-local `Context` object which manages all the device `Runtime` objects used by this thread. A `Runtime` object is a resource manager for a device, and `Context` will create (with lazy initialization) a single `Runtime` object for each device. You can set and switch between them using `setDevice` function in `Context`. Only one device will be active at a time for each thread. Check `src/core/context.hpp` for more details.
+事实上，LLAISYS 是一个支持同构硬件的框架。使用时，每个线程会创建一个线程唯一的 **Context** 对象，管理该线程使用的所有设备 **Runtime**。**Runtime** 对象是设备的资源管理器，**Context** 会为每个设备（以延迟初始化的方式）创建唯一的 **Runtime**。你可以用 ``setDevice`` 在不同设备间切换，每个线程同一时间只会激活一个设备。详情见 ``src/core/context.hpp``。
-### Implement CUDA Runtime APIs
-Each `Runtime` object is intialized with a set of generic functions called `Runtime APIs`. You will need to implement CUDA version of these APIS. Check `src/device/cpu/cpu_runtime_api.cpp` to see how these functions are implemented for CPU and look for CUDA APIs to use in [`CUDA Runtime documentation`](https://docs.nvidia.com/cuda/cuda-runtime-api/index.html).
+### 实现 CUDA Runtime API
-You can see in `src/device/runtime_api.hpp` that `nvidia::getRuntimeAPI()` is guarded by `ENABLE_NVIDIA_API` macro.
+每个 **Runtime** 对象都会初始化一组通用的 **Runtime API**。你需要实现 CUDA 版本的 API。参考 ``src/device/cpu/cpu_runtime_api.cpp`` 看 CPU 的实现方式，查阅 [`CUDA Runtime 文档`](https://docs.nvidia.com/cuda/cuda-runtime-api/index.html) 找到对应 API。
+
+在 ``src/device/runtime_api.hpp`` 中，``nvidia::getRuntimeAPI()`` 被 ``ENABLE_NVIDIA_API`` 宏保护：
```c++
#ifdef ENABLE_NVIDIA_API
@@ -368,9 +368,9 @@ const LlaisysRuntimeAPI *getRuntimeAPI();
#endif
```
-This macro is defined in `xmake.lua` as a switch to enable/disable CUDA support. CUDA codes will not be compiled if the switch is off. In `xmake/` directory, create a `nvidia.lua` that configs your compiling process. (Similar to `cpu.lua` for CPU.) Search online to learn how to do it with Xmake.
+该宏的定义在 ``xmake.lua`` 中，用于开关 CUDA 支持。若关闭，CUDA 代码不会被编译。你需要在 ``xmake/`` 下新建 ``nvidia.lua``，配置编译流程（参考 ``cpu.lua``）。查阅资料学习如何用 Xmake 配置。
-After you implement the CUDA Runtime APIs, config your xmake with `--nv-gpu=y` to enable CUDA support and recompile your program. Run runtime tests to see if your implementation works.
+完成 CUDA Runtime API 后，用 ``--nv-gpu=y`` 打开 CUDA 支持并重新编译，运行测试：
```bash
xmake f --nv-gpu=y -cv
@@ -379,53 +379,54 @@ xmake install
python test/test_runtime.py --device nvidia
```
-### Implement CUDA Operators
-Create a `nvdia/` sub-directory in each operator source directory and implement a cuda version. Check `src/ops/add/op.cpp` to see how to include your cuda implementations. Remeber to define the compiling procedures in the xmake files. Run the operator tests with `--device nvidia` flag to test your CUDA implementation.
+### 实现 CUDA 算子
+
+在每个算子目录下新建 ``nvidia/`` 子目录，写 CUDA 版本实现。参考 ``src/ops/add/op.cpp`` 看如何包含 CUDA 实现。别忘了在 xmake 文件中定义编译流程。用 ``--device nvidia`` 参数运行测试。
-You can use CUDA libraries like cuBLAS, cuDNN, etc. to accelerate your operators. Check their documentations to see how to use them. You can store extra device resources in `src/device/nvidia/nvidia_resource.cu`.
+你可以使用 cuBLAS、cuDNN 等 CUDA 库来加速算子，额外的设备资源可以放在 `src/device/nvidia/nvidia_resource.cu`。
-Modify your model codes to support CUDA inference.
+最后,修改模型代码，支持 CUDA 推理：
```bash
python test/test_infer.py --model [dir_path/to/model] --test --device nvidia
```
-## Project #3: Build an AI chatbot
+## 项目#3：构建 AI 聊天机器人
-In this project you will build an AI chatbot that can do live conversations with single user with LLAISYS.
+本项目中，你将用 LLAISYS 构建一个能与单用户实时对话的聊天机器人。
-### Random Sampling
+### 随机采样
-So far we have been testing our model with argmax sampling. This is good enough for testing, but a chatbot should be able to generate more natural responses. Implement a random sample operator. Try to add supports for **Temperature**, **Top-K** and **Top-P**.
+目前我们只用过 argmax 采样，这在测试时够用，但聊天机器人需要更自然的回复。请实现一个随机采样算子，并尽量支持 **Temperature**、**Top-K**、**Top-P**。
-### Build a Chatbot Server
+### 搭建聊天服务器
-In your Python frontend, implement a server that can receive http requests from user and send responses back. You can use frameworks like FastAPI to build the server. You should follow the OpenAI chat-completion APIs. Try to support streaming responses if you can. You can assume, for now, that the server is only serving one user, and block the endpoint until the previous request is served.
+在 Python 前端里，实现一个能接收 HTTP 请求并返回响应的服务器。可以用 FastAPI 等框架。接口最好遵循 OpenAI 的 chat-completion API。如果可以，尽量支持流式输出。你可以先假设只有一个用户在使用，每次请求可以阻塞直到处理完成。
+### 交互式聊天 UI
-### Interactive Chat UI
+实现一个 UI，能向服务器发送请求并接收回复。可以是命令行界面，也可以是 Web 界面。要能通过连续发送消息与机器人保持对话。
-Build a UI that send requests to and receive responses from the chatbot server. You can build a simple command-line interface or a fancy web interface. You should be able to keep a conversation going with the chatbot by sending messages and receiving responses consecutively.
+### （可选）会话管理
-### (Optional) Chat Session Management
+实际应用中，用户可以开启多个对话并在它们之间切换，还能修改历史问题让 AI 重新生成回答。扩展 UI，支持这些功能。实现一个支持前缀匹配的 KV-Cache 池，尽可能复用已有结果。
-In real-world AI applications, users are allowed to start new conversations and switch between them. Users can also edit a past question and let the AI regenerate an answer. Enhance your UI to support these features. Implement a KV-Cache pool with prefix matching to reuse past results as much as possible.
+## 项目#4：多用户推理服务
+在做这个项目之前，你需要完成 ``项目#3`` 并实现流式输出。
-## Project #4: Multi-user Inference Service
+### 支持多用户
-You need to finish **Project #2** and achieve streaming response first before proceeding to this project.
+现实中推理服务要同时为多个用户提供服务，请求可能随时到来。你的服务端需要将请求加入请求池/队列，并用单独的循环线程/进程来处理。
-### Serving Multiple Users
+### 连续批处理
-In real-world scenarios, an inference service will serve multiple users. Requests can come in at any time, and the service should be able to handle them concurrently. Your endpoint should add a new request to a request pool or queue and have a another looping process or thread to serve the requests.
+为了最大化吞吐量，你需要做批处理，而不是逐一处理。由于每个请求长度不同，需要实现连续的迭代级批处理机制：每轮从池中取出若干请求组成批次（batch），执行一次批量推理，再把未完成的请求放回池中。推理时尽量用批量矩阵乘法加速。注意每个请求需要绑定不同的 KV-Cache，应实现支持前缀匹配的 KV-Cache 池来复用结果。
-### Continous Batching
-To maximize the throughput of your inference service, you need to batch your requests instead of serving them one by one. Since each request can have different length, you will need a continous and iteration-level batching mechanism. For each interation you extract several requests from pool to form a batch, do one round of batch inference, and then return the unfinished requests back to the pool. Use batched matrix multiplication when possible to speed up your inference. Note that every request in the batch need to bind with a different KV-Cache. You should build a KV-Cache pool with prefix matching to reuse past results as much as possible.
+## 项目#5：分布式推理
-## Project #5: Distributed Inference
-Introduce Tensor Parallelism to LLAISYS. Shard your model across multiple devices and implement distributed model inference. Support NCCL in LLAISYS if your are uing Nvidia GPUs, or MPI if you are using CPUs.
+在 LLAISYS 中引入张量并行。把模型分片到多个设备上，实现分布式推理。如果用 Nvidia GPU，需要支持 NCCL；如果用 CPU，需要支持 MPI。
-## Project #6: Support New Models
+## 项目#6：支持新模型
-Support another model type than the one we use for homework in LLAISYS.
+在 LLAISYS 中支持除作业所用模型以外的其他模型。
diff --git a/README_ZN.md b/README_ZN.md
deleted file mode 100644
index 7704dbd5b..000000000
--- a/README_ZN.md
+++ /dev/null
@@ -1,432 +0,0 @@
-# 欢迎使用 LLAISYS
-
-