[FEAT] Support fast engine recovery through weight cache - #27139

Merged

ShangmingCai merged 78 commits into

sgl-project:mainfrom

QiuMike:fast_recovery

Jul 25, 2026

Commits on May 28, 2026

[Core] Add weight cache daemon for fast engine restart via CUDA IPC

QiuMike
and
claude
committed
7066879

Commits on May 29, 2026

[Core] Fix weight cache daemon distributed init for TP > 1

QiuMike
and
claude
committed
52c0e0f
[Test] Add weight cache unit tests and E2E test

QiuMike
and
claude
committed
13fb9c4
[Core] Fix weight cache daemon: use hf_config.architectures and optimize loading

QiuMike
and
claude
committed
e6f1497
[Core] Fix IpcModelLoader fallback: reset load_format to AUTO for disk load

QiuMike
and
claude
committed
d074b6b
[Core] Fix quant_config_hash mismatch: compute fingerprint before model loading

QiuMike
and
claude
committed
e2c273e
[Core] Fix IpcModelLoader import: set_default_torch_dtype is in model_loader.utils

QiuMike
and
claude
committed
1cde404
[Core] Fix IpcModelLoader: FileNotFoundError is a builtin, not socket attribute

QiuMike
and
claude
committed
9e0ec4b
[Core] Zero-copy IPC loading: use meta device init and daemon release protocol

QiuMike
and
claude
committed
d3d91c4

Commits on May 30, 2026

[Core] Fix zero-copy IPC weight loading for post-quant params and non-persistent buffers

QiuMike
and
claude
committed
9dd4f6d

Commits on Jun 1, 2026

[Core] Optimize IPC weight loading: 4.7s → 0.95s by pre-building param/buffer dicts

QiuMike
and
claude
committed
6e8c14a
[Core] Add CUDA graph serialization for fast restart (WIP)

QiuMike
and
claude
committed
2125f23
[Core] Add pool offset mechanism for intermediate tensor address translation

QiuMike
and
claude
committed
c226c60
[Test] Add pool offset validation tests

QiuMike
and
claude
committed
071a6e5

Commits on Jun 3, 2026

Merge branch 'sgl-project:main' into fast_recovery
QiuMike
authored
3d8d5a4

Commits on Jun 10, 2026

fix lint
liusy58
committed
d4b45e2
Revert "fix lint"
QiuMike
committed
3719180
Revert "[Test] Add pool offset validation tests"
QiuMike
committed
c3a5c13
Revert "[Core] Add pool offset mechanism for intermediate tensor address translation"
QiuMike
committed
2705ceb
Revert "[Core] Add CUDA graph serialization for fast restart (WIP)"
QiuMike
committed
db07672
fix lint
QiuMike
committed
cdf0cc3

Commits on Jun 11, 2026

fix
liusy58
committed
80d07ca
Merge branch 'main' into fast_recovery
liusy58
committed
e0f27de
Remove copy mode from weight cache and add single-command daemon launcher

QiuMike
and
claude
committed
296a27e
[Core] Use subprocess.Popen instead of multiprocessing.Process for daemon launcher

QiuMike
and
claude
committed
382a0e7
[Core] Don't capture daemon subprocess stdout/stderr

QiuMike
and
claude
committed
6fcd0b2
[Doc] Update weight_cache_design.md: remove copy mode, add launch docs

QiuMike
and
claude
committed
ca7cbaf
[Doc] Adjust IPC zero-copy performance to <1s based on实测

QiuMike
and
claude
committed
7db93fd

Commits on Jun 12, 2026

fix lint
liusy58
committed
f3f1b8f

Commits on Jun 13, 2026

delete
liusy58
committed
1443e8f

Commits on Jun 15, 2026

fix
liusy58
committed
6d848a8
support node>1 and pp
liusy58
committed
7ad2974
fix
liusy58
committed
c2f1929

Commits on Jun 16, 2026

Commits on Jun 22, 2026

Merge branch 'main' into fast_recovery
liusy58
committed
c750c35
fix
liusy58
committed
cf009e2

Commits on Jun 23, 2026

Merge branch 'main' into fast_recovery
liusy58
committed
38760ff
fix
liusy58
committed
ce4e231

Commits on Jun 25, 2026

Merge branch 'main' into fast_recovery
liusy58
authored
1edceb6

Commits on Jun 27, 2026

Commits on Jun 30, 2026

fix
liusy58
committed
a960d88

Commits on Jul 1, 2026

fix
liusy58
committed
24dfd14
fix commnets
liusy58
committed
641777f

Commits on Jul 2, 2026

fix
liusy58
committed
e9c4de7
add ci
liusy58
committed
09f303b

Commits on Jul 5, 2026

fix
liusy58
committed
4bdac78

Commits on Jul 6, 2026

Merge branch 'main' into fast_recovery
liusy58
authored
218ff9f
fix
liusy58
committed
e3a7e03
fix
liusy58
committed
fca72f7

Commits on Jul 16, 2026

Merge branch 'main' into fast_recovery
liusy58
committed
555ac4d
fix
liusy58
committed
0f03a83
fix
liusy58
committed
87303b5
fix
liusy58
committed
cb91f54
fix
liusy58
committed
32220c4
fix
liusy58
committed
b5cb36f

Commits on Jul 17, 2026

fix
liusy58
committed
0f5393f
Merge branch 'main' into fast_recovery
liusy58
authored
57c8447

Commits on Jul 18, 2026

add tp1 test
liusy58
committed
967caa6
Merge branch 'fast_recovery' of https://github.com/QiuMike/sglang into fast_recovery
liusy58
committed
4d87356

Commits on Jul 20, 2026

Merge branch 'main' into fast_recovery
liusy58
authored
9c2ab2d

Commits on Jul 21, 2026

Merge branch 'main' into fast_recovery
alexnails
authored
b303932

Commits on Jul 22, 2026

fix
liusy58
committed
6512c36

Commits on Jul 23, 2026

fix
liusy58
committed
d905875
Merge branch 'main' into fast_recovery
liusy58
authored
173626e
fix
liusy58
committed
6a9c343

Commits on Jul 24, 2026

fix
liusy58
committed
0512100
fix
liusy58
committed
6744dce
fix
liusy58
committed
572cd28
fix
liusy58
committed
a9b7300
Merge branch 'main' into fast_recovery
liusy58
authored
6580f9d
Merge branch 'main' into fast_recovery
liusy58
authored
25b9b78
Merge branch 'main' into fast_recovery
liusy58
authored
d618069
Merge branch 'main' into fast_recovery
liusy58
authored
318f310
Merge remote-tracking branch 'upstream/main' into fast_recovery
liusy58
committed
3476444