Project overview · Inference · Robot deployment · Training core
RoboGesture's automated checks cover offline model loading, motion generation, MuJoCo rendering, package isolation and synthetic training steps. They do not connect to a physical robot or use cloud-service credentials.
From the repository root, after setup:
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python -m training_examples.demoAfter retrieving the trained checkpoints and downloading both official models, run the full GPU acceptance suite:
.venv/bin/python scripts/verify_offline.py --output acceptance-outputUse an empty output directory. The suite checks the installation, runs local Qwen + LoRA and Mimi + motion inference, and renders all 15 examples plus the new motion. Each resulting video's complete audio and video streams are decoded; frame counts must match the source NPZ. Reports, logs, generated motion and videos stay in the ignored output directory.
The GPU setup was tested on Linux x86-64 with Python 3.10.6, an RTX 4090 (24 GB)
and NVIDIA driver 535.183.01. The 67 pinned dependency versions are recorded in
constraints.txt; pip check passed. Python dependencies,
CycloneDDS and FFmpeg resolved inside the checkout and its own .venv.
Official model revisions are pinned in the model guide. The installation check validates the trained checkpoint hashes and all 15 example pairs against their release manifests. All 68 bundled robot meshes match the public sources documented in the asset provenance and modification notice.
The offline suite validates real model execution, package isolation and video integrity. Checks pass with networking disabled, other code checkouts and installation caches inaccessible, and Python/model-path overrides unset.
| Check | Result |
|---|---|
| Unit suite | 40 passed |
| CUDA | Real tensor operation on the RTX 4090 passed |
| CLI and module entry points | robogesture and python -m robogesture work from outside the checkout |
Disconnected stream |
No microphone, model, DDS or robot access |
| Qwen + LoRA | Nonempty, action-tagged English response |
| Mimi + motion model | 300 finite, nonconstant frames at 30 FPS |
| Reproducibility | Motion arrays were identical across two independent installations |
| MuJoCo + FFmpeg | 16 complete videos rendered and both streams fully decoded |
The 16 videos contain 6,180 frames / 206 seconds at 960×720, 30 FPS, H.264/AAC. This is offline validation, not a robot safety certification or a general throughput benchmark.
The unit suite also checks environment-variable names, default checkpoint and asset paths, Python 3.8 robot-module syntax, and the fixed DDS topic/type names with actual message serialization round trips. These are software-only checks; they do not start a robot service or send motion commands.
Local-clone checks validate checkpoint transfer and git lfs fsck. They do not
validate transport through a hosted Git server.
If local-path cloning fails with Git LFS 2.9.2, use this command-scoped endpoint override for local-file transport:
SOURCE_REPO=/absolute/path/to/source/RoboGesture
CHECK_PARENT=/absolute/path/to/a/new/location
git -c lfs.url="file://$SOURCE_REPO/.git" clone \
"$SOURCE_REPO" "$CHECK_PARENT/RoboGesture"
git -C "$CHECK_PARENT/RoboGesture" lfs install --local
git -C "$CHECK_PARENT/RoboGesture" lfs fsckDo not commit this local transport setting or use it for a public remote. See the Git LFS documentation for local standalone-file transport.
The robot environment recipe is verified on x86-64 / Python 3.8.10 with a checkout-local CycloneDDS 0.10.2 source build. Software checks cover dependencies, module origins and an audio-message IDL serialization round trip without starting robot services.
These results do not validate ARM64 installation, physical G1 behavior or cloud credentials. Actual deployment requires the host drivers, robot services, credentials and supervised preflight described in the robot guide.
The two-stage examples use prepared synthetic tensors and the checkout's own environment. CPU and GPU checks pass without networking, external code checkouts, model checkpoints or installation caches.
| Check | Result |
|---|---|
| Training-core unit tests | 15 passed |
| CPU demo | Tiny model, batch size 2, two updates per stage |
| RTX 4090 demo | Full architecture, batch size 1, one update per stage |
| Gradients and parameters | Finite, nonzero gradients; active probes changed and frozen probes did not |
| Objective checks | Losses, gradients, time sampling and single-step updates matched reference formulas |
| Failure checks | Invalid batches and nonfinite losses/gradients rejected before updates |
| Checkpoint independence | No learned checkpoint loaded or saved |
The training-core guide and objective specification describe the tensor contract and settings. These results demonstrate execution of the training core, not convergence, useful generated motion, training-data availability or reproduction of the paper's experiments. The demo does not train Qwen or LoRA.