Model upgrades // Architecture improvements // Policy enhancements

[keivalya]

META ISSUE FOR

Improving architecture:

• Replace simple CNN with Mini-ViT encoder
• Add optional CLIP/SigLIP embedding support
• Add FiLM conditioning for text → vision modulation
• Add temporal stacking (sequence of frames)

Diffusion Policy Enhancements

• Support flow-matching policy head
• Add denoising visualization (step-by-step)
• Add action noise scheduling & different betas
• Add multi-step action horizon (predict k steps)

A lot can be improved, but remember: the upgrades shall not compromise simplicity. ALWAYS document modules that you add in the simplest way possible.

[keivalya]

[UPDATE]

The following improvements in the architecture were made in a subsequent branch:

• Replace simple CNN with Mini-ViT encoder
• Add optional CLIP/SigLIP embedding support

Branch to refer: mini-VLA/vision

Refer to mini-VLA/models/vision/VISION.md to understand how to use these features from the CLI.

[nikhil-nakhate]

Adding a PR for flow matching head as linked above