What
Fit FLAME 3D morphable model parameters from MediaPipe's 478 face landmarks, producing a textured 3D mesh from a single 2D photo.
Why
This is a key step toward the 3D roadmap. FLAME fitting from 2D landmarks enables:
- 3D-space surgical deformations (anatomically grounded, not pixel-level)
- Multi-view rendering of predicted results
- Interactive 3D previews for patient consultations
Technical context
- MediaPipe outputs 478 3D landmarks (normalized coords + depth estimate)
- FLAME has ~5K vertices; need a landmark-to-vertex correspondence mapping
- Existing tools like DECA or EMOCA solve similar problems
- The challenge is making it work from MediaPipe landmarks specifically (not DLIB 68 or FAN 98)
Deliverables
landmarkdiff/flame_fitting.py — FLAME parameter estimation from MediaPipe landmarks
- Landmark-to-FLAME vertex correspondence mapping
- Basic textured mesh output (OBJ or PLY)
- Tests with synthetic landmarks
References
Difficulty: 🔴 Advanced — requires 3D vision experience
What
Fit FLAME 3D morphable model parameters from MediaPipe's 478 face landmarks, producing a textured 3D mesh from a single 2D photo.
Why
This is a key step toward the 3D roadmap. FLAME fitting from 2D landmarks enables:
Technical context
Deliverables
landmarkdiff/flame_fitting.py— FLAME parameter estimation from MediaPipe landmarksReferences
Difficulty: 🔴 Advanced — requires 3D vision experience