Immersive Mixed Reality Training Concept for Mastering Surgical Knot-tying - Prototype Implementation
This repository contains a prototype implemtation of our mixed reality training concept designed to enhance medical students’ acquisition of surgical knot-tying skills. Utilizing a virtual reality headset with video passthrough functionality, the system provides adaptive visual instructions tailored to the user’s hand movements during the knot-tying process. This prototype is implemented in Unity3D Version 2022.3 LTS fpr the Varjo XR-3 headset. The prototype can be adapted to other headsets as well, but these systems need to be equipped with a Ultraleap Handtracking sensor unit.
- Git with Git LTS :
- Volumetric Video sequences downloaded and unzipped :
- Latest Unity 2022 LTS version
- Varjo XR-3 Headset
- Varjo Base installed and hand tracking enabled
- A thread for tying the knots
- Ideally, a knot tying bench, or a similar object which you can use to tye the thread around
- Best case: The Digital knot tying bench
- Clone this repository with Git
- Put the folder "PointClouds" from the unzipped volumetric video sequences into the "Assets/Streamingassets" folder of the Unity Project
- Open the project with Unity 2022 LTS
- Open the scene "Assets/Scenes/BMBF_KnotbAR_INTRO.unity"
- Click play! Make sure to have audio enabled, as the voiceover contains important instructions
Coming sooon
Licensed under the GNU General Public License Version 3