Skip to content

cgsaxner/HoloLens2-Unity-ResearchModeStreamer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 

Repository files navigation

HoloLens2-Unity-ResearchModeStreamer

Unity Plugin for accessing HoloLens2 Research Mode sensors and video camera, and streaming them to desktop. It builds upon the official HoloLens2ForCV and HoloLensForCV repos.

Currently, only Depth AHAT and video camera streams are enabled, but an extension to other RM sensors should be straight forward and will probably be added in the future. The image, as well as rig2world transforms for AHAT and and pv2world, fx and fy for video camera are transmitted for each frame.

Dependencies

Plugin

Using the Plugin

  1. Open the plugin solution in Visual Studio
  2. Add path to Eigen under Project -> Properties -> C/C++ -> General -> Additional Include Directories or copy the folder containing Eigen headers to HL2RmStreamUnityPlugin/Dependencies.
  3. Build the solution for Release, ARM64.
  4. In your Unity Project, create a folder Assets/Plugins/WSAPlayer/ARM64.
  5. Copy the HL2RmStreamUnityPlugin.dll from HL2RmStreamUnityPlugin/ARM64/Release/HL2RmStreamUnityPlugin into the folder from step 3.
  6. To call the Initialize function from the DLL, add this statement to one of your Unity scripts:
[DllImport("HL2RmStreamUnityPlugin", EntryPoint = "Initialize", CallingConvention = CallingConvention.StdCall)]
public static extern void InitializeDll();
  1. You can call InitializeDll() from Unity. An example can be found in UnityHL2RmStreamer.
  2. Before building the Unity Project, go to Build Settings -> Player Settings the following Capabilities are enabled:
    • InternetClient, InternetClientServer, PrivateNetworkClientServer, WebCam, SpatialPerception.
  3. Build the Unity Project and open the solution in Visual Studio.
  4. After building the Unity Project for the first time: Open the package.appxmanifest in the solution in a text editor to enable the research mode sensors:
    • add the rescap package to Package: xmlns:rescap="http://schemas.microsoft.com/appx/manifest/foundation/windows10/restrictedcapabilities"
    • add rescap to the Ignorable Namespaces: IgnorableNamespaces="... rescap"
    • add rescap to Capabilities: <rescap:Capability Name="perceptionSensorsExperimental" />
    • save and close Package.appxmanifest
  5. Build solution for Release, ARM64 and deploy to HoloLens2.

Python Client

A simple client written in python for receiving and displaying the frames is available in hololens2_simpleclient.py.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages