MRTK 2.7 Integration Guide for Unity 2020.3.x / MLSDK 0.26.0

Last Updated: 7/2/2021

Overview

MRTK-MagicLeap is an extension to Microsoft's open source Mixed Reality Toolkit (MRTK) for Unity. It adds compatibility for the Magic Leap platform, including head and hand tracking, and controller support. Using this plug-in, applications built using MRTK will be able to add support for Magic Leap devices. Some features such as native keyboard support are not yet implemented. View the notes for development tips and information about limitations.

In this Article

• Features
• Installation Guide
• Examples and Scene Settings
• Build Your Application
• Notes

Prerequisites

• Magic Leap Unity SDK

Features

This following MRTK Features are supported in the latest release.

Supported Features in MRTK

• Hand Tracking
• Hand Skeleton / Gestures
• Magic Leap Controller
• Voice Input (Using Google speech to text)
• Scene Meshing
• Gaze Input (aka Eye Tracking)

Unsupported Features in MRTK

• Hand Meshing
• Planes Detection
• Native Keyboard

This package also includes support for Zero Iteration, which allows you to test your application without having to build and install it on your device.

Installation Guide

Platform Settings

To use Magic Leap's MRTK feature, your Unity project needs to have Lumin as the build target.

1. Create a new Unity 2020.2.x project.
2. In the menu, go to File and select Build Settings.
3. Under Platform, select Lumin.
4. Click Switch Platform.

Configure XR Plugin Management

Configure Unity's XR plug-in framework to integrate Magic Leap into Unity’s engine and make full use of its features.

1. To install the latest version of the Magic Leap XR Plugin, open the Package Manager Window > Package Manager.
2. Select Unity Registry from the package registry dropdown to view all packages provided by Unity.
3. Locate the Magic Leap XR Plugin, select the arrow to expand the package options then See other versions. Select version 6.2.2 and click Install.
4. After the package is installed, open the XR Plugin Management settings File > Build Settings > Player Settings > XR Plug-in Management and enable Magic Leap as a Plug-in Provider on the Lumin Platform.

Microsoft Mixed Reality Toolkit 2.7

1. Download version 2.7.x of MRTK Foundation and MRTK Examples from the MRTK GitHub.
2. Import the MRTK Foundation 2.7.x package into your Unity project. Apply the recommended MRTK settings from the popup window that appears after doing so.
3. Next, import the MRTK Examples 2.7.x package into your project.

Text Mesh Pro

1. Import the TMP Essential Resources by selecting Window > TextMeshPro > Import TMP Essential Resources.

Install MRTK Magic Leap

After all of the dependencies are installed, download and install the MRTK Magic Leap package.

1. Download the latest version of the MRTK Magic Leap package from the release sections on the GitHub page.
2. Import the MRTK MagicLeap Unity Package by going to Assets > Import Package > Custom Package. Import all of its contents.

* If you are upgrading from an previous version, follow the instructions provided in the Upgrade Guide.

Examples and Scene Settings

This project includes two pre-configured scenes:

• SpatialAwarenessMeshDemoMagicLeap
• HandInteractionExamplesMagicLeap

These scenes do not require additional configuration and serve as a blueprint for Magic Leap's MRTK integration. They can also be used as a guide for creating custom MRTK Configuration Profiles.

Add support to existing scenes.

If you want to test other MRTK Scenes, additional configuration is required. The steps below explain how to configure other scenes to support the Magic Leap platform. Please note that some Magic Leap features are still not supported. See the limitations section for more details.

1. Open the HandInteractionExamples scene.
2. Select the MixedRealityToolkit in the Hierarchy. Set the configuration to the MagicLeap1 ConfigurationProfile.
3. Select the Main Camera and in the MixedRealityInputModule select Force Module Active is enable. This enables interaction with Unity's canvas components.
4. Verify that the Camera has a TrackedPoseDriver with the default values:
   • Generic XR Device
   • Center Eye - HMD Reference
   • Rotation & Position
   • Update & Before Render
5. Some examples may require additional controller and hand configuration to insure proper interactions - see the Input section for details.

Build Your Application

Publishing Settings

1. First, set the project's identity and certificate settings. Navigate to Edit > Project Settings > Player.
   1. Set Company Name and Product Name.
   2. Under Other Settings > Identification > Bundle Identifier, enable Override Default Bundle Identifier to allow text entry.
   3. Set the Bundle Identifier. Make sure to use lowercase letters only. ex: com.yourcompanyname.xxx
   4. Under Publishing Settings, set the developer cert. You can generate a developer certificate by going to the Publish section of the Magic Leap website and selecting certificates.

Permissions

1. For your application to have access certain Magic Leap features, you need to configure your project's permissions. In Project Settings window, navigate to Magic Leap > Manifest Settings and add the following privileges:
   • ControllerPose
   • GesturesConfig
   • GesturesSubscribe
   • HandMesh (optional)
   • Internet (optional)
   • PcfRead (optional)
   • WorldReconstruction
   • LocalAreaNetwork

Build and Run

1. Open the Build Settings window and add the scenes you want to build to the Scenes In Build.
2. Select Build And Run.

Note: To publish your app to the store, the Version Name, in the Player Settings > Identification needs to be set to at least one decimal place (ex: Version Name: 1.0). This step is not required for development builds.

Upgrade Guide

If you are upgrading from a previous version of the MRTK Magic Leap Package follow the steps below.

1. Delete existing the following folders:
   • Assets/MRTK-MagicLeap
   • Assets/MagicLeap-Tools (If present)
2. If you are upgrading to a newer version of both MRTK and the MRTK Magic Leap, follow Microsoft's MRTK Upgrade Guide to update the MRTK components. Otherwise, continue to the next step.
3. Download and install the MRTK Magic Leap package by following the Install MRTK Magic Leap instructions.

Notes

Development Tips

Track Both Hands

• To enable hand tracking on both hands programmatically, set the Magic Leap Device Manager's hand settings to Both by adding the following line to one of your script's Start method:

MagicLeapDeviceManager.Instance.CurrentHandSettings == MagicLeapDeviceManager.HandSettings.Both

Set the Magic Leap controller ray to "Always On"

• To prevent the controller's ray from being disabled when close to 3D objects, change the motion controller pointer behaviour to AlwaysOn by adding the following line to one of your script's Start method:

PointerUtils.SetMotionControllerRayPointerBehavior(PointerBehavior.AlwaysOn);

Spatial Awareness

• Magic Leap's specific mesh features such as Vertex Confidence and Planarization can be enabled in the inspector when selecting the Magic Leap's SpatialAwarenessMeshOberverProfile.
• The MRTK spatial mesh shaders require Force Multipass to be enabled. This can be done in Project Settings > Magic Leap Settings.
• Meshing Settings can be changed at runtime using Magic Leap's MeshingSettings API.
• Scene Understanding and Plane Finding are not supported.

Input

• To ignore specific Controllers or Hands, edit the _CurrentHandSettings and _CurrentControllerSettings values before building in the MagicLeapDeviceManager.cs script. Run-time settings changes are not yet implemented.
• You can use the Trigger, Bumper, and HomeTap as digital inputs. However, no response is provided for HomeTap down. Instead, HomeTap down and HomeTap up are both executed simultaneously on HomeTap up. This is due to it being a system-level button.
• You can also use the Touchpad Touch, Touchpad Press, and Touchpad position as input.

Limitations

• This package is experimental and subject to errors, use at your own risk. We recommend backing up your project before implementing.
• The Hand Mesh feature is not fully implemented.
• The System Keyboard is not currently integrated.
• Run-time input settings changes are not yet implemented.
• No response is provided for the HomeTap down input. This is due to it being a system-level button.