Microsoft Mixed Reality Spectator View v1.1.0

What's New

v1.1.0 contains changes to improve both the developer and end user experiences when working with Spectator View. Bug fixes were made to both the mobile and video camera filming experiences. Mobile users now have more information on the networking and spatial alignment states of their device in the shared experience. Developers now have better tools for understanding the performance impacts of spectator view's state synchronization components in their HoloLens application.

Release Notes:

1. UI now exists to inform users on whether or not their mobile device has lost tracking.
2. UI now exists to inform users on whether or not their mobile device has completed spatial alignment.
3. Users can now choose to rerun spatial alignment when their device loses tracking or their local application origin drifts.
4. Spectator View now supports multi-scene experiences for HoloLens applications.
5. UI for setting up network connections is now re-shown to the user on their mobile device during network disconnects. This is intended to better convey to the user the networking state of their spectating mobile device.
6. Asset Caches are now created in a deterministic manner. This means that different development PC can create their own Asset Caches that work with one another.
7. Spectator View now contains improved tools and documentation for performance tuning state synchronization components in the HoloLens application.
8. Bug fixes were made related to video camera recording. Users should see improved quality for videos recorded with the video camera version of spectator view.
9. Spectator View now contains playmode tests.
10. Bug fixes were made to better cleanup spatial alignment logic on network disconnects. Spatial alignment components should now better handle network disconnects/reconnects.

Breaking Changes:

#213 Add tests for video camera video recording and fix subsequent recording

This change fixes video encoding issues that occur when recording videos with the Video Camera filming solution. It redefines the UnityCompositorInterface and is therefore a breaking change (requires recompiling associated dlls). This break allows the editor to know the name of the video file recorded, which allows for playmode test cleanup.

Migration Instructions:

To migrate to codebases using this change, users need to do the following:

1. x64 Release versions of SpectatorView.Compositor.dll and SpectatorView.Compositor.UnityPlugin.dll.
2. Close any instances of Unity
3. Run tools\Scripts\CopyPluginsToUnity.bat (this will update the dlls within SpectatorView.Native within your local SpectatorView.

#223 Show device tracking status to the user on mobile devices and cleanup UI

This change has two primary breaking changes.

1. Mobile UI prefabs have been replaced, this was done to get a cleaner UI experience that works and better across different device scaling.

Migration Instructions:

1. Any references to Mobile Recording Service Visual and Mobile Network Configuration Visual will need to be updated to Default Mobile UI and Default Mobile Network Configuration Visual. Code changes may be required if any of the SpectatorView\Scripts\UI scripts were referenced/used outside of these prefabs.

2. HolographicCamera.Unity will need to be recompiled/deployed for tracking/spatial coordinate information to be correctly reported to the spectator view compositor. This now uses the ITrackingObserver MonoBehaviour.

#234 Enable resetting spatial alignment

This review introduces breaking changes into ISpatialLocalizationSession and SpatialLocalizationInitializer. It also changes some public facing functions on SpectatorView for starting and resetting localization sessions.

Migration Instructions:

1. Update any custom implementations of ISpatialLocalizationSessions to expose their associated SpatialCoordinateSystemParticipants as IPeerConnetions.
2. Update any custom child classes for SpatialLocalizationInitializer to implement updated functions. This will require returning success/failure values when running localization sessions.

#247 Enable cancelling localization sessions

This review introduces breaking changes to ISpatialLocalizationSession. Contributors felt there was a need to be able to cancel sessions, which led to this change.

Migration Instructions:

1. Update custom implementations of ISpatialLocalizationSession to support a Cancel function.

#264 Add deterministic asset id generation process for asset caches.

StringGuid keys for asset caches have been updated to AssetIds so that they can be deterministically created on different computers. This will break old AssetCaches.

Migration Instructions:

1. Delete asset caches in your Generated.StateSynchronization.AssetCaches\Resources folder (these will be the .asset files).
2. Call Spectator View -> Update All Asset Caches in your project to regenerate asset caches.
3. Recompile all player flavors (UWP, iOS, Android) of your application.

#291 Change build name to avoid errors for iOS and fix TextMeshPro

TextMeshPro was upgraded from 1.3.0 to 1.4.1 to fix iOS synchronization.

Migration Instructions:

1. Open the Unity package manager (Window -> Package Manager).
2. Update TextMesh Pro dependencies from 1.3.0 to 1.4.1

Microsoft Mixed Reality Spectator View v1.0.0

What's New

Spectator View v1.0.0 builds off of the functionality provided in v1.0.0-beta. In addition to the mobile filming experience, users will now have the ability to film HoloLens 2 applications with higher end video cameras. In addition to this new functionality, v1.0.0 contains improved documentation and developer workflows.

Release Notes

1. Investments have been made in documentation. We now have improved instructions for setting up Spectator View as a submodule, video camera calibration, video camera compositing, building for Android, and building for iOS.
2. Mobile devices can now specify HoloLens IP Addresses at runtime to connect to the shared experience. Users will no longer need to recompile applications with HoloLens IP Addresses change.
3. DLL dependencies now play well with one another when building for different device flavors (x86, ARM, etc). Previously, some dependencies for HoloLens 2 caused errors for HoloLens builds and vice versa.
4. Camera rig models have been included in the repository. Said models can now be 3D printed or machined to mount a HoloLens 2 device to a video camera for capture card filming.
5. QR Code detection logic has been updated to consume newer binaries.
6. OpenCV dependencies used in video camera calibration and ArUco marker detection have been updated from OpenCV version 3.4.3 to 4.1.1
7. Improvements were made to Android video playback. Choosing to view a video now launches the last recorded video compared to opening the photo app to its landing page.
8. Video camera filming now supports recording 4K video comprised of 4 1920x1080 video streams. This allows post production color correction for camera and hologram feeds.
9. Spectator View now has a documentation website: https://microsoft.github.io/MixedReality-SpectatorView/README.html

Microsoft Mixed Reality Spectator View v1.0.0-beta

Spectator View enables users to view and record both HoloLens and HoloLens 2 experiences from secondary devices, such as mobile phones and DSLR cameras.

The v1.0.0-beta release converges all pre-existing Spectator View projects into the same code base. Content has been relocated from MixedRealityToolkit, MixedRealityToolkit-Unity and MixedRealityCompanionKit into the MixedReality-SpectatorView repo. v1.0.0-beta is intended to allow for initial feedback before the first official release (anticipated mid August 2019).

What's New

With v1.0.0-beta, users can view and record HoloLens experiences on Android and iOS devices. The v1.0.0-beta release also establishes the groundwork for filming experiences with DSLR cameras. Formal support for filming with DSLR cameras will be provided in v1.0.0.

Release Notes

1. Spectator View has been updated to support HoloLens 2.
2. Spatial coordinate systems are now shared across devices using Microsoft Mixed Reality Spatial Alignment interfaces. This will allow easier adoption of new spatial alignment strategies as they become available.
3. Spectator View now supports uni-directional state synchronization. Unity content running on HoloLens 2 and HoloLens devices can be automatically synced to mobile devices.
4. Android is now a supported mobile platform. Previous releases of Spectator View only supported iOS.
5. Azure Spatial Anchors can now be used for sharing spatial coordinate systems.
6. Spectator View now contains sample code from the //BUILD 2019 demo.
7. QR Code detection can now be used for sharing spatial coordinate systems on HoloLens 2 devices.