Pointers disappear when the scale of a gameObject with a "NearInteractionTouchableUnityUI" is zero

[ArDevKarl]

Description

In my application, I can create Buttons and other holograms. In some cases they appear invisible. In this case their scale is at zero.
Now the problem is, everytime I instantiate a button (or any other interactalbe object with nearinteraction) and set its scale to zero, the pointers disappear. I cannot use any UI elements with handrays and/or nearinteraction rendering my application usless.

Workaround

Make sure the NearInteractionTouchableUnityUI is disabled while scale is at zero.

To reproduce

Create an object that uses "NearInteractionTouchableUnityUI" to be interacted with and scale it down to Vector3.zero. Then try to use pointers.

Expected behavior

Scale of various objects or components won't break the application by making pointes not work.

Setup

• Unity Version: 2022.3.7f1 (Happened also in previous versions)
• MRTK Version: 2.8.3.0 (Happened also in previous versions)

Target platform

• HoloLens 2 & Unity Editor

[AMollis]

@ArDevKarl, does the Unity log show any errors when the pointers aren't functioning?

[ArDevKarl]

@AMollis No, there is no exception or warning.

[AMollis]

The problem is this bit of code in NearInteractionTouchableUnityUI.

public override float DistanceToTouchable(Vector3 samplePoint, out Vector3 normal)
{
    normal = transform.TransformDirection(-LocalPressDirection);


    Vector3 localPoint = transform.InverseTransformPoint(samplePoint);

    // touchables currently can only be touched within the bounds of the rectangle.
    // We return infinity to ensure that any point outside the bounds does not get touched.
    if (!rectTransform.Value.rect.Contains(localPoint))
    {
        return float.PositiveInfinity;
    }

    // Scale back to 3D space
    localPoint = transform.TransformSize(localPoint);

    return Math.Abs(localPoint.z);
}

In particular this line:

Vector3 localPoint = transform.InverseTransformPoint(samplePoint);

When the scale is zero, localPoint is always (0, 0, 0). This results in NearInteractionTouchableUnityUI, whose scale is zero, to return a non-infinite value, thus indicating it is being touched. When this happens, only the touch pointer is enabled and all others are disabled.

A quick fix could be to add the following to the if statement:

rectTransform.Value.lossyScale == Vector3.zero

For the resulting code:

public override float DistanceToTouchable(Vector3 samplePoint, out Vector3 normal)
{
    normal = transform.TransformDirection(-LocalPressDirection);


    Vector3 localPoint = transform.InverseTransformPoint(samplePoint);

    // touchables currently can only be touched within the bounds of the rectangle.
    // We return infinity to ensure that any point outside the bounds does not get touched.
    if (rectTransform.Value.lossyScale == Vector3.zero ||
        !rectTransform.Value.rect.Contains(localPoint))
    {
        return float.PositiveInfinity;
    }

    // Scale back to 3D space
    localPoint = transform.TransformSize(localPoint);

    return Math.Abs(localPoint.z);
}

Unfortunately, since there is an available workaround (disabling NearInteractionTouchableUnityUI), this issue doesn't meet the bar for MRTK2.

If you have any other comments or concerns, please reach out. Thank you for the issue, and for participating in the community. We also encourage you to check out MRTK3 at https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity. More information about MRTK3 can be found at https://mixedrealitytoolkit.org.

[ArDevKarl]

Well, the given workaround causes bigger problems due to unrelated reasons. Me and my colleagues kept running into this over and over again, so I finally decided to just post it as a bug.

Thank you for the quick fix, this is is propably way better, then just switching everything on and off all the time. It won't keep my colleagues from falling into this trap in fresh projects, though. ;)

MRTK3 hopefully won't have this issue, when it's out of preview stage. Unfortunately it is not an option right now.