mrtk3 finger tracking and clicking

[giotab]

Describe the issue

New to mrtk3 (building an app for Quest2), it is not clear how to execute code when clicking on a virtual object.
I would like to enable clicking using different fingers (e.g. thumb, index and pinky), and know which finger clicked my object.

Feature area

Clear instructions how to track fingers.

Existing doc link

N/A

Additional context

I am not sure if this is lacking documentation in mrtk3, or perhaps this is found someplace else. Any assistance to attach a script and track clicks and which finger clicked will be helpful :)

[Zee2]

Hi @giotab, I migrated this to a discussion.

I think I need more info here; what exactly do you mean by "finger clicking"? Do you mean determining when a finger touches/intersects an object? Or do you mean detecting air-tap-like gestures, but with non-index-tip fingers?

[giotab]

Hi @Zee2 ,

I mean detecting when (and which) a finger intersects with/touches an object.
Scenario: I have a cube which I want to paint in red if I touch it with pinky finger, green if I touch it with index finger, yellow if I touch it with middle finger.

[Zee2]

Lots of ways to do things like this

• Write your own collider/trigger script that you put on each finger you care about. You can add the FollowJoint script to make a GameObject follow a particular finger joint. You'd also write a script for the cube that can talk to your collider/trigger script and negotiate the color change.
• Do the same (with FollowJoint) but use PokeInteractors instead of your own script, and then tag each interactor with some tag that indicates which finger it is. You'd make your cube an interactable, and listen to OnHoverEnter/etc events to then apply your color changes.