[DRAFT] XR Editor: Proof-of-concept immersive editing using the debugger

[dsnopek]

In the Godot XR editor running standalone on the Meta Quest headset, you can launch your game immersively, and then bring the 2D editor window back up, and make edits to the current scene which will be immediately reflected in the immersive environment.

This is great! It's super helpful for doing level design, so you can see exactly how something will look in VR, especially when precise spacing is important.

However, even though the objects you're manipulating may be directly in front of you in the immersive environment, you still need to use the editor in the 2D panel to manipulate them.

Wouldn't it be great if you could just grab the objects in the immersive environment and move them around?

This PR is a proof-of-concept which shows how that could be implemented :-)

Check out a quick video demo here:

https://www.youtube.com/watch?v=S_VliP5-3OU

How to use it:

• You press the "Menu" button to switch into "immersive editing mode", which puts a ray in your right hand and a flat panel in your left hand
• You can point the ray at an object in the scene, and press the trigger to select it - it'll get a selection box around it
• You can then press and hold the grip button, and move the controller to move the object immersively
• If the object you selected is from the current scene that's open in the editor, once you release the grip button, it'll update that object in the scene in the editor. If it's where you want it, you can just save those changes! It does its updates using the normal undo/redo system, so you can easily undo it in the editor too, if you want

NOTE: The idea I had for the flat panel in your left hand, is that maybe it could show the inspector with the properties of the selected object and you could do some light editing there, but it's not implemented and I don't know if that's even a good idea.

How this works:

• It's using the debugger protocol to do the selection and send the edits back to the editor. This is very similar to how the current game view works, where you can select objects in the game view, and then it selects them in the editor. I think this overall approach is a good way to do this, and so is the main focus of the PoC. (Of course, the specific implementation of this approach is very early.)
• It's doing some unholy things with XROrigin3D and other XR nodes to enable/disable the immersive editing mode, and doesn't take the OpenXR Action Map into account at all. This will probably need to be completely redone in a real implementation - please don't pay too much attention to this part :-)

Some things I'd like to have in the finished implementation:

• Objects selected in immersive editing mode should show the same object selected in the scene in the editor
• A 3D transform gizmo that can be used to only translate or rotate on a specific axis, for fine-tuned editing. (It still makes sense to have a mode to just grab and freely move the object, but just not as the only option)
• The ability to launch a single scene, and have it inject the XROrigin3D and related nodes, and initialize OpenXR. Maybe this could even start in immersive editing mode? This way you could directly edit a specific scene, rather than launching the full game and having to find your way to the scene you want to edit
• Some form of locomotion that isn't connected with the game. Sometimes you may want to edit something that you can't physically reach, or the game's locomotion won't let you move to
• The ability to scale the immersive environment. When doing level design, it can be very useful to "zoom out" or "zoom in", and in an immersive context, I think that means scaling the world! Although, this would need to actually be tested to see just how useful it is
• The ability to place a cursor in the immersive environment, which can be used as a position to spawn a new node from the editor. This way you can create new things near yourself without having manually move them there in the 2D panel
• Proper integration with the OpenXR Action Map. I think a special Action Set should be injected when the game is launched with immersive editing enabled (and this code should only exist in editor builds) and these actions should be used rather than the default Godot ones

Not all those things need to be in one big PR though :-)