Scene Graph - Transform Gizmo

[OlivierLDff]

Code is available at: https://github.com/OlivierLDff/cxx-qt-experiments

CleanShot.2025-05-18.at.1.48.30.mp4

I got time to have more play-around with cxx-qt, so I keep up on implementing QQuickItem based item. My goal here was to port be able to use transform_gizmo as a qml object.

transform-gizmo is a framework-agnostic Rust crate that provides a feature-rich and customizable 3D transformation gizmo for manipulating the position, rotation and scale of 3D entities.

I will split my feedback in 3 parts:

• doing the rendering (updatePaintNode)
• interaction with the mouse/hovering
• multi-selection/feedback API for qml

Rendering via updatePaintNode

The crate have a simple draw function that return a GizmoDrawData:

pub struct GizmoDrawData {
    pub vertices: Vec<[f32; 2]>,
    pub colors: Vec<[f32; 4]>,
    pub indices: Vec<u32>,
}

We need to render that using QSG API. Like in my previous test (bezier curve) I went with interacting from C++ as they are no bindings for that. As I found before I don't know if doing 1:1 mapping brings anything to the table.

    unsafe extern "C++" {
        include!("gizmo.h");
        unsafe fn gizmo_update_paint_node(
            old_node: *mut QSGNode,
            vertices: &[[f32; 2]],
            colors: &[[f32; 4]],
            indices: &[u32],
        ) -> *mut QSGNode;
	  }

And this call the following cpp code:

QSGNode *gizmo_update_paint_node(QSGNode *oldNode,
                                 rust::Slice<std::array<float, 2> const> vertices,
                                 rust::Slice<std::array<float, 4> const> colors,
                                 rust::Slice<std::uint32_t const> indices)

Then this is just regular Qt with C++. I've found myself a bit in uncharted water because they are not much online example/documentation on how to create something else than bezier curve and Efficient custom shapes in Qt Quick. But I guess this is mainly my lack of knowledge of graphic programming.

Interaction with the Mouse

The transform_gizmo crate expect a GizmoInteraction to describe the interaction:

pub struct GizmoInteraction {
    pub cursor_pos: (f32, f32),
    pub hovered: bool,
    pub drag_started: bool,
    pub dragging: bool,
}

So I wanted to have interaction with the mouse click/move/hover information. For that I needed to inherit 6 functions and manipulate QMouseEvent* & QHoverEvent*. I didn't know what I needed before hand so trying to solve making new bindings and figuring out what needed to be done was too much at once.
I went with the same trick as before: writing the stuff in C++ that doesn't bring much value, and call the logic written in rust.

So my base class:

class GizmoInteractionItem : public QQuickItem
{
    Q_OBJECT

public:
    GizmoInteractionItem(QQuickItem *parent = nullptr)
        : QQuickItem(parent)
    {
        setAcceptHoverEvents(true);
        setAcceptedMouseButtons(Qt::LeftButton);
    }
    virtual ~GizmoInteractionItem() = default;

protected:
    void hoverEnterEvent(QHoverEvent *event) override;
    void hoverLeaveEvent(QHoverEvent *event) override;
    void hoverMoveEvent(QHoverEvent *event) override;
    void mousePressEvent(QMouseEvent *event) override;
    void mouseReleaseEvent(QMouseEvent *event) override;
    void mouseMoveEvent(QMouseEvent *event) override;

    virtual void updateInteraction(QPointF position, bool hovered, bool dragStarted, bool dragging) = 0;
    virtual bool pickPreview(QPointF position) = 0;
};

This make life so much easier since I just need to "inherit" from rust updateInteraction and pickPreview (that check if we are hovering a clickable part of the gizmo).

#[cxx_override]
#[cxx_name = "updateInteraction"]
fn update_interaction(
		self: Pin<&mut Gizmo>,
		cursor_position: QPointF,
		hovered: bool,
		drag_started: bool,
		dragging: bool,
);

#[cxx_override]
#[cxx_name = "pickPreview"]
fn pick_preview(self: Pin<&mut Gizmo>, cursor_position: QPointF) -> bool;

I'm not sure what is the best way to provide bindings for QMouseEvent and QHoverEvent, but I'm sure the new upcast feature will help.

Api for Targets Manipulation

That part is very opinionated. The role of the gizmo is to update translation/rotation/scale of 1 or N objects. At first I thought about doing some QQmlListProperty or something using QObject for each targe that need to be manipulated. But that sounded like a hassle so I went with something way simpler to avoid any binding: QVariant, or more precisely a QList<QMap<QString, QVariant>>.

So user could just pass a list of js object, and get back a list of update object.

But somehow I struggle with lots of cryptic error message, I would really like your feedback on how to write that using only rust because I think this is possible with what is in cxx-qt.

Since I had trouble with QVariant usage from rust I went the C++ route again:

    unsafe extern "C++" {
        include!("gizmo.h");
        fn extract_target_count_from_qvariant(targets: &QVariant) -> usize;

        fn extract_targets_from_qvariant(
            targets: &QVariant,
            positions: &mut [QVector3D],
            rotations: &mut [QVector4D],
            scales: &mut [QVector3D],
        );

        fn transforms_to_qvariant(
            positions: &[QVector3D],
            rotations: &[QVector4D],
            scales: &[QVector3D],
        ) -> QVariant;
		}

And the code I would love to be able to write in rust:

std::size_t extract_target_count_from_qvariant(const QVariant &targets)
{
    const auto targetsList = targets.toList();
    return targetsList.size();
}

void extract_targets_from_qvariant(const QVariant &targets, rust::Slice<QVector3D> positions, rust::Slice<QVector4D> rotations, rust::Slice<QVector3D> scales)
{
    const auto targetsList = targets.toList();

    assert(std::size_t(targetsList.size()) == positions.size());
    assert(std::size_t(targetsList.size()) == rotations.size());
    assert(std::size_t(targetsList.size()) == scales.size());

    for (int i = 0; i < targetsList.size(); ++i)
    {
        const auto target = targetsList.at(i);

        const QMap<QString, QVariant> targetMap = target.toMap();
        const QVariant position = targetMap.value("position", QVector3D());
        const QVariant rotation = targetMap.value("rotation", QVector4D(0.f, 0.f, 0.f, 1.f));
        const QVariant scale = targetMap.value("scale", QVector3D(1.f, 1.f, 1.f));

        positions[i] = qvariant_cast<QVector3D>(position);
        rotations[i] = qvariant_cast<QVector4D>(rotation);
        scales[i] = qvariant_cast<QVector3D>(scale);
    }
}

QVariant transforms_to_qvariant(rust::Slice<QVector3D const> positions, rust::Slice<QVector4D const> rotations, rust::Slice<QVector3D const> scales)
{
    assert(positions.size() == rotations.size());
    assert(positions.size() == scales.size());

    QVector<QVariant> transforms;
    for (std::size_t i = 0; i < positions.size(); ++i)
    {
        QMap<QString, QVariant> transform;

        transform.insert("position", positions[i]);
        transform.insert("rotation", rotations[i]);
        transform.insert("scale", scales[i]);

        transforms.emplace_back(transform);
    }

    return transforms;
}

Also I wanted to use CxxVector or rust::Vector but somehow this gave me linking error about a missing cxx symbol. Is qt cxx bridge linking to cxx library?

What Have I Been Missing from cxx-qt

• QVariant examples
• QQuaternion for rotation bindings, I went with QVector4D
• QVariant with QtGui types (QVector3D, QVector4D)

I don't know if there is a right way to bind all the QSG & QEvent object, so manipulating them from C++ is okish.

I've also seen there is a qt-lib-extracrate so I will see if I can contribute some work for QQuickItem.

[LeonMatthesKDAB]

Awesome work @OlivierLDff !

In a sense your work shows that we can now do almost anything with CXX-Qt, as long as you're willing to write the odd C++ function every now and then, which was our goal with CXX-Qt.

Also thank you for your feedback about what could be improved.
We do have a small QVariant example that shows how to use it with a custom type here:
https://github.com/KDAB/cxx-qt/blob/main/examples/qml_features/rust/src/types.rs

Generics (QVariant/QList/QMap)

You can implement a trait for any type you want to store in the QVariant.
In your case this is QList<QMap<QString, QVariant>>, which you unfortunately cannot implement this trait for, as you will run into Rusts orphan rule that states you cannot implement a foreign trait on a foreign type.

The issue we have with this in general is that C++ templates are very difficult to represent in Rust with generics.
We basically have to list all concrete instantiations that you could ever need.
And something like QList<QMap<QString,QVariant>> we simply have not considered "widely used", so we won't add it to cxx-qt-lib.

You can maybe get around this with some newtype wrappers, but it would be somewhat cumbersome.

Alternative solution: destructuring the property

If I understand the code correctly, the targets property is an input property that is only written in QML, not read, right?

In that case a solution could be to deconstruct the property into multiple Rust method calls:

• clearTargets()
• addTarget(position, rotation, scale)
• finalizeTargets()

Which could then be called from QML/JS:

        function updateTargets() {
            clearTargets();
            for (model of view.pickedModels) {
                addTarget(model.position, model.rotation.toVector4d(), model.scale);
            }
            finalizeTargets();
        }

This would move the data deconstruction from C++ into QML, which is not an ideal API, but should allow you to get rid of the C++ code.

[LeonMatthesKDAB]

Any other ideas @ahayzen-kdab ?

[OlivierLDff]

Thanks for the hint about alternative solution. The problem I see is that it kills composability a bit. Like I can't do another rust qobject that expose a qproperty(QVariant, targets) for example. Writing js in qml is always fast at first but then turn into a pain to maintain.

One thing I didn't understand is why I couldn't use rust::Vector from c++ without a compilation error (undefined symbol to resize something). I will try to recreate a minimal reproducible example.

[LeonMatthesKDAB]

Right, a property is definitely a nicer API from QML.

You might have to manually tell CXX that you need support for the given concrete Vec<...> implementation with a shim impl:
https://cxx.rs/extern-c++.html#explicit-shim-trait-impls

I don't know if that's needed for rust::Vec though, as it only mentions std::vector 🤔 .

And you may not be able to do this outside of cxx-qt-lib due to the orphan rule :/

Templates/Generics are just a bit of a pain in Rust/C++ interop...

[ahayzen-kdab]

Lists in QML are kinda funky too :-) At least with Qt 6 it is able to understand things like QVector/QList as the type, but you have to be careful about keeping things notifiable. Eg in QML you might do like object.prop.push(value) but that might not trigger a notify signal for the property. This is where you see a lot of QML code using the following that causes the setter to be called and therefore the notify.

let arr = object.prop;
arr.push(value);
object.prop = arr;

This is also where QQmlListProperty can help you in a normal QML/C++ world.

The other route is instead as you said to change the API to have a method for mutating and then you could potentially still have a read-only notifiable property (so that you can do object.prop to read but you need to do object.mutateProp(value) to change it.

But you also have a lot of nested types going on here, so variant might be the simplest way without modelling it differently, i'd probably need to look more deeply at the example to understand what exactly is needed 😅

[OlivierLDff]

Basically the type is

struct Transform {
  position: QVector3D,
  rotation: QVector4D,
  scale: QVector3D,
}
struct Targets(Vec<Transform>)

Going with the readonly property and a mutate function sound like a great idea.

I thought about going the QQmlListProperty way, but this sound like more work to create the value from qml (ie I need to define a QObject, I need a "constructor" function exposed to qml, etc...)