The state of math libraries

[Osspial]

The big math libraries are cgmath and nalgebra. Both of them have deal-breaking flaws for many, namely (imo):

• cgmath has a weird, idiosyncratic design that simultaneously does too much and too little, and tries to shoehorn in genericism with complex traits that are difficult to internalize.
• nalgebra stretches the type system so far that the documentation is nigh impossible to understand without extreme patience and a deep understanding of generics.

As a result, several people have gone and made their own math libraries that try and solve those problems, which don't get adoption because the ecosystem hasn't grown up around them. I'd like this issue to house discussion on how exactly we can go about designing a library that addresses those issues, while developing enough consensus that we can potentially replace both of the current libraries.

[Osspial]

At the risk of making a worn-out reference, I can't deny that we're at risk of falling into this trap:

Regardless, I think there has been a lot of cool research done in the community, but the only way I can see the fruits of that research reaching a wide audience is if it has some amount of official backing, which we (as the gamedev working group) can provide.

---

Anyhow, time for my opinions: I think we should take the standard library's approach of building a small, fast, and relatively uncontroversial core that the rest of the ecosystem can build around. With that in mind, I'd like to propose a scope for the standard math crate and guidelines for its design:

Scope

• "Standard Math" should only contain Vector/Point/Matrix types. Iterating over designs and moving those types to 1.0 should be our highest priority.
  • This explicitly does not include Quaternion or Euler or anything relating specifically to rotations. There's a lot of design space to explore there and I think figuring those out is considerably more complex than figuring out the three types listed above.
• All other work expanding upon the base types should initially be done in other crates, so that experimentation can be done without harming ecosystem interoperability.
  • Other work potentially includes (but is not limited to):
    • High-level matrix constructors, such as cgmath::PerspectiveFov.
    • Rotation and quaternions.
    • Higher-level geometric constructs (e.g. rectangles, circles, spheres, capsules, etc.)

Design

I've got various design issues I've seen in various math crates that I'd like a standard math library to address, and I'll list them here:

1. .x, .y, .z accessors are a must.
2. Swizzling is a must.
   • We've made plenty of swizzle-based accessors, but I haven't seen any research done into swizzle-based constructors. Macros could be helpful here.
3. Genericism over dimensionality is nice to have, but shouldn't sacrifice point 1.
   • We can potentially use an nalgebra-like Deref<Target=Coordinates, since despite nalgebra's complexity that's actually a really good solution.
   • Const generics are exciting here but relegate us to nightly.
4. The documentation should be easy to understand. nalgebra has done a lot of really interesting work with embedding math into the type system, but the result is that you need a comprehensive understanding of both to make sense of it.
   • To that extent, we should minimize the usage of traits, since traits tend to fragment documentation and make things harder to understand.

There are also a few design points that I think are worth discussing, but aren't as cut-and-dry as I've been making the above points out to be. Namely,

• Should we integrate units into a standard math library? The main library I'm aware of that does that is euclid, but I'm not a huge fan of their API, and I think there's a way of adding units without being as intrusive as euclid makes it.
• Should we split Vector and Point types into a different type, or should they be the same type? There's points to be made on both sides here, and I haven't managed to convince myself that either approach is better.

[Lokathor]

(I typed this up while Osspial was typing their first reply, so i'll comment on that a bit separately in a moment)

How do we feel about glam? Or at least "the glam approach" as we'll perhaps call it.

• No traits, no generics, just plain structs and functions/methods.

It's very easy to think about.

Now, currently glam is incomplete in some areas:

• there's only f32 types not f64, i32, and i64
• there's a fixed opinion on rotations and coordinate systems which might not match all systems people might want to use, but that's also a matter of perhaps renaming some functions to be clear that they're left-handed and adding a right-handed version or something like that (like GLM / nalgebra-GLM does).

but besides the fact that things are missing how do we feel about the approach?

[Lokathor]

reply to Osspial's proposal

• We kinda have a "standard math" lib, it's called mint, and most of the other math libs have optional interop with it. Well, except that mint has no operations, only data types. Still, a starting point perhaps.
• I disagree on (1) that it has to be field accessors as a must. Particularly, that means that you can't just use a SIMD lane as the rep of a Vec or Mat, which means a performance hit. I'll honestly take the faster version without direct field access, and I bet a lot of others would too.
• (2) Swizzling is a must? Really? I mean it's neat, but a must? But, sure, it's easy enough to make them all, and it doesn't matter if they're accessors or constructors, you can do both, it's just a lot of copy and paste work basically.

[icefoxen]

Like Lokathor said, we already have the solution thanks to kvark, it's mint. The lack of operations is a feature, not a bug; it's not a math library, it's a math TYPE library. The intended usage is what ggez 0.5 does, just have all library functions take Into<mint::Whatever> instead of nalgebra::Whatever or cgmath::Whatever. The user uses whatever library they feel like and their types silently vanish into the maw of Into impl's and it's like the library is using whatever math library they prefer. Works great, all the math libraries that matter already have feature support for it, and it's not hard to add to new libraries.

Sorry! I feel like the problem's already been solved. It's not perfect, since my measurements suggest there IS some run-time overhead to mint's conversions that doesn't get entirely optimized out, but it's negligible for my purposes.

[Osspial]

@Lokathor

How do we feel about glam? Or at least "the glam approach" as we'll perhaps call it.

* No traits, no generics, just plain structs and functions/methods.

It's very easy to think about.

On a high level, I quite like glam's approach. However, I'm hesitant to say that we should fully adopt what glam is doing, since I disagree with their SIMD-centric approach:

• Universally using SIMD-backed storage makes it extremely difficult to cleanly implement generics in a coherent and unsurprising way (at least on stable Rust, without specialization). Having separate Vector/Point/Matrix types for each primitive is unacceptable in a library that's designed for everyone.
• Forcing 16-byte alignment on 3-element types makes it significantly harder to do unsafe operations on the types. You couldn't, for instance, re-interpret a &[f32] as a &[Vector<3, f32>], since you'd skip every fourth float and floats are 4-byte aligned.
  • glam's current approach of "optional SIMD" also means that types don't have a reliable layout, which breaks unsafe code for reasons that I don't feel I need to describe. I realize that inconsistent layout isn't inherent to the idea of SIMD-based types, but it's still an issue I have with the current implementation.
• SIMD-backed storage makes GPU interop much more awkward. The space wastage is unquestionably detrimental in that context since there's no speed increase to be had, and VRAM can still be constrained enough that it's an actual problem. I'd like to be able to use standard math types when creating Vertex types, and SIMD storage makes that infeasible.

I suppose the base problem I have there is that SIMD-backed types improves speed at the cost causing direct usability issues in every other context, and I don't think that sort of tradeoff is acceptable for a base, standard math library that everyone can build upon.

We kinda have a "standard math" lib, it's called mint, and most of the other math libs have optional interop with it. Well, except that mint has no operations, only data types. Still, a starting point perhaps.

The fact that mint is designed as an interop library, and not a base library for direct usage, makes it infeasible for direct use. Admittedly, it still has gotten ecosystem traction so adopting it and expanding upon it may be a decent idea, but I don't hugely like the high-level design of its API and changing it would induce significant breakage.

I disagree on (1) that it has to be field accessors as a must. Particularly, that means that you can't just use a SIMD lane as the rep of a Vec or Mat, which means a performance hit. I'll honestly take the faster version without direct field access, and I bet a lot of others would too.

Eh, different priorities I guess. I've made my point on SIMD so I'm not going to reiterate that, but direct field access makes the API significantly nicer to use in pretty much every way and I don't want to throw that out without extensively proving that it's worth it.

(2) Swizzling is a must? Really? I mean it's neat, but a must? But, sure, it's easy enough to make them all, and it doesn't matter if they're accessors or constructors, you can do both, it's just a lot of copy and paste work basically.

Swizzing is a feature that I've consistently found useful when implementing code, and it's something you notice when it's missing. I don't want to have to think about about using it and explicitly import it - I just want to have it available, and there's really only one way to do it.

[Osspial]

@icefoxen

Sorry! I feel like the problem's already been solved. It's not perfect, since my measurements suggest there IS some run-time overhead to mint's conversions that doesn't get entirely optimized out, but it's negligible for my purposes.

I think that the fact mint needs to exist shows how much of a problem there is in the ecosystem. Saying that mint is a solution is like saying "The Rust standard library doesn't have an array type, but everything takes Into<GenericArrayWithoutImplementations<T>>, so it's okay" (yes I know that Rust has arrays but that isn't the point). Mint clunkily solves the problem that you can't use some libraries with others, but it doesn't solve the issue is that nobody's designed a solution that everyone can actually use. Maybe that's impossible, but I have a hard time believing that.

Besides, since Mint does so little, it means that you can't actually build higher-level constructs around Mint without sacrificing the ability to sanely design an internal and external API. I can't, say, build a higher-level geometry library around Mint (e.g. a library that provides rectangles, circles, etc. and the ability to transform them) without either throwing away all the math functions every other library provides or clumsily doing conversions internally every time I want to perform any trivial transformation on my datatypes.

I don't particularly care about single-nanosecond losses of performance here. I care about designing APIs that people enjoy using, and standardizing around Mint makes it extremely difficult to do that.

[Lokathor]

Hot Take: Generics are a necessary evil, not an inherent good. The ideal vec/mat library for sizes 1-4 is probably one that is fully written out over time for all possible combinations and 100% non-generic. There are sufficiently few combinations that it's actually possible to write them all down, so you might as well write them all down (and your re-compilation times will actually improve if you do this).

Now, you have a bit of a point with the SIMD stuff, but I do wonder how often you're doing math operations on all the vertexes in a model, and not just on its transforms (presumably a much smaller amount of data overall). It might not be insane to make a set of model/vertex data and then have the faster SIMD types for CPU-side usage.

[icefoxen]

I think that the fact mint needs to exist shows how much of a problem there is in the ecosystem.

Contrast with C/C++, or C#, or Python, or literally anything else, where there is either ONE library that everyone uses even when it kinda sucks, or there is >1 library and nobody EVER lets ANYTHING interoperate between different libraries. The fact that mint even can exist is heckin' amazing.

... it doesn't solve the issue is that nobody's designed a [single] solution that everyone can actually use. Maybe that's impossible, but I have a hard time believing that.

It's impossible to have a single solution that everyone can use, because different people have different goals. Different people just think in different ways. This is okay. People will naturally agglomerate towards the de-facto standard because that's how humans work, but there will always be the outliers that do things differently that work better for some people or use cases. Currently the de-facto standard is nalgebra, which wouldn't have been my first choice, but mint means that it is POSSIBLE to have a heterogeneous ecosystem that still works together.

Edit: The current state of math libraries, as I see it, is that there's lots of good choices and they all work together. Go us! 🎉

[Lokathor]

I mean even my wanting the Fast SIMD version and Osspial wanting the Slow Plain version shows that we need Mint somewhere in the system.

[kvark]

@Osspial Trying to make that ideal API math library is an interesting task for sure, attempted by many in the past. Maybe you'll get better luck at it, who knows. I'd be interested to watch the progress and potentially contribute.

What I don't expect though is for that effort to eliminate the other solutions. People using nphysics will always use nalgebra, no matter how good the new hotness is. People would always disagree on proper SIMD usage, on Y up versus down, on million other things (looks like @icefoxen just brought this point as well while I was writing).

So mint will always be needed.... but this isn't a concern, it works fine.

When I initially raised the concern during the call, it was specifically about cgmath: it's in a sad state, both in terms of API and maintenance. Writing an entirely new math library may be helpful in the longer term, but in the shorter - we need to find a way to maintain it or declare bankrupsy.

[Osspial]

Hot Take: Generics are a necessary evil, not an inherent good. The ideal vec/mat library for sizes 1-4 is probably one that is fully written out over time for all possible combinations and 100% non-generic.

I'm going to have to disagree with you there. Generics are a powerful tool for reducing API surface area, letting you see what APIs work everywhere and which only work in some places at a quick glance without having to go through pages of API docs on different types. Different types for everything fundamentally sacrifices that.

Currently the de-facto standard is nalgebra, which wouldn't have been my first choice, but mint means that it is POSSIBLE to have a heterogeneous ecosystem that still works together.

🤨

...that may have been more snarky than needed, but I guess in general my goal isn't to replace nalgebra. nalgebra has a place, and it's certainly widely used, but the complexity it introduces is a deal-breaker for many, including myself.

I'll put some extra emphasis on the following point, since this post is currently a cluttered wall of text: Replacing nalgebra is an infeasible task, and something we probably don't want to do. Replacing cgmath, on the other hand, is absolutely feasible, as abandoning it leaves a hole in the ecosystem that currently goes unfilled.

My goal would be to provide a replacement to cgmath that has a sane API. Maybe the solution to that is to overhaul cgmath in-tree, pulling out the cruft and simplifying the API in a way that lets other people expand upon it without being overly opinionated.

Edit: The current state of math libraries, as I see it, is that there's lots of good choices and they all work together. Go us! 🎉

I guess my problem is that there are a lot of different math libraries, but none of them actually do what I need them to do. They either sacrifice usability for functionality, aren't actually usable, or are used by so few people that exposing them in a public API is a burden upon everyone else.

I mean even my wanting the Fast SIMD version and Osspial wanting the Slow Plain version shows that we need Mint somewhere in the system.

You could certainly do it cleaner than via Mint. There's no reason a single library couldn't provide both SIMD and non-SIMD types and provide clean interop between the two, so that you can do Vector2 + Vector2SIMD and have everything Just Work.

When I initially raised the concern during the call, it was specifically about cgmath: it's in a sad state, both in terms of API and maintenance. Writing an entirely new math library may be helpful in the longer term, but in the shorter - we need to find a way to maintain it or declare bankruptcy.

Ideally, we can get it to the point where it doesn't need active maintenance. If it's got a clearly defined scope and lets more complex design problems get evolved and solved out-of-tree, I think it'd be in a pretty decent person. It just doesn't do that right now, and the design problems are there and left unsolved.

---

Out of curiosity, why do we need Mint's non-matrix types when the standard arrays exist and have wider ecosystem compatibility?

[Lokathor]

Think of them as newtype over arrays

[AlexEne]

Anyhow, time for my opinions: I think we should take the standard library's approach of building a small, fast, and relatively uncontroversial core that the rest of the ecosystem can build around.

I guess my problem is that there are a lot of different math libraries, but none of them actually do what I need them to do. They either sacrifice usability for functionality, aren't actually usable, or are used by so few people that exposing them in a public API is a burden upon everyone else.

I don't have the full context of what you actually need them to do and I think it would be helpful for readers of this thread if you would be describing what these things are in a bit more detail and how these things will be achieved by the new library (and cannot be achieved by current libraries like nalgebra/glam/etc.).

For example, the initial description mentions fast, but then SMID approach that glam seems to be critiqued. While the critique might be fair, how does that affect speed for this new library if we don't use SIMD? Do you propose another approach to SIMD ? Do we want to not use it at all? What's the proposal? I already highlighted the lack of a --ffast-math equivalent in Rust as far as I can tell.

What are the speed targets? glam-level speed? slower/faster. By how much?

[Lokathor]

Note that not all platforms support the same SIMD, or even SIMD at all. Particularly, WASM doesn't yet support SIMD, though there's progress in this area with a proposal written out, and there's experimental implementations, so we might have something in probably a year or two.

So, not all libs want to try to express themselves as SIMD operations, and then they'll just have very different runtime profiles based on what LLVM can divine about what's going on or not.

Other libs want to try to express themselves as SIMD and have fallbacks when it's not available for that platform (get it together ASM intrinsics team!!).

Two points I want to highlight:

• Any answer that's "for everyone" needs to work on Stable rust. Anything that's Nightly only is a 2nd rate citizen in the Rust world. Nightly optional for some bonus is okay, but Nightly only is no good.
• The glam person made a benchmark suite, mathbench, and I haven't looked into it to see how realistic the benchmarks are and such, but that could be an area to investigate

While I'm here I guess I'll give a ping to @bitshifter and see if they even want is swooping in on their lib to drown them with issues and PRs and such. If they want to just do their work in peace that's okay too.