Skip to content

Add 64-bit versions of core power of 2 functions #106606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

aaronfranke
Copy link
Member

@aaronfranke aaronfranke commented May 19, 2025

This PR adds 64-bit versions of the power of 2 functions to core. Almost all uses throughout the codebase are still using the 32-bit versions of the functions, but now casts have been added to make this explicit.

A 64-bit version of next_power_of_2 is required to fix FileAccessCompressed with file sizes above 2 GiB. As for the other functions, I figured it would be consistent to change all of them at once, and this enables any other parts of the codebase to migrate to 64-bit if needed. But if it's not desired, I can remove most of the 64-bit versions of the functions.

The 32-bit versions of the functions previously used unsigned int, but this PR updates them to uint32_t.

I changed nearest_shift to use a do/while loop, so that the i variable can be unsigned, avoiding casting from signed to unsigned every loop iteration. The old >= 0 check would always be true for unsigned ints.

I believe this code is correct, but since these are important core functions, we should test this.

@Ivorforce
Copy link
Member

Related to #91664, which also touched these functions.

Copy link
Member

@Ivorforce Ivorforce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes total sense to have uint64_t versions.
I think this is a good opportunity to clean up surrounding mess. You cleaned up some of it already, but I'd also like to see at least constexpr being added to all functions (instead of static and inline).

While you're at it, please also remove CowData::next_po2, and call next_power_of_2 instead.

I'm also a bit uneasy about all of this rarely used boilerplate being in typedefs.h. Math namespace doesn't seem appropriate either. Perhaps we should add a new file for this, and include it where needed?

@aaronfranke
Copy link
Member Author

@Ivorforce Yeah, I agree, typedefs.h feels like the wrong place for these functions. Since, you know, they're not typedefs. But I am not proposing to change that in this PR.

Copy link
Member

@Ivorforce Ivorforce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks for making sure all callers are still calling what they were calling earlier.

Just one more comment for a small change.

Yeah, I agree, typedefs.h feels like the wrong place for these functions. Since, you know, they're not typedefs. But I am not proposing to change that in this PR.

That's fine. We can propose a move some other time.

@@ -39,7 +39,7 @@ class FileAccessCompressed : public FileAccess {
bool writing = false;
uint64_t write_pos = 0;
uint8_t *write_ptr = nullptr;
uint32_t write_buffer_size = 0;
uint64_t write_buffer_size = 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sneaking some changes in here already? :P

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making FileAccessCompressed call the 64-bit version is the motivation behind the rest of the changes, and is actually enough to make the function not crash for the test in Calinou's comment (to my surprise).

@Ivorforce Ivorforce modified the milestones: 4.x, 4.5 May 19, 2025
@fire fire requested a review from a team May 19, 2025 22:48
Copy link
Contributor

@Repiteo Repiteo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would benefit from templates for the sake of consolidation, but this code should probably be migrated to math_funcs.h in the future anyway, so it's not a blocker here.

@Ivorforce
Copy link
Member

Ivorforce commented May 20, 2025

This would benefit from templates for the sake of consolidation, but this code should probably be migrated to math_funcs.h in the future anyway, so it's not a blocker here.

We already have a templated version of next_power_of_2, but it's kinda complicated and the current implementation ended up incorrect (inefficient because of wrong math). I think this is one case where it's better to just copy paste some code and call it a day.

@fire
Copy link
Member

fire commented May 20, 2025

I don’t suggest changing code but what was the reasoning between unsigned and signed 64 bit integers.

@aaronfranke
Copy link
Member Author

@fire These functions exclusively work with only positive numbers due to the way they are implemented with bit shifting, so having them be unsigned is essentially self-documenting this. Negative numbers would not work. Making these unsigned also means you can go up to 2^31 and 2^63 whereas signed ints can only hold up to 2^31 - 1 and 2^63 - 1.

Copy link
Member

@hpvb hpvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM discussed in the core meeting.

Copy link
Member

@Calinou Calinou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me. Combining this PR with #106190 makes the testing project from #106190 (review) work 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants