Skip to content

24929: Updates the reduce_data algorithm, MAJOR #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
cademack merged 43 commits into from
Jan 20, 2026
Merged

24929: Updates the reduce_data algorithm, MAJOR #743

cademack merged 43 commits into from
Jan 20, 2026

Conversation

@cademack
Copy link
Member

@cademack cademack commented Jan 20, 2026 •
edited by howsoRes
Loading

Updates reduce_data to use an entirely new algorithm. This new algorithm does not yet support the usage of the prediction stat threshold parameters, so the following parameters are removed from the reduce_data endpoint:
abs_threshold_map, delta_threshold_map, and rel_threshold_map and batch_size

cademack and others added 30 commits January 13, 2026 16:12
@cademack
grab random cases to remove
51049cc
@cademack
remove some unused stuff
495f44a
@cademack
one last unused var
f815bf1
@howsoRes
randomize test
5fbefc9
@cademack
make it one pass
457700d
@cademack
crappy python translation
f887770
@cademack
fixed
67ee26d
@cademack
msimatch paren
7448590
@cademack
impl
e3b1b93
@cademack
udpaters
ca752b8
@cademack
now testing this version
3a539a0
@cademack
nitpicking
022331a
@cademack
optimization
f967c97
@howsoRes
blur weights, 2 passes
e5769dd
@howsoRes
Merge branch 'weight-blur' into 24929-coreset-lit
8d0781e
@howsoRes
added comment
f99543d
@cademack
switch to min mode
b8c4aea
@howsoRes
remove zero weight and duplicates
6dd2e65
@howsoRes
Merge branch 'weight-blur-2' into 24929-coreset-lit
cc57cc7
@cademack
updating comments and such
902e976
@howsoRes
comment out blurring, add performance hack
df2eb1d
@howsoRes
cleanup
d0e43c1
@howsoRes
Merge branch 'main' into 24929-coreset-res
a2619d6
@howsoRes
update extremely large value
4cb468b
@cademack
batch size smarts
c550725
@cademack
dynamic batch size, case cleanup, and more
698ffca
@cademack
unnecessary trunc
1043b5e
@howsoRes
ensure reduce_max_cases is an int
ced55b5
@howsoRes
fixes early out logic
d8cf3ac
@howsoRes
code cleanup, removed unused parameters
fc1ae9e
@cademack cademack requested a review from a team as a code owner January 20, 2026 16:39
@howsohazard howsohazard enabled auto-merge (squash) January 20, 2026 20:31
@howsohazard howsohazard disabled auto-merge January 20, 2026 20:33
@cademack cademack enabled auto-merge (squash) January 20, 2026 20:35
@cademack cademack merged commit 44022e9 into main Jan 20, 2026
10 checks passed
@cademack cademack deleted the 24929-coreset-res branch January 20, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@howsoRes howsoRes howsoRes approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@cademack @howsoRes