Skip to content

Issues: NVIDIA/NeMo-Curator

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

66 Open 113 Closed
66 Open 113 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

support writing compressed JSON format enhancement New feature or request jira
#571 opened Feb 25, 2025 by chenrui17
@ayushdg
1
Re-add test_read_data_different_columns_blocksize bug Something isn't working jira
#557 opened Feb 18, 2025 by sarahyurick
1
@praateekmahajan
Extend support to non-English languages for PII Deidentifier enhancement New feature or request jira
#554 opened Feb 18, 2025 by hamsarajan
1
[FEA] Add Sampling-Based Clustering in SemDedup enhancement New feature or request jira
#538 opened Feb 11, 2025 by VibhuJawa
@praateekmahajan
[FEA] Remove GPU-related messages on CPU-only servers enhancement New feature or request jira
#535 opened Feb 10, 2025 by miguelusque
The output list length of open question pipeline is wrong ？ bug Something isn't working jira
#533 opened Feb 10, 2025 by leadtekleadtek
@ryantwolf
1
Running Curator under SLURM Cluster
#531 opened Feb 7, 2025 by philm001
@ryantwolf
6
Exact / Fuzzy Duplicate Removal Improvements at Scale
#529 opened Feb 7, 2025 by praateekmahajan
1
Refactor separate_by_metadata and Partition On to use the same code paths. enhancement New feature or request jira
#524 opened Feb 5, 2025 by VibhuJawa
@VibhuJawa
torch.OutOfMemoryError: CUDA out of memory. while performing peft curation with sdg on default configs bug Something isn't working jira
#520 opened Feb 5, 2025 by mohit5tech
1
8
Unifying Deduplication API Modules enhancement New feature or request jira
#516 opened Feb 4, 2025 by praateekmahajan
4
Inconsistent filter modules behavior enhancement New feature or request jira
#515 opened Feb 4, 2025 by zxnie
1
Consecutive execution of fuzzy deduplication on different columns fails with errors bug Something isn't working jira
#501 opened Jan 29, 2025 by sarahyurick
@ayushdg
3
[FEA] Enable Best Fit Packing enhancement New feature or request jira
#492 opened Jan 21, 2025 by VibhuJawa
Post to internal slack if nightly tests fail enhancement New feature or request jira
#488 opened Jan 17, 2025 by praateekmahajan
nemo_curator.utils.distributed_utils.read_data doesn't work for my own parquet dataset unless cleaning text by myself bug Something isn't working jira
#482 opened Jan 16, 2025 by RickyShi46
@ayushdg
false_positive_check=True need to add in ThWiki tutorial bug Something isn't working jira
#481 opened Jan 15, 2025 by sophiayyya
@ayushdg
1
fuzzy_dedup OOM issue bug Something isn't working jira
#471 opened Jan 7, 2025 by chenrui17
@ayushdg
2
Fuzzy Duplicates Identification fails on batched_merge_and_write when document dataset is read with blocksize bug Something isn't working jira
#462 opened Jan 2, 2025 by praateekmahajan
1
Enable test_aegis_classifier and test_instruction_data_guard_classifier jira
#456 opened Dec 23, 2024 by sarahyurick
@sarahyurick
Remove dependency on convert_str_id_to_int in FuzzyDedup Scripts enhancement New feature or request jira
#447 opened Dec 20, 2024 by praateekmahajan
Fuzzy dedup - minhash buckets and jaccard_map_buckets bug Something isn't working jira
#430 opened Dec 13, 2024 by ms-leemina
@ayushdg
1
Update minhash API after 25.02 enhancement New feature or request jira
#426 opened Dec 11, 2024 by ayushdg
@praateekmahajan
PII Modifier fails to load on worker sporadically raising cannot reshape array of size bug Something isn't working jira
#424 opened Dec 11, 2024 by praateekmahajan
@singhva
1
Pii Modifier should work with DocumentDataset on cudf enhancement New feature or request jira
#418 opened Dec 10, 2024 by praateekmahajan
ProTip! Updated in the last three days: updated:>2025-03-05.