Skip to content

Conversation

@juaristi22
Copy link
Collaborator

@juaristi22 juaristi22 commented Jul 22, 2025

Fix #4
Fix #9
Fix #7

This pr also addresses the changelog issue in the merging workflow

@juaristi22 juaristi22 changed the title Attempting to generalize SingleYearDataset and MultipleYearDataset classes Generalize SingleYearDataset and MultipleYearDataset classes Jul 22, 2025
@baogorek
Copy link
Collaborator

Hi @juaristi22 , I just pulled the code and tried things out. I don't have a lot to add on this one. I would like to try out the download_from_gcs function but I didn't know what paths are available. Is there a token needed? For the new data set classes, I'll need to work with them on a task to really get a feel for them, but everything seemed to work fine.


# Create SingleYearDataset for each year
for year, entities in years_entities.items():
self.datasets[year] = SingleYearDataset(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests for loading and saving datasets pls

Copy link
Collaborator Author

@juaristi22 juaristi22 Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean to hugging face? or the load and save methods? (the load and save methods are used in the test_dataset_classes.py tests)

Copy link
Collaborator

@nikhilwoodruff nikhilwoodruff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it- really clean!

@nikhilwoodruff nikhilwoodruff merged commit 5f71b32 into main Jul 28, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants