Skip to content

Updates to public quilt data #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Conversation

mfs4rd
Copy link

@mfs4rd mfs4rd commented Nov 4, 2024

Purpose

This updates the quilt paths used to download data used in the paper to match what is on the publicly available dataset

Key changes

  • Downloads of full datasets have been updated. This includes downloading multiple directories at once when the original data has been split into multiple numbered folders within the package's root directory
  • Downloading of small test datasets has been removed as there is no subset of test data provided within the public dataset
  • Additional updates to analysis code and data retrieval

@mfs4rd mfs4rd requested a review from vianamp November 4, 2024 21:21
@vianamp
Copy link
Collaborator

vianamp commented Nov 4, 2024

@mfs4rd can you add some description of what this PR is doing? For example why removing the test dataset downloading?

@mfs4rd
Copy link
Author

mfs4rd commented Nov 5, 2024

@vianamp Updated PR description. Test data downloading has been removed since there are no subsets of the data provided in the public dataset as in the private dataset. We would need to implement a method of properly sampling the dataset and trimming the relevant manifests if we want to keep this feature.

Copy link
Collaborator

@vianamp vianamp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, Filip. Looks great!

@@ -48,7 +48,8 @@ make_cellprofiler_image_set \
--defaults_json fish_morphology_code/cellprofiler/cellprofiler_image_set_defaults.json \
--path_key rescaled_2D_fov_tiff_path \
--local_path ./quilt_data_contrasted_test \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove test from the path since there is no longer a test set?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants