Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(lineage_fate_correlation): use cached clone trajectory data #659

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e49ee6d
chore(scripts): add script to generate clone trajectory data
cameronraysmith Mar 25, 2025
e2b4e5c
fix(datasets): add clone trajectory data sets
cameronraysmith Mar 25, 2025
695a98c
fix(time_fate_correlation): refactor create_time_lineage_fate_correla…
cameronraysmith Mar 26, 2025
07b0ee3
fix(workflows): use create_time_lineage_fate_correlation_plot task fu…
cameronraysmith Mar 26, 2025
18e70f9
chore(workflows): revert cache to `2024.8.15`
cameronraysmith Mar 26, 2025
e7f2196
chore(workflows): bump lineage fate correlation cache to `2024.8.15.6`
cameronraysmith Mar 26, 2025
ef3518b
fix(constants): disable demo flag
cameronraysmith Mar 26, 2025
776ed2c
fix(main_workflow): reenable all data sets
cameronraysmith Mar 26, 2025
d194fea
fix(lineage_fate_correlation): absorb nan cleanup from plotting funct…
cameronraysmith Mar 28, 2025
4f557a9
fix(tasks): compute clone trajectories in time lineage fate correlati…
cameronraysmith Mar 28, 2025
54178c3
refactor(_trajectory): migrate analysis code from plots to analysis
cameronraysmith Mar 28, 2025
71174d2
fix(trajectory): use input times and error on empty centroids
cameronraysmith Mar 28, 2025
16d382e
fix(analysis): include trajectory functions
cameronraysmith Mar 28, 2025
38809fc
fix(plots): use trajectory functions from analysis package
cameronraysmith Mar 28, 2025
6f50788
fix(tasks): use trajectory functions from analysis package
cameronraysmith Mar 28, 2025
5c38692
fix(scripts): use trajectory functions from analysis package
cameronraysmith Mar 28, 2025
d14d288
chore(workflows): lineage fate correlation cache `2024.8.15.7`
cameronraysmith Mar 28, 2025
9fcd1af
fix(make): support cleaning up workflow artifacts for multiple data sets
cameronraysmith Mar 28, 2025
5693fe1
chore(archive): add larry data assembly archive
cameronraysmith Mar 30, 2025
deb5fca
chore(archive): add larry data subpopulation extraction archive
cameronraysmith Mar 30, 2025
3bb108b
chore(analysis): add subpopulation extraction from archive
cameronraysmith Mar 30, 2025
634278e
test(analysis): add subpopulation extraction smoke test
cameronraysmith Mar 30, 2025
9b9491c
fix(fixture): add function to generate larry data fixture
cameronraysmith Mar 30, 2025
cb4064c
test(data): add larry multilineage 50x6 fixture
cameronraysmith Mar 30, 2025
0b0388a
test(conftest): add adata_larry_multilineage_50_6 fixture
cameronraysmith Mar 30, 2025
43a9aef
test(trajectory): add smoke tests for trajectory module
cameronraysmith Mar 30, 2025
ce26b54
fix(tasks): use vendored data for multilineage clone trajectory
cameronraysmith Mar 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 25 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -228,13 +228,32 @@ run-async: ## Run registered workflow (async).
run-check-image: ## Check workflow image exists.
crane ls $(WORKFLOW_IMAGE) | grep "$(GIT_REF)\|$(GIT_SHA)\|$(GIT_SHA_SHORT)"

TEST_DATASET_NAME ?= simulated
TEST_DATASET_NAMES ?= simulated larry larry_neu larry_mono larry_multilineage

workflow-clean-outputs: ## Clean testing dataset and model outputs. make workflow-clean-outputs TEST_DATASET_NAME=simulated.
rm data/external/$(TEST_DATASET_NAME).h5ad || true
rm data/processed/$(TEST_DATASET_NAME)_* || true
rm -r models/$(TEST_DATASET_NAME)_model* || true
rm -r reports/$(TEST_DATASET_NAME)_model* || true
# Function to clean a single dataset
define clean-dataset
@echo "Cleaning outputs for $(1)..."
rm -f data/processed/$(1)_* || true
rm -rf models/$(1)_model* || true
rm -rf reports/$(1)_model* || true

endef

# Function to print external dataset remove command
define print-rm-external
@echo " rm -f data/external/$(1).h5ad"

endef

workflow-clean-outputs: ## Clean testing dataset and model outputs. make workflow-clean-outputs TEST_DATASET_NAMES="simulated pancreas"
@echo "Cleaning outputs for datasets: $(TEST_DATASET_NAMES)"
$(foreach dataset,$(TEST_DATASET_NAMES),$(call clean-dataset,$(dataset)))
@echo ""
@echo "To remove external datasets, run one of these commands:"
$(foreach dataset,$(TEST_DATASET_NAMES),$(call print-rm-external,$(dataset)))
@echo ""
@echo "Or remove all with:"
@echo " rm -f data/external/{$(shell echo $(TEST_DATASET_NAMES) | tr ' ' ',')}.h5ad"

workflow-clear-cache: ## Clear local cache db from ~/.flyte/local-cache/cache.db.
pyflyte local-cache clear
Expand Down

Large diffs are not rendered by default.

Loading