Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine: make immediate recovery to recover immediately #4069

Merged

Conversation

btovar
Copy link
Member

@btovar btovar commented Feb 18, 2025

The m.tune parameter immediate-recovery was meant to immediately create recovery tasks when the only worker containing a temp file was lost. However, it was only activated for the input files of tasks currently running at the lost worker. This pr fixes that.

The use case is workflows such as dynamic map reduce where temporary files are undeclared as soon as they are not needed anymore, and any declared file will be eventually used in the future. Since tasks are created dynamically, the management of files cannot be left to taskvine proper.

Merge Checklist

The following items must be completed before PRs can be merged.
Check these off to verify you have completed all steps.

  • make test Run local tests prior to pushing.
  • make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
  • make lint Run lint on source code prior to pushing.
  • Manual Update: Update the manual to reflect user-visible changes.
  • Type Labels: Select a github label for the type: bugfix, enhancement, etc.
  • Product Labels: Select a github label for the product: TaskVine, Makeflow, etc.
  • PR RTM: Mark your PR as ready to merge.

@btovar btovar requested a review from dthain February 18, 2025 17:31
@btovar btovar merged commit 54aaae7 into cooperative-computing-lab:master Feb 20, 2025
10 checks passed
@btovar btovar deleted the fix_immediate_recovery branch February 20, 2025 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant