Repository navigation
Add Lassen trace conversion and data-driven synthetic job generation - #80
Merged
Merged
Conversation
JaeseungYeom
commented
Oct 5, 2026
JaeseungYeom
commented
Collaborator
- convert Lassen CSM pbatch records to Unix epoch timestamps
- resolve Daylight Saving Time ordering anomalies
- generate simulation traces with duration in place of begin/end times
- add configurable filtering and empirical sampling for synthetic jobs
- document input schemas, outputs, sampling logic, and usage
- convert Lassen CSM pbatch records to Unix epoch timestamps - resolve Daylight Saving Time ordering anomalies - generate simulation traces with duration in place of begin/end times - add configurable filtering and empirical sampling for synthetic jobs - document input schemas, outputs, sampling logic, and usage
- treat the application requirements table as a workload allowlist - exclude applications absent from apps.csv during table generation and dispatch - retain all compatible measured modes in ground-truth performance tables - allow prediction tables to have different mode coverage from ground truth - generate application-average predictions for measured modes - generate RAJAPerf predictions for every system mode - filter systems whose actual runtime exceeds the maximum wall-time limit - prefer any feasible system over infeasible systems regardless of queue wait - double underestimated predicted time limits until they cover actual runtime - cap submitted limits at the configured platform maximum - drop and report jobs when no system can complete them within the maximum - record submitted limits, doubling counts, and dropped-job totals - document the optimistic assumption that failed limit attempts consume no resources - add a three-policy prediction-study runner with validated completion markers - summarize ideal, application-average, and RAJAPerf results across ten traces - relocate production tables and configuration under experimental/multi-cluster - move dispatcher test fixtures into a dedicated testdata directory - source generated traces from experimental/synthesize/lassen/synthetic_traces - ignore generated traces and prediction-study results - emit generated CSV traces with Unix line endings - add Python and MPI regression coverage for allowlists, wall-time correction, infeasible placement, dropped jobs, and prediction coverage
- make exit_status optional unless --successful-only is requested - validate that historical time limits are positive - sort eligible jobs by submission time before sampling arrival windows - add --max-time-limit to enforce a target platform wall-time cap - normalize runtimes that exceed their effective time limits - apply minimum-duration filtering after runtime normalization - sample time limits conditionally from the selected runtime bucket - cap sampled runtimes when fractional bucket matching selects a shorter limit - guarantee duration does not exceed time_limit in generated jobs - emit generated CSV files with Unix line endings - document time-limit units, normalization, sampling, and Lassen’s 12-hour cap - add coverage for default limits, custom caps, invalid limits, and line endings
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.