Add performance profiling harness and baseline document - #91
Merged
Conversation
Reproducible cProfile + tracemalloc harness with parameterized workloads covering both training and generation paths. Baseline numbers for NorESM2-MM (training + six generation workloads: global timeseries at varying N, multi-scale aggregation, degree-days impacts, no-noise climatology, and gridded output) are captured in docs/profiling_baseline.md alongside a ranked list of optimization targets to be tackled in follow-up PRs. The harness is standalone — one workload per invocation, output to scripts/profiling/results/ as .prof + .json for post-hoc analysis via scripts/profiling/analyze.py. No library behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
maritsandstad
approved these changes
Jul 30, 2026
maritsandstad
left a comment
maritsandstad
left a comment
Collaborator
There was a problem hiding this comment.
This looks good and very useful to me, and doesn't really interfere with anything. The problem with the notebook not running is not to do with this PR, so as far as I can see this can probably just go in as is
Collaborator
|
Alright, so the failing notebook here, fails because it is too resource intensive and github actions kills it. Don't think I will bother to solve that now before merging... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What's here
scripts/profiling/run_profile.py— one workload per invocation, output toscripts/profiling/results/as.prof+.jsonscripts/profiling/analyze.py— summary table, hotspot leaderboard, and per-run drill-downdocs/profiling_baseline.md— baseline results and identified targetsNo library behavior change
Follow-up PRs land the optimizations that populate the "Optimization outcomes" placeholder in the doc.
Test plan
python scripts/profiling/run_profile.py --listprints workloadspython scripts/profiling/run_profile.py --workload gen_no_noise --tag smokecompletes and writes a.prof+.jsonpython scripts/profiling/analyze.pyprints a coherent table🤖 Generated with Claude Code