Add blog plot generation with real compilation results#44
Merged
Conversation
The loop variable `rec` was a float (from keep_us list) but the code tried to access rec.code_hash. Fixed by zipping with the actual _BenchmarkRecord objects. https://claude.ai/code/session_01CZ2Vu6Y6vjtqEZLAsjcMF3
- generate_blog_plots.py: runs compile + optimize + benchmark pipeline - Normal model: 38.5x faster than PyTensor (0.06 vs 2.46 us/eval) - LinReg model: 7.1x faster (0.49 vs 3.50 us/eval) - Hierarchical model: 8.8x faster (0.57 vs 5.04 us/eval) - Optimization progress plots (autoresearcher-style) - Waterfall and timeline plots for both models - Benchmark comparison bar chart https://claude.ai/code/session_01CZ2Vu6Y6vjtqEZLAsjcMF3
Adds make_gp_model() factory using pm.gp.Latent with ExpQuad kernel, includes GP in the compile+optimize pipeline and benchmark comparison. https://claude.ai/code/session_01CZ2Vu6Y6vjtqEZLAsjcMF3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
examples/generate_blog_plots.pythat runs the full compile → optimize → benchmark pipeline on real modelsplot_optimization_progresswhere annotation loop used float values instead of_BenchmarkRecordobjectsBenchmark Results (real runs)
Generated Plots
benchmark_comparison.png— side-by-side speed + speedup barsopt_progress_normal.png/opt_progress_hierarchical.png— autoresearcher-style optimization progressopt_waterfall_*.png— contribution of each optimization stepopt_timeline_*.png— wall-clock event timelineTest plan
uv run python examples/generate_blog_plots.pyplot_optimization_progressworks from TSVhttps://claude.ai/code/session_01CZ2Vu6Y6vjtqEZLAsjcMF3