perf(data_loader): cache JSON data in memory to avoid repeated disk reads - #13
Conversation
…eads Implements a simple module-level cache for projects.json to eliminate repeated disk I/O on every call to load_all_projects(). Changes: - Add _projects_cache module variable - Modify load_all_projects() to use cache after first read - Add clear_cache() function for test isolation - Update tests to call clear_cache() in setup_module() - Add benchmark script showing 847x speedup Fixes komalharshita#11
komalharshita
left a comment
There was a problem hiding this comment.
This is a solid improvement overall.
The caching logic is simple, clean, and fits the project really well. I also like that you added clear_cache() for test isolation and included benchmark results to show the performance improvement. The code stays beginner-friendly while still improving efficiency, which is exactly the direction we want for DevPath.
I reviewed the updated files and everything looks good to merge.
|
Nice work on this optimization. |
Problem
As described in #11, utils/data_loader.py currently reads projects.json from disk on every call to load_all_projects().
Solution
Implemented in-memory cache with module-level variable:
Benchmarks
Testing
All 25 tests pass.
Fixes #11