Feat: Optimize data loading with LRU cache and schema validation

[DebasmitaBose0]

Closed #595

Problem:
The utility in data_loader.py reads and parses the projects.json file from disk on every invocation. This disk I/O overhead slows down the recommendation pipeline and is inefficient.
Acceptance Criteria:

• Decorate the data loading function with functools.lru_cache to memoize the loaded JSON object in memory.
• Introduce a startup check to validate the JSON file's integrity and schema before caching.

[vercel]

@DebasmitaBose0 is attempting to deploy a commit to the komalsony234-1530's projects Team on Vercel.

A member of the Team first needs to authorize it.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds caching to the project JSON loader to avoid repeatedly reading/parsing the same file.

Changes:

• Introduces functools.lru_cache on load_all_projects()
• Adds a new functools import near the loader function

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[komalharshita]

Good optimization direction overall — reducing repeated disk reads for projects.json using lightweight in-memory caching makes sense for this project and keeps the implementation simple.

Things done well:

• Properly scoped backend-only change
• Lightweight dependency-free optimization
• Avoided unnecessary architectural complexity
• Good use case for lru_cache

However, a few important issues should be addressed before merge:

• The issue requested both caching and schema/integrity validation, but the PR currently only implements caching.
• There is no cache invalidation strategy, so stale project data may persist during runtime if projects.json changes.
• Since load_all_projects() takes no arguments, maxsize=128 is unnecessary/confusing — only one cached entry can exist.
• import functools should be moved to the top import section for consistency.
• No tests were added for caching behavior or validation logic.

The optimization idea is good, but the implementation should be completed/refined before approval.