Perf/optimize tfidf

[OmBapodra]

🚀 Performance Improvements (Benchmark Results)

I wrote a benchmark script to test the time complexity of get_recommendations before and after moving the tokenization and _idf scoring outside of the single-project scoring loop.

Testing constraints: 22 Projects loaded in the dataset.

Before Optimization (O(N²))

• Time taken: 0.1537 seconds per request
• Scaling: Would grow exponentially slower as the database of projects increases.

After Optimization (O(N))

• Time taken: 0.0103 seconds per request
• Scaling: Linear. Safe for large datasets.

📈 Result

• Speedup: ~14.93x faster on just 22 projects!

[vercel]

Someone is attempting to deploy a commit to the komalsony234-1530's projects Team on Vercel.

A member of the Team first needs to authorize it.

[komalharshita]

Thanks for the optimization! Moving the tokenization and IDF computation outside the per-project scoring loop is a clean improvement that removes redundant work while keeping the recommendation logic unchanged.

The benchmark results clearly demonstrate the performance gains (~15x faster on the provided dataset), and the optimization should scale much better as the project dataset grows. The implementation is well-scoped and focused on performance without introducing unnecessary complexity.

Overall, this is a valuable optimization for the recommendation engine. Nice work! 🚀

Approved for merge.