Issue 3 (Trending topics service: robustness + offline mode + source options)

[abby-ql]

Owner:

What’s already done: app/core/trends.py already supports NewsAPI or RSS fallback; extraction logic exists; DB stores TrendTopic.

Goal: Make trends ingestion reliable for demo + backtesting without needing live APIs.

Tasks

• Add offline ingest mode:

  • if TRENDS_SOURCE=snapshot, load data/trends_snapshot_today.json and store it as TrendTopic rows

• Improve extraction determinism:

  • ensure stable sorting (volume desc, tie-break alphabetical)
  • filter out obvious generic terms with config (move drop list into config file)

• Optional (nice to have): add GOV.UK Atom feed option as an alternative open-licensed UK source

Acceptance criteria

• POST /trends/ingest works in 3 modes:

  1. RSS default
  2. NewsAPI if key present
  3. Snapshot mode with no network

• GET /trends/current returns consistent, sorted list

Tests

• Unit test: extract_trend_terms() deterministically returns same top terms given a fixed input headline list
• Integration-ish test: snapshot ingest populates DB and endpoint returns them

Dependencies

• None (snapshot mode avoids any key/network dependency)

[daohongliu]

I don't understand the need for having an offline ingestion mode. The snapshot is already being stored when we seed the database, so nothing new is ever added by doing this.
I have implemented stable sorting.

[abby-ql]

@daohongliu Thanks! stable sorting is definitely part of the solution.The snapshot mode is not quite about adding new trends, but about reproducibility. though we seed the DB we still need a deterministic /trends/ingest -> /alerts/run flow that works without network/API keys and behaves the same in demo and CI. feel free to let me know if i am unclear or wrong in anything

[daohongliu]

@abby-ql Thanks! I've implemented snapshot mode and added the atom feed option.