feat(citations): add source ranking to tests and tooling (closes #933)

[arcgod-design]

What

Adds a source-ranking algorithm to the citation-utils module and tests/tooling — closing issue #933's "Add source ranking to tests and tooling".

Previously build_sources() returned sources in the order ChromaDB returned them (cosine-similarity nearest-to-farthest), with no weighting or tie-breaking. This PR adds rank_sources() which produces a deterministic, explainable ranking from three human-intuitive signals: relevance, recency, and authority.

Why

ChromaDB retrieval returns sources ordered by pure cosine distance. In the real world:

• A very recent doc may be more important than a marginally more similar older one.
• A README.md or docs/official-spec.md carries more authority than notes.txt.
• Equal-score source sets (same doc across multiple chunks) need a deterministic tie-breaker for reproducible frontend rendering.

The ranking function exposes a tunable RankWeights dataclass so a maintainer can adjust the signal mix, and the tooling CLI lets a contributor experiment offline without ChromaDB.

What changed

Backend service

• backend/services/citation_utils.py — the existing file is extended (backward-compatible; build_sources unchanged):

  New dataclasses:

  • RankWeights (relevance=0.65, recency=0.10, authority=0.25; .normalised() handles zero-sum gracefully)
  • RankedSource (source dict, composite score, components breakdown)

  New free functions:

  • _parse_isoformat(value) — ISO-8601 / naive / aware → UTC datetime
  • _relevance_score(distance) — cosine distance 0..2 → 1..0; NaN/non-numeric → 0
  • _recency_score(created_at, now) — 365-day linear decay; future clock-skew → 1; missing → 0
  • _authority_score(filename, rules, base) — filename-pattern matching with a default ruleset
  • rank_sources(sources, *, relevance_distances, recency_timestamps, weights, authority_rules, authority_base, now) — composite score, deterministic tie-break (filename then chunk ascending)

  Default authority rules: README.md 0.55, docs/**/*.md 0.70, backend/services/ 0.55, backend/routes/ 0.45, backend/tests/ 0.30, frontend/src/ 0.40, keyword official/spec/rfc 0.85, unknown 0.40.

Tooling

• scripts/rank_sources.py — offline CLI that reads a JSON source list (stdin or file), pulls optional distance and created_at metadata from each entry, calls rank_sources, and outputs ranked JSON or plaintext. Accepts --relevance-weight, --recency-weight, --authority-weight. Exit 2 on bad input, 1 on empty list. Example:

  $ cat sources.json | python scripts/rank_sources.py --plain
  Ranked 3 source(s) (relevance_w=0.65 recency_w=0.10 authority_w=0.25):
     1. [0.512] docs/csrf-protection.md chunk=0 (rel=0.50 rec=0.20 auth=0.70)
     2. [0.463] README.md              chunk=2 (rel=0.50 rec=0.00 auth=0.55)
     3. [0.400] uncategorized.txt      chunk=0 (rel=0.50 rec=0.00 auth=0.40)

Tests

• backend/tests/test_citation_ranking.py — 41 pytest cases, all pass:
  • TestParseIsoformat (5): ISO-Z, naive, aware UTC, missing, garbage → None.
  • TestRelevanceScore (5): 0=1 / 2=0 / 1=0.5 / NaN=0 / non-numeric=0.
  • TestRecencyScore (8): today=1, 365d=0, 400d=0, future=1, missing=0, 180d~0.5.
  • TestAuthorityScore (9): README.md 0.55, docs/foo.md 0.70, frontend/src/ 0.40, backend/services/ 0.55, unknown 0.40, official keyword 0.85, empty 0.40, custom rules override.
  • TestRankWeights (3): defaults, zero-sum normalised → all-0, proportional scale 0.7/0.1/0.2 unchanged, as_dict round-trip.
  • TestRankSources (8): empty→[], single source has score, higher relevance→first, tie-break by filename+chunk, all-zero weights→key sort, recency bias, now-kwarg determinism, custom authority override, missing relevance→0.5, short dist list padded.
  • TestRankSourcesCLI (3): JSON input→ranked list, plain mode output, missing input→exit 2.
  • Existing test_citations.py (21 cases) still passes unmodified — backward-compatible.

Verification

$ cd backend && pytest tests/test_citation_ranking.py tests/test_citations.py -v
============================= 62 passed in 4.05s ==============================
      (41 new + 21 existing)

$ ruff check backend/services/citation_utils.py \
           backend/tests/test_citation_ranking.py scripts/rank_sources.py
All checks passed!

$ ruff format --check <same files>
3 files already formatted

$ python scripts/rank_sources.py --plain < sources.json
Ranked 2 source(s) (relevance_w=0.65 recency_w=0.10 authority_w=0.25):
    1. [0.500] docs/a.md chunk=0 (rel=0.50 rec=0.00 auth=0.70)
    2. [0.463] README.md chunk=1 (rel=0.50 rec=0.00 auth=0.55)

The retrieve_context flow in rag_service.py is not changed — the ranking function is opt-in. A future PR can wire rank_sources into the chat pipeline after build_sources (out of scope per issue #933's "tests and tooling" title).

Acceptance criteria

• Implement the change in the tests and tooling.
• Add or update tests, docs, or validation for the touched path.

Closes #933.

[vercel]

@arcgod-design is attempting to deploy a commit to the Darshan's projects Team on Vercel.

A member of the Team first needs to authorize it.

[imDarshanGK]

@arcgod-design