Skip to content

fix(insights): replace density threshold with mean intra-degree - #752

Open
HUAN2022A wants to merge 1 commit into
nashsu:mainfrom
HUAN2022A:fix/issue-731-replace-density-threshold-with-m
Open

HUAN2022A wants to merge 1 commit into
nashsu:mainfrom
HUAN2022A:fix/issue-731-replace-density-threshold-with-m

Conversation

@HUAN2022A

Copy link
Copy Markdown

Fixes #731

Problem

On large wikis the Sparse cluster knowledge-gap warning fires on nearly every community. The reporter's math is correct and confirmed against the code: cohesion is the classic density 2E / (n(n-1)), which decays as O(1/n). With the fixed threshold cohesion < 0.15, no community larger than ~55 pages (at the reporter's mean degree of 8.1) can ever pass, so real, well-linked topic clusters are permanently flagged — and the suggested remedy ("add links") pushes users toward filler wikilinks.

Fix (reporter's option 1 — scale-invariant)

  • CommunityInfo gains meanIntraDegree = 2 * intraEdges / nodeCount (average links per page into the same community), computed alongside the existing cohesion in detectCommunities (cohesion itself is kept — it is asserted in existing tests and remains meaningful).
  • New shared constant SPARSE_CLUSTER_MIN_MEAN_INTRA_DEGREE = 2 in wiki-graph.ts, consumed by both flag sites so they cannot drift apart:
    • detectKnowledgeGaps (graph-insights.ts): condition becomes meanIntraDegree < 2 && nodeCount >= 3, and the description now states something actionable: "N pages average X.X links to other pages in this cluster".
    • the community panel "!" marker in graph-view.tsx, which had a second, independent hard-coded 0.15.

The reporter's own numbers validate the boundary: at mean degree 8.1, a 55-page community scores cohesion exactly 0.15 (n=56 drops below) — precisely the "~55 pages max" they measured.

Testing

  • wiki-graph-analysis.test.ts: two 4-cliques + a bridge → each community meanIntraDegree === 3, plus a nodeCount-weighted conservation assertion (Σ n·meanIntraDegree === 2 × intra-community edges) that holds for any Louvain partition.
  • graph-insights.test.ts (new): a 500-node community with mean intra-degree 8 (cohesion 0.016) is not flagged (previously always flagged); a 10-node community at 0.8 is flagged with the new description; boundary at exactly 2.0 not flagged; <3 nodes not flagged; isolated/bridge paths unregressed.
  • npm run test:mocks: 133 files / 1883 tests passed · npm run typecheck: clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Insights] "Sparse cluster" is unactionable on large wikis — cohesion is raw graph density, so every real topic cluster is flagged

1 participant