diff --git a/.env.example b/.env.example index 328fcd3..179fd68 100644 --- a/.env.example +++ b/.env.example @@ -2,3 +2,7 @@ # 註冊:https://stackapps.com/apps/oauth/register # 複製此檔為 .env 並填入 key(.env 已 gitignored) SE_API_KEY= + +# OpenAI API key — Module D LLM roles (gate / extraction / aggregator / narrative) +# Provider 走 provider-neutral seam,預設 OpenAI(ADR 0002 / PLAN ▼Q2) +OPENAI_API_KEY= diff --git a/.gitignore b/.gitignore index 3db4967..e070552 100644 --- a/.gitignore +++ b/.gitignore @@ -25,3 +25,6 @@ data/* # OS .DS_Store + +# Brainstorming visual companion (local mockups) +.superpowers/ diff --git a/CONTEXT.md b/CONTEXT.md index 3b0ccc7..2285862 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -5,7 +5,7 @@ ## Language **Query**: -一段 developer-facing 自然語言問題,研究的測量入口。Demo 預備一組 query 作為 finding 載體。 +一段 developer-facing 自然語言問題,研究的測量入口。**是開放的實作問題(「這問題有哪些實作解法?」),不預設已命名的 X-vs-Y 選擇;「選擇」從 practice breakdown emergent(ADR 0006)。** Demo 預備一組 query 作為 finding 載體。 _Avoid_: question(保留給 SO post 那種),prompt(保留給 LLM input) **Question**: @@ -17,16 +17,16 @@ _Avoid_: query,post(太泛) _Avoid_: topic(已棄用 BERTopic 框架),cluster(保留給 practice 側) **Answer**: -SO 上一篇 answer post(body)。Raw embedding clustering 在 answer body 上 probe 已證為失敗路線。 +SO 上一篇 answer post(body)。Raw embedding clustering 在 answer body 上 probe 已證為失敗路線。Dashboard 上標為 **"original answer"**,與我們抽出的 **practice** 做明確區分(answer = SO 原文,practice = 我們抽的單位)。 _Avoid_: response, comment **Practice**: -從一篇 answer 用 LLM 抽出的結構化 normalization:`{practice: 1 sentence, conditions: short list, evidence_type: prose|code|both}`。**Answer 側的 normalization 單位**,分群在 practice 上做、不在 raw answer 上做。Step 1 of hierarchical map-reduce (ADR 0002)。 +從一篇 answer 用 LLM 抽出的結構化 normalization:`{practice: 1 sentence, conditions: short list, evidence_type: prose|code|both}`。**Answer 側的 normalization 單位**,分群在 practice 上做、不在 raw answer 上做。Step 1 of hierarchical map-reduce (ADR 0002)。**一篇 answer 可抽出多個 practice(每個是一票;ADR 0007)。practice 專指推薦的實作 action(prescription);純解釋成因(diagnosis)的答案不產 practice(ADR 0006)。** _Avoid_: stance(opinion-mining 借詞、不擬合 SO Q&A 的「推薦 action」結構,且和 project README 「community-consensus practice」標題不一致),opinion,view,claim **Practice cluster**: -在一個 query-equivalent canonical group 內,多個 answer 的 practice 句聚成的群。**Consensus shape 的呈現單位**。Discovery 用 hierarchical map-reduce (ADR 0002):Step 2 LLM aggregator + companion deterministic SBERT+HDBSCAN run,兩者 agreement 報為 method-defensibility 指標。 -_Avoid_: position cluster, answer cluster, stance cluster +在一個 query-equivalent canonical group 內,多個 answer 的 practice 句聚成的群。**Consensus shape 的呈現單位**。Discovery 用 hierarchical map-reduce (ADR 0002):Step 2 LLM aggregator + companion deterministic SBERT+HDBSCAN run,兩者 agreement 報為 method-defensibility 指標。Dashboard 上以 **"community"** 呈現(一群提供相近 practice 的 answers)—— 注意這是「做法社群」,**勿與整個 reactjs SO「社群/community」(專案標題的那個 community) 混淆**。 +_Avoid_: position cluster, answer cluster, stance cluster;UI 的 "community" 不指 SO 全體群眾 **Breakdown / Shape**: 一個 canonical group 內各 practice cluster 的 size + authority share 分布。**取代 Convergent/Divergent hard label**;finding 是「shape 長這樣」,不是「label 是 X」。 @@ -44,8 +44,9 @@ _Avoid_: reputation alone(單純 SO native reputation 不是 authority signal - A **Query** retrieves one **Query-equivalent canonical group** - A **Query-equivalent canonical group** contains many **Questions** - A **Question** has one or more **Answers** -- An **Answer** is normalized to one **Practice**(`{practice, conditions, evidence_type}`) +- An **Answer** is normalized to **one or more Practices**(每個 `{practice, conditions, evidence_type}`;ADR 0007) - **Practices** within a canonical group cluster into multiple **Practice clusters** +- 歸屬單位是 **Practice**,不是 Answer:一篇 Answer 的多個 Practice 可落在**不同 Practice cluster(community)**,故 Answer 沒有單一「所屬 cluster」 - Each **Practice cluster** has size + **Authority** share → forms the **Breakdown** ## Example dialogue @@ -61,3 +62,4 @@ _Avoid_: reputation alone(單純 SO native reputation 不是 authority signal - 早期討論「question 的聚合」混用了「BERTopic topic」與「canonical group」兩個概念 — 已釐清:BERTopic 路線棄用(ADR 0001),測量單位是 query-equivalent canonical group。 - 「Consensus」的判定方式:曾在「Convergent/Divergent hard label with 0.6 threshold」與「shape-as-finding」之間 — 已釐清:取後者(ADR 0001)。 - Answer 側的 normalization 單位曾叫 `stance` — 已改名為 `practice`(2026-05-29)。理由:`practice` 是 project 原生詞(README 標題就是 "community-consensus practice");`stance` 是 opinion-mining 借詞、預設「對固定 target claim 支持/反對」的二分結構,不擬合 SO Q&A 的「每個 answer 自行提議一個 action / approach」結構。 +- **Dashboard 目前的 authority 與 grouping 是 SO proxy,非真 B/C**:authority = SO vote score(pre-differentiator、age-confounded),grouping = SO 原生 duplicate links(非真 canonical RAG+gate)。兩者藏在 `providers/so_proxy.py` 的 provider seam 後、可無痛替換(ADR 0008);UI 帶 provenance/PRE-DIFFERENTIATOR 標記,**勿把 proxy 數字當 finding**。 diff --git a/README.md b/README.md index 4d148a2..cabf40e 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Mapping community-consensus practice landscape in framework-level Stack Overflow Stack Overflow 上「一個 question 多個 answer」的結構,本身就是 collective expert opinion 的 implicit aggregator。不同 answer 間的 practice 分布加上作者社群權威,能定量描述某社群對某類技術議題的共識狀態。 -**2026-05-29 pivot 後**(見 `docs/adr/0001`–`0005`):對 developer query 採 LLM-canonicalized retrieval + hierarchical map-reduce practice extraction + community-network-derived authority overlay,揭示 React 社群在 implementation choice 場景下的 **practice breakdown shape**(emergent convergent / divergent / shifting / authority-disputed);不再有 hard typology label,shape 從 breakdown 自然 emergent。Demo = 6–8 stratified-selected developer queries。在資料覆蓋充足的 query 上做 per-query trajectory case study。 +**2026-05-29 pivot 後**(見 `docs/adr/0001`–`0007`):對 developer query 採 LLM-canonicalized retrieval + hierarchical map-reduce practice extraction + community-network-derived authority overlay,揭示 React 社群在 implementation choice 場景下的 **practice breakdown shape**(emergent convergent / divergent / shifting / authority-disputed);不再有 hard typology label,shape 從 breakdown 自然 emergent。Demo = 6–8 stratified-selected developer queries。在資料覆蓋充足的 query 上做 per-query trajectory case study。 完整研究設計見: - **`docs/spec.md`** — team-internal implementation reference(流程、演算法、DB schema、實作待辦)— **post-pivot 已對齊** ADR 0001–0005 @@ -52,7 +52,9 @@ cp .env.example .env │ │ ├── 0002-practice-clustering-hierarchical-map-reduce.md │ │ ├── 0003-authority-as-deterministic-weight-plus-narrative-overlay.md │ │ ├── 0004-default-view-beeswarm-with-overlay.md -│ │ └── 0005-demo-query-selection-stratified-hybrid.md +│ │ ├── 0005-demo-query-selection-stratified-hybrid.md +│ │ ├── 0006-query-framing-open-implementation-problem.md +│ │ └── 0007-extraction-multipractice-clustering-method.md │ └── reference/ │ └── se-api-overview.html # SE API reference ├── probe/ # 探索性檢驗 diff --git a/docs/adr/0002-practice-clustering-hierarchical-map-reduce.md b/docs/adr/0002-practice-clustering-hierarchical-map-reduce.md index 381d20d..7c46ddd 100644 --- a/docs/adr/0002-practice-clustering-hierarchical-map-reduce.md +++ b/docs/adr/0002-practice-clustering-hierarchical-map-reduce.md @@ -1,6 +1,6 @@ # 0002 — Practice clustering via hierarchical map-reduce (LLM + companion deterministic run) -**Status:** accepted (2026-05-29, terminology updated practice/stance 2026-05-29) +**Status:** accepted (2026-05-29, terminology updated practice/stance 2026-05-29). **Refined by ADR 0007 (2026-05-29):** Step-1 emits a *list* of practices per answer; companion = matched-k agglomerative (not HDBSCAN-from-scratch); `majority_vote_on_assignments` = k=3 co-association consensus. Model/provider now OpenAI GPT-5.4 family (PLAN ▼Q2), not Haiku. ## Context @@ -20,7 +20,7 @@ Independent WebSearch synthesis (2024–2026 literature) ratified the hierarchic **Primary method:** Hierarchical map-reduce on practice statements. -- **Step 1 — Practice extraction (parallel, per answer)**: LLM extracts structured normalization from each answer. Output schema: `{practice: 1 sentence, conditions: short list, evidence_type: prose|code|both}`. The 1-sentence practice is the unit; the conditions field exists to prevent minority-position erasure (e.g., "recommends X *only if* state is local"). Few-shot React-specific anchoring; explicit instruction to interpret code blocks as practice evidence; temp=0. +- **Step 1 — Practice extraction (parallel, per answer)**: LLM extracts structured normalization from each answer. Output schema: `{practice: 1 sentence, conditions: short list, evidence_type: prose|code|both}`. The 1-sentence practice is the unit; the conditions field records scope caveats the answer **explicitly states** (e.g., "recommends X *only if* state is local"). Few-shot React-specific anchoring; explicit instruction to interpret code blocks as practice evidence; temp=0. **[Refined by ADR 0007: Step-1 emits a *list* of practices per answer (one answer can vote for several approaches); `conditions` is a displayed annotation that does NOT affect cluster boundaries.]** - **Step 2 — Practice aggregator (one-shot)**: LLM aggregator over N short practice statements (≤2K tokens for N≤100; far below lost-in-the-middle danger zone). Output: cluster assignment per answer + cluster names + brief breakdown description, JSON-schema-enforced. Temp=0 + **k=3 multi-sample voting** on cluster assignments. @@ -38,7 +38,7 @@ Independent WebSearch synthesis (2024–2026 literature) ratified the hierarchic ## Consequences -- Per-query cost ≈ $0.05 (Haiku 4.5 + parallel Step 1 + single Step 2); demo 8 queries total ≈ $0.40 / ~1 min wall clock. Not a constraint. +- Per-query cost (OpenAI GPT-5.4 family, measured in Module-D probe): extraction k=1 over ~163 pooled answers ≈ $0.21 (gpt-5.4-mini); aggregator k=3 one-shot ≈ $0.04 (gpt-5.4); narrative negligible. Demo 8 queries ≈ a few $. Not a constraint. (Old ~$0.05 estimate was bound to Haiku 4.5 + N=50.) - LLM non-determinism is contained in two well-defined steps with explicit reproducibility controls (temp=0, structured output, k=3 voting). Step-1 schema (practice + conditions + evidence_type) prevents minority erasure. - Companion deterministic run buys: (a) a reproducibility floor that does not depend on LLM availability, (b) a quantitative agreement number for the methodology section, (c) protection against silent LLM drift across model versions. - Practice extraction has its own failure modes (information loss, code interpretation, domain jargon); these go in limitations with prompt-engineering mitigations recorded in implementation. diff --git a/docs/adr/0004-default-view-beeswarm-with-overlay.md b/docs/adr/0004-default-view-beeswarm-with-overlay.md index 0084ff1..d8950c9 100644 --- a/docs/adr/0004-default-view-beeswarm-with-overlay.md +++ b/docs/adr/0004-default-view-beeswarm-with-overlay.md @@ -12,7 +12,7 @@ ADR 0003 鬆綁了 cluster authority 的單一公式,把 dashboard 重新定 Default view = **beeswarm**: -- **個別 answer = 1 點**(first-class visual unit) +- **個別 answer = 1 點**(first-class visual unit)—— 注:ADR 0007 後一答可含多 practice;default view 仍以該答案的 **primary** practice 一答一點,「一答多點」viz 延後(D-5 再定) - **點水平位置** = practice cluster 分欄(cluster boundary 由 ADR 0002 決定) - **點大小** = author authority(default yearly PageRank;可切 reputation / full-period PageRank / vote score / answer length 等) - **點顏色** = year band(default;可切 accept_status / evidence_type / cluster_id 等) @@ -30,3 +30,38 @@ Default view = **beeswarm**: - **失去 of**:原 spec § 3.1 的 "tag-level landscape heatmap"(30 topic × 7 year 整面 heatmap)—— 那是 corpus-wide framing 的產物,ADR 0001 已棄 - **實作風險**:beeswarm with size / color 多 axis switching 在 Streamlit 上要 Plotly + 自訂 layout,不是 one-liner。Demo 8 query 可接受 - **可訪問性**:點大小編碼對視障使用者不友善,需要 alt text / sortable table fallback(保留 LineUp-style table 為 secondary tab,per 候選 D 的精神) + +## Amendment (2026-05-30) — D-5 interactive viz exploration + +Probe `src/module_d/probe_d/viz/` explored the default-view idiom space on real data (q54069253 +slice, 46 practices). The tentative beeswarm default is refined into a small composed system: + +- **Overview / default = clustered circle-pack**, realized as a **d3-force floating layout** with + **organic gooey/metaball cluster blobs** — the cluster boundary is an irregular shape that grows + along its member nodes, not a circle. Nodes draggable (fling); whole bubble draggable; collide + keeps nodes non-overlapping. (`viz/make_force.py` → `force.html`) +- **Single-practice temporal = click a practice → 2D continuous-time trajectory**: that cluster's + answers pin onto a real **date axis (old→new)**, beeswarm-stacked, others fade. This is the + PRIMARY analytical temporal tool. A single-practice trajectory is just a 1-D slice through the + space-time cube, so the cube is **not** needed for the core finding. (`viz/make_traj.py`; also + folded into `force.html`) +- **Time = expandable dimension, demoted from colour.** Ordered variable → ordered channel + (position / facet), never colour. Default is time-collapsed (pooled); time is revealed on demand + (per-practice trajectory, or per-year small-multiples). **This drops the original "point colour = + year band" default (§ 2.9)**; colour is freed (→ evidence_type / neutral). +- **3D Space-Time Cube = optional demo showpiece ONLY** (engagement, not a quantitative figure): + orthographic keeps cross-year sizes fair; empty gap-years shown honestly; gated on the 2019–2022 + backfill. (`viz/make_stc_cube.py`; static concept = `data/d5_stc_concept.png`) +- **Static finding figure** stays beeswarm / circle-pack PNG (paper-printable). + +Shared encoding (all idioms): **disk size ∝ log1p(vote)** (raw vote is power-law and can be +negative; size carries only ordinal "clearly bigger" — the exact authority/concentration number +goes in narrative/tooltip text); **accepted = ring**; **long-tail singletons collapse into one +labelled group** (not erased — long-tail mass is itself the divergent-shape signal). Authority here +is the **SO vote proxy — PRE-DIFFERENTIATOR and age-confounded** (older answers accrue votes); real +network authority (yearly PageRank) enters only with Module B (+ backfill for this fixture). + +Tech: prototypes use matplotlib (static PNG) + D3/Three.js (interactive HTML). The production +dashboard leans **D3 / SVG** (force + trajectory) over the originally-assumed Streamlit + Plotly, +and the architecture stays open to a JS/WebGL stack (D3 / Three.js / deck.gl) for the cube +showpiece. Layout / animation polish is deferred (tune later). diff --git a/docs/adr/0005-demo-query-selection-stratified-hybrid.md b/docs/adr/0005-demo-query-selection-stratified-hybrid.md index bdc78ed..9fb44ff 100644 --- a/docs/adr/0005-demo-query-selection-stratified-hybrid.md +++ b/docs/adr/0005-demo-query-selection-stratified-hybrid.md @@ -16,7 +16,7 @@ Candidate strategies: Adopt **stratified hybrid (a)+(c)**:pre-state criteria → 跑 candidate pool → data-driven 篩入 final N。 -1. **Persona definition (pre-stated)**: "Stuck developer facing implementation choice — has a real implementation problem, doesn't know how to implement or which of several approaches to use." 這個 persona 自然 implies 多種 approach 存在,是 community consensus measurement 的有效場景。 +1. **Persona definition (pre-stated)**: "Stuck developer facing implementation choice — has a real implementation problem, doesn't know how to implement or which of several approaches to use." 這個 persona 自然 implies 多種 approach 存在,是 community consensus measurement 的有效場景。**(Broadened by ADR 0006:query 是開放實作問題「這問題有哪些實作解法」、不預設已命名的 X-vs-Y choice;此 implementation-choice persona 為其特例。Stratification machinery 不變。)** 2. **Stratification dimensions (pre-stated)**: - **Shape stratum** (predicted breakdown shape): convergent / divergent / shifting trajectory / authority-disputed diff --git a/docs/adr/0006-query-framing-open-implementation-problem.md b/docs/adr/0006-query-framing-open-implementation-problem.md new file mode 100644 index 0000000..af2661a --- /dev/null +++ b/docs/adr/0006-query-framing-open-implementation-problem.md @@ -0,0 +1,24 @@ +# 0006 — Query framing: open implementation problem, surface community practices + +**Status:** accepted (2026-05-29) — broadens the persona in ADR 0005 §Decision.1; spec § 1.1 / § 2.10 updated to match + +## Context + +ADR 0005 fixed the demo-query persona as "stuck developer facing implementation **choice** — doesn't know which of several approaches to use." Module-D probe grilling (on the `q54069253` "useState not updating" canonical chain) showed this is too narrow: + +- The answer-rich, recurring React questions in the corpus — and the persona-real ones — are **open problems** ("why doesn't this work / how do I do X"), not pre-named A-vs-B choices. The "Redux vs Context"-style explicit choice debates are scarce in the 2023–2026 window (they sit in the pre-backfill 2019–2021 era). +- Yet such open problems DO surface multiple community approaches in their answers — the "choice" is **emergent in the breakdown**, not stated in the query. Forcing a choice framing onto the query would exclude most of the usable corpus. + +## Decision + +A **query** is an open developer-facing implementation problem — *"what are the implementation approaches for this problem?"* — NOT a pre-named choice between alternatives. The research output is the **emergent community-practice breakdown** (authority-weighted) for that problem; any "choice" is read off the breakdown, not presupposed by the query. + +- This **broadens** ADR 0005's persona (it subsumes the implementation-choice case) and keeps the rest of ADR 0005 intact: the shape / topic / temporal strata and the data-driven within-stratum selection still apply. +- Query selection still favours problems with **practice diversity** (multiple legitimate approaches); single-fix bug questions (one obvious answer) are filtered out — their breakdown is trivial. +- Downstream (ADR 0007): the consensus object is the **prescription** (recommended action). Pure-diagnosis answers (explaining *why* without proposing an action) are excluded — diagnosis convergence is correctness, not a preference-consensus signal. + +## Consequences + +- More of the 2023–2026 corpus becomes valid demo material (how-to / debugging problems with diverse fixes), reducing dependence on the 2019–2022 backfill for demonstrating *mechanics* (cross-year *trajectory* still needs backfill). +- CONTEXT.md `Query` term updated with this framing. +- The ADR 0005 selection-bias limitation persists: chosen problems are still researcher-selected, not universal. diff --git a/docs/adr/0007-extraction-multipractice-clustering-method.md b/docs/adr/0007-extraction-multipractice-clustering-method.md new file mode 100644 index 0000000..5956d0c --- /dev/null +++ b/docs/adr/0007-extraction-multipractice-clustering-method.md @@ -0,0 +1,26 @@ +# 0007 — Multi-practice extraction; clustering = LLM aggregator + matched-k companion + k=3 co-association consensus + +**Status:** accepted (2026-05-29) — refines ADR 0002 (Step-1 cardinality, companion method, voting definition). Validated by the Module-D probe (`src/module_d/probe_d/`). + +## Context + +ADR 0002 fixed hierarchical map-reduce: Step-1 one practice per answer; companion SBERT + HDBSCAN; "k=3 voting on cluster assignment, majority decides" — but left `majority_vote_on_assignments` undefined. The Module-D probe on the `q54069253` canonical chain (46 practices from 35 answers) surfaced three concrete gaps. + +## Decision + +1. **Step-1 yields a LIST of practices per answer (0..N), not one.** Rich answers genuinely propose several approaches; keeping only one undercounts community support for the others and distorts the breakdown — multi-position visibility is the point (RQ2). An **empty list** means the answer proposes no implementation practice (pure diagnosis / "me too" / thanks → excluded, per ADR 0006). The default beeswarm plots each answer at its **primary** practice; the one-answer-multiple-points visual is deferred (ADR 0004 stays tentative on this). + +2. **Companion = matched-k partitioning, not HDBSCAN-from-scratch.** Probe: `HDBSCAN(min_cluster_size=2)` on normalized, short, topically-homogeneous practice sentences **degenerates** (44/46 in one blob; ARI 0.009) — there are no density valleys. Given **k = the LLM aggregator's cluster count**, agglomerative / k-means on the *same* embeddings recovers moderate agreement (ARI ~0.38; head-only, n≥2 clusters, ~0.45). The embedding is not the bottleneck (`text-embedding-3-small` MTEB ~62 > spec's SBERT `mpnet` ~58 / `MiniLM` 56); density-from-scratch was the wrong tool. The companion borrows only `k` from the LLM (shared granularity) — the grouping decision stays embedding-only. The **production** companion should still use **local SBERT** for LLM-provider independence (ADR 0002's reproducibility-floor rationale), accepting equal-or-slightly-lower quality vs the OpenAI embedder. + +3. **`majority_vote_on_assignments` = co-association consensus.** Run the aggregator k=3; for each practice pair, count how often the runs co-cluster them; pairs co-occurring in a **majority (≥ 2/3)** are linked; **connected components** form the consensus clustering. This is label-alignment-free — immune to differing cluster counts / IDs across runs. Probe: per-run counts 17/16/16 (unstable granularity) but consensus pair-level agreement **0.99** (97% of within-cluster pairs unanimous across the 3 runs); head clusters all 1.00. + +4. **Report two agreement axes separately** (spec § 4 conflated them): + - **(i) LLM self-stability** across the k=3 runs — the co-association agreement (~0.99). A *reproducibility* metric. + - **(ii) cross-method agreement** vs the independent embedding companion — matched-k ARI (~0.38). A *method-defensibility* metric. + They answer different questions. ARI ~0.38 falls in spec § 4's pre-existing "< 0.5 → report & discuss" branch. + +## Consequences + +- ADR 0002 deltas: Step-1 "1 sentence practice" → list; HDBSCAN companion → matched-k agglomerative; undefined majority vote → co-association consensus. +- DB schema (spec § 3.2): `practice_extractions` becomes one row per (answer, practice) with a primary/rank flag, not per-answer; `practice_clusters.voting_agreement` = co-association agreement. +- Numbers above are from a **single-chain probe** (`q54069253`, 46 practices); full-scale (163 pooled answers) and other queries are pending. The qualitative findings (HDBSCAN degenerates; co-association stable; two distinct axes) are expected to generalise; exact ARI will vary. diff --git a/docs/adr/0008-module-d-bc-data-contract-provider-seams.md b/docs/adr/0008-module-d-bc-data-contract-provider-seams.md new file mode 100644 index 0000000..5619c63 --- /dev/null +++ b/docs/adr/0008-module-d-bc-data-contract-provider-seams.md @@ -0,0 +1,43 @@ +# 0008 — Module D ↔ B/C data contract: provider seams + normalized Breakdown + +**Status:** accepted (2026-06-01) — realizes the PLAN "對外接口" for Module D. Validated by the D-5 dashboard on the `q54069253` slice (46 practices / 8 communities / 28 answers) with zero API/DB calls at runtime. + +## Context + +Module D (answer-side practice breakdown + authority overlay + dashboard) depends on two other teams' outputs that do not exist yet (PLAN § 對外接口): + +- **Module B** — network authority (yearly PageRank + multi-source author authority lookup). +- **Module C** — `query → canonical_group` (which SO questions/answers form one measurement unit). + +D still needed real data to validate the D-5 dashboard idioms now. Two bad options were on the table: (a) block D until B/C ship, or (b) wire the dashboard directly to stand-in signals — SO vote score as authority, SO native duplicate links as the grouping — and let those proxy assumptions soak into the dashboard code. Both are wrong, because the proxies are **explicitly throwaway**: vote-as-authority is pre-differentiator and age-confounded (older answers accrue votes; the differentiator only exists once *network* authority enters the weighting — ADR 0003), and SO duplicate links are not the real canonical RAG+gate grouping. They must be swappable for the real providers without touching the dashboard. + +## Decision + +Decouple D from B/C behind two **provider seams**, mirroring the provider-neutral `llm_call` seam already adopted for the LLM tier (PLAN ▼Q2). + +1. **Ports** (`ports.py`) — two `Protocol`s are the only contact surface with B/C: + - `AuthorityProvider.score(answer) -> float` (Module B) + - `CanonicalGroupProvider.fetch(group_id) -> CanonicalGroup` (Module C) + +2. **Normalized contract** (`contract.py`) — the dashboard reads *only* a `Breakdown`: + - `authority` is a **first-class per-point field**. Both the circle-pack geometry (`r`) and the dashboard's node size read this one field — there is no second, divergent size formula. (The earlier probe re-derived `log(vote)` in JS *and* in the packer; that split is gone.) + - **provenance** (`authority_source` / `group_source`) travels with the data, so the UI can honestly badge "proxy" and a swap is visible. + - per-answer detail cards (body + author) for the click-to-detail panel. + +3. **Proxies are quarantined** (`providers/so_proxy.py`) — `VoteAuthorityProxy` and `DuplicateChainGroups` are the *only* place the proxy assumptions live; nothing else in Module D imports them. `breakdown.py` is the single integration point that joins D's own offline outputs (extraction + clustering) with B (authority) and C (group), then lays it out via `layout.py`. + +**Swap = replace one provider + re-run `dashboard/build`.** `contract` / `layout` / `render` are untouched. + +## Consequences + +- **D is unblocked on B/C.** The full D-5 dashboard idiom (force-floating organic bubbles · click-to-detail · community-swimlane timeline) was validated on real practice data with no runtime API/DB dependency — the committed `viz_data.json` lets the generators run standalone. +- **Swap blast radius is confined** to `providers/so_proxy.py` (+ one build re-run). The dashboard never learns where authority/grouping came from beyond the provenance string — the same blast-radius discipline as the LLM seam. +- **Honesty burden.** The proxy numbers are *wrong on purpose* (vote ≠ network authority; SO dup links ≠ canonical grouping). Guarded by the provenance fields + PRE-DIFFERENTIATOR labels in the UI, but a reader must not mistake proxy values for findings. +- **Known proxy artifact:** because one answer yields multiple practices that can land in different communities (ADR 0007), a community's `Σ votes` double-counts an answer across communities. Acceptable for the proxy; revisit when real per-author network authority replaces the per-answer vote sum. +- **Not done here:** Module B real authority (yearly PageRank + the 2019–2022 backfill needed to cover this fixture's early authors) and Module C real grouping. This ADR fixes only the *seam*; the providers behind it stay proxies until those land. + +## Relationships to other ADRs + +- **0003** (authority as deterministic post-weight, not boundary) — the proxy occupies the same slot the real network authority will; swapping it does not move cluster boundaries. +- **0004** (default-view idiom + amendment) — the contract is what that dashboard consumes. +- **0007** (multi-practice extraction) — the reason `Σ votes` double-counts; the contract carries one point per practice, deduping answers only for the detail cards. diff --git a/docs/spec.md b/docs/spec.md index ccc2284..51552d4 100644 --- a/docs/spec.md +++ b/docs/spec.md @@ -3,7 +3,7 @@ > **用途:** team-internal implementation reference — 給所有開發者看的設計依據,包含所有設計決策及其理由。 > **對應:** `../deliverables/proposal.md`(對外 proposal、給老師看的精簡版;2026-05-29 pivot 後 proposal 也待改) > **與 Framing A 的關係:** 並列存在,最終擇一。Framing A 的 spec 在 `../sentiment-divergence/spec.md`。 -> **Binding decisions:** `docs/adr/0001` – `docs/adr/0005`(pivot 後的鎖定設計,本 spec 是它們的整本化展開) +> **Binding decisions:** `docs/adr/0001` – `docs/adr/0007`(pivot 後的鎖定設計,本 spec 是它們的整本化展開;0006/0007 為 Module-D probe 後的 refinement) > **Last updated:** 2026-05-29(post-pivot rewrite) --- @@ -12,7 +12,7 @@ ### 1.1 一句話描述 -從 Stack Overflow 的 `[reactjs]` 討論中,**對 developer query 採 LLM-canonicalized retrieval + hierarchical map-reduce practice extraction + community-network-derived authority overlay,揭示 React 社群在 implementation choice 場景下的 community-consensus practice breakdown shape(emergent convergent / divergent / shifting / authority-disputed),並在資料覆蓋充足的 query 上做 per-query trajectory case study**。Demo = 6–8 stratified-selected developer queries,持 "stuck developer facing implementation choice" persona。 +從 Stack Overflow 的 `[reactjs]` 討論中,**對 developer query 採 LLM-canonicalized retrieval + hierarchical map-reduce practice extraction + community-network-derived authority overlay,揭示 React 社群在 implementation choice 場景下的 community-consensus practice breakdown shape(emergent convergent / divergent / shifting / authority-disputed),並在資料覆蓋充足的 query 上做 per-query trajectory case study**。Demo = 6–8 stratified-selected developer queries。Persona 為「開放實作問題 → 揭露社群有哪些 practice」(broadened,見 ADR 0006;原 "implementation choice" 為其特例)。 ### 1.2 研究問題 @@ -121,7 +121,7 @@ Binding ADR: **0002** #### Step 1 — Practice extraction(per answer, parallel) -- LLM 對每篇 answer 抽出 `{practice: 1 sentence, conditions: short list, evidence_type: prose|code|both}` +- LLM 對每篇 answer 抽出 **一個 practice list**(0..N,空 = 無實作解法;ADR 0007),每個 `{practice: 1 sentence, conditions: short list, evidence_type: prose|code|both}` - Few-shot React-specific anchoring;prompt 明說 code block 也是 practice evidence - temp=0、parallel calls(stateless per answer) - 預期 wall clock:N=50 ~5s @@ -131,13 +131,13 @@ Binding ADR: **0002** - 把 N 個 practice statement 一次塞 LLM(≤ 2K token,遠在 lost-in-the-middle danger zone 之下) - LLM 做 cluster discovery + naming - JSON schema enforced output:cluster assignment per answer + cluster name + brief description -- temp=0、**k=3 voting on cluster assignment**;majority decides,記錄 voting agreement +- temp=0、**k=3 voting on cluster assignment**;majority 以 **co-association consensus** 定(pair 共現 ≥2/3 → 連通分量,label-alignment-free;ADR 0007);voting agreement = 共識群內平均共現率(probe 實測 0.99) #### Companion deterministic run(每個 canonical group) -- 同 Step-1 practice statement → SBERT embed → HDBSCAN -- Report cluster assignment agreement(Adjusted Rand Index)vs Step-2 LLM aggregator -- 作為 method-defensibility 量化指標進論文 method 段 +- 同 Step-1 practice statement → SBERT embed → **matched-k agglomerative**(k = LLM aggregator 群數;HDBSCAN-from-scratch 對同質短句退化、已棄,ADR 0007) +- Report cluster assignment agreement(Adjusted Rand Index)vs Step-2 LLM aggregator(probe 實測 ~0.38) +- 作為 **cross-method** defensibility 指標進論文(與 k=3 self-stability 0.99 是不同軸,分開報;ADR 0007 §4) #### 為何不選別的路線 @@ -384,6 +384,8 @@ CREATE TABLE canonical_group ( PRIMARY KEY (query_id, question_id) ); +-- ADR 0007: 一答可多 practice → 實際改為每 (answer, practice) 一列(加 practice_rank,PK 改 autoincrement id)。 +-- 下方 DDL 為 pre-0007 single-practice 版,待 implementer 依 ADR 0007 調整。 CREATE TABLE practice_extractions ( answer_id INTEGER PRIMARY KEY REFERENCES answers(id), query_id INTEGER REFERENCES queries(id), @@ -517,6 +519,8 @@ Prompt 設計細節(few-shot examples、邊緣案例判則)見 § 10。「 ### 3.6 Hierarchical map-reduce practice clustering +> **ADR 0007 refinement(post-probe):** 下方 code 的 `companion_deterministic`(HDBSCAN) 對同質短 practice 句退化(44/46 一坨),已改為 **matched-k agglomerative**(k = LLM 群數);`majority_vote_on_assignments` = **k=3 co-association consensus**(pair 共現 ≥2/3 → 連通分量,label-alignment-free)。Extraction 也改吐 **practice list**(一答多解)。參考實作見 `src/module_d/probe_d/d2_cluster.py` / `d2_consensus.py`。 + ```python from sklearn.cluster import HDBSCAN from sklearn.metrics import adjusted_rand_score @@ -626,8 +630,8 @@ Dashboard (Streamlit + Plotly) 依 § 2.9 default view 將上述資料 render | **RAG retrieval recall** | Top-K vs SO confirmed-dup subset 對齊度 | local | recall@100 ≥ 65%、recall@500 ≥ 85%(per probe 09) | | **LLM equivalence gate 對齊度** | 抽 30 (query, candidate) pair 人工 vs LLM | 人工 + LLM | agreement > 70% → 信任 LLM 全 corpus;否則退回純人工或加 voting | | **Practice extraction 品質** | 30 answer 抽樣,人工驗 practice/conditions/evidence_type | 人工 | 一致率 > 75% | -| **Step-2 aggregator 內部 reproducibility** | k=3 voting agreement metric | 內建 | majority agreement > 0.7 per cluster | -| **Companion deterministic agreement** | LLM aggregator vs HDBSCAN,Adjusted Rand Index | sklearn | ARI > 0.5 → method-defensible;若低,paper 報並討論 emergent vs deterministic 各自價值 | +| **Step-2 LLM self-stability(k=3)** | co-association agreement(共識群內平均共現率;ADR 0007)| 內建 | > 0.7 per cluster;probe 實測 **0.99**(head 全 1.00)| +| **Companion cross-method agreement** | LLM aggregator vs **matched-k agglomerative** embedding companion,Adjusted Rand Index(ADR 0007;非 HDBSCAN)| sklearn | ARI > 0.5 → strong;probe 實測 **~0.38** → 落「< 0.5 → report & discuss」。與上列 self-stability 是**不同軸** | | **RQ3 baseline comparison** | 並列 5 條件 demo per query | 人工 + 5 條件各自的 API | **不評分**、並列表 + raw output 進 appendix | | Practitioner contrast study | per query 5 條件下 final-pick answer / consensus surface 並列討論 | 人工 + LLM | 6–8 query × 5 condition × 質性比較段 | | 整體意義 | 6–8 demo query case study + cross-query patterns | 人工撰寫 | 報告含具體 case + 跨 case 觀察 | @@ -652,13 +656,13 @@ Dashboard (Streamlit + Plotly) 依 § 2.9 default view 將上述資料 render | 儲存 | SQLite | 內建 | | Sentence embedding | `sentence-transformers` (`all-mpnet-base-v2` 或更新) | >= 2.5 | | Question canonicalization | 自寫 RAG + LLM gate | LLM provider § 10 | -| Practice clustering | LLM (Step 1 + 2) + sklearn HDBSCAN (companion) | sklearn >= 1.3, HDBSCAN >= 0.8.33 | +| Practice clustering | LLM aggregator (Step 1 + 2) + sklearn matched-k agglomerative (companion; ADR 0007,HDBSCAN-from-scratch 退化已棄) | sklearn >= 1.3 | | 圖分析 | NetworkX + python-louvain | networkx >= 3.0, python-louvain >= 0.16 | -| Reproducibility(k=3 voting)| 自寫 majority vote utility | — | +| Reproducibility(k=3 voting)| co-association consensus(ADR 0007)| — | | 視覺化 | Streamlit + Plotly | streamlit >= 1.30 | | GPU | Google Colab / Kaggle | 免費版 | | Python 環境 | uv(per 使用者習慣) | latest | -| LLM API(gate / extraction / aggregator / narrative)| Claude Haiku 4.5 / Gemini 2.5 Flash / Groq Llama;provider 實作期決定(§ 10) | latest | +| LLM API(gate / extraction / aggregator / narrative)| **OpenAI GPT-5.4 family**(gate nano / extraction mini / aggregator+narrative standard);strict structured outputs;provider-neutral seam(ADR 0002 / PLAN ▼Q2) | latest | --- @@ -673,7 +677,7 @@ Dashboard (Streamlit + Plotly) 依 § 2.9 default view 將上述資料 render | Unique users | ~80k–130k | scale from tag-level + window-span probes | | SBERT inference 時間(全 corpus)| ~6–9 hr Colab GPU | 4M+ embeddings | | SQLite DB 大小 | ~6–10 GB | 含 raw + embeddings + post-pivot tables | -| **Per query pipeline LLM cost** | ~$0.05 | Step-1 N=50 parallel + Step-2 k=3 + narrative | +| **Per query pipeline LLM cost** | ~$0.25 | OpenAI GPT-5.4:extraction k=1(~163 pooled A,mini)≈$0.21 + aggregator k=3(standard)≈$0.04 + narrative;Module-D probe 實測 | | **Demo 全跑 LLM cost** | ~$5–10 | 15–20 candidate × 5 baseline 條件 | | 完整 pipeline 一次跑時間 | ~15–24 hr | 含 backfill fetch + NLP + per-query 全 pool | @@ -733,7 +737,7 @@ Dashboard (Streamlit + Plotly) 依 § 2.9 default view 將上述資料 render |----------|------|------| | **高** | **2019–2022 SO 資料 backfill** | 跑 SE API paginated fetch;30–60 min connectivity;不影響 Module B / C 設計 | | **高** | **RAG retrieval top-K + embedding model 校準** | 實作 mini-probe:K ∈ {100, 200, 500} × MiniLM vs mpnet;驗 recall vs cost | -| **高** | **LLM provider / model 選擇** | 涵蓋四角色:equivalence gate / practice extraction / Step-2 aggregator / narrative。候選 Claude Haiku 4.5、Gemini 2.5 Flash、Groq Llama;prompt 固定版本、temp=0 + k=3 voting reproducibility 保證 | +| ✅ 定 | **LLM provider / model(resolved)** | **OpenAI GPT-5.4 family**(gate nano / extract mini / aggregator+narrative standard),strict structured outputs,call site 走 provider-neutral seam(ADR 0002 / PLAN ▼Q2)。temp=0 + k=3 voting reproducibility。prompt 版本 pinning 仍待(G12)| | 中 | PageRank vs SO native multi-source overlap mini-probe | § 2.6 校準步驟,~30 min | | 中 | **Practice extraction prompt design** | Few-shot React-specific anchor 選哪 5–10 個範例;code-as-evidence 規則;conditions field 取捨;測 30 answer 對齊度 | | 中 | **Step-2 aggregator prompt design** | JSON schema 鎖法;cluster naming style 規則;k=3 disagreement resolution(majority vote 失敗時 fallback) | diff --git a/docs/superpowers/specs/2026-06-01-dashboard-editorial-ink-design.md b/docs/superpowers/specs/2026-06-01-dashboard-editorial-ink-design.md new file mode 100644 index 0000000..d02ca10 --- /dev/null +++ b/docs/superpowers/specs/2026-06-01-dashboard-editorial-ink-design.md @@ -0,0 +1,99 @@ +# Dashboard visual design — "The Editorial · Ink" + +**Date:** 2026-06-01 **Status:** approved (brainstorming) **Scope:** Module D dashboard look-and-feel (color + type + texture) **and** the answer-detail panel (practice-led redesign + SO-markdown content rendering). Adds `conditions`/`evidence_type` to the data contract; does **not** change the bubble-field idiom, encodings, or the B/C provider seams. + +## Decision + +Adopt a single, intentional visual identity for the practice-breakdown dashboard: + +> **The Editorial · Ink** — a warm-paper, journal-typeset surface where saturated, colorblind-safe data sits on a quiet ink-and-paper chrome. The memorable idea: *rigorous scientific data that feels hand-set / printed.* + +This replaces the inherited "generic dark data-viz" defaults (which were never a deliberate choice). + +### How we got here (narrowing path) + +1. Broad sweep — 4 directions: Refined Dark, Light Research-Paper, Colorblind-safe Scientific, Warm Brand. +2. User kept **B (Research-Paper)** + **C (Scientific)** → the "credibility" half of the spectrum. +3. Within B/C, 3 high-fidelity mocks (Journal / Instrument / Editorial) → user chose **Editorial**. +4. Within Editorial, 3 chrome-accent options (Ink / Plum / Pine) → user chose **Ink**. + +## Design tokens + +Implement as CSS custom properties in `:root` so a future dark variant is a one-block override. + +### Foundation +| token | value | role | +|---|---|---| +| `--paper` | `#FBF8F2` | canvas / app background (warm paper) | +| `--panel` | `#FFFFFF` | cards, panels, toggle | +| `--panel2` | `#F5F0E7` | inset stat cells, answer-body block | +| `--ink` | `#20242B` | primary text | +| `--muted` | `#6B7280` | secondary text | +| `--muted2` | `#9AA0A8` | faint mono micro-labels | +| `--line` | `#E7E0D4` | borders / separators | +| paper grain | `radial-gradient(rgba(120,100,70,.06) 1px,transparent 1px)` @ 5px | subtle texture (atmosphere, not flat) | + +### Type (three roles) +| role | family | usage | +|---|---|---| +| display | **Fraunces** upright 500–600 | community names (community card + practice cards) | +| body | **Newsreader** serif 400–500 | answer content, descriptions | +| data/label | **IBM Plex Mono** 400–600 | numbers, `COMMUNITY`/`ORIGINAL ANSWER` micro-labels (uppercase, letter-spacing .1em), axis ticks, toggle, small map labels | + +Rationale: a display serif carries the "editorial" character; mono for all data keeps figures aligned and nods to the developer/StackOverflow origin; small map labels use mono (not display) for legibility at ≤11px. Community names are **upright, not italic** — italic hurt readability on the long, sentence-style cluster names. + +### Community data palette — Paul-Tol *muted* (CVD-safe) +Applied in cluster-size order; `long-tail` is always neutral grey. +``` +#332288 #88CCEE #44AA99 #117733 #DDCC77 #CC6677 #882255 #AA4499 +long-tail → #9AA0A6 +``` +- Bubbles use `mix-blend-mode:multiply` so overlaps darken organically (the gooey idiom on a light ground). +- The two light fills (`#88CCEE`, `#DDCC77`) are low-contrast on paper; the existing paper-colored node stroke (`#FBF8F2`, 2.4px) already cuts each node out cleanly, which covers it. + +### Accent & semantic +| token | value | role | +|---|---|---| +| `--accent` | `#241A12` (warm ink-black) | toggle-active, title rule, emphasis numbers, (future) selection ring | +| `--accepted` | `#C8A24A` (gold) | accepted marker (gold ★ / ✓) in hover + detail surfaces — semantic "endorsed" | + +**Accepted is not marked in the bubble field.** It is a secondary, age-confounded signal (like vote) and must not compete with authority (= bubble size); a per-practice ring would also multiply across a multi-practice answer. It surfaces only on hover (tooltip) and in the community card / answer panel, as a gold ★ / ✓. Decided 2026-06-01 after comparing on-field options (gold ring · corner ✓ badge · center pip · fill-weight · ink etch · double keyline · letterpress emboss · deepen-shade · gold seal) — every on-field marker cluttered the organic field or over-emphasized a weak signal. The ink **selection/focus** ring stays deferred to the formal frontend. + +### Encodings (unchanged — data contract / ADR 0004) +Bubble radius = `authority` (first-class contract field, shared by layout `r` and the front-end). Bubble fill = community color. The **bubble-field** idiom (hover tooltip · click node → answer panel · click blob → community card · ⏱ timeline swimlanes · inline/push reflow) is untouched; the answer panel's **internal** layout was redesigned — see *Answer detail panel* below. + +### Honesty markup (ADR 0008) +`authority = SO vote · proxy` and `PRE-DIFFERENTIATOR` stay rendered in muted mono so the proxy caveat survives the prettier skin — proxy values must not read as findings. + +## Answer detail panel — practice-led (added 2026-06-01) + +Clicking a bubble selects a **practice**, but the panel was answer-led, burying the unit of analysis (CONTEXT.md: "歸屬單位是 Practice"). Since **hover already gives the practice quick-look**, the panel's distinct job is the **source answer**. Division: *hover = practice · click = its original answer*. + +- **Contract plumbing:** `conditions` + `evidence_type` (per practice, from extraction) now flow through `contract.py` → `breakdown.py` → `build` (previously dropped). Re-run build to populate. +- **Uniform practice cards** (no per-click highlight): community = a small colored **dot + name**; a left accent bar read as generic "AI-dashboard" and was dropped. `evidence_type` renders as **chips** `⟨⟩ code` / `¶ prose` — two chips when both kinds back it (the literal word "both" was unclear). `conditions` render as a `WHEN ·` line only when present. +- **No persistent click-highlight:** darkening the clicked card over-emphasised a redundant cue and looked heavy; orientation is a **transient flash + auto-scroll** to the clicked practice on open. +- **Answer content = SO markdown, not HTML.** `mdToHtml` renders an escape-first subset (fenced → `
`, inline code, bold, links, lists, blockquotes, paragraphs): code reads as code, prose as paragraphs. Escape-first → any raw HTML in the body is shown literally, never executed. + +## Accessibility +- Paul-Tol muted is colorblind-safe across common CVD types. +- `--ink` on `--paper` ≈ 13:1 contrast; mono micro-labels use `--muted`/`--muted2` kept ≥ ~4.5:1 on their backgrounds. + +## Scope & non-goals +- **This round: light theme only.** Tokens are authored as `:root` variables so a dark variant is a later override, not a rewrite. +- The current `render.py` restyle is a **preview** to validate the identity on real data; a more formal frontend will be built later. Keep the restyle a focused token swap — no architectural rewrite. + +## Mapping to `render.py` +Single-file change: rewrite the ` + + + + ++ + + + + + +++ ◐ + + + + ++ + ++ +