RWE & cross-deliverable enhancements: 8 new tools (v1.17.0–v1.20.0) - #3
Merged
Merged
Conversation
Adds rwe.method_select, a deterministic tool that recommends a real-world-
evidence study design for a research question. Scores the five core RWE
methodologies (retrospective database analysis, survey, literature review,
chart review, social-media listening) against the research objective, the
data the user can access, the decision context, and the rigour required.
Returns a primary recommendation + ranked alternatives, each with its
results-validity tier, intrinsic bias caveats, and the downstream tools in
this server that operationalise it (literature.search, pv.signal_workflow,
evidence.population_adjusted, data.claims_query, ...) — so it doubles as a
router into the rest of the toolset. Hard feasibility gate excludes methods
whose required data is unavailable; flags rigor_satisfiable=false when no
feasible design meets the requested rigour; suggests triangulation across
complementary designs.
Derived from an RWE method-comparison framework; validity tiers per
ISPOR/ISPE RWE Good Procedural Practices + the FDA RWE framework. Pure
logic, <200ms, no external calls. Mirrors the pv.classify architecture
(registry + decision tree + tool). 22 tests.
New: src/rwe/{types,methodRegistry,decisionTree}.ts,
src/tools/rweMethodSelect.ts, tests/tools/rweMethodSelect.test.ts.
Wired into server.ts (ListTools + dispatch). Docs + version bumped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
Two RWE-focused tools derived from real-world class-safety and RCT-vs-RWE
medical-affairs deliverables.
pv.comparative_safety — class-level comparative safety profile from
spontaneous-report data (FAERS/EudraVigilance/VigiBase). Ranks top-N
adverse events per product by reporting rate per 1,000 exposed (or raw
count), lays products side-by-side in a class-comparison matrix, and emits
observations: shared class profile, product-level differentiators, and
explicit events_of_interest call-outs (e.g. "cardiovascular not in any
product's top 10"). Optional disproportionality (PRR/ROR/IC/EBGM) per
drug-AE pair, reusing computeAllStats. Complements pv.signal_workflow
(single drug x single AE). Spontaneous-report rates are reporting
behaviour, not incidence — flagged as hypothesis-generating.
evidence.triangulation — per-outcome RCT-vs-RWE concordance. Classifies
agreement (concordant/partial/discordant/single-source) and, when both
bodies give an estimate on the same measure, whether the real-world effect
is larger (long-term use, broad populations) or smaller (efficacy-
effectiveness gap) — sign-normalised by benefit_direction so "larger"
always means more benefit for both lower-is-better and higher-is-better
outcomes. Builds the "RCTs and RWE key message per outcome" deliverable.
RWE framed as complementary to, not a substitute for, RCTs.
Both pure deterministic logic, no external calls, submission disclosure
default, registered in server.ts. 26 tests.
New: src/pv/comparativeSafety.ts, src/tools/pvComparativeSafety.ts,
src/triangulation/{types,concordance}.ts,
src/tools/evidenceTriangulation.ts, + tests. Docs + version bumped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
Implements the lowest-validity RWE method (social-media listening) from
rwe.method_select end-to-end, without live scraping — avoiding platform
ToS, network-egress, and PV-liability exposure.
rwe.social_listening_protocol — generates a study protocol + compliance
checklist: objectives, search strategy, inclusion/exclusion, analysis plan
(sentiment + thematic + MedDRA-coded AE extraction), data-governance/
privacy/ethics (GDPR Art. 6/9 and HIPAA items gate on jurisdiction), a
MANDATORY GVP Module VI pharmacovigilance-handling section (applies to any
systematic digital-media review regardless of objective/channel ownership;
escalated for MAH-managed channels), deliverables, checklist, limitations.
pv.social_listening_triage — GVP Module VI reportability triage of
already-collected posts. Caller supplies the four-element ICSR assessment
per post; tool applies the deterministic validity test (all four -> valid
reportable ICSR, 15d serious / 90d non-serious; product+event -> follow-up;
no AE -> qualitative insight), tallies sentiment/themes/AE terms, surfaces
reporting obligations + low-validity caveats. No scraping, no in-tool NLP —
keeps output auditable and anti-fabrication.
Both pure logic, no external calls, submission disclosure default,
registered in server.ts. 21 tests.
New: src/social/{types,reportability,protocol}.ts,
src/tools/{pvSocialListeningTriage,rweSocialListeningProtocol}.ts, + tests.
Docs + version bumped to 1.19.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
Single-source-of-truth layer so evidence figures stay consistent across
dossiers, publications, and payer materials. Closes the cross-deliverable
linking gap: claims are linked by reference, not copy-paste, and drift is
detectable.
evidence.claim_registry — author a claim once (ICER, effect estimate,
prevalence) and reference it by ID. Persisted in the project KB at
<project>/claims/registry.json. Actions upsert/list/get/remove; each claim
carries numeric_value + value_display + unit + keywords + citation +
source_tool/run provenance + status (draft/verified/superseded).
evidence.consistency_check — scans deliverables for registered (or inline)
claims, locates each by keyword, and flags drift (a different number within
+/-70 chars of the keyword, e.g. an ICER updated in the model but not the
dossier) or absence. Returns a claim x deliverable matrix + drifting-claims
list. Keyword-anchored + numeric; necessary-not-sufficient, human review
still required.
publication.draft — drafts an abstract/manuscript(IMRaD)/poster/plain-
language summary that reuses registry claims so published figures match the
dossier. Auto-selects the reporting guideline by study design (CONSORT/
STROBE/PRISMA/CHEERS), enforces per-type word limits, emits a GPP2022 +
ICMJE compliance checklist; flags over-limit drafts, unregistered RCTs, and
superseded claims; notes AI cannot be a named author.
All pure logic + a project-registry read, registered in server.ts. 13
tests. New: src/claims/{types,consistency,publication}.ts,
src/knowledge/claimStore.ts (+ getClaimsDir),
src/tools/{claimRegistry,consistencyCheck,publicationDraft}.ts, + tests.
Docs + version bumped to 1.20.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
Completes the "living evidence intelligence — from review to
reimbursement" architecture: one living source of truth feeding every
downstream deliverable.
evidence.gap_analysis (iEGP) — integrated Evidence Generation Plan.
Assesses the evidence base across 11 HEOR domains (each robust/limited/
absent/discordant) and returns the gaps, a recommended generation activity
per gap, the tool that operationalises it, the deliverable it unblocks, and
a severity-prioritised plan (decision-critical domains escalate). Folds in
discordant/single-source outcomes from evidence.triangulation; reports a
readiness score.
workflow.living_evidence — orchestrates SLR -> living knowledge base ->
JCA/HTA deliverables as an ordered runbook. baseline = full chain;
refresh = only the steps triggered by living_review/consistency signals, so
an unchanged refresh is a near no-op. Deterministic runbook generator, not
an executor — the calling agent runs the steps; the host owns persistence +
the refresh schedule (server is stateless).
Both pure logic, no external calls, registered in server.ts. 17 tests.
New: src/iegp/{types,domainRegistry,planner}.ts,
src/orchestration/livingEvidence.ts,
src/tools/{evidenceGapAnalysis,workflowLivingEvidence}.ts, + tests.
Docs + version bumped to 1.21.0 (44 tools).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
Implementation-ready spec for the medvera web-tier home screen: 6 blocks (Living Evidence Intelligence hero, single-source-of-truth, capability grid, 44-tool catalog, internal-only RWD block, trust strip). One codebase gated by NEXT_PUBLIC_HEOR_ENABLE_INTERNAL_CLAIMS for the external (public) vs internal (Azure) releases. Guardrail: generic/de-branded only — no proprietary data, product names, or client slides. Lives here as the source of the work; components belong in medvera (out of this session's scope until added). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
The two fetchNiceTa tests hit the live NICE endpoint with no timeout override, so a slow-but-successful response trips Jest's 5s default and fails CI (PR #3 run on a160f2d: "Exceeded timeout of 5000 ms"). Unrelated to the PR's changes — the prior commit was docs-only. Bump to 15000ms to match the existing wiley.test.ts pattern; assertions are unchanged, so a truly broken endpoint or wrong-shaped data still fails. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
The only tests making unmocked third-party calls (niceTa, wiley live cases) were flaking CI on slow/unreachable endpoints — PR #3 flaked twice this way. Gate them behind RUN_LIVE_TESTS=1 via tests/helpers/live.ts (liveDescribe/liveIt). `npm test` is now fully deterministic (live tests skipped); `npm run test:live` runs them. Deterministic cases on those providers (mocked 429, key-absent, query sanitisation) still always run. Supersedes the niceTa timeout band-aid. No runtime/tool behaviour change. Verified: full suite 1409 passed, 5 skipped, 0 failed in a network-blocked environment (previously 3 wiley failures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
…truth) [#5]
Adds an 'import' action to evidence.claim_registry so the claim registry
self-populates from tool results instead of manual upserts. Pass
import_from { source_tool, result } with the structured output of
models.cost_effectiveness (-> ICER, incremental QALYs, incremental cost) or
models.budget_impact (-> net budget impact); claims are registered with
provenance (source_tool/source_run_id, tagged "imported").
Additive and low-risk: a new pure extractor (src/claims/extract.ts) reads
the raw result or a {content:...} envelope, skips missing/non-finite
fields, and never touches the source tools — their default output is
unchanged. Imported claims carry the right keywords, so
evidence.consistency_check detects drift of those figures across
deliverables immediately (verified end-to-end).
New: src/claims/extract.ts; extended src/tools/claimRegistry.ts. 5 new
tests (claim-layer suite 17 passing). Docs + version bumped to 1.22.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
A GVD refresh now regenerates only the sections whose figures changed, not
the whole dossier.
snapshot: record which registry claims each GVD section is built from
(sections [{name, claim_ids}]), capturing each claim's current value as a
baseline manifest persisted at <project>/gvd/manifest.json.
refresh: diff the snapshot against the live claim registry by value (0.5%
tolerance / exact) and return the stale sections to regenerate (old->new per
claim), also flagging sections whose claims were removed (claim_missing) or
superseded.
Builds on evidence.claim_registry and is the Living GVD step of
workflow.living_evidence. Computes the diff + regeneration list only —
regeneration stays in hta.dossier (that tool is untouched). Additive,
project-scoped persistence.
New: src/gvd/{types,livingGvd}.ts, src/knowledge/gvdManifestStore.ts
(+ getGvdDir), src/tools/htaLivingGvd.ts, + tests. 9 tests. 45 tools.
Docs + version bumped to 1.23.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
…s sync workflow.living_evidence now reflects the tools built after it was written: - baseline registers claims via the claim registry import action (auto-extract from model runs) and adds a "Snapshot the Living GVD" step (hta.living_gvd). - refresh adds an always-on "Living GVD section diff" step and makes regeneration section-level (regenerate only stale GVD sections, re-snapshot). No schema change — the orchestrator runbook now mirrors the real toolchain. Docs: README + package description synced to 45 tools / v1.23.x with a "Living Evidence Intelligence" summary of the v1.17–v1.23 additions. Tests updated (orchestrator suite asserts the new steps). Full suite 1421 passed, 5 skipped, 0 failed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Eight new tools across four releases, derived from a real-world-evidence / medical-affairs review (RCT-vs-RWE, class safety, RWE methodology, social listening) plus the cross-deliverable traceability follow-ups. All pure deterministic logic, no new external API calls, ISPOR AI-disclosure blocks throughout, registered in
server.ts.rwe.method_selectpv.comparative_safety·evidence.triangulationrwe.social_listening_protocol·pv.social_listening_triageevidence.claim_registry·evidence.consistency_check·publication.draftDesign notes
pv.signal_workflowstatistics; triage applies rules to caller-supplied assessments rather than inventing extractions.<project>/claims/registry.json); consistency check is keyword-anchored + numeric (necessary-not-sufficient, human review still required).Tests
wileylive-network tests (Crossref blocked by egress policy) — unrelated to this change.Files
src/rwe/*,src/triangulation/*,src/social/*,src/claims/*,src/knowledge/claimStore.ts(+getClaimsDir),src/pv/comparativeSafety.ts, and 8 tools undersrc/tools/+ tests. Docs (FEATURES.md,CHANGELOG.md) and version (1.16.0 → 1.20.0) updated.🤖 Generated with Claude Code
https://claude.ai/code/session_017QpgqeqCEf6oX8NKexi8i1
Generated by Claude Code