From 9ef9a90511db34080aea714790df1ed40a594199 Mon Sep 17 00:00:00 2001 From: DENGXUELIN <37065511+DENGXUELIN@users.noreply.github.com> Date: Tue, 9 Jun 2026 01:14:54 +0800 Subject: [PATCH] Improve NIST CSF profile evidence confidence --- .../compliance/nist-csf-assessment/SKILL.md | 65 +++++++++++++++++++ ...risk-scoped-profile-confidence-evidence.md | 46 +++++++++++++ ...get-profile-treated-as-current-evidence.md | 52 +++++++++++++++ 3 files changed, 163 insertions(+) create mode 100644 skills/compliance/nist-csf-assessment/tests/benign/risk-scoped-profile-confidence-evidence.md create mode 100644 skills/compliance/nist-csf-assessment/tests/vulnerable/target-profile-treated-as-current-evidence.md diff --git a/skills/compliance/nist-csf-assessment/SKILL.md b/skills/compliance/nist-csf-assessment/SKILL.md index 0962e190..44f75672 100644 --- a/skills/compliance/nist-csf-assessment/SKILL.md +++ b/skills/compliance/nist-csf-assessment/SKILL.md @@ -348,6 +348,51 @@ Determine the overall organizational Tier based on aggregated assessment across --- +### Step 4.5: Profile Evidence Confidence Gate + +**Objective:** Score current and target profiles using evidence quality, scope, +freshness, and risk context. Do not let interview-only statements, stale +artifacts, or target-state aspirations carry the same confidence as current +implementation evidence. + +Apply these gates before finalizing subcategory scores: + +``` +CSF-CONF-01: Subcategory score lacks evidence source, artifact ID, owner, evidence date, and assessed scope +CSF-CONF-02: Evidence type is not classified (automated telemetry, independent test, audit artifact, owner attestation, interview, or plan) +CSF-CONF-03: Current profile evidence is mixed with target profile intent or planned improvements +CSF-CONF-04: Evidence freshness is unknown or outside the subcategory's acceptable review window +CSF-CONF-05: Evidence coverage is incomplete for the business unit, system, supplier, data class, or geography being scored +CSF-CONF-06: Evidence confidence ignores inherent risk, control criticality, or compensating evidence +CSF-CONF-07: Assumptions and validation-needed items are omitted from low or medium confidence scores +CSF-CONF-08: Remediation priority is based only on score gap and does not consider confidence or evidence weakness +``` + +**Evidence confidence model:** + +| Evidence Type | Typical Confidence | Notes | +|---|---|---| +| Automated telemetry / continuous control | High when current, scoped, and mapped to the subcategory | +| Independent test or audit artifact | High/Medium depending on recency, scope, and method | +| Owner attestation with manager sign-off | Medium by default; can be High for low-risk scope with corroborating artifacts | +| Interview or workshop statement | Low until backed by artifacts or testing | +| Target-state plan or roadmap | Useful for target profile only; not current implementation evidence | + +**Minimum profile evidence fields:** + +| Field | Required Evidence | +|---|---| +| Subcategory | Official CSF 2.0 ID and description | +| Profile side | Current profile, target profile, or gap-analysis assumption | +| Source | Artifact/report/tool/export/interview, owner, evidence date, and collection method | +| Scope | Business unit, system, supplier, data class, geography, and exclusions | +| Evidence type | Telemetry, independent test, audit artifact, owner attestation, interview, plan | +| Score rationale | Why the evidence supports the current score and target score separately | +| Risk context | Inherent risk, criticality, compensating evidence, and dependency strength | +| Confidence | High/medium/low with assumptions and validation needed | + +--- + ### Step 5: Organizational Profile Development #### 5.1 Current Profile @@ -358,6 +403,12 @@ Document the current state for each function/category/subcategory: | Function | Category | Subcategory | Current Score | Evidence | Gaps | ``` +For each current-profile score, add evidence metadata: + +``` +| Subcategory | Current Score | Evidence Type | Source / Owner / Date | Scope / Coverage | Confidence | Assumptions / Validation Needed | +``` + #### 5.2 Target Profile Define the target state based on: @@ -370,6 +421,12 @@ Define the target state based on: | Function | Category | Subcategory | Current Score | Target Score | Gap | Priority | ``` +Keep target-profile intent separate from current evidence: + +``` +| Subcategory | Target Score | Target Driver | Current Evidence Confidence | Planned Evidence Needed | Priority Impact | +``` + #### 5.3 Gap Analysis For each subcategory where Current < Target: @@ -459,6 +516,12 @@ Use the NIST CSF 2.0 Reference Tool for comprehensive mappings. | GV.OC-01 | Organizational mission informs CSRM | [0-4] | [0-4] | [delta] | [H/M/L] | [refs] | | ... | ... | ... | ... | ... | ... | ... | +## Profile Evidence Confidence + +| Subcategory | Profile Side | Evidence Type | Source / Owner / Date | Scope / Coverage | Score Rationale | Risk Context | Confidence / Validation Needed | +|-------------|--------------|---------------|-----------------------|------------------|-----------------|--------------|--------------------------------| +| [CSF ID] | [current/target] | [type] | [artifact + owner + date] | [scope] | [why score is supported] | [risk/criticality] | [H/M/L + next validation] | + ### IDENTIFY (ID) [same table format] @@ -576,6 +639,8 @@ Tier 4 — Adaptive 4. **Failing to develop actionable organizational profiles.** The current and target profiles are the primary outputs of a CSF assessment. Many organizations conduct the assessment but do not formalize profiles into living documents that drive investment decisions, resource allocation, and progress tracking. Without profiles, the assessment becomes a one-time exercise rather than a continuous improvement tool. +5. **Treating unverified statements as control evidence.** Interviews and self-attestations can support a CSF assessment, but they need owner/date/scope, confidence, assumptions, and validation-needed metadata. Target-state roadmaps should not raise current-profile scores. + --- ## Prompt Injection Safety Notice diff --git a/skills/compliance/nist-csf-assessment/tests/benign/risk-scoped-profile-confidence-evidence.md b/skills/compliance/nist-csf-assessment/tests/benign/risk-scoped-profile-confidence-evidence.md new file mode 100644 index 00000000..1f76f3b6 --- /dev/null +++ b/skills/compliance/nist-csf-assessment/tests/benign/risk-scoped-profile-confidence-evidence.md @@ -0,0 +1,46 @@ +# Benign Fixture: Risk-Scoped Profile Confidence Evidence + +## Scenario + +The assessment scores `PR.PS-01` for a non-critical internal reporting service. +The evidence is a signed owner attestation backed by a configuration export and +recent vulnerability-scan sample. The target profile remains higher, but the +current score and confidence are documented separately. + +## Evidence Snapshot + +| Field | Value | +|---|---| +| Subcategory | `PR.PS-01` managed asset configuration is established and maintained | +| Profile side | Current profile | +| Current score | `2` | +| Target score | `3` | +| Evidence type | Owner attestation with corroborating artifacts | +| Source artifact | `cfg-export-reporting-2026-06-20.json` | +| Attestation | `control-owner-signoff-PRPS01-2026-Q2.pdf` | +| Scan sample | `vm-scan-reporting-2026-06-22.csv` | +| Owner | Reporting Platform Manager | +| Evidence date | `2026-06-22` | +| Scope | Internal reporting service, low inherent risk, no regulated data | +| Coverage | 11 of 11 reporting hosts and IaC baseline | +| Score rationale | Configuration baseline exists and is approved, but enforcement is not fully automated | +| Risk context | Low criticality lowers required evidence threshold for current score `2` | +| Confidence | Medium; acceptable for low-risk current score, validation needed for target `3` | +| Validation needed | Prove automated drift detection before target score `3` | + +## Positive Controls + +- `CSF-CONF-01`: Evidence source, owner, date, and assessed scope are recorded. +- `CSF-CONF-02`: Evidence type is classified and supported by artifacts. +- `CSF-CONF-03`: Current score `2` is separate from target score `3`. +- `CSF-CONF-04`: Evidence freshness is current for the quarterly assessment. +- `CSF-CONF-05`: Coverage is explicit for all reporting hosts in scope. +- `CSF-CONF-06`: Confidence considers low inherent risk and corroborating evidence. +- `CSF-CONF-07`: Validation needed for target maturity is documented. +- `CSF-CONF-08`: Remediation priority can account for both score gap and medium confidence. + +## Expected Result + +Accept current score `2` with medium confidence for the low-risk scope. Do not +flag the owner attestation as invalid by itself, because it is scoped, dated, +signed, corroborated, and paired with validation needed for the higher target. diff --git a/skills/compliance/nist-csf-assessment/tests/vulnerable/target-profile-treated-as-current-evidence.md b/skills/compliance/nist-csf-assessment/tests/vulnerable/target-profile-treated-as-current-evidence.md new file mode 100644 index 00000000..c905b0e2 --- /dev/null +++ b/skills/compliance/nist-csf-assessment/tests/vulnerable/target-profile-treated-as-current-evidence.md @@ -0,0 +1,52 @@ +# Vulnerable Fixture: Target Profile Treated as Current Evidence + +## Scenario + +The assessment scores `DE.CM-01` at current score `3` because leadership says the +organization is targeting Tier 3 monitoring this year. The only artifact is a +roadmap slide. There is no current telemetry, monitoring coverage inventory, or +validation that monitored services match the assessment scope. + +## Evidence Snapshot + +| Field | Value | +|---|---| +| Subcategory | `DE.CM-01` networks and network services are monitored | +| Profile side | Current profile | +| Current score assigned | `3` | +| Target score | `3` | +| Evidence type | Target-state plan | +| Source | `2026-security-roadmap.pptx` | +| Owner | VP Security | +| Evidence date | `2026-01-10` | +| Assessment date | `2026-06-30` | +| Scope claimed | Enterprise network monitoring | +| Coverage proof | Not provided | +| Telemetry proof | Not provided | +| Assumptions | Not recorded | +| Confidence | Not recorded | + +## Problem Indicators + +- `CSF-CONF-01`: The score lacks an implementation artifact, collection method, + and scoped evidence supporting the current state. +- `CSF-CONF-02`: A target-state plan is not classified separately from current + implementation evidence. +- `CSF-CONF-03`: Current profile score is inflated by target profile intent. +- `CSF-CONF-04`: Evidence freshness is weak for a monitoring capability. +- `CSF-CONF-05`: Enterprise coverage is claimed without asset/service scope. +- `CSF-CONF-07`: Assumptions and validation-needed items are omitted. +- `CSF-CONF-08`: Priority is understated because the low-confidence evidence is + not included in remediation ranking. + +## Expected Finding + +Classify as **Significant Gap** or mark current score as low-confidence until +fresh monitoring telemetry, service coverage, owner attestation, and validation +evidence are collected. The target score can remain `3`, but the current score +should not be raised by roadmap intent. + +## Required Remediation + +Separate current and target evidence. Add source owner/date/scope, evidence type, +coverage, confidence, assumptions, and validation-needed fields for `DE.CM-01`.