Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -616,6 +616,19 @@ retention, repository-to-workload correlation, alert, typed intent, PEP handoff,
execution. The cache-backed `sith investigate` path cannot produce R9 until a future reader supplies
validated workflow-run graph facts and explicit TIMELINE coverage.

The brain's existing R3 CrashLoop rule can also consume the bounded Elasticsearch
`search/ecs-v1` facts produced from an already-fetched, complete Search API response. The graph
bridge requires exact Elasticsearch source/provenance, an attached Pod identity, a SHA-256
native/resource identity that recomputes from the retained sanitized aggregate and Pod, and the
closed `logs.cause` values `panic`, `missing-config`, or
`dependency-failure`. It revalidates source bounds, discards count/container/window metadata, and
preserves only the Pod, cause, last classified event time, source, and stale flag. Fact presence
does not infer TELEMETRY coverage, and evidence for one Pod cannot strengthen another. Raw logs,
index/document IDs, query text, labels, URLs, credentials, and user data do not enter the brain or
CLI output. This path adds no Elasticsearch HTTP client, endpoint/index configuration, credential,
query execution, persistence, fleet correlation, typed intent, mutation, or execution; the
cache-backed `sith investigate` command still does not fetch Elasticsearch data.

Every verdict includes its rule, exact cited signals, confidence state, missing lenses, and an
advisory command or PR change for the operator to inspect and run. The brain performs no I/O and
imports no connector planning, execution, intent, PEP, MCP, or local-operation path.
Expand Down
15 changes: 15 additions & 0 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,17 @@ sensitive human inspection guidance and adds no GitHub client, token, storage, a
typed intent, policy handoff, mutation, or execution. The cache-backed local CLI still has no
workflow-run fetch path.

The existing **R3** rule now also accepts the reviewed Elasticsearch `search/ecs-v1` graph fact
through a narrow bridge. Only an attached Pod TELEMETRY `FactDerived` with exact Elasticsearch
source/provenance, matching scope and namespace, a revalidated SHA-256 native/resource identity
bound to the retained workspace, Pod, aggregate, and collection fields, and a closed `logs.cause`
payload can become an observation. The accepted values remain `panic`,
`missing-config`, and `dependency-failure`; count, container, event-window metadata, and the source
fact payload are discarded after validation. The last classified event time and stale flag are
preserved, while TELEMETRY coverage remains entirely caller-declared. Evidence attached to another
Pod cannot strengthen the CrashLoop verdict. This bridge adds no Elasticsearch client, endpoint,
index, query execution, credential, persistence, correlation, typed intent, mutation, or execution.

## Integration waves (E12) — the connector coverage the brain needs

Connectors ship in four waves (`docs/specs/E2-readfed-brain-integrations.md` §4), each scored by
Expand Down Expand Up @@ -372,6 +383,10 @@ need:**
Raw messages are classified in memory and discarded.
Missing cluster identity, partial or failed shards, `_source`, unknown fields, and ambiguous values
fail closed. This slice adds no HTTP client, index discovery, credentials, persistence, or writes.
Issue #280 connects those already-reviewed facts to R3 without widening the source contract:
exact `elasticsearch` / `search/ecs-v1` provenance and Pod identity are validated again, only the
closed cause classification enters the brain, cross-Pod evidence stays separate, and declared
TELEMETRY coverage and staleness remain authoritative.
Issue #216 establishes the AWS autoscaler-evidence contract: one already-fetched EKS
`DescribeNodegroup` response becomes a bounded LIVE inventory fact and a bounded LIVE
provider-health fact attached to an already-trusted Sith cluster. The response's partition,
Expand Down
42 changes: 38 additions & 4 deletions docs/adr/0008-deterministic-advisory-brain.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,40 @@ contract in [Workflow runs](https://docs.github.com/en/rest/actions/workflow-run
and the closed conclusion vocabulary in
[Checks](https://docs.github.com/en/rest/guides/using-the-rest-api-to-interact-with-checks).

### 2026-07-18 extension: Elasticsearch log-cause graph bridge for R3

R3 continues to use its existing generic `logs.cause` TELEMETRY strengthener. The extension is a
narrow graph admission path for the already-reviewed Elasticsearch projector, not a new rule.
`FromGraphFacts` first validates the workspace-bounded graph, then admits only an attached
TELEMETRY `FactDerived` whose resource is an Elasticsearch `LogSignal`, source kind and provenance
adapter are both `elasticsearch`, protocol is `search/ecs-v1`, source/scope/namespace agree, and
the entity carries exactly one Pod identity. The source hash uses only the retained workspace,
scope, namespace, Pod, optional container, cause, count, first/last event times, and collection
time, allowing the bridge to recompute both the full native ID and resource-name prefix. Extra
resource attributes, display fields, entity dimensions, cross-Pod retargeting, and malformed,
noncanonical, or mismatched SHA-256 identities fail closed.

The exact-case payload is closed to `key`, `value`, `count`, `first_event_at`, `last_event_at`, and
optional `container`. The bridge revalidates the source projector's byte, count, event-window,
clock-skew, and text bounds, accepts only `logs.cause` with `panic`, `missing-config`, or
`dependency-failure`, then discards count, container, and the source payload. The resulting
observation retains only exact Pod identity, the closed cause, last classified event time, source,
and stale flag. Graph fact presence never creates TELEMETRY coverage; missing, unavailable, stale,
or observation-stale TELEMETRY evidence therefore cannot produce a fully confirmed R3 result.
Entity grouping uses exact scope, namespace, kind, and name, so evidence attached to one Pod cannot
strengthen another Pod or create a fleet-wide cause claim.

Raw messages are classified and discarded by the source projector before this bridge runs. Index
and document IDs, query text, labels, URLs, credentials, user data, and container/count metadata do
not enter the observation, citation, replay, or renderer. The extension adds bounded in-memory JSON
validation only: no HTTP/TLS client, endpoint/index configuration, API key, mapping discovery,
query execution, pagination, retention, socket, filesystem, database, process, correlation, typed
intent, dispatch, mutation, or cloud resource. Elastic documents the underlying Search API and
selected-field behavior in [Search](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search)
and [Retrieve selected fields](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/retrieve-selected-fields),
and the normalized identity fields in [ECS orchestrator fields](https://www.elastic.co/docs/reference/ecs/ecs-orchestrator)
and [Filebeat Kubernetes fields](https://www.elastic.co/docs/reference/beats/filebeat/exported-fields-kubernetes-processor).

## Consequences

- Investigations are offline, reproducible, replayable, and inspectable. The same observation
Expand All @@ -115,12 +149,12 @@ and the closed conclusion vocabulary in
telemetry-dependent cause variant. This is intentionally less confident than an opaque guess.
- Phase L does not auto-port-forward to Prometheus/Loki, retain telemetry series, read Git desired
state, or infer absent evidence. Those connectors can later emit the same observation contract.
- The existing cache-backed CLI does not fetch Argo Applications or GitHub workflow runs. R8 and R9
are available only to callers that already possess validated graph facts and explicitly declare
TIMELINE coverage.
- The existing cache-backed CLI does not fetch Argo Applications, GitHub workflow runs, or
Elasticsearch logs. R8, R9, and Elasticsearch-strengthened R3 are available only to callers that
already possess validated graph facts and explicitly declare the relevant lens coverage.
- There is no hosted or cloud cost. Runtime cost is one existing tier-1 hydration pass plus
in-memory rule evaluation over the returned records or an existing bounded graph. No extra
Kubernetes watch, Argo/GitHub request, storage, or network egress is introduced.
Kubernetes watch, Argo/GitHub/Elasticsearch request, storage, or network egress is introduced.
- Advisory strings are not shell execution. Operators remain responsible for reviewing and
running a suggestion with their own kubeconfig identity.

Expand Down
16 changes: 15 additions & 1 deletion docs/specs/E2-readfed-brain-integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,20 @@ hub). Weights are indicative, to be tuned against real incidents.
advisory* accordingly.
- **Coverage gate.** LIVE + logs(TELEMETRY) are the crux; DESIRED/TIMELINE disambiguate.

**Implemented Elasticsearch graph seam (#280).** The bounded `search/ecs-v1` projector described
in #214 can feed this existing signal without exposing raw logs to the brain. `FromGraphFacts`
accepts only an attached Pod TELEMETRY `FactDerived` whose source kind and provenance adapter are
both `elasticsearch`, protocol is exact, source/scope/namespace agree, and the SHA-256
native/resource identity recomputes from the retained workspace, Pod, aggregate, and collection
fields. The payload contains only `key`, `value`, `count`,
`first_event_at`, `last_event_at`, and optional `container`. The key must be `logs.cause`; the value
must be `panic`, `missing-config`, or `dependency-failure`; source projector count, time-window,
clock-skew, and optional-container bounds are revalidated. Only the Pod identity, cause, last event
time, source, and stale flag survive as an observation. Count and container metadata are discarded.
Caller-declared coverage is copied exactly and never inferred from fact presence. Different Pods
remain different evaluator entities, so log evidence for one Pod cannot strengthen another Pod's
CrashLoop. This is an in-memory graph bridge, not an Elasticsearch reader or freshness claim.

#### R4 — Config drift (live diverged from desired)

- **Symptom.** Live ≠ desired for a workload/Application (Argo `OutOfSync`, or `diff` verb shows a
Expand Down Expand Up @@ -595,7 +609,7 @@ the *reasoning*. Decision deferred to the owner (see §7).
| **GitHub** | RA + TA | di, rd, qy, df, **pl/ex** (`gitops.open-pr`), vf | DESIRED (manifests), TIMELINE (commits/PR/deploys) | **T1** | read=local, write=hub | Bounded pure projectors now normalize caller-fetched merged-PR and completed workflow-run failure evidence; the HTTP/token reader remains future. `gitops.open-pr` is the first governed write (P2). |
| **ArgoCD** | RA + BR + TA | di, rd, qy, **df**, pl/ex (`argocd.sync`,`argocd.rollback`), vf | DESIRED, LIVE, TIMELINE (sync history), drift | **T1** | read=local, sync=hub | Richest single connector — 3 lenses + the exemplar of the `diff` verb. Central to R1, R4. Application CRDs read via kubeconfig. |
| **Prometheus** | RA + query-through | di, **qy**, rd (alerts) | TELEMETRY | **T1** | local-if-reachable / hub | Query-through, not retained. Central to R2, R5, R6 and R1 validation. |
| **Elasticsearch** | RA + query-through | di, **qy** | TELEMETRY (logs) | **T2** | hub (local if creds) | Log search for R3. Auth/index-mapping variance → T2. |
| **Elasticsearch** | RA + query-through | di, **qy** | TELEMETRY (logs) | **T2** | hub (local if creds) | Bounded sanitized `search/ecs-v1` cause facts now bridge into R3; HTTP/auth/index/query execution remains future. Auth/index-mapping variance → T2. |
| **AWS** | RA (enum/cred) | di, rd, (qy CloudWatch later) | LIVE (nodes/infra), TIMELINE (CloudTrail later) | **T2** | cluster-enum local; deep facts hub | Enumeration + short-lived token minting (no long-lived keys). Feeds R6 (nodegroup/autoscaler). |

#### Wave 2 — the desired-state / diff pipeline
Expand Down
Loading