diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 39e0acf..87329a6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -67,7 +67,7 @@ jobs: - name: Install package with dev extras run: | python -m pip install --upgrade pip - python -m pip install -e .[dev] + python -m pip install -e .[dev,atlassian] - name: Run tests run: python -m pytest -q diff --git a/.rac/config.yaml b/.rac/config.yaml index 998cc13..d2554c3 100644 --- a/.rac/config.yaml +++ b/.rac/config.yaml @@ -1 +1,8 @@ repository_key: LCON + +# External ticketing provider (ADR-087, ADR-093). Roadmap intent lives in the +# corpus; execution is tracked in GitHub issues, linked from a live artifact's +# `## Related Tickets` section. `rac validate` format-lints those references as +# `owner/repo#123` or a URL — offline; the engine never fetches issue state. +ticketing: + provider: github diff --git a/CHANGELOG.md b/CHANGELOG.md index e767439..7d272df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,32 @@ All notable changes to `rac-connectors` are recorded here. Versions are CalVer (`YYYY.M.N`, ADR-008); the version is derived from the git tag by setuptools-scm, and the published distribution is `rac-connectors`. +## Unreleased + +The Atlassian suite connector — Jira + Confluence (rac-core ADR-090), the +first export-direction integration. Release gate: the docs page's live +smoke test against a real Cloud site (itsthelore/rac-connectors#10). + +### Added + +- The `atlassian` module and the nested CLI verbs: `rac-connect atlassian + verify` (read-only Jira `related_tickets` existence/state checks over + `rac export --graph`, exit 3 on findings — the CI gate; ADR-010) and + `rac-connect atlassian publish` (idempotent managed Confluence pages over + `rac export --documents`, keyed by the `lore.artifact_id` content + property with body-hash skip and conflict surfacing; ADR-011). +- The `[atlassian]` extra — an internal `httpx` client (no Atlassian SDK) + with Basic auth from `ATLASSIAN_*` environment variables and capped, + jittered backoff honouring `Retry-After`. +- The `TicketVerifier` / `PagePublisher` seams and `VerifySummary` beside + the existing `Connector` / `GraphConnector` shapes (ADR-010). + +### Changed + +- The graph reader now surfaces the export's `external` / `provider` edge + markers (rac-core ADR-087/096) — additive; existing connectors are + unaffected. + ## 2026.6.1 First published release of **rac-connectors** — the integrations companion for diff --git a/README.md b/README.md index 826d40f..7f6c36c 100644 --- a/README.md +++ b/README.md @@ -467,6 +467,146 @@ Design + decision: [`rac/designs/`](rac/designs) (graph-connector-shape) and +### Workspace & ticketing + +
+Atlassian — Jira related_tickets verification + Confluence managed-page publish over the export contracts + +The Atlassian suite connector (rac-core ADR-090): **verify** that every Jira +reference in the corpus still points at a real, reachable issue, and +**publish** the corpus into a Confluence space as managed pages. Both verbs +are thin consumers of the export contracts; the engine never talks to +Atlassian (rac-core ADR-087), and the connector only ever contacts the +instance you configure (rac-core ADR-086). No Atlassian SDK — an internal +`httpx` client, see [ADR-010](rac/decisions/adr-010-atlassian-verify-publish-seams.md). + +```bash +pip install 'rac-connectors[atlassian]' +export ATLASSIAN_BASE_URL=https://yourorg.atlassian.net +export ATLASSIAN_EMAIL=you@example.com +export ATLASSIAN_API_TOKEN=... # id.atlassian.com API token +export ATLASSIAN_CONFLUENCE_SPACE=DOCS # publish only; or pass --space + +rac export rac/ --graph | rac-connect atlassian verify # check Jira refs +rac export rac/ --graph | rac-connect atlassian verify --dry-run # list, no calls +rac export rac/ --documents | rac-connect atlassian publish # mirror pages +rac export rac/ --documents | rac-connect atlassian publish --dry-run # plan, no calls +``` + +## `verify` — read-only Jira reference checks + +Selects the `--graph` projection's ticket edges by contract markers +(`external: true`, `provider: "jira"` — set from your repo's +`ticketing.provider`, rac-core ADR-087), dedupes the issue keys (bare +`PROJ-123` or full `/browse/` URLs), fetches them 100 at a time through +Jira's bulk-fetch endpoint with `fields=["status"]`, and reports each as +**exists** (with status and statusCategory), **missing**, or **forbidden** +— attributed back to the referencing artifacts. `verified_by` edges +(rac-core ADR-096) and other providers' tickets are counted as skipped, +never guessed at. It writes nothing, anywhere. + +| Exit code | Meaning | +|---|---| +| 0 | Every checked reference exists. | +| 1 | The input was not a valid `--graph` payload. | +| 2 | Credentials missing from the environment. | +| 3 | One or more references are missing or forbidden — the CI gate. | + +| Flag | Meaning | +|---|---| +| `--dry-run` | List the references and batches that would be checked; no client, no calls. | +| `--input`, `-i` | Read the `--graph` JSON from a file (default: stdin; `-` also means stdin). | +| `--verbose`, `-v` | Print per-reference results on a live verify too (findings always print). | + +## `publish` — managed Confluence pages + +Mirrors the `--documents` stream into one space, idempotent on the canonical +artifact id ([ADR-011](rac/decisions/adr-011-confluence-page-identity-and-idempotency.md)): + +- **Page identity is a content property** (`lore.artifact_id`) plus a + `lore-managed` label — never the title, so artifact renames are ordinary + updates; and no page id is ever written back into the corpus (write-back + is propose-only via human PR, rac-core ADR-065). +- **Unchanged pages are skipped without a write.** The property stores a + sha256 of the rendered body; a second publish over an unchanged corpus + performs zero writes. +- **Conflicts are surfaced, never clobbered.** Updates send + `version + 1`; a 409 means a human edited the page — it is reported as a + skip and left alone. +- **Rendering is deterministic and escape-first.** A small Markdown subset + (headings, paragraphs, emphasis, code, fenced blocks, flat lists, links) + becomes storage format; corpus content is untrusted input, so hostile + HTML/macro text stays inert and only `http`/`https`/`mailto` links become + anchors. Tables are not yet rendered (they degrade to escaped text). + +| Flag | Meaning | +|---|---| +| `--space` | Confluence space key (default: `ATLASSIAN_CONFLUENCE_SPACE`). | +| `--dry-run` | Print the pages that would be upserted; no client, no calls. | +| `--input`, `-i` | Read JSONL from a file (default: stdin; `-` also means stdin). | +| `--strict` | Fail on a malformed line instead of skipping it. | +| `--verbose`, `-v` | Print per-page actions on a live publish too. | + +Exit codes are the standard 0 (done) / 1 (malformed input) / 2 (missing +credentials or space). + +## Auth + +API token + Basic auth against Atlassian Cloud — mint a token at +id.atlassian.com and set the three `ATLASSIAN_*` variables; one credential +serves Jira and Confluence. Tokens expire; rotate them like any secret. +Data Center (Bearer PAT, v2 Jira endpoints), OAuth, inbound Confluence +ingest, and Jira comment-mode are named deferrals on the +[`atlassian-connector`](rac/roadmaps/atlassian-connector.md) roadmap. + +### Python API + +```python +from rac_connectors import parse_documents, parse_graph +from rac_connectors.atlassian import ( + AtlassianPublisher, + AtlassianVerifier, + client_from_env, +) + +client = client_from_env() +report = AtlassianVerifier(client).verify(parse_graph(open("graph.json").read())) +summary = AtlassianPublisher(client, space_key="DOCS").publish( + parse_documents(open("corpus.jsonl")) +) +``` + +### Live smoke test + +The connector is wired and unit-tested against fakes and a mock transport, +but the live path (a real Cloud site) is unproven until someone runs it — +this page is `drafted (live run pending)`. To validate end to end: + +1. **Configure the environment** with a real site, account, and API token + (all four variables above; pick a scratch Confluence space). +2. **Verify, dry-run first:** + `rac export rac/ --graph | rac-connect atlassian verify --dry-run`, then + live. With a corpus referencing one known-good and one deleted issue, + confirm the exists/missing split and exit code 3. +3. **Publish twice into the scratch space:** + `rac export rac/ --documents | rac-connect atlassian publish`. First run + creates every page (property + `lore-managed` label set); the second run + must report all pages `unchanged` and perform **zero writes**. +4. **Rename check:** change one artifact's title, re-publish, and confirm + the same page updates in place (no duplicate). +5. **Conflict check:** hand-edit a managed page in Confluence, re-publish a + changed body for that artifact, and confirm the run reports a version + conflict and leaves the human edit alone. +6. **429 behaviour** (optional): run against a busy site and confirm + retries honour `Retry-After` rather than hammering. + +Then flip this page's `status` to `shipped` — and only then consider a +release tag (the gate recorded on itsthelore/rac-connectors#10). + +**Full page:** [`docs/connectors/atlassian.md`](docs/connectors/atlassian.md) + +
+ ## Run it in CI diff --git a/docs/connectors/atlassian.md b/docs/connectors/atlassian.md new file mode 100644 index 0000000..c5f2a82 --- /dev/null +++ b/docs/connectors/atlassian.md @@ -0,0 +1,140 @@ + +# Atlassian (Jira + Confluence) + +The Atlassian suite connector (rac-core ADR-090): **verify** that every Jira +reference in the corpus still points at a real, reachable issue, and +**publish** the corpus into a Confluence space as managed pages. Both verbs +are thin consumers of the export contracts; the engine never talks to +Atlassian (rac-core ADR-087), and the connector only ever contacts the +instance you configure (rac-core ADR-086). No Atlassian SDK — an internal +`httpx` client, see [ADR-010](../../rac/decisions/adr-010-atlassian-verify-publish-seams.md). + +```bash +pip install 'rac-connectors[atlassian]' +export ATLASSIAN_BASE_URL=https://yourorg.atlassian.net +export ATLASSIAN_EMAIL=you@example.com +export ATLASSIAN_API_TOKEN=... # id.atlassian.com API token +export ATLASSIAN_CONFLUENCE_SPACE=DOCS # publish only; or pass --space + +rac export rac/ --graph | rac-connect atlassian verify # check Jira refs +rac export rac/ --graph | rac-connect atlassian verify --dry-run # list, no calls +rac export rac/ --documents | rac-connect atlassian publish # mirror pages +rac export rac/ --documents | rac-connect atlassian publish --dry-run # plan, no calls +``` + +## `verify` — read-only Jira reference checks + +Selects the `--graph` projection's ticket edges by contract markers +(`external: true`, `provider: "jira"` — set from your repo's +`ticketing.provider`, rac-core ADR-087), dedupes the issue keys (bare +`PROJ-123` or full `/browse/` URLs), fetches them 100 at a time through +Jira's bulk-fetch endpoint with `fields=["status"]`, and reports each as +**exists** (with status and statusCategory), **missing**, or **forbidden** +— attributed back to the referencing artifacts. `verified_by` edges +(rac-core ADR-096) and other providers' tickets are counted as skipped, +never guessed at. It writes nothing, anywhere. + +| Exit code | Meaning | +|---|---| +| 0 | Every checked reference exists. | +| 1 | The input was not a valid `--graph` payload. | +| 2 | Credentials missing from the environment. | +| 3 | One or more references are missing or forbidden — the CI gate. | + +| Flag | Meaning | +|---|---| +| `--dry-run` | List the references and batches that would be checked; no client, no calls. | +| `--input`, `-i` | Read the `--graph` JSON from a file (default: stdin; `-` also means stdin). | +| `--verbose`, `-v` | Print per-reference results on a live verify too (findings always print). | + +## `publish` — managed Confluence pages + +Mirrors the `--documents` stream into one space, idempotent on the canonical +artifact id ([ADR-011](../../rac/decisions/adr-011-confluence-page-identity-and-idempotency.md)): + +- **Page identity is a content property** (`lore.artifact_id`) plus a + `lore-managed` label — never the title, so artifact renames are ordinary + updates; and no page id is ever written back into the corpus (write-back + is propose-only via human PR, rac-core ADR-065). +- **Unchanged pages are skipped without a write.** The property stores a + sha256 of the rendered body; a second publish over an unchanged corpus + performs zero writes. +- **Conflicts are surfaced, never clobbered.** Updates send + `version + 1`; a 409 means a human edited the page — it is reported as a + skip and left alone. +- **Rendering is deterministic and escape-first.** A small Markdown subset + (headings, paragraphs, emphasis, code, fenced blocks, flat lists, links) + becomes storage format; corpus content is untrusted input, so hostile + HTML/macro text stays inert and only `http`/`https`/`mailto` links become + anchors. Tables are not yet rendered (they degrade to escaped text). + +| Flag | Meaning | +|---|---| +| `--space` | Confluence space key (default: `ATLASSIAN_CONFLUENCE_SPACE`). | +| `--dry-run` | Print the pages that would be upserted; no client, no calls. | +| `--input`, `-i` | Read JSONL from a file (default: stdin; `-` also means stdin). | +| `--strict` | Fail on a malformed line instead of skipping it. | +| `--verbose`, `-v` | Print per-page actions on a live publish too. | + +Exit codes are the standard 0 (done) / 1 (malformed input) / 2 (missing +credentials or space). + +## Auth + +API token + Basic auth against Atlassian Cloud — mint a token at +id.atlassian.com and set the three `ATLASSIAN_*` variables; one credential +serves Jira and Confluence. Tokens expire; rotate them like any secret. +Data Center (Bearer PAT, v2 Jira endpoints), OAuth, inbound Confluence +ingest, and Jira comment-mode are named deferrals on the +[`atlassian-connector`](../../rac/roadmaps/atlassian-connector.md) roadmap. + +### Python API + +```python +from rac_connectors import parse_documents, parse_graph +from rac_connectors.atlassian import ( + AtlassianPublisher, + AtlassianVerifier, + client_from_env, +) + +client = client_from_env() +report = AtlassianVerifier(client).verify(parse_graph(open("graph.json").read())) +summary = AtlassianPublisher(client, space_key="DOCS").publish( + parse_documents(open("corpus.jsonl")) +) +``` + +### Live smoke test + +The connector is wired and unit-tested against fakes and a mock transport, +but the live path (a real Cloud site) is unproven until someone runs it — +this page is `drafted (live run pending)`. To validate end to end: + +1. **Configure the environment** with a real site, account, and API token + (all four variables above; pick a scratch Confluence space). +2. **Verify, dry-run first:** + `rac export rac/ --graph | rac-connect atlassian verify --dry-run`, then + live. With a corpus referencing one known-good and one deleted issue, + confirm the exists/missing split and exit code 3. +3. **Publish twice into the scratch space:** + `rac export rac/ --documents | rac-connect atlassian publish`. First run + creates every page (property + `lore-managed` label set); the second run + must report all pages `unchanged` and perform **zero writes**. +4. **Rename check:** change one artifact's title, re-publish, and confirm + the same page updates in place (no duplicate). +5. **Conflict check:** hand-edit a managed page in Confluence, re-publish a + changed body for that artifact, and confirm the run reports a version + conflict and leaves the human edit alone. +6. **429 behaviour** (optional): run against a busy site and confirm + retries honour `Retry-After` rather than hammering. + +Then flip this page's `status` to `shipped` — and only then consider a +release tag (the gate recorded on itsthelore/rac-connectors#10). diff --git a/pyproject.toml b/pyproject.toml index 2685b82..f5f284a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -57,6 +57,11 @@ cognee = ["cognee>=1,<2"] # fake-driven test-suite stay dependency-free. Embeddings come from an external # endpoint over stdlib HTTP (no SDK), so only qdrant-client is required here. qdrant = ["qdrant-client>=1.7,<2"] +# The live Atlassian verify/publish needs httpx (an internal client, no +# Atlassian SDK — ADR-010); the core install and the fake-driven test-suite +# stay dependency-free. CI installs this extra so the mock-transport wire +# tests run; they importorskip httpx and pass silently without it. +atlassian = ["httpx>=0.27,<1"] dev = ["pytest>=7.0", "ruff", "mypy"] [project.scripts] @@ -96,5 +101,6 @@ module = [ "letta_client.*", "cognee.*", "qdrant_client.*", + "httpx.*", ] ignore_missing_imports = true diff --git a/rac/decisions/adr-010-atlassian-verify-publish-seams.md b/rac/decisions/adr-010-atlassian-verify-publish-seams.md new file mode 100644 index 0000000..fa3b49b --- /dev/null +++ b/rac/decisions/adr-010-atlassian-verify-publish-seams.md @@ -0,0 +1,156 @@ +--- +schema_version: 1 +id: LCON-KWGHWM0W2DMV +type: decision +--- +# ADR-010: Atlassian Verify and Publish Seams, No SDK + +## Context + +rac-core ADR-090 places the Atlassian suite in this repo — `atlassian/`, one +subdir spanning both directions on shared Cloud auth — and rac-core ADR-087 +explicitly delegates ticket **existence and state checks** here: the engine +format-lints `related_tickets` references offline and never contacts a +ticketing system. The engine side is live: `rac export --graph` marks ticket +edges `external: true` with the configured `provider` (rac-core ADR-074). + +Two of the suite's surfaces are scheduled (roadmap `atlassian-connector`): +verifying Jira references, and publishing corpus artifacts to Confluence. +Neither fits the existing seams. `Connector.push` (ADR-002) is an outbound +memory upsert; `GraphConnector.push_graph` (ADR-003) loads a graph image. +Verification is *read-shaped* — it fetches issue state and writes nothing — +and Confluence publish, while outbound, has page-identity and versioning +semantics no memory backend has. ADR-002's rule that a connector "pushes, and +never pulls" guards against one specific shape: a recall/re-rank surface that +would couple reading agents to a backend. An operator-facing state check is +not that shape, but the distinction must be recorded, not assumed. + +## Decision + +- **Two sibling seams, not overloads.** Following ADR-003's precedent: + + ```python + class TicketVerifier(Protocol): + name: str + def verify(self, graph: Graph, *, dry_run: bool = False) -> VerifySummary: ... + + class PagePublisher(Protocol): + name: str + def publish(self, records: Iterable[Record], *, dry_run: bool = False) -> PushSummary: ... + ``` + + `Connector`, `GraphConnector`, their readers, and the existing CLI + subcommands are untouched (additive — rac-core ADR-007, ADR-063). +- **`verify` does not reopen ADR-002.** It is an operator-facing state check + delegated by rac-core ADR-087, run against the operator's own instance + (rac-core ADR-086); it exposes no recall, search, or re-rank surface to a + reading agent, and it never writes — not to Atlassian, not to the corpus. +- **Verify selects edges by contract markers.** Only edges with + `external: true` and `provider == "jira"` are checked. `verified_by` edges + (rac-core ADR-096) carry `provider: null` and are excluded by construction; + other providers are skipped and counted, never guessed at. +- **No Atlassian SDK.** The connector needs roughly six endpoints; a thin + internal client on `httpx` (the `[atlassian]` extra) keeps the supply chain + one dependency wide and makes contract tests trivial via a mock transport. + This deviates from ADR-002's "backend SDK behind a thin client Protocol" + pattern deliberately: there is no official Atlassian Python SDK worth + pinning, and the wrapper libraries chase API churn we would then chase + second-hand. The client Protocols still exist; only the adapter behind + them is ours. +- **Cloud first, API-token Basic auth.** Credentials come from the + environment (`ATLASSIAN_BASE_URL`, `ATLASSIAN_EMAIL`, + `ATLASSIAN_API_TOKEN`), never hard-coded. The Data Center profile (Bearer + PAT, v2 Jira endpoints) is a named deferral; the auth seam leaves the slot. +- **Jira reads use bulk fetch.** Verification batches issue keys through + `POST /rest/api/3/issue/bulkfetch` (100 keys, narrow fields). The classic + `/search` endpoint no longer exists in Jira Cloud; if search is ever + needed, it is `POST /search/jql` with cursor pagination. +- **CLI verbs nest under one backend.** `rac-connect atlassian verify` and + `rac-connect atlassian publish` — the suite will grow verbs (comment-mode, + ingest), and a flat `atlassian-verify` would fork the backend namespace. +- **Verify exits 3 on findings.** 0 = all references verified, 1 = malformed + input, 2 = missing credentials (the existing vocabulary), and 3 = one or + more references missing or forbidden — distinct so CI can gate on state + without conflating it with operator error. + +## Consequences + +### Positive + +- ADR-087's delegated obligation is discharged on the published contract + alone; the engine stays offline and pure. +- The seam split keeps the rejected recall/re-rank shape structurally + impossible: `verify` returns a report, not records. +- Contract tests drive the real request/response wire shape through a mock + transport instead of faking an SDK's object model. + +### Negative / trade-offs + +- Four seams now live in `base.py` (`push`, `push_graph`, `verify`, + `publish`). Accepted: the inputs and outputs are genuinely different + shapes, and each seam is a handful of lines. +- Owning the HTTP client means owning retry/backoff behaviour. Accepted: + that behaviour is the part worth testing anyway, and 429/`Retry-After` + handling is recorded in the design and covered offline. + +### Risks + +- Atlassian Cloud API churn lands on us directly rather than via a wrapper + release. Mitigation: a pinned, deliberately small endpoint set; the docs + page carries a live smoke-test checklist that gates any release tag. +- The verify report tempts growth toward a state-sync. Mitigation: the seam + returns a summary only; mirroring ticket state into the corpus is + explicitly out (rac-core ADR-017). + +## Status + +Accepted + +## Category + +Architecture + +## Alternatives Considered + +### Route verification through `GraphConnector.push_graph` + +Rejected: verification writes nothing; pretending a read is a push would +falsify the seam's contract and its `PushSummary` shape. + +### Use `atlassian-python-api` or `jira` (pycontribs) + +Rejected: kitchen-sink wrappers with heavy transitive dependencies, uneven +Cloud/Data-Center behaviour, and their own lag behind Atlassian's endpoint +removals — for six endpoints, a wrapper is more surface than the client it +wraps. + +### Flat CLI subcommands (`atlassian-verify`, `atlassian-publish`) + +Rejected: the suite is one backend with several verbs (ADR-090 names more to +come); nesting keeps `rac-connect ` one namespace. + +### Report findings via exit 0 and output parsing + +Rejected: the whole point of verify is a CI gate; a distinct exit code is the +deterministic contract for that. + +## Related Decisions + +- adr-002 +- adr-003 +- adr-008 +- adr-011 + +## Related Designs + +- atlassian-connector-shape + +## Related Roadmaps + +- atlassian-connector + +## Review Date + +Revisit when the Data Center profile or a second verb family (comment-mode, +ingest) is scheduled, or if Atlassian ships an official Python SDK worth +adopting. diff --git a/rac/decisions/adr-011-confluence-page-identity-and-idempotency.md b/rac/decisions/adr-011-confluence-page-identity-and-idempotency.md new file mode 100644 index 0000000..60c7c87 --- /dev/null +++ b/rac/decisions/adr-011-confluence-page-identity-and-idempotency.md @@ -0,0 +1,129 @@ +--- +schema_version: 1 +id: LCON-KWGHWM84VSVH +type: decision +--- +# ADR-011: Confluence Page Identity and Idempotent Publish + +## Context + +The `publish` seam (ADR-010) mirrors corpus artifacts into a Confluence +space. Every existing connector is idempotent on the canonical artifact `id` +(ADR-002), and publish must be too — but Confluence has no native +upsert-by-external-id. The mature Markdown publishers show the failure +modes: title-keyed lookup orphans pages on rename and creates duplicates; +writing the page id back into the source binds cleanly but mutates the +input. Our input is a read-only export stream, and corpus writes are +propose-only via human PR review (rac-core ADR-065) — id write-back is not +available to us even if we wanted it. + +Two further Confluence realities shape the decision: the API normalizes +stored XHTML, so read-body-and-compare is not a reliable change detector; +and updates are optimistic-concurrency versioned, so a stale version number +means someone else edited the page. + +## Decision + +- **Page identity is a content property, never the title.** Each managed + page carries the content property `lore.artifact_id` holding the canonical + artifact id, plus a `lore-managed` label. Upsert resolves artifact → + page by that property within the configured space; the title is display + only and free to change with the artifact. +- **No state outside Confluence.** The artifact-to-page mapping lives on + the pages themselves — no local state file, no id write-back into the + corpus (rac-core ADR-065). +- **Change detection is a body hash.** The property also stores a sha256 of + the rendered storage-format body. Unchanged hash → the record is skipped + without a write, so a second publish over an unchanged corpus performs + zero writes. +- **Updates are version-checked and never forced.** `PUT` sends + `version.number = current + 1`; a 409 conflict is recorded as a conflict + action in the summary and the stream continues. The connector never + overwrites an edit it did not make. +- **Rendering is deterministic and escape-first.** Artifact Markdown renders + to storage format through a fixed, deliberately small subset (headings, + paragraphs, emphasis, code, fenced blocks, lists, links). All corpus + content is HTML-escaped before any markup is emitted — artifact content is + untrusted input (rac-core ADR-065), so no corpus text can smuggle a macro + or raw XHTML into the page. +- **Jira backlinks ride the same idempotency.** When publish is asked to + link referenced issues back to their artifacts, it upserts a remote issue + link with `globalId = "lore:"` — Jira's native + same-`globalId` upsert makes re-runs update in place, never duplicate. + +## Consequences + +### Positive + +- Re-publish is safe by construction, mirroring every other connector's + idempotent-on-`id` contract; renames are ordinary updates. +- Human edits to managed pages are surfaced as conflicts, not silently + clobbered — consistent with the trust boundary that humans, not tools, + have the last word. +- The hash check makes the steady-state publish cheap: reads only. + +### Negative / trade-offs + +- A page manually stripped of its property/label becomes invisible to the + connector and a fresh publish creates a sibling. Accepted: the label makes + managed pages discoverable, and adopting orphans is a listable follow-up. +- The rendering subset drops constructs (tables first among them) that + full-fat publishers support. Accepted: fidelity grows additively; golden + tests pin what is supported. + +### Risks + +- Confluence content-property APIs differ subtly between Cloud v2 and Data + Center. Mitigation: Cloud first (ADR-010); the property/label scheme + itself is portable. +- Hash-skip hides drift if Confluence rewrites stored XHTML more + aggressively than expected. Mitigation: the hash is of *our* rendered + output, not the read-back body, so skips are decided entirely on our side. + +## Status + +Accepted + +## Category + +Architecture + +## Alternatives Considered + +### Title-keyed page lookup + +Rejected: renames orphan pages and duplicate on next publish — the exact +failure mode the property scheme exists to prevent. + +### Page-id write-back into the corpus + +Rejected: the export stream is a read-only contract surface, and corpus +writes are propose-only via human PR (rac-core ADR-065). + +### Read the stored body and diff for change detection + +Rejected: Confluence normalizes stored XHTML, so byte comparison flaps; +hashing our own rendered output is deterministic. + +### Force updates on version conflict + +Rejected: a conflict means a human edited the page; the connector defers to +humans by contract. + +## Related Decisions + +- adr-010 +- adr-002 + +## Related Designs + +- atlassian-connector-shape + +## Related Roadmaps + +- atlassian-connector + +## Review Date + +Revisit when orphan adoption, table rendering, or the Data Center profile is +scheduled, or if Confluence ships a native external-id upsert. diff --git a/rac/designs/atlassian-connector-shape.md b/rac/designs/atlassian-connector-shape.md new file mode 100644 index 0000000..ca37fb9 --- /dev/null +++ b/rac/designs/atlassian-connector-shape.md @@ -0,0 +1,173 @@ +--- +schema_version: 1 +id: LCON-KWGHWMFFTHY7 +type: design +--- +# Atlassian Connector Shape: Client, Verify, Publish + +## Context + +ADR-010 fixes the seams (`TicketVerifier.verify`, `PagePublisher.publish`, +no SDK, Cloud first) and ADR-011 fixes the publish identity model. This +design works the *how*: module layout, client shape, transport behaviour, +the verify report, the rendering subset, and the CLI tree. It is the +Atlassian sibling of `graph-connector-shape`. + +## User Need + +An operator with a corpus full of `related_tickets` Jira references wants a +CI-runnable check that every reference still points at a real, reachable +issue — and a way to mirror the corpus into the Confluence space where the +rest of the organisation reads, without hand-copying or clobbering human +edits. Both must run from the export contract alone, with `--dry-run` +needing no credentials. + +## Design + +Module layout, mirroring the per-backend convention: + +```text +src/rac_connectors/atlassian/ + __init__.py re-exports: AtlassianVerifier, AtlassianPublisher, BACKEND, + MissingCredentialsError, client_from_env + client.py env credentials, JiraClient / ConfluenceClient Protocols, + HttpAtlassianClient (httpx, lazy import), backoff + jira.py verify_graph(): edge selection, key extraction, bulk fetch, + classification; remote-link upsert helper + render.py render_storage(markdown) -> str, body_hash(storage) -> str + confluence.py publish_records(): property lookup, create/update/skip + connector.py AtlassianVerifier / AtlassianPublisher seam implementations +``` + +`VerifySummary` sits beside `PushSummary` in the shared `base.py`, where the +`TicketVerifier` and `PagePublisher` Protocols also live (the ADR-003 +precedent: seams are shared shape, implementations are per-backend). + +**Client.** `HttpAtlassianClient` implements both Protocols over one `httpx` +client (imported lazily so the core install stays dependency-free). Base +URLs derive from `ATLASSIAN_BASE_URL`: Jira under `/rest/api/3/`, Confluence +under `/wiki/api/v2/`. Auth is Basic `email:api_token`. Every request runs +through one `_request()` helper: timeout ~30s, sequential (no concurrency), +and a retry loop for 429/5xx honouring `Retry-After` with capped, jittered +exponential backoff (at most 4 retries; the sleep function is injectable so +tests run instantly). 409 on a page update raises `VersionConflictError`; +401/403 raise a permission error the callers classify. + +**Verify.** `verify_graph(graph, client, *, dry_run)`: + +1. Select edges with `external and provider == "jira"` (ADR-010); count + everything else it skips (other providers, `verified_by`, in-corpus). +2. Extract issue keys from edge targets — bare `PROJ-123` or a full browse + URL — and dedupe while remembering every source artifact per key. +3. Batch keys 100 at a time through `POST /issue/bulkfetch` with + `fields=["status"]`. +4. Classify each key: `exists` (with status name and statusCategory), + `missing`, or `forbidden`; attribute results back to source artifacts in + the action log (`RAC-XXXX -> PROJ-123: missing`). + +`VerifySummary` carries counts (`checked/exists/missing/forbidden/skipped`), +the per-key action log, a `summary_line()`, and `exit_code` (0 clean, 3 +findings). Dry-run lists the keys and batches it would check — no client, +no network. + +**Render.** A deterministic Markdown → storage-format subset: ATX headings, +paragraphs, bold/italic/inline code, fenced code blocks (a `code` +structured macro with CDATA-safe escaping), ordered/unordered lists, links. +Everything is HTML-escaped before markup is emitted; unknown constructs +degrade to escaped text. Same input, same bytes — `body_hash` is sha256 +over the rendered output. + +**Publish.** For each record: render, hash, look up the page by the +`lore.artifact_id` content property in the configured space. Not found → +create (title from the record, optional parent), then set the property +(artifact id + body hash) and the `lore-managed` label. Found, hash equal → +skip as `unchanged`. Found, hash differs → `PUT` with `version.number + 1` +and refresh the property; a 409 becomes a `conflict` action and the stream +continues. The summary is the shared `PushSummary`. + +**CLI.** + +```text +rac-connect atlassian verify --input graph.json [--dry-run] [--verbose] +rac-connect atlassian publish --input docs.jsonl [--dry-run] [--strict] + [--space KEY] [--verbose] +``` + +Nested verbs under one `atlassian` subparser, each with its own +`set_defaults(func=...)` — `main()` untouched. Exit codes: verify +0/1/2/3 per ADR-010; publish the standard 0/1/2. Missing credentials name +the exact environment variables. `--space` overrides +`ATLASSIAN_CONFLUENCE_SPACE`. + +**Testing.** Connector behaviour and CLI tests drive in-memory fakes of the +two Protocols (the repo's standard pattern). Client contract tests use +`httpx.MockTransport` behind `pytest.importorskip("httpx")`: auth header +shape, bulkfetch body, 429 retry with patched sleep, 409 mapping. Render +has golden fixtures including hostile input (`") + assert "