Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion docs/api/pytest-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,3 @@ hook to reconcile per-worker Result counts. See
- deserialize_trial_specs
- finalize_worker
- handle_testnodedown
- discover_sinks_from_conftest
2 changes: 1 addition & 1 deletion docs/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ RAMPART registers as a pytest plugin automatically when installed. It provides:
- **Markers**: `@pytest.mark.harm(...)` for categorization, `@pytest.mark.trial(n=...)` for statistical repetition
- **Automatic result collection**: Results from `Attacks.*` and `Probes.*` are collected without manual wiring
- **Terminal summary**: A safety summary printed after the standard pytest output
- **Report sinks**: Structured output via the `pytest_rampart_sinks` hook (the `rampart_sinks` fixture is deprecated)
- **Report sinks**: Structured output via the `pytest_rampart_sinks` hook

See [pytest Markers & Fixtures](../usage/pytest-integration.md) for setup details.

Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ See [pytest Markers & Fixtures](../usage/pytest-integration.md) for the full mar

## Step 4: Add Reporting

Register report sinks with the `pytest_rampart_sinks` hook in your `conftest.py` so RAMPART writes structured JSON reports. See [pytest Markers & Fixtures](../usage/pytest-integration.md#pytest_rampart_sinks-hook) for the setup. (The older `rampart_sinks` fixture is still supported but deprecated.)
Register report sinks with the `pytest_rampart_sinks` hook in your `conftest.py` so RAMPART writes structured JSON reports. See [pytest Markers & Fixtures](../usage/pytest-integration.md#pytest_rampart_sinks-hook) for the setup.

---

Expand Down
3 changes: 0 additions & 3 deletions docs/usage/ci-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,6 @@ def pytest_rampart_sinks(config):

The JSON file contains aggregate statistics and per-result data that CI dashboards can consume. The hook is resolved on the controller, so it behaves identically in single-process and [`pytest-xdist`](xdist.md) CI runs. See [Registering Sinks](pytest-integration.md#pytest_rampart_sinks-hook).

!!! warning "Deprecated"
The older `rampart_sinks` fixture still works but is deprecated and will be removed in `0.3.0`. Prefer the `pytest_rampart_sinks` hook above.

---

## Pytest Options
Expand Down
61 changes: 6 additions & 55 deletions docs/usage/pytest-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,60 +73,13 @@ async def test_with_threshold(adapter):

---

## Fixtures

### `rampart_sinks`

!!! warning "Deprecated"
The `rampart_sinks` fixture is deprecated and will be removed in `0.3.0`.
Use the [`pytest_rampart_sinks` hook](#pytest_rampart_sinks-hook) instead — it
behaves identically in single-process and `pytest-xdist` runs and accepts the
active `pytest.Config`. Defining the fixture now emits a `DeprecationWarning`.

Define this **session-scoped** fixture in your `conftest.py` to configure report output:

```python
from pathlib import Path
import pytest
from rampart.reporting import JsonFileReportSink, ReportSink


@pytest.fixture(scope="session")
def rampart_sinks() -> list[ReportSink]:
return [JsonFileReportSink(output_dir=Path(".report"))]
```

If you don't define this fixture, RAMPART still prints the terminal summary — but no structured report files are written. You can provide multiple sinks:

```python
@pytest.fixture(scope="session")
def rampart_sinks() -> list[ReportSink]:
return [
JsonFileReportSink(output_dir=Path(".report")),
MyCustomDatabaseSink(connection_string="..."),
]
```

!!! warning "xdist compatibility"
Under [`pytest-xdist`](xdist.md), the controller process discovers fixture-based sinks by calling `rampart_sinks` directly. Fixtures that depend on other fixtures (e.g., `tmp_path_factory`, `request`) cannot be resolved on the controller and are skipped with a warning. Use a parameterless fixture or a module-level list to remain compatible:

```python
# Resolved on the xdist controller (controller-only — single-process
# discovery needs the fixture form above, or the hook below)
rampart_sinks = [JsonFileReportSink(output_dir=Path(".report"))]
```

For sinks that need configuration or dependencies, prefer the
`pytest_rampart_sinks` hook below — it is resolved on the controller and works
identically in single-process and parallel runs.

---
## Registering Sinks

### `pytest_rampart_sinks` hook

For sinks that need configuration — or to register sinks in a way that behaves
identically in single-process and `pytest-xdist` runs — implement the
`pytest_rampart_sinks` hook in your `conftest.py`:
Implement the `pytest_rampart_sinks` hook in your `conftest.py` to register the
report sinks RAMPART emits to. It behaves identically in single-process and
`pytest-xdist` runs:

```python
# conftest.py
Expand All @@ -143,10 +96,8 @@ The hook receives the active `pytest.Config`, so you can build
sinks from CLI/ini options or environment variables. Multiple implementations are
supported; RAMPART emits to the **union** of every returned sink.

**Precedence:** when any `pytest_rampart_sinks` implementation exists, it is
authoritative and the `rampart_sinks` fixture path is skipped entirely (so a
project that defines both does not double-register). The fixture remains the
single-process fallback when no hook implementation is present.
If you don't register any sinks, RAMPART still prints the terminal summary — but
no structured report files are written.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/usage/results-and-reporting.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ class MyDatabaseSink:
Register the `pytest_rampart_sinks` hook in your `conftest.py`. See [pytest Markers & Fixtures](pytest-integration.md#pytest_rampart_sinks-hook) for the setup and examples with multiple sinks.

!!! note "Parallel execution"
Under [`pytest-xdist`](xdist.md), workers send their results to the controller, which emits sinks **once** with a unified [`TestRunReport`][rampart.reporting.sink.TestRunReport]. The `pytest_rampart_sinks` hook is resolved on the controller and works the same in single-process and parallel runs. The deprecated `rampart_sinks` fixture is still supported as a single-process fallback, but on the controller it cannot depend on other fixtures. See [Registering Sinks](xdist.md#registering-sinks-the-pytest_rampart_sinks-hook) for details.
Under [`pytest-xdist`](xdist.md), workers send their results to the controller, which emits sinks **once** with a unified [`TestRunReport`][rampart.reporting.sink.TestRunReport]. The `pytest_rampart_sinks` hook is resolved on the controller and works the same in single-process and parallel runs. See [Registering Sinks](xdist.md#registering-sinks-the-pytest_rampart_sinks-hook) for details.

---

Expand Down
57 changes: 3 additions & 54 deletions docs/usage/xdist.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,57 +131,9 @@ def pytest_rampart_sinks(config):
- Non-`ReportSink` items (or a non-list return) are dropped with a warning, so one
malformed implementation cannot break emission.

### Precedence vs the `rampart_sinks` fixture

!!! warning "Deprecated"
The `rampart_sinks` fixture is deprecated and will be removed in `0.3.0`.
Prefer the `pytest_rampart_sinks` hook above. Resolving the fixture emits a
`DeprecationWarning` in both single-process and controller discovery.

The legacy `rampart_sinks` fixture is still supported as a **single-process
fallback**. The rule is:

- If **any** `pytest_rampart_sinks` hook implementation exists, the hook is
authoritative and the fixture path is skipped entirely (so a project that
defines both does **not** double-register).
- If **no** hook implementation exists, RAMPART falls back to the fixture. On the
xdist controller this fallback scans registered conftest modules for a
`rampart_sinks` attribute.

### Fixture fallback constraints (no hook present)

When you rely on the fixture fallback under xdist, pytest's fixture machinery
does not run on the controller. RAMPART therefore unwraps a **parameterless**
`rampart_sinks` fixture and calls its underlying function directly, so these
shapes resolve:

```python
# Parameterless session fixture — resolves single-process AND on the
# xdist controller.
@pytest.fixture(scope="session")
def rampart_sinks():
return [JsonFileReportSink(output_dir=Path(".report"))]

# Plain list assigned at module level — resolved on the xdist controller
# only. Single-process discovery looks up a *fixture* named rampart_sinks,
# so a bare module-level list is silently ignored there; use the fixture
# form above (or the hook) for single-process runs.
rampart_sinks = [JsonFileReportSink(output_dir=Path(".report"))]
```

A **fixture with dependencies** cannot be resolved on the controller and is
skipped with a warning:

```python
# Not resolvable on the controller — use the hook instead
@pytest.fixture(scope="session")
def rampart_sinks(my_sink_config, db_connection):
return [DatabaseSink(connection=db_connection)]
```

If your sinks need dependencies, **use the `pytest_rampart_sinks` hook** — it
receives the `pytest.Config` and runs on the controller, so you can build sinks
from `config` values or environment variables there.
If your sinks need dependencies, build them inside the hook — it receives the
`pytest.Config` and runs on the controller, so you can build sinks from `config`
values or environment variables there.

---

Expand Down Expand Up @@ -259,9 +211,6 @@ does not discard normal Results from that worker.

## Limitations

- Sinks discovered through the **fixture fallback** on the controller cannot depend
on other pytest fixtures — use the `pytest_rampart_sinks` hook instead (see
[Registering Sinks](#registering-sinks-the-pytest_rampart_sinks-hook)).
- Results recorded only during fixture teardown are outside the report-streaming
boundary and are not included.
- A worker that dies can lose Results whose eligible reports had not reached
Expand Down
56 changes: 0 additions & 56 deletions rampart/common/deprecation.py

This file was deleted.

4 changes: 2 additions & 2 deletions rampart/pytest_plugin/_hooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ def pytest_rampart_sinks(config: pytest.Config) -> list[ReportSink]: # ruff: ig

Implement this hook in your ``conftest.py`` to register sinks in a
way that works identically in single-process and ``pytest-xdist``
runs. Unlike the ``rampart_sinks`` fixture, hook implementations are
resolved on the xdist controller, which never executes fixtures.
runs. Hook implementations are resolved on the xdist controller,
which never executes fixtures.

Multiple implementations are supported; RAMPART emits to the union
of every returned sink.
Expand Down
Loading
Loading