quarkus-goblin

Chaos engineering extension for Quarkus -- inject latency, exceptions, HTTP failures, and dependency degradation into your running application without touching a single line of source code.

Why?

Quarkus has excellent resilience primitives (MicroProfile Fault Tolerance, Mutiny reactive timeouts, health probes), but no built-in way to trigger the failures these mechanisms are supposed to handle. Goblin fills that gap.

Features

• Latency injection -- Artificial delay before processing (configurable min/max ms)
• Exception injection -- Throw configurable exceptions before method execution
• HTTP status forcing -- Return specific HTTP status codes (503, 500, etc.)
• Dependency degradation -- Simulate downstream service failures
• Response body injection -- Truncate or inflate the response entity (TRUNCATE keeps the first N%, INFLATE pads it) to break strict JSON clients and length-validating consumers. Applies to String, CharSequence and byte[] entities; an object serialized by a JSON provider (e.g. a POJO) is left untouched
• Response header injection -- Set or remove headers on emitted responses (SET forces the value, replacing an existing header or adding it when absent; REMOVE deletes it when present)
• Client-side assaults -- Inject latency and exceptions into outgoing MicroProfile / Quarkus REST Client calls (quarkus-rest-client) and Vert.x WebClient calls (GoblinWebClient.enable(...), opt-in at client creation)
• Metrics -- Optional quarkus-goblin-metrics module exposing assault activity as Micrometer / Prometheus metrics (goblin_assaults_total, goblin_latency_injected_seconds, goblin_active, see the Metrics guide)
• Tracing -- Optional quarkus-goblin-opentelemetry module emitting one goblin.assault span per assault with goblin.assault.* attributes, linked to the parent request span (see the Tracing guide)
• Multiple types simultaneously -- Enable latency + exception together for slow failure simulation
• Targeting -- By package, by annotation, by percentage of requests
• Dev UI -- Toggle assaults, edit config, view history -- all in real time
• State persistence -- Dev UI config changes survive restarts automatically (.goblin-state.json, dev mode); the master on/off toggle is not written to that file: a deactivation survives the live reloads of the running dev process, and a new process starts from quarkus.goblin.enabled
• Markdown report export -- Generate a factual report of config + assault history, ready to hand to an LLM for resilience review
• Multi-layer chaos -- Arm the DATABASE, MESSAGING, SERVICE, HTTP_OUT and HTTP_IN layers independently (Dev UI switches): DATABASE fails or delays JDBC connection acquisition (Agroal, below Hibernate / Panache), MESSAGING faults @Incoming consumers, and SERVICE injects latency/exceptions on business beans inside MicroProfile Fault Tolerance (@Priority(4100)), so @Retry, @Fallback, @Timeout and @CircuitBreaker react for real
• Dev by default, tests on opt-in -- Chaos activates under quarkus:dev, and under @QuarkusTest only with quarkus.goblin.test.enabled=true; in a production build the engine stays inactive and no bean is woven with the service interceptor

Quick start

Add the dependency:

<dependency>
    <groupId>io.quarkiverse.goblin</groupId>
    <artifactId>quarkus-goblin</artifactId>
    <version>${goblin.version}</version>
</dependency>

Start in dev mode:

./mvnw quarkus:dev

Open the Dev UI at http://localhost:8080/q/dev and look for the Goblin card.

Demo application

quarkus-goblin-demo -- The Falling Whale -- is a complete Quarkus application to try Goblin on: a medieval tavern whose services are guarded by MicroProfile Fault Tolerance (@Retry, @Fallback, @Timeout, @CircuitBreaker, @RateLimit), with a database, an outgoing REST client, and an observability stack. It also ships an AGENTS.md to let an AI agent run the chaos experiments through Dev MCP.

Configuration

# Enable/disable (default: true, only ever active in dev mode and, on opt-in, in test mode)
quarkus.goblin.enabled=true

# Opt in to chaos in @QuarkusTest (default: false, so adding the extension never slows down nor breaks your tests)
# quarkus.goblin.test.enabled=true

# Assault type enabled at startup (can be changed at runtime via Dev UI)
quarkus.goblin.assault.type=LATENCY

# Predefined composite assault mode: NONE, SLOW_FAILURE, INTERMITTENT, TIMEOUT
# Overrides assault.type when not NONE. Individual assaults stay user-overridable.
# At startup, a non-NONE profile takes precedence over the static assault toggles/params above.
quarkus.goblin.assault.profile=NONE

# Latency settings
quarkus.goblin.assault.latency.min-milliseconds=100
quarkus.goblin.assault.latency.max-milliseconds=5000

# Exception settings
quarkus.goblin.assault.exception.type=java.lang.RuntimeException
quarkus.goblin.assault.exception.message=Goblin chaos: simulated exception

# HTTP status settings
quarkus.goblin.assault.http-status.code=503
quarkus.goblin.assault.http-status.message=Service Unavailable (Goblin chaos)

# Response body settings
quarkus.goblin.assault.body.mode=truncate
quarkus.goblin.assault.body.percentage=50

# Response header rules (actions: set, remove)
# SET forces the header with the value (replacing an existing one or adding it when absent)
# REMOVE deletes the header when present (value ignored)
quarkus.goblin.assault.headers.X-Chaos.action=set
quarkus.goblin.assault.headers.X-Chaos.value=goblin
quarkus.goblin.assault.headers.Cache-Control.action=set
quarkus.goblin.assault.headers.Cache-Control.value=no-store

# Target level (0-100% of requests affected)
quarkus.goblin.target.level=100

# Optional: include/exclude packages
# quarkus.goblin.target.include-packages=com.example.api
# quarkus.goblin.target.exclude-packages=com.example.health

# Optional: exclude annotated methods
# quarkus.goblin.target.exclude-annotations=org.eclipse.microprofile.faulttolerance.Timeout

Metrics (optional)

Add the quarkus-goblin-metrics module to expose the assault activity through Micrometer, so it shows up in your Prometheus / Grafana dashboards:

<dependency>
    <groupId>io.quarkiverse.goblin</groupId>
    <artifactId>quarkus-goblin-metrics</artifactId>
    <version>${goblin.version}</version>
</dependency>

The module depends on the Micrometer API only: add the registry you use, e.g. io.quarkus:quarkus-micrometer-registry-prometheus, whose /q/metrics endpoint then exposes:

• goblin_assaults_total -- counter of every fired assault, tagged by type and source (server, service, rest-client, webclient, database, messaging)
• goblin_latency_injected_seconds -- timer of the delays actually injected, tagged by source, published as a histogram (_sum/_count/_max plus _bucket series; see the guide for bucket tuning)
• goblin_active -- gauge, 1 while the engine is active, 0 otherwise

Tracing (optional)

Add the quarkus-goblin-opentelemetry module to surface every assault as an OpenTelemetry span in your existing tracing backend:

<dependency>
    <groupId>io.quarkiverse.goblin</groupId>
    <artifactId>quarkus-goblin-opentelemetry</artifactId>
    <version>${goblin.version}</version>
</dependency>

Each assault produces one span named goblin.assault:

• kind INTERNAL (the OpenTelemetry default) for every assault: the span annotates an in-process moment and makes no network call of its own, so CLIENT/SERVER labels would fabricate phantom dependency edges or duplicate the request topology
• attributes goblin.assault.type, .source (server, service, rest-client, webclient, database, messaging), .target.method, the injected value (.latency_ms, .status_code, .exception) and the .config snapshot
• linked to the parent request span; for latency, the injected delay is back-dated so it is attributed to the span; exception assaults mark the span ERROR (see the guide)

Dev UI

The Chaos Dashboard provides:

• Master toggle -- Activate/deactivate all chaos, with an optional auto-off (5 to 60 minutes) enforced by the engine even when the Dev UI is closed
• Profile selector -- Switch a whole server-side assault setup (NONE, SLOW_FAILURE, INTERMITTENT, TIMEOUT) in one click: a profile turns every server-side assault off, then enables its own; client-side toggles and layers are untouched and individual toggles stay overridable
• Assault type toggles -- Independent on/off for Latency, Exception, HTTP Status, Dependency Degradation, Response Body, and Response Header
• Client-side assault toggles -- client latency and client exception for outgoing REST Client and Vert.x WebClient calls
• Config sections -- Edit parameters per type (a summary of the current values when the type is off)
• Target level -- Adjust percentage of affected requests
• History -- Live chaos-testing console: 2-second auto-refresh, newest-first ordering, filters (assault type, method, time period), a summary band with totals and average injected latency, and expandable Active Config cells
• Markdown report -- "Export Markdown" button in the History panel generates a factual report of the current configuration and assault history (copy or download it), handy for pasting into an LLM assistant (e.g. Claude) for a resilience review

All changes apply instantly with WARN logs in the console and are persisted to .goblin-state.json across restarts (except the master toggle: a deactivation survives the live reloads of the running dev process, and a new process starts from quarkus.goblin.enabled). Invalid values are never applied: Goblin logs a clear message and applies a safe fallback -- inverted latency ranges are swapped, out-of-range HTTP status codes (100-599) fall back to 503, unknown exception classes fall back to RuntimeException, the response body percentage is clamped to its mode's valid range (0-100 for TRUNCATE, 101-1000 for INFLATE), response header rules with an unknown action or an invalid name are rejected, and the target level is clamped to 0-100. In the dashboard, a warning toast explains the applied correction.

Safety

• Dev by default, tests on opt-in -- Chaos activates in quarkus:dev. In @QuarkusTest it stays off unless quarkus.goblin.test.enabled=true, so adding the extension never slows down nor breaks an existing test suite; a single test can also switch it on with AssaultEngine.setActive(true).
• Zero code modification -- No annotations needed for the server-side and REST Client assaults; Vert.x WebClient instances are armed with one GoblinWebClient.enable(...) call.
• Explicit logging -- WARN log emitted when chaos is active.

Repository layout

Each module carries its own README for contributors:

• runtime -- the assault abstraction and how to add a new assault (the extension SPI)
• metrics -- optional Micrometer / Prometheus metrics for assault activity
• opentelemetry -- optional OpenTelemetry traces for each assault
• runtime-dev -- the Dev UI JSON-RPC backend (dev mode only)
• deployment -- build steps, bean registration, and Dev UI wiring
• integration-tests -- the @QuarkusTest suite and how to extend it
• docs -- the Antora documentation sources

Documentation

The full AsciiDoc guide lives in docs/modules/ROOT/pages/ (index.adoc), covering assault types, targeting, configuration validation, the Dev UI (with screenshots), an end-to-end example, a JSON-RPC reference, and a FAQ.

Screenshots of the Dev UI are stored in docs/modules/ROOT/assets/images/.

Coverage

Test coverage is aggregated by JaCoCo (report-aggregate on integration-tests). On every push to main, the coverage CI job recomputes the instruction coverage and publishes coverage.json to the badges branch, which feeds the badge above through a shields.io endpoint.

To inspect the full HTML report locally:

mvn clean install -Dno-format
open integration-tests/target/site/jacoco-aggregate/index.html

The badge color follows the coverage: brightgreen >= 90%, green >= 80%, yellowgreen >= 70%, yellow >= 60%, red < 60%.

Requirements

• Java 25+
• Quarkus 3.38+

License

Apache License 2.0