User story

As a Code PushUp user, I want to track accessibility compliance using axe-core so that I can ensure my web applications meet WCAG 2.1 AA and EAA requirements.

Acceptance criteria

Plugin configuration

• The plugin function axePlugin() accepts a required config parameter specifying the URL(s) to analyze in three formats:
  • string ('https://example.com')
  • array (['https://example.com', 'https://another-example.com'])
  • weighted object ({ 'https://example.com': 2, 'https://another-example.com': 1 })
• The plugin function accepts an optional second parameter options object that contains:
  • A preset field (string) that determines which rules to run (default: 'wcag21aa')
  • A scoreTargets field for defining custom pass/fail thresholds (optional)
• All configuration inputs are validated using Zod schemas:
  • The config schema validates URL format(s)
  • The options schema validates preset values and scoreTargets structure

const axePluginConfigSchema = z.union([
  z.url(),
  z.array(z.url()),
  z.record(z.url(), z.number()),
]);

const axePluginOptionsSchema = z.object({
  preset: z
    .enum(['wcag21aa', 'wcag22aa', 'best-practice', 'all'])
    .default('wcag21aa'),
  scoreTargets: pluginScoreTargetsSchema.optional(),
});

axe-core integration

• The plugin uses two packages from the axe ecosystem:
  • axe-core
  • @axe-core/playwright
• Both packages are installed as direct dependencies
• In the plugin factory, the plugin calls axe.getRules() from axe-core to load all available rules and filter them based on the selected preset
• In the runner function, the plugin launches a headless Chromium browser using playwright-core (a peer dependency)
• For each configured URL, the runner navigates to the page and executes new AxeBuilder({ page }).withTags(...).analyze() from @axe-core/playwright to run the accessibility scan
• The runner collects all results from all URLs before closing the browser
• The Logger is used throughout execution to log info when starting, debug for each URL tested, errors if browser or axe fails, and warnings if axe returns incomplete results

Audit generation

• The plugin loads all axe rules using axe.getRules() API and filters them based on the selected preset:
  • For WCAG presets: rules are filtered by WCAG level tags
  • For best-practice: rules are filtered by best-practice tag
  • For all: all rules are included
• Each axe rule is transformed to a Code PushUp audit where:
  • rule.ruleId maps to audit.slug
  • rule.help maps to audit.title
  • rule.description maps to audit.description
  • rule.helpUrl maps to audit.docsUrl

Result transformation

• The plugin transforms axe execution results (containing violations, passes, incomplete, and inapplicable arrays) into AuditOutputs where each rule produces an AuditOutput with:
  • The slug set to the rule ID
  • A binary score of 1 when no violations exist, or 0 when any violations exist
  • The value set to the count of violation nodes
  • A displayValue formatted as human-readable text (e.g., "3 violations")
• Violations are transformed into issues where:
  • Each violation node becomes an Issue object with the message extracted from the node failureSummary or rule.help
  • Issue messages include the page URL and CSS selector from the node target or snippet from the node html for context
  • Axe impact levels map to Code PushUp severity:
    • critical/serious ➡️ error
    • moderate ➡️ warning
    • minor ➡️ info
  • Source location is not used

Group generation

• When using WCAG presets (wcag21aa, wcag22aa), the plugin generates groups including all audits tagged with the corresponding WCAG level, and all refs have weight 1:
  • wcag21aa ➡️ tags: [wcag2a, 'wcag21a', 'wcag2aa, wcag21aa] (WCAG 2.1 AA compliance)
  • wcag22aa ➡️ tags: [wcag2a, 'wcag21a', 'wcag2aa, wcag21aa, wcag22aa] (WCAG 2.2 AA compliance)
• When using the best-practice preset, the plugin extracts unique category tags (cat.*) from rules and creates a group for each category:
  • cat.aria ➡️ ARIA
  • cat.color ➡️ Color & Contrast
  • cat.forms ➡️ Forms
  • cat.keyboard ➡️ Keyboard
  • cat.language ➡️ Language
  • cat.name-role-value ➡️ Names & Labels
  • cat.parsing ➡️ Parsing
  • cat.semantics ➡️ Semantics
  • cat.sensory-and-visual-cues ➡️ Visual Cues
  • cat.structure ➡️ Structure
  • cat.tables ➡️ Tables
  • cat.text-alternatives ➡️ Text Alternatives
  • cat.time-and-media ➡️ Media
• When using the all preset, the plugin combines WCAG groups with key category groups
• Before returning the plugin configuration, empty groups (those with refs.length === 0) are filtered out to prevent validation errors

Plugin metadata

• The plugin includes metadata:
  • slug axe
  • title Axe Accessibility
  • icon accessibility
  • description explaining EAA/WCAG compliance
  • docsUrl linking to the package documentation on npmjs.com
  • packageName and version from package.json

Unit tests

• Configuration schema validation tests verify that valid URL formats are accepted, invalid URLs are rejected, preset values are validated, and scoreTargets are validated correctly
• Preset logic tests verify that each preset returns the correct rule count, and rule filtering works correctly
• Transformation function tests verify axe violations map to Issues correctly, impact maps to severity correctly, score calculation produces binary 0 or 1, and displayValue is formatted properly
• Group generation tests verify that correct groups are generated for each preset, empty groups are filtered out, and group refs include correct audit slugs

E2E tests

• A test fixture exists containing an HTML file with known accessibility violations (missing alt text, poor color contrast, invalid ARIA attributes, missing form labels)
• A code-pushup.config.ts file uses the axe plugin with default options
• When the collect command runs, it generates report.json that validates against the schema and contains expected violations with specific audits (image-alt, color-contrast, aria-allowed-attr) showing score: 0
• The generated report contains the expected groups (wcag21-level-a and wcag21-level-aa) with correct audit refs

Documentation

• The README.md contains installation instructions, a quick start example with zero configuration, an explanation of each preset option, scoreTargets usage examples, an API reference section, and links to axe-core rules documentation, WCAG guidelines, and EAA compliance information
• Where necessary, exported functions include TSDoc comments
• The package is configured for publishing to npm as @code-pushup/axe-plugin with correct package.json metadata, peer dependencies, and build configuration

Implementation details

Follow the Lighthouse plugin pattern:

• Direct library integration
• RunnerFunction with programmatic axe-core execution
• Output transformation pipeline
• Auto-generate groups from rule metadata

Basic file structure

packages/plugin-axe/
├── src/
│ ├── lib/
│ │ ├── axe-plugin.ts      Main plugin function
│ │ ├── config.ts          Zod schemas
│ │ ├── constants.ts       Plugin metadata
│ │ ├── processing.ts
│ │ ├── runner/
│ │ │ ├── runner.ts        Runner function
│ │ │ ├── run-axe.ts       Axe-core execution
│ │ │ └── transform.ts     Result transformation
│ │ └── meta/
│ │   └── transform.ts     Helpers loading and transforming Axe rules
│ │ 
│ └── index.ts
├── README.md
├── package.json
└── vitest.unit.config.ts

e2e/plugin-axe-e2e/
├── tests/
│ └── collect.e2e.test.ts
├── mocks/
│ └── fixtures/
│     └── default-setup/
│        ├── code-pushup.config.ts
│        └── test-app.html
└── vitest.e2e.config.ts

Related resources

• Axe-core API documentation
• WCAG 2.1 guidelines
• EAA information