Skip to content

[BUG] CI reports success when the Vitest suite fails #3349

Description

@nyxsky404

Bug Description

The automated test workflow masks a failing Vitest run. Its test step runs pnpm test || echo "Baseline mock unit test execution passes completed successfully.", so the successful echo replaces Vitest’s non-zero exit code and the job passes.

A pull request with failing unit tests can therefore receive a green CI result even though the repository advertises this workflow as its automated test gate.

Steps to Reproduce

  1. Add a deliberately failing Vitest test on a branch targeting main.
  2. Push the branch or open a pull request.
  3. Inspect the “Run Automated Unit Testing Framework Suites” step.
  4. Observe that pnpm test fails, the fallback message is printed, and the workflow step succeeds.

Affected Area

Other

Screenshots

Not applicable; the behavior is defined in .github/workflows/automated-tests.yml.

Browser & OS

Not applicable; this runs on GitHub Actions.

Environment

Both

Additional Context

Remove the || echo fallback so pnpm test preserves its exit status. If a diagnostic message is useful, emit it in a separate if: failure() step. The following audit step is also only an echo, so it cannot validate or restore test-gate behavior.

Metadata

Metadata

Assignees

Labels

gssoc:assignedGSSoC: Issue assigned to a contributor

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions