Skip to content

ci: make the credentialed Ollama showcase fail closed #375

Description

@gnanirahulnutakki

Context

The workflow named E2E Showcase (real Ollama) does not currently install the Ollama Python client. The test module treats an import failure as cloud unavailability and skips every model-gated test. The job is also informational and excluded from the blocking aggregate, so it cannot currently act as a release-promotion gate.

Fresh audit base: dev at 3a5015b00a78604f8db4fc8f65433e0e883668a2. No open issue or PR covers this dependency and fail-closed gap.

Exact evidence

Acceptance criteria

  • Add a bounded Ollama client optional dependency or an equally explicit pinned workflow install, with the dependency rationale recorded.
  • Install that dependency in the credentialed showcase job.
  • Preflight client import, configured model, and credential presence without printing or persisting secret values.
  • Fail this job if any Ollama-gated showcase test is skipped after preflight.
  • Add a credential-free regression for the fail-closed skip-detection logic.
  • Explicitly decide and document whether this remains informational or becomes a required pre-promotion gate.
  • Keep ordinary pull-request CI free of private credentials and paid-provider calls.
  • Bound provider calls and note cost implications for manual or release runs.
  • Ruff, format validation, secret scanning, and relevant targeted tests pass.

Security and cost boundary

Credentials must remain environment-only and redacted. The job must not print token values or provider payloads containing secrets. Live calls should remain bounded because they consume external inference quota.

Related: #367 and #371.

Metadata

Metadata

Assignees

No one assigned

    Labels

    release-hygieneRelease process & versioning hygiene

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions