Skip to content

[AX] Make real macOS keychain tests opt-in to avoid interactive password prompts #326

@ankitranjan7

Description

@ankitranjan7

Summary

Some tests currently exercise the real OS keychain by default. On macOS, that can trigger repeated login/keychain permission prompts during a normal pytest run, especially when running from a new worktree or a different virtualenv path.

This makes the default developer test experience interactive and noisy, even though the vast majority of the suite is otherwise hermetic.

Observed behavior

A normal local uv run pytest can prompt for macOS login / keychain access multiple times.

This is especially noticeable when:

  • running tests from a new git worktree
  • running through a different .venv path than one that was previously granted access
  • the tests create fresh keychain service/account identifiers for isolation

In practice, a developer may see no prompts on one checkout and repeated prompts on another, even though the code under test is the same.

Likely cause

tests/vault/test_crypto.py includes tests that intentionally hit the real OS keyring via KeyringCrypto().

Those tests are valuable as integration coverage, but they are not pure unit tests. On macOS, keychain approvals are tied to the requesting process/binary context, so a new worktree or virtualenv path can cause the OS to ask again.

Why this is a problem

For the default local test command, interactive OS prompts are a poor developer experience:

  • pytest should be non-interactive by default
  • CI should not depend on human approval dialogs
  • developers should not need to click through keychain prompts to validate unrelated changes

Proposed direction

Keep real keychain coverage, but make it opt-in.

Recommended shape:

  • default test runs should use mocks/stubs for OS keychain behavior
  • real keychain tests should be marked as integration tests or gated behind an explicit opt-in env var
  • documentation should explain how to run the real keychain tests locally when needed

Examples of reasonable approaches:

  • pytest -m keychain_integration
  • AUTHSOME_RUN_KEYCHAIN_TESTS=1 uv run pytest ...

Acceptance criteria

  • uv run pytest is non-interactive on macOS by default
  • real OS keychain coverage is still available through an explicit opt-in path
  • tests that exercise require_os_auth(...) or keychain access do not require user interaction in the default suite
  • developer docs mention how and when to run the opt-in keychain/integration tests

Notes

This is not about removing keychain coverage.

It is about separating:

  • deterministic unit/default-suite behavior
  • interactive or environment-sensitive integration behavior

That split is a common and healthy testing practice for code that touches OS security prompts, credential stores, browsers, or other external system UI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions