Add TestPyPI workflow for safe release testing

[itdove]

Overview

Create a separate GitHub Actions workflow for publishing to TestPyPI to enable safe testing of the release process before publishing to production PyPI.

Why TestPyPI?

Benefits:

• ✅ Test the entire release process without touching production
• ✅ Verify PyPI Trusted Publishing configuration works
• ✅ Catch packaging/metadata issues early
• ✅ Safe experimentation with workflow changes
• ✅ Standard best practice for Python projects

Implementation Checklist

1. TestPyPI Account Setup

• Create account at https://test.pypi.org/account/register/
• Verify email address

2. Configure Trusted Publishing on TestPyPI

• Go to https://test.pypi.org/manage/account/publishing/
• Click "Add a new pending publisher"
• Configure publisher settings:
  • PyPI Project Name: devaiflow
  • Owner: itdove
  • Repository name: devaiflow
  • Workflow name: publish-test.yml
  • Environment name: testpypi
• Save configuration

3. Create GitHub Environment

• Go to https://github.com/itdove/devaiflow/settings/environments
• Click "New environment"
• Name: testpypi
• (Optional) Add protection rules:
  • Required reviewers: yourself
  • Deployment branches: only release-* branches
• Save environment

4. Create Workflow File

• Create .github/workflows/publish-test.yml with the following content:

# GitHub Actions CI/CD - TestPyPI Publish Workflow
# Publishes package to TestPyPI for testing

name: Publish to TestPyPI

on:
  push:
    tags:
      - 'v*-test*'  # Trigger on test tags (e.g., v2.2.0-test, v2.2.0-test1)
  workflow_dispatch:  # Allow manual trigger

jobs:
  build-and-publish-test:
    name: Build and Publish to TestPyPI
    runs-on: ubuntu-latest
    environment: testpypi  # Use GitHub environment for Trusted Publishing
    permissions:
      id-token: write  # Required for Trusted Publishing
      contents: read

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install build dependencies
        run: |
          python -m pip install --upgrade pip
          pip install build twine

      - name: Build distribution packages
        run: |
          python -m build

      - name: Check distribution packages
        run: |
          twine check dist/*
          ls -la dist/

      - name: Publish to TestPyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          repository-url: https://test.pypi.org/legacy/
          skip-existing: true
          verbose: true
          print-hash: true

• Commit and push the workflow file

5. Test the Workflow

• Create test release branch:

  git checkout -b release-2.2-test main

• Update version to test version in pyproject.toml:

  version = "2.2.0-test1"

• Update version in devflow/__init__.py:

  __version__ = "2.2.0-test1"

• Commit changes:

  git add pyproject.toml devflow/__init__.py
  git commit -m "chore: bump version to 2.2.0-test1 for TestPyPI"

• Create and push test tag:

  git tag -a v2.2.0-test1 -m "Test release for TestPyPI"
  git push origin v2.2.0-test1

• Monitor GitHub Actions: https://github.com/itdove/devaiflow/actions

• Verify publication on TestPyPI: https://test.pypi.org/project/devaiflow/

• Test installation from TestPyPI:

  pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ devaiflow==2.2.0-test1
  daf --version

6. Documentation

• Update RELEASING.md with TestPyPI workflow instructions
• Add section on testing releases before production
• Document tag naming convention: v*-test* for TestPyPI, v* for production
• Update docs/developer/publishing-to-pypi.md with automated workflow reference

Usage Examples

Testing a release:

# 1. Create test release branch
git checkout -b release-2.2-test main

# 2. Update version to test version
# pyproject.toml: version = "2.2.0-test1"
# devflow/__init__.py: __version__ = "2.2.0-test1"

# 3. Tag and push
git add pyproject.toml devflow/__init__.py
git commit -m "chore: bump version to 2.2.0-test1 for TestPyPI"
git tag -a v2.2.0-test1 -m "Test release 2.2.0"
git push origin v2.2.0-test1

# 4. Verify TestPyPI publication
# Check: https://test.pypi.org/project/devaiflow/

# 5. Test installation
pip install --index-url https://test.pypi.org/simple/ \
  --extra-index-url https://pypi.org/simple/ \
  devaiflow==2.2.0-test1

# 6. If successful, proceed with production release using v2.2.0 tag

Manual workflow trigger:

• Go to https://github.com/itdove/devaiflow/actions/workflows/publish-test.yml
• Click "Run workflow"
• Select branch
• Click "Run workflow" button

Tag Naming Convention

• TestPyPI: v2.2.0-test, v2.2.0-test1, v2.2.0-test2
• Production PyPI: v2.2.0, v2.2.1, v2.3.0

The -test suffix ensures the workflow only triggers for test releases.

Expected Outcomes

After completing this issue:

• ✅ Safe testing environment for release process
• ✅ Ability to verify Trusted Publishing works
• ✅ Catch issues before production release
• ✅ Confidence in release automation
• ✅ Best practice Python packaging workflow

References

• TestPyPI Documentation
• PyPI Trusted Publishing Guide
• GitHub Environments
• ai-guardian#14 - Reference implementation

Acceptance Criteria

• Workflow file created and committed
• TestPyPI account created and configured
• Trusted Publishing configured on TestPyPI
• GitHub environment testpypi created
• Test release published successfully to TestPyPI
• Package installable from TestPyPI
• Documentation updated (RELEASING.md and publishing-to-pypi.md)

[itdove]

Pull request created: #368

[itdove]

✅ Session Complete

Project: devaiflow
Goal: #367: Add TestPyPI workflow for safe release testing
Time Spent:
0h 9m total

• dvernier: 0h 8m
• unknown: 0h 0m
  Branch: 367

Perfect! I've updated AGENTS.md to clarify when to run pytest. Here's what changed:

✅ AGENTS.md Testing Guidelines Updated

New "When to Run Tests" Section Added:

**When to Run Tests**:
- ✅ Run pytest when making code changes (Python files in devflow/, tests/, integration-tests/)
- ⏭️ Skip pytest for documentation-only changes (.md files, docs/, CHANGELOG.md, README.md)
- ⏭️ Skip pytest for workflow/config-only changes (.github/workflows/, .yml files without code changes)
- ⚠️ Always run pytest if you modified any .py file, even if it seems minor

Updated Key Sections:

1. Critical Testing Requirement: Added exception for documentation/workflow-only changes
2. Testing Requirements: Clarified "for code changes" instead of "for all changes"
3. Example Testing Workflow: Added documentation-only change example showing pytest is not needed

Benefits:

• ✅ Saves 4+ minutes when making documentation-only changes
• ✅ Reduces unnecessary CI/CD runs
• ✅ Clear guidance for when testing is required vs optional
• ✅ Maintains test quality for actual code changes

This makes perfect sense for changes like the TestPyPI workflow we just added - since it was only workflow files and documentation, we didn't need to run the full 3600+ test suite!

Generated by DevAIFlow