Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
b83a4fd
Add Azure OIDC authentication to GitHub Actions workflow
Jun 4, 2026
30aa72e
Refactor AzureOpenAIApi to support auto-detection of DefaultAzureCred…
Jun 4, 2026
64046d2
Refactor AzureOpenAIApi to utilize DefaultAzureCredential for authent…
Jun 4, 2026
18d2a4f
Disable ghcp tests in workflow until OIDC keyless auth is fully imple…
Jun 4, 2026
29f4751
Temporarily disable ghcp tests in workflow until OIDC keyless auth is…
Jun 4, 2026
d3eca92
Add step to reinstall Azure CLI in GitHub Actions workflow
Jun 4, 2026
85c3d42
Refactor test fixture to use DefaultAzureCredential for token provide…
Jun 4, 2026
ed08e08
Update GitHub Actions workflow to re-enable ghcp tests and enhance BY…
Jun 4, 2026
1f53142
Enhance BYOK authentication in CopilotBot to support Azure AD keyless…
Jun 4, 2026
0cc0e55
Refactor BYOK authentication in CopilotBot to utilize DefaultAzureCre…
Jun 4, 2026
4c5060c
Fallback to native GitHub auth when provider type is absent and updat…
Jun 4, 2026
7ce8837
Refactor AzureOpenAIApi
Jun 4, 2026
8c562bd
Update tests to mock Azure OpenAI API key and handle missing token pr…
Jun 4, 2026
6ec3738
fix: pass token_provider to MicroBot in test_llm_finds_symbols_using_…
Jun 4, 2026
5e60395
add Azure AD dependencies for keyless authentication in tests
Jun 4, 2026
00b19ab
fix: install azure-identity for all CI test types
Jun 4, 2026
34e8989
refactor: remove DefaultAzureCredential and token_provider from bot t…
Jun 4, 2026
bd415ff
remove trailing commas in bot tests
Jun 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 25 additions & 9 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ on:
jobs:
test:
runs-on: ubuntu-latest
permissions:
id-token: write # required for Entra OIDC authentication
contents: read
strategy:
matrix:
# Installing ollama model in GitHub Actions runner requires significant disk space.
Expand Down Expand Up @@ -45,6 +48,10 @@ jobs:
docker-images: false
swap-storage: false

- name: Reinstall Azure CLI (removed by disk cleanup)
if: matrix.test-type == 'slow-browser' || matrix.test-type == 'slow-other' || matrix.test-type == 'ollama_local'
run: sudo apt-get install -y azure-cli

- name: Checkout code
uses: actions/checkout@v4

Expand Down Expand Up @@ -94,6 +101,10 @@ jobs:
run: |
pip install "github-copilot-sdk==0.3.0"

- name: Install Azure AD / keyless auth dependencies
run: |
pip install "azure-identity>=1.15.0"

- name: Build Docker images for integration tests
if: matrix.test-type != 'unit'
run: |
Expand Down Expand Up @@ -129,13 +140,21 @@ jobs:
env:
response: ${{ steps.model.outputs.response }}

- name: Azure Login (Entra OIDC)
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

- name: Run ${{ matrix.test-type }} tests
env:
# Azure OpenAI API Configuration
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
# Azure OpenAI API Configuration (key-free via Entra OIDC)
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME }}
AZURE_OPENAI_ENDPOINT: ${{ vars.AZURE_OPENAI_ENDPOINT }}
AZURE_OPENAI_API_VERSION: ${{ vars.AZURE_OPENAI_API_VERSION }}
# Enable DefaultAzureCredential auto-detection for all non-unit test types
AZURE_AUTH_METHOD: ${{ matrix.test-type != 'unit' && 'azure_ad' || '' }}
BROWSER_USE_LLM_MODEL: "gpt-5"
BROWSER_USE_LLM_TEMPERATURE: 1
#Anthrpic API Configuration
Expand All @@ -145,15 +164,12 @@ jobs:
#Local Model Configuration
LOCAL_MODEL_NAME: "qwen2.5-coder:latest"
LOCAL_MODEL_PORT: 11434
# CopilotBot Configuration (only populated for ghcp tests)
COPILOT_BYOK_API_KEY: ${{ matrix.test-type == 'ghcp' && secrets.AZURE_OPENAI_API_KEY || '' }}
COPILOT_BYOK_BASE_URL: ${{ matrix.test-type == 'ghcp' && vars.OPENAI_ENDPOINT || '' }}
# CopilotBot Configuration (keyless OIDC — only populated for ghcp tests)
COPILOT_BYOK_BASE_URL: ${{ matrix.test-type == 'ghcp' && vars.AZURE_OPENAI_ENDPOINT || '' }}
COPILOT_BYOK_PROVIDER_TYPE: ${{ matrix.test-type == 'ghcp' && 'azure' || '' }}
COPILOT_BYOK_MODEL: ${{ matrix.test-type == 'ghcp' && vars.AZURE_OPENAI_DEPLOYMENT_NAME || '' }}
COPILOT_BYOK_API_VERSION: ${{ matrix.test-type == 'ghcp' && vars.AZURE_OPENAI_API_VERSION || '' }}
COPILOT_BYOK_WIRE_API: ${{ matrix.test-type == 'ghcp' && 'openai' || '' }}
# Passing BYOK as arguement. Test: TestCopilotBotBYOKOpenAIIntegration::test_byok_openai_simple_task
OPENAI_ENDPOINT: ${{ matrix.test-type == 'ghcp' && vars.OPENAI_ENDPOINT || '' }}
COPILOT_BYOK_AZURE_API_VERSION: ${{ matrix.test-type == 'ghcp' && vars.AZURE_OPENAI_API_VERSION || '' }}
COPILOT_BYOK_WIRE_API: ${{ matrix.test-type == 'ghcp' && 'completions' || '' }}
run: |
python -m pytest ${{ matrix.pytest-args }} \
-n auto \
Expand Down
19 changes: 17 additions & 2 deletions src/microbots/bot/CopilotBot.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
_BYOK_ENV_BASE_URL = "COPILOT_BYOK_BASE_URL"
_BYOK_ENV_API_KEY = "COPILOT_BYOK_API_KEY"
_BYOK_ENV_BEARER_TOKEN = "COPILOT_BYOK_BEARER_TOKEN"
_BYOK_ENV_WIRE_API = "COPILOT_BYOK_WIRE_API" # openai / responses
_BYOK_ENV_WIRE_API = "COPILOT_BYOK_WIRE_API" # completions / responses
_BYOK_ENV_AZURE_API_VERSION = "COPILOT_BYOK_AZURE_API_VERSION"
_BYOK_ENV_MODEL = "COPILOT_BYOK_MODEL"

Expand Down Expand Up @@ -171,6 +171,21 @@ def resolve_auth_config(
logger.info("🔑 BYOK auth resolved via environment variables (type=%s)", provider["type"])
return env_model, None, provider

# ── 2.5. BYOK base_url set but no key/bearer → auto-detect via DefaultAzureCredential ──
# Only fires when COPILOT_BYOK_PROVIDER_TYPE is also set (explicit BYOK intent).
# Without it, a bare COPILOT_BYOK_BASE_URL would hijack tests that expect native Copilot auth.
env_provider_type = os.environ.get(_BYOK_ENV_PROVIDER_TYPE)
if env_base_url and env_provider_type and not (env_api_key or env_bearer_token) and not token_provider:
try:
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
_credential = DefaultAzureCredential()
token_provider = get_bearer_token_provider(
_credential, "https://cognitiveservices.azure.com/.default"
)
logger.debug("DefaultAzureCredential configured for BYOK — falling through to step 3")
except Exception as e:
logger.debug("DefaultAzureCredential not available for BYOK: %s", e)

# ── 3. Token provider (e.g. Azure AD) ────────────────────────────
if token_provider:
if not callable(token_provider):
Expand All @@ -196,7 +211,7 @@ def resolve_auth_config(
azure_api_version=azure_api_version or os.environ.get(_BYOK_ENV_AZURE_API_VERSION),
)
logger.info("🔑 BYOK auth resolved via token_provider (type=%s)", provider["type"])
return model, None, provider
return os.environ.get(_BYOK_ENV_MODEL, model), None, provider

# ── 4. Native GitHub Copilot auth ────────────────────────────────
resolved_github_token = (
Expand Down
84 changes: 65 additions & 19 deletions test/bot/test_copilot_bot.py
Original file line number Diff line number Diff line change
Expand Up @@ -722,6 +722,44 @@ def test_anthropic_provider_type(self):
)
assert provider["type"] == "anthropic"

def test_env_base_url_only_credential_fails_falls_through_to_github(self, monkeypatch):
"""Step 2.5: COPILOT_BYOK_BASE_URL + PROVIDER_TYPE set but DefaultAzureCredential raises
→ falls through to native GitHub Copilot auth."""
from microbots.bot.CopilotBot import resolve_auth_config

monkeypatch.setenv("COPILOT_BYOK_BASE_URL", "https://env-endpoint.com/v1")
monkeypatch.setenv("COPILOT_BYOK_PROVIDER_TYPE", "azure")
# No COPILOT_BYOK_API_KEY, no COPILOT_BYOK_BEARER_TOKEN

with patch("azure.identity.DefaultAzureCredential", side_effect=Exception("no credential")), \
patch("azure.identity.get_bearer_token_provider") as mock_gbt:
model, gh_token, provider = resolve_auth_config(
model="gpt-4.1",
github_token="ghp_fallback",
)

mock_gbt.assert_not_called()
assert provider is None
assert gh_token == "ghp_fallback"

def test_env_base_url_without_provider_type_skips_step25(self, monkeypatch):
"""Step 2.5 does NOT fire when COPILOT_BYOK_PROVIDER_TYPE is absent —
falls through to native GitHub Copilot auth without touching DefaultAzureCredential."""
from microbots.bot.CopilotBot import resolve_auth_config

monkeypatch.setenv("COPILOT_BYOK_BASE_URL", "https://env-endpoint.com/v1")
# No COPILOT_BYOK_PROVIDER_TYPE — step 2.5 should be skipped

with patch("azure.identity.DefaultAzureCredential") as mock_cred:
model, gh_token, provider = resolve_auth_config(
model="gpt-4.1",
github_token="ghp_native",
)

mock_cred.assert_not_called()
assert provider is None
assert gh_token == "ghp_native"


@pytest.mark.unit
class TestCopilotBotBYOKInit:
Expand Down Expand Up @@ -1489,7 +1527,7 @@ def test_mount_additional_copy_succeeds(self, copilot_bot):
# ---------------------------------------------------------------------------


@pytest.mark.integration
@pytest.mark.unit
class TestCopilotBotSDKLayoutIntegration:
"""Integration-level tests that drive a full ``run()`` against each
supported SDK module layout."""
Expand Down Expand Up @@ -1646,7 +1684,14 @@ def test_simple_task(self, test_repo, issue_1):
# ---------------------------------------------------------------------------

def _byok_openai_available():
"""Check if OpenAI BYOK credentials are configured via env vars."""
"""Check if OpenAI BYOK credentials are configured.

Accepts either:
- Explicit key: AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT
- Keyless OIDC: COPILOT_BYOK_BASE_URL set (DefaultAzureCredential handles auth)
"""
if os.environ.get("COPILOT_BYOK_BASE_URL"):
return True
return bool(
os.environ.get("AZURE_OPENAI_API_KEY")
and os.environ.get("AZURE_OPENAI_ENDPOINT")
Expand All @@ -1655,7 +1700,7 @@ def _byok_openai_available():

_skip_no_byok_openai = pytest.mark.skipif(
not _byok_openai_available(),
reason="OpenAI BYOK not configured (set env variables AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT)",
reason="OpenAI BYOK not configured (set COPILOT_BYOK_BASE_URL or AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT)",
)

def _azure_ad_auth_available():
Expand Down Expand Up @@ -1684,31 +1729,35 @@ class TestCopilotBotBYOKOpenAIIntegration:
"""End-to-end integration tests for CopilotBot with OpenAI BYOK."""

def test_byok_openai_simple_task(self, test_repo, issue_1):
"""CopilotBot can fix a simple syntax error using OpenAI BYOK credentials."""
"""CopilotBot can fix a simple syntax error using Azure AD keyless BYOK credentials."""
_restore_real_copilot_modules()
from microbots.bot.CopilotBot import CopilotBot
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

issue_text = issue_1[0] + "\nFix the error in the original file."
verify_function = issue_1[1]

api_key = os.environ["AZURE_OPENAI_API_KEY"]
base_url = os.environ["OPENAI_ENDPOINT"]
base_url = os.environ["AZURE_OPENAI_ENDPOINT"]
model = os.getenv(
"AZURE_OPENAI_DEPLOYMENT_NAME", "mini-swe-agent-gpt5"
)
credential = DefaultAzureCredential()
token_provider = get_bearer_token_provider(
credential, "https://cognitiveservices.azure.com/.default"
)

bot = CopilotBot(
model=model,
folder_to_mount=str(test_repo),
permission="READ_WRITE",
api_key=api_key,
base_url=base_url,
provider_type="openai",
provider_type="azure",
token_provider=token_provider,
)

try:
assert bot._provider_config is not None
assert bot._provider_config["type"] == "openai"
assert bot._provider_config["type"] == "azure"
assert bot.github_token is None

result = bot.run(
Expand All @@ -1729,32 +1778,29 @@ def test_byok_openai_simple_task_with_token_provider(self, test_repo, issue_1):
issue_text = issue_1[0]
verify_function = issue_1[1]

api_key = os.environ["AZURE_OPENAI_API_KEY"]
base_url = os.environ["AZURE_OPENAI_ENDPOINT"]
model = os.getenv(
"AZURE_OPENAI_DEPLOYMENT_NAME", "mini-swe-agent-gpt5"
)

from azure.identity import DefaultAzureCredential
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
credential = DefaultAzureCredential()
def get_token():
return credential.get_token(
"https://cognitiveservices.azure.com/.default"
).token
token_provider = get_bearer_token_provider(
credential, "https://cognitiveservices.azure.com/.default"
)

bot = CopilotBot(
model=model,
folder_to_mount=str(test_repo),
permission="READ_WRITE",
api_key=api_key,
base_url=base_url,
provider_type="openai",
token_provider=get_token,
provider_type="azure",
token_provider=token_provider,
)

try:
assert bot._provider_config is not None
assert bot._provider_config["type"] == "openai"
assert bot._provider_config["type"] == "azure"
assert bot.github_token is None

result = bot.run(
Expand Down
20 changes: 14 additions & 6 deletions test/bot/test_microbot.py
Original file line number Diff line number Diff line change
Expand Up @@ -502,7 +502,8 @@ def test_tool_usage_instructions_appended_to_system_prompt(self):
mock_env.execute.return_value = Mock(return_code=0, stdout="", stderr="")

# Mock the environment and LLM creation to avoid actual Docker/API calls
with patch('microbots.llm.azure_openai_api.AzureOpenAI'):
with patch('microbots.llm.azure_openai_api.api_key', 'test-key'), \
patch('microbots.llm.azure_openai_api.AzureOpenAI'):
# Create a MicroBot with the mock tool
bot = MicroBot(
model="azure-openai/test-model",
Expand Down Expand Up @@ -599,7 +600,8 @@ def test_run_json_output_with_content_key(self):
return_code=0, stdout=json.dumps(json_content), stderr=""
)

with patch('microbots.llm.azure_openai_api.AzureOpenAI'):
with patch('microbots.llm.azure_openai_api.api_key', 'test-key'), \
patch('microbots.llm.azure_openai_api.AzureOpenAI'):
bot = MicroBot(
model="azure-openai/test-model",
system_prompt="test prompt",
Expand Down Expand Up @@ -634,7 +636,8 @@ def test_run_json_output_without_content_key(self):
return_code=0, stdout=raw_stdout, stderr=""
)

with patch('microbots.llm.azure_openai_api.AzureOpenAI'):
with patch('microbots.llm.azure_openai_api.api_key', 'test-key'), \
patch('microbots.llm.azure_openai_api.AzureOpenAI'):
bot = MicroBot(
model="azure-openai/test-model",
system_prompt="test prompt",
Expand Down Expand Up @@ -668,7 +671,8 @@ def test_run_non_json_output_json_decode_error(self):
return_code=0, stdout=raw_stdout, stderr=""
)

with patch('microbots.llm.azure_openai_api.AzureOpenAI'):
with patch('microbots.llm.azure_openai_api.api_key', 'test-key'), \
patch('microbots.llm.azure_openai_api.AzureOpenAI'):
bot = MicroBot(
model="azure-openai/test-model",
system_prompt="test prompt",
Expand Down Expand Up @@ -702,7 +706,8 @@ def test_run_json_parse_blanket_exception(self, caplog):
return_code=0, stdout=raw_stdout, stderr=""
)

with patch('microbots.llm.azure_openai_api.AzureOpenAI'):
with patch('microbots.llm.azure_openai_api.api_key', 'test-key'), \
patch('microbots.llm.azure_openai_api.AzureOpenAI'):
bot = MicroBot(
model="azure-openai/test-model",
system_prompt="test prompt",
Expand Down Expand Up @@ -807,12 +812,15 @@ def test_azure_ad_env_raises_import_error_when_azure_identity_missing(self):
)

def test_no_azure_ad_env_leaves_token_provider_none(self):
"""When AZURE_AUTH_METHOD is not set, token_provider defaults to None."""
"""When AZURE_AUTH_METHOD is not set, token_provider is None if api key is present."""
mock_env = Mock()
mock_env.execute.return_value = Mock(return_code=0, stdout="", stderr="")

env_without_azure = {k: v for k, v in os.environ.items() if k != 'AZURE_AUTH_METHOD'}
# Ensure an api key is present so DefaultAzureCredential is not attempted
env_without_azure['AZURE_OPENAI_API_KEY'] = 'test-key'
with patch.dict('os.environ', env_without_azure, clear=True), \
patch('microbots.llm.azure_openai_api.api_key', 'test-key'), \
patch('microbots.llm.azure_openai_api.AzureOpenAI'):
bot = MicroBot(
model="azure-openai/test-model",
Expand Down
Loading