Skip to content

Add real Excel adversarial extension provider coverage#654

Merged
tmustier merged 2 commits into
mainfrom
test/adversarial-extension-provider
Jul 16, 2026
Merged

Add real Excel adversarial extension provider coverage#654
tmustier merged 2 commits into
mainfrom
test/adversarial-extension-provider

Conversation

@tmustier

Copy link
Copy Markdown
Owner

Summary

  • add a repeatable real-Excel adversarial smoke for pasted-code sandbox model providers
  • drive permission grants, host connection setup, dynamic discovery, model selection, streaming, endpoint upgrades, unload races and hostile endpoint registration through the tokened taskpane bridge
  • keep test credentials inside instrumented local HTTPS gateways and return only bounded pass/fail summaries

Defects the smoke found and fixes in this PR

  1. Sandbox bootstrap did not execute in Excel — an unescaped newline made the generated module syntactically invalid; generated JavaScript now has a parser regression test.
  2. Discovery was unbounded — responses are now limited to 2 MiB, 2,000 entries and 256 characters per model ID while retaining the last safe cache on rejection.
  3. Same-ID endpoint upgrades could retain stale session metadata — provider refreshes now reconcile runtime metadata, and reselection updates transport metadata even when provider/model identity is unchanged.
  4. In-flight unload could leave a removed provider selected — model reconciliation retries after agent_end once the action queue unwinds.
  5. Delayed discovery could resurrect cache after unregister — dynamic providers now own lifecycle abort controllers and cancel discovery before cache deletion.

Real Excel evidence

npm run smoke:extension-provider passed against a pasted inline extension running in sandbox-iframe mode with two instrumented HTTPS gateways:

  • default capability denial followed by explicit grants
  • host-owned discovery/inference credentials; sandbox secret read denied
  • exact EXTENSION_PROVIDER_A_OK, EXTENSION_PROVIDER_CACHE_OK, EXTENSION_PROVIDER_B_OK, and EXTENSION_PROVIDER_DELAYED_OK
  • offline cache fallback and oversized catalogue rejection
  • gateway A → B cache rebinding with no replacement credential sent to A
  • in-flight unload, idle uninstall and delayed-discovery race recovery
  • non-allowlisted endpoint host rejected

Evidence: docs/release-smoke-runs/2026-07-16-macos-extension-provider.md

Automated verification

  • npm run check
  • npm test — 937 passing (60 model, 745 context, 108 security, 16 manifest)
  • npm run build
  • npm run validate
  • npm audit --audit-level=high — 0 high/critical; existing inherited moderate findings only

@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
pi-for-excel Ignored Ignored Jul 16, 2026 5:50pm

@tmustier
tmustier merged commit 567fef1 into main Jul 16, 2026
10 checks passed
@tmustier
tmustier deleted the test/adversarial-extension-provider branch July 16, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant