feat(codex): pull an authenticated remote catalog into local Codex state

[rrmlima]

Area

Catalog / models

What are you trying to accomplish?

I need to keep a Codex CLI or Desktop client synchronized with the complete model catalog generated by a remotely hosted OpenCodex instance.

OpenCodex already exposes the persisted catalog through the authenticated GET|HEAD /v1/catalog data-plane endpoint. The missing workflow is safely materializing that snapshot into the remote client's local CODEX_HOME, rebuilding models_cache.json, and reporting when a running Codex app-server or Desktop process still holds the previous catalog in memory.

This would make centrally hosted OpenCodex deployments usable without maintaining deployment-specific web roots, cron jobs, PowerShell downloaders, or ad hoc cache-rebuild scripts.

What prevents this today?

model_catalog_json accepts a local filesystem path, not a URL. ocx sync builds a catalog from locally configured providers, while ocx export --client generates client-specific configuration; neither consumes a complete catalog from another OpenCodex server.

Operators currently need to implement the entire client-side lifecycle themselves:

1. authenticate to /v1/catalog;
2. download into a same-directory temporary file;
3. validate JSON structure and model slug uniqueness;
4. preserve the last-known-good catalog on network, HTTP, or validation failure;
5. atomically replace the local catalog;
6. rebuild $CODEX_HOME/models_cache.json;
7. detect stale Codex app-server/Desktop processes;
8. schedule the workflow separately on Windows, macOS, and Linux.

This duplicates security-sensitive filesystem and credential handling that OpenCodex already implements elsewhere and makes reliable Desktop synchronization unnecessarily difficult for non-technical client users.

What should OpenCodex do?

Provide an explicit remote-catalog pull workflow that consumes the existing /v1/catalog contract and installs the result as a local Codex catalog.

Observable requirements:

• accept an HTTPS OpenCodex catalog URL;
• read authentication from an environment reference or existing secure credential mechanism, never a plaintext CLI argument;
• support conditional requests through ETag / If-None-Match, treating 304 as a no-op;
• reject URL-embedded credentials and avoid forwarding authorization across origins;
• enforce response-size and timeout bounds;
• validate the complete catalog before the first filesystem mutation, including a non-empty models array and unique, valid slugs;
• write through the existing catalog serialization and atomic replacement primitives;
• preserve the last-known-good catalog and cache on every download or validation failure;
• update models_cache.json in the same coordinated operation;
• preserve mtimes and avoid process handling when the pulled catalog is unchanged;
• report stale Codex processes after a real update;
• keep --restart-codex and Windows --restart-desktop-app explicit and opt-in, because either can interrupt active work;
• never mutate server configuration, provider credentials, or the source catalog.

The first implementation can stay intentionally narrow: one-shot pull plus cache update. A managed cross-platform scheduled integration can be discussed separately after the command contract is stable.

Example usage or interface

export OPENCODEX_CATALOG_AUTH_TOKEN='...'

ocx catalog pull \
  https://proxy.example.com/v1/catalog \
  --auth-env OPENCODEX_CATALOG_AUTH_TOKEN \
  --restart-codex

Machine-readable automation:

ocx catalog pull \
  https://proxy.example.com/v1/catalog \
  --auth-env OPENCODEX_CATALOG_AUTH_TOKEN \
  --json

Expected no-op result after an unchanged conditional request:

{
  "status": "unchanged",
  "catalogWritten": false,
  "cacheSynced": false,
  "codexRestarted": false
}

The exact command and environment-variable names are proposed for discussion; the important contract is authenticated remote-to-local materialization with coordinated catalog/cache writes and opt-in process restart.

Alternatives or workarounds

• Serve a deployment-specific /dl directory: works, but duplicates the existing authenticated /v1/catalog endpoint and makes nginx/web-root permissions part of catalog correctness.
• Cron plus shell/PowerShell scripts: currently workable, but each operator must reimplement validation, locking, atomic replacement, cache generation, error handling, and secret hygiene.
• Point model_catalog_json at an HTTPS URL: incompatible with Codex's local-path contract and startup behavior.
• Extend ocx export --client: misleading because that command exports client-specific configuration rather than the complete Codex catalog.
• Server-side watcher or publisher daemon: does not solve client-side models_cache.json or in-memory Desktop staleness and adds another service lifecycle.
• Automatically restart Codex Desktop: rejected as a default because it can terminate active conversations; restart must remain explicit.

Additional context

Related existing work solves adjacent, but different, layers:

• Serve the generated Codex model catalog over the management API for remote clients #709 / feat(api): serve Codex catalog at GET /api/catalog #772 introduced the management catalog endpoint.
• [Feature]: add least-privilege GET /v1/catalog for remote Codex clients #809 and the current /v1/catalog implementation address least-privilege remote catalog delivery.
• [Bug]: Routed models are loaded by Codex app-server but missing from Desktop model picker #241 / fix(codex): preserve routed models in desktop picker #1244 cover routed-model catalog preservation in Codex App.
• Windows: model picker stays stale after ocx sync --restart-codex until full desktop app restart #2292 / feat(cli): add an opt-in Windows desktop-app restart for a stale model picker #2382 establish that full Desktop restart is separate, disruptive, and opt-in.
• [Bug][2.36.0] /api/models drops entitled GPT-5.6 rows after the 5-minute entitlement cache expires #3023 / fix(codex): refresh expired management entitlements #3054 harden model entitlement freshness for exported/client-visible catalogs.

A production deployment currently uses a validated atomic catalog publisher plus a Windows downloader/cache rebuilder. That proves the workflow is useful, but the proposal deliberately avoids upstreaming deployment-specific paths, nginx configuration, cron syntax, or public unauthenticated /dl behavior.

Security boundaries for implementation review:

• no token in argv, output, errors, redirects, or persisted catalog metadata;
• HTTPS by default;
• no cross-origin credential forwarding;
• bounded body acquisition;
• validation before mutation;
• catalog/cache writes only under the existing per-CODEX_HOME serialization permit;
• no implicit process termination.

Checks

• I searched existing issues and documentation.
• This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
• I removed secrets and personal data.

[lidge-jun]

리뷰 · 우선순위 54 / 80

이 이슈는 원격 OpenCodex가 이미 만들어 둔 모델 목록(/v1/catalog)을, 로컬 Codex의 CODEX_HOME 카탈로그·models_cache.json에 안전하게 받아 심는 한 방 워크플로를 요청합니다. 지금 dev(HEAD af344a28e, 패키지 2.44.0)에는 서버 쪽 조각은 있습니다. #809로 GET|HEAD /v1/catalog가 데이터 플레인에 있고, 직렬화는 src/server/catalog-download.ts가 /api/catalog와 같은 바이트를내도록 공유합니다. 크기 상한은 원격 경로만 MAX_REMOTE_CATALOG_BYTES(256 MiB)입니다. 로컬 쪽에는 model_catalog_json 주입(src/codex/inject.ts), withCatalogWriteSerialization 원자 교체, invalidateCodexModelsCacheWithPermit 캐시 갱신, ocx sync / ocx sync-cache의 --restart-codex·Windows --restart-desktop-app(src/cli/dispatch.ts, src/codex/desktop-app-restart.ts, #2292 계열)이 이미 있습니다. #3699의 src/integrations/catalog-refresh.ts는 Pi/Aside처럼 로컬에서 고른 카탈로그를 통합에 다시 밀어 넣는 쪽이고, 원격 HTTPS 카탈로그를 받아 Codex 홈에 심는 명령은 아닙니다.

이슈가 말하는 빈틈도 맞습니다. model_catalog_json은 로컬 경로만 받고, ocx sync는 이 머신의 설정된 공급자로 카탈로그를 만들고, ocx export --client는 클라이언트용 설정 쪽입니다. 허브 연결(src/client/connect.ts 등)은 다른 계약의 원격 카탈로그 동기화가 있지만, “임의 HTTPS /v1/catalog URL + --auth-env로 한 번 pull” 운영자 UX는 없습니다. 제안 계약(토큰을 argv에 안 넣기, 검증 후 쓰기, 실패 시 last-known-good 보존, 변경 없을 때 mtime/프로세스 손대지 않기, 재시작은 명시 opt-in)은 기존 sync/restart 관례와 잘 맞습니다.

다만 조건부 요청(ETag / If-None-Match / 304) 요구는 현재 dev의 /v1/catalog 설계와 정면으로 어긋납니다. src/server/index.ts는 자격 증명마다 본문이 달라질 수 있어 공유 캐시·검증자 재검증이 위험하다고 보고, 의도적으로 cache-control: no-store, ETag 없음, 304 없음을 고수합니다. 관리면 GET /api/catalog만 ETag를 줍니다. src/client/connect.ts 주석도 “/v1/catalog는 validator를 안 낸다(Phase 1)”고 적고, 로컬 해시로 소유권을 확인합니다. 그래서 이슈의 304 계약 그대로 구현하려면 서버 정책을 바꾸거나, 클라이언트만 content-hash로 “unchanged”를 판정하는 쪽으로 범위를 줄여야 합니다. 첫 구현을 one-shot pull + cache로 좁히자는 제안은 타당하고, 스케줄러/크로스플랫폼 데몬은 뒤로 미루는 편이 맞습니다.

우선순위 54인 이유다. 중앙 호스팅 배포에는 실사용 가치가 있고 보안 경계도 잘 적혀 있지만, 지금 dev 최적화 축(macOS CI·릴리스 후보)과 겹치지 않는 새 CLI 표면 + 설계 합의가 필요합니다. 구현 전에 서버 304 정책을 유지할지부터 정해야 합니다. types.ts/config.ts 대분할에 통째로 무효화될 성격은 아니고, 중복 이슈로 바로 닫을 대상도 아닙니다(#3630은 로컬 공급자 자동 refresh라 인접하지만 다른 문제).

경로/심볼 - src/server/index.ts GET|HEAD /v1/catalog - 인증·origin·크기 507·no-store·ETag/304 없음 (이슈의 조건부 요청과 충돌)
경로/심볼 - src/server/catalog-download.ts serializePersistedCatalog / catalogEtag / MAX_REMOTE_CATALOG_BYTES - 바이트·etag 계산은 공유 모듈에 있으나 원격 라우트가 validator를 응답에 안 실음
경로/심볼 - src/cli/dispatch.ts ocx sync / sync-cache - 로컬 공급자 동기화 + 재시작 플래그 관례. ocx catalog pull 서브커맨드는 없음
경로/심볼 - src/codex/catalog-write-serialization.ts / src/codex/catalog/sync.ts - 원자 교체·models_cache.json 무효화 재사용 후보
경로/심볼 - src/codex/desktop-app-restart.ts / #2292 - Desktop 전체 재시작은 대화 중단이므로 기본 off·명시 플래그 (이슈와 동일)
경로/심볼 - src/integrations/catalog-refresh.ts (#3699) - Pi/Aside 등 로컬 선택 반영. 원격 pull 대체재 아님
경로 - #809 / #709 / #3023 / #2292 - 인접 계층. 이 이슈는 클라이언트 materialization UX

메인테이너의 판단이 필요한 지점

• /v1/catalog에 ETag·304를 열지(보안 주석과 거래), 아니면 클라이언트가 로컬 해시로 unchanged를 판정하고 서버는 매번 전체 본문을 줄지
• 명령 표면: 새 ocx catalog pull vs ocx sync --from-url 확장 vs hub-connect 전용 경로만 강화
• 인증: --auth-env만 허용할지, 기존 OPENCODEX_API_AUTH_TOKEN / 키체인과 어떻게 맞출지
• pull이 model_catalog_json이 가리키는 파일만 바꾸는지, Codex config 주입·통합 refresh(catalog-refresh)까지 묶을지
• 첫 PR 범위: one-shot + cache + stale 프로세스 보고까지만 두고 restart는 기존 플래그 재사용할지

너의 추천
바로 구현 PR을 받기보다 설계 한 줄을 먼저 고정하세요. 권장: 서버 /v1/catalog의 no-ETag/no-304는 유지하고, 클라이언트는 다운로드 후 로컬 SHA로 unchanged를 판정하며, 쓰기는 기존 withCatalogWriteSerialization + cache invalidate + 기존 restart 플래그 재사용. 합의되면 help wanted / needs-design로 두고 좁은 CLI PR을 받습니다. 지금은 이슈를 열린 채로 두세요. 중복 close 대상 아님.

이 댓글은 grok-bot이 작성했습니다

[rrmlima]

Thanks for the detailed review. I agree that the first implementation should preserve the current /v1/catalog security contract rather than couple this client workflow to a server-side validator change.

Proposed Phase 1 contract

I suggest fixing the design as follows:

1. Keep /v1/catalog exactly as it is today: authenticated, Cache-Control: no-store, no ETag, and no 304 contract.

2. Add a one-shot client command:

   ocx catalog pull <https-url> [--auth-env <NAME>] [--json] [--restart-codex]

3. Download and validate the complete response outside catalog serialization.

4. Determine unchanged by comparing a locally computed SHA-256/content snapshot with the active local catalog.

5. When unchanged, preserve catalog/cache mtimes and do not inspect or terminate processes.

6. When changed, commit the active catalog and models_cache.json through the existing per-CODEX_HOME serialization permit and atomic writer paths.

7. Preserve the last-known-good catalog/cache on transport, HTTP, size, validation, lock, or write failure.

8. Read credentials only through an environment reference or an existing secure credential resolver; never accept a literal token argument or forward authorization across origins.

9. Keep process handling explicit. --restart-codex may reuse the current app-server mechanism after a real write. Full Windows Desktop restart remains out of Phase 1 and can only be considered later as an explicit opt-in.

10. Exclude scheduling, daemon behavior, GUI, config persistence, /dl, server changes, and managed integration from the first PR.

This keeps the server policy unchanged, avoids expanding the authentication boundary, and limits the initial patch to client-side materialization plus cache convergence.

Decisions requested before implementation

My recommendation is:

• command surface: new ocx catalog pull namespace, because ocx sync builds from local providers and ocx export --client emits client-specific configuration;
• authentication: optional --auth-env <NAME>, falling back to OPENCODEX_API_AUTH_TOKEN only if that matches the project's desired credential convention;
• destination: the active local path already resolved by model_catalog_json / CODEX_HOME; no arbitrary output path in Phase 1;
• config ownership: do not inject or rewrite Codex config in Phase 1; only update an already resolved active catalog path and cache;
• process behavior: report stale app-servers by default and restart only with the existing explicit flag.

If this contract matches the maintainer direction, I can prepare the focused regression-first PR against the latest dev without changing /v1/catalog.