Skip to content

refactor(api): split inference and control services#21

Merged
connorcarpenter15 merged 5 commits into
mainfrom
build/buf-release
Jul 15, 2026
Merged

refactor(api): split inference and control services#21
connorcarpenter15 merged 5 commits into
mainfrom
build/buf-release

Conversation

@connorcarpenter15

@connorcarpenter15 connorcarpenter15 commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

  • define openengine.v1.Inference for generation, embedding, classification, and scoring
  • define openengine.v1.Control for discovery, lifecycle, LoRA management, KV coordination, and runtime events
  • rename engine.proto to server.proto
  • rename EngineInfo and GetEngineInfo to ServerInfo and GetServerInfo
  • make the hosted engine role explicit as ServerInfo.engine_role while retaining the semantically accurate EngineRole enum
  • advertise decode-context parallelism through ParallelismInfo.decode_context_parallel_size
  • update the API reference and project documentation for the new names and service boundary

Why

The split keeps inference traffic on a small data-plane surface while coordination operations live on a separate control-plane service. Implementations can register both services on one listener or isolate Control by listener and access policy.

Inference is broader and more accurate than Generate because it also owns the non-generative task RPCs. The package-qualified names openengine.v1.Inference and openengine.v1.Control remain concise without losing context.

server.proto now matches its primary ServerInfo message. EngineRole remains engine-specific because aggregated, prefill, and decode describe execution roles rather than administrative server roles.

Compatibility

This is an intentional pre-adoption API refinement:

  • RPC paths become /openengine.v1.Inference/* and /openengine.v1.Control/*
  • GetEngineInfo, GetEngineInfoRequest, and EngineInfo become GetServerInfo, GetServerInfoRequest, and ServerInfo
  • ServerInfo.role becomes ServerInfo.engine_role
  • declarations in engine.proto move to server.proto
  • decode_context_parallel_size is an additive optional field; when present it must be at least one, and one means DCP is disabled

Validation

  • Buf lint
  • direct protoc descriptor compilation
  • Markdown lint
  • documentation link checks
  • DCO sign-off

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / schema (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed⏩ skippedJul 15, 2026, 4:11 PM

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@connorcarpenter15 connorcarpenter15 changed the title build(buf): publish OpenEngine schema refactor(api): separate inference and control services Jul 15, 2026
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@connorcarpenter15 connorcarpenter15 changed the title refactor(api): separate inference and control services refactor(api): split inference and control services Jul 15, 2026
@connorcarpenter15 connorcarpenter15 marked this pull request as ready for review July 15, 2026 16:29
@connorcarpenter15 connorcarpenter15 merged commit 8c93c23 into main Jul 15, 2026
5 checks passed
@connorcarpenter15 connorcarpenter15 deleted the build/buf-release branch July 15, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant