Skip to content

feat(racer): inject racer-object sidecars via racer-controller webhook - #939

Open
Jordan Olshevski (jveski) wants to merge 33 commits into
mainfrom
racer-object-sidecar-injection
Open

Jordan Olshevski (jveski) wants to merge 33 commits into
mainfrom
racer-object-sidecar-injection

Conversation

@jveski

@jveski Jordan Olshevski (jveski) commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Adds in opt-in racer-object sidecar injection for pods that need an S3 or Azure blob interface to Racer.
Also allows the Racer client UDSs to be used by non-root processes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Injection can broaden existing required node affinity and bypass workload placement restrictions.

1 open issue
What changed in this PR

Adds opt-in racer-object sidecar injection through racer-controller, with operator-managed webhook trust and non-root origin support.

Changes:

  • Injects native sidecars using pod labels and validated annotations.
  • Installs the webhook, synchronizes its CA bundle, and configures its image.
  • Moves origins to dataplane-managed directories owned by UID/GID 65532.
File Description
pkg/​racersdk/​origin.go Documents origin directory ownership.
internal/​racer/​racer.go Registers and configures the webhook server.
internal/​racer/​racer_test.go Tests webhook configuration validation.
internal/​racer/​injector/​injector.go Implements sidecar injection and validation.
internal/​racer/​injector/​injector_test.go Tests injection, placement, and admission responses.
internal/​racer/​hints_test.go Supplies the required object image.
internal/​racer/​controller_test.go Isolates integration-test webhook listeners.
internal/​operator/​imagecoverage_test.go Includes the racer-object image.
internal/​operator/​components/​racer/​webhook_test.go Tests webhook installation and trust maintenance.
internal/​operator/​components/​racer/​tls.go Synchronizes retained webhook trust.
internal/​operator/​components/​racer/​tls_test.go Checks webhook trust during rotation.
internal/​operator/​components/​racer/​racer.go Installs and watches webhook configuration.
internal/​operator/​components/​racer/​racer_test.go Checks image and port wiring.
internal/​operator/​components/​racer/​identity_test.go Supplies the required object image.
images/​racer-object/​Containerfile Uses UID/GID 65532.
images/​racer-loadgen/​Containerfile Uses UID/GID 65532.
docs/​content/​reference/​racer.md Documents non-root origin ownership.
docs/​content/​reference/​racer-object.md Documents injection and non-root deployment.
docs/​content/​guides/​racer-object.md Updates origin and sidecar examples.
deploy/​racer/​mutating-webhook.yaml.tmpl Defines scoped, fail-closed admission.
deploy/​racer/​embed.go Embeds webhook and image defaults.
deploy/​racer/​embed_test.go Tests rendered webhook configuration.
deploy/​racer/​controller.yaml.tmpl Exposes the webhook port.
deploy/​racer/​config.yaml.tmpl Adds webhook and image settings.
deploy/​racer-loadgen/​direct/​render_test.py Checks updated ownership and mounts.
deploy/​racer-loadgen/​direct/​README.md Documents non-root socket access.
deploy/​racer-loadgen/​direct/​daemonset-patch.yaml Uses separate, preexisting socket directories.
cmd/​racer-object/​main.go Validates dataplane-created origin directories.
cmd/​racer-object/​main_test.go Tests directory safety and ownership.
cmd/​racer-loadgen/​README.md Documents origin directory requirements.
cmd/​racer-loadgen/​origin_uds.go Validates directories without creating them.
cmd/​racer-loadgen/​origin_dir_test.go Tests origin directory validation.
cmd/​racer-loadgen/​container_test.go Checks the non-root image user.
cmd/​racer-dataplane/​src/​client/​tests.rs Updates directory ownership expectations.
cmd/​racer-dataplane/​src/​client/​origin_directory.rs Creates, repairs, and migrates origin directories.
cmd/​racer-dataplane/​src/​client/​listener.rs Prepares origin directories during cache setup.
cmd/​racer-dataplane/​src/​client.rs Exposes the origin directory module.
.github/​workflows/​nightly.yaml Adds nightly racer-object image builds.

Review effort: Balanced

Comment thread internal/racer/injector/injector.go
Comment thread cmd/racer-dataplane/src/client/origin_directory.rs Dismissed
Re-own trusted leftover entries while the old root owner still holds the
directory, then narrow the mode, then chown the directory. The origin user
can no longer swap entries during migration, and a run that stops before
the handoff retries the migration. Skip migration when the dataplane is
unprivileged so entries never change when the directory cannot.
The sidecar requires Linux. Only the kubernetes.io/os nodeSelector was
checked, so a pod with spec.os.name=windows was mutated and then failed
API validation with an unclear error.
@jveski
Jordan Olshevski (jveski) marked this pull request as ready for review October 19, 2026 15:41
@jveski
Jordan Olshevski (jveski) requested a review from a team October 19, 2026 15:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Privileged ownership migration and fail-closed admission require human upgrade validation, alongside resolution of the retained correctness issues.

2 open issues
1 resolved since last review

Review effort: Balanced

Comment thread internal/racer/injector/injector.go Outdated
Comment thread cmd/racer-dataplane/src/client/listener.rs
Kubelet expands $(VAR) in container args. The namespace annotation may
contain "$", so a value like store$(KUBERNETES_SERVICE_HOST) reached
the sidecar changed, giving a different cache identity. Write each "$"
as "$$" so kubelet passes the namespace through unchanged.
Root origins could leave /run/racer/<cache>/client as 0700 or 0750, so
nonroot SDK clients could not reach the 0666 client socket. After removing
group and other write bits, add group and other search bits through the
pinned descriptor and verify the inode, owner, and mode. Read and write
bits are not added, ownership stays with the dataplane, and the shared
uds-endpoint helper is unchanged.
Add an O_PATH case for a secure 0700 client directory. It passes
restrict_directory and fails in the search grant fchmod, leaving the mode
and owner unchanged. Clarify that the search bits are for users other
than the directory owner.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Privileged ownership migration has an unresolved recovery failure, and the admission and upgrade paths need final human validation.

1 open issue
2 resolved since last review

Review effort: Balanced

Comment thread cmd/racer-dataplane/src/client/origin_directory.rs Outdated
A list or ownership error during origin entry migration was only logged,
and the directory was still handed to the origin user. The next run then
saw an owned directory and never retried, so a root-owned lock left by a
root origin blocked the SDK origin.

Return the first migration error after trying every entry, and skip chmod
and chown on failure. The directory keeps its old trusted owner, so the
next bind retries. Skipped symlinks, hard links, and other types are still
not errors.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

An unresolved network-isolation issue blocks approval, and privileged ownership migration and fail-closed admission require human validation.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread internal/racer/injector/injector.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Fail-closed admission and privileged ownership migration require human validation, and a permission-test descriptor bug remains unresolved.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Clear FD_CLOEXEC when duplicating directory onto descriptor 3

cmd/​racer-dataplane/​src/​client/​tests.rs:301

If parallel tests release descriptor 3 before directory is opened, directory.as_raw_fd() can be 3. The directory is opened with O_CLOEXEC (cmd/racer-dataplane/uds-endpoint/src/directory.rs:34), and dup2(3, 3) leaves that flag set. Exec then closes the descriptor, so the post-migration assertion at line 309 fails even when permissions are correct. Clear FD_CLOEXEC on descriptor 3 after duplication, including the self-duplication case.

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Fail-closed admission integration and privileged filesystem ownership migration require final human validation.

No open issues 🎉

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants