security: Prevent session hijacking through a malicious notebook. by sven-ernw · Pull Request #3550 · kubeflow/community-distribution

sven-ernw · 2026-07-15T00:15:57Z

✏️ Summary of Changes

This pull request addresses the security issue described in GHSA-qjw6-hpc7-w36h together with the notebook's pull request !562. It prevents session hijacking through a malicious notebook.

This will be achieved by the following changes:

Add an EnvoyFilter envoy-filter-gateway.yaml in the oauth2-proxy overlay that strips the authenticating cookie from requests before it is forwarded to backend services like a malicious notebook. This prevents malicious notebooks from accessing other users' authentication credentials. This solution was discussed with @thesuperzapper during a Notebooks WG meeting.
Add a description of how to set up the multi-domain support for notebooks in proposals/20260705-secure-notebook-setup.md and the configuration parameter (disabled by default). This setup is required because even without authentication credentials, an attacker can control the user agent (browser) through a malicious notebook to make requests to the Kubeflow APIs.
Add a Security Considerations section which describes important security measures for a secure production setup.

📦 Dependencies

The notebook's pull request !562 is required to make the described multi-domain setup work. However, the multi-domain setup is not enabled by default. So, this change will not break any existing functionality even if !562 is not released.

🐛 Related Issues

GitHub Security advisory GHSA-qjw6-hpc7-w36h

✅ Contributor Checklist

I have tested these changes with kustomize. See Installation Prerequisites.
All commits are signed-off to satisfy the DCO check.
I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.). (Answer: No, bc we are not adopters but independent security researchers and consultants.)

CC: @lorinl @thesuperzapper @juliusvonkohout

github-actions · 2026-07-15T00:16:06Z

Welcome to the Kubeflow Community Distribution Repository

Thanks for opening your first PR. Your contribution means a lot to the Kubeflow community.

Before making more PRs:
Please ensure your PR follows our Contributing Guide.
Please also be aware that many components are synchronized from upstream via the scripts in /scripts.
So in some cases you have to fix the problem in the upstream repositories first, but you can use a PR against kubeflow/community-distribution to test the platform integration.

Community Resources:

CNCF Slack channels: https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels
Community meetings: https://www.kubeflow.org/docs/about/community/

Thanks again for helping to improve Kubeflow.

Copilot

Pull request overview

This pull request hardens Kubeflow notebook access against session hijacking.

Changes:

Adds gateway cookie filtering.
Documents production security and multi-domain notebook configuration.
Adds notebook-controller subdomain parameters.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
`README.md`	Adds production security guidance.
`proposals/20260705-secure-notebook-setup.md`	Documents multi-domain notebook setup.
`common/oauth2-proxy/README.md`	Documents cookie-name coupling.
`common/oauth2-proxy/base/oauth2_proxy.cfg`	Adds subdomain cookie guidance.
`common/istio/istio-install/overlays/oauth2-proxy/kustomization.yaml`	Includes the gateway filter.
`common/istio/istio-install/overlays/oauth2-proxy/envoy-filter-gateway.yaml`	Filters the session cookie.
`applications/notebooks-v1/upstream/notebook-controller/manager/params.env`	Adds subdomain parameters.
`applications/notebooks-v1/upstream/notebook-controller/manager/manager.yaml`	Injects the parameters into the controller.

sven-ernw · 2026-07-15T01:22:38Z

Addressed all comments from Copilot.

sven-ernw · 2026-07-15T01:27:49Z

/assign @juliusvonkohout

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 15 comments.

thesuperzapper · 2026-07-16T18:37:03Z

/ok-to-test

google-oss-prow · 2026-07-19T14:22:53Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from juliusvonkohout. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sven-ernw · 2026-07-19T14:42:52Z

@thesuperzapper I have reviewed and fixed all suggestions from the CoPilot review.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

proposals/20260705-secure-notebook-setup.md:100

Remove the duplicated word section.

This will make the setup work. Please ensure that you follow the recommendations in the [_Security Considerations_ section](../README.md#security-considerations) section for guidance on a secure setup.

sven-ernw · 2026-07-23T14:47:30Z

Fixed all feedback from review, pull request can be merged once kubeflow/notebooks#562 is merged.

…icious notebook. Add the EnvoyFilter ``envoy-filter-gateway.yaml`` in the `oauth2-proxy` overlay that strips the authentication cookie from requests before they are forwarded to backend services, such as a malicious notebook. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

…icious notebook. Add a description of how to set up multi-domain support for notebooks, including the configuration parameter (disabled by default). Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

…icious notebook. Add a Security Considerations section to the README.md which describes important security measures for a secure production setup. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

…icious notebook. Implemented suggestions from Copilot review. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

…icious notebook. Implemented suggestions from the second Copilot review. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Suppressed comments (3)

applications/notebooks-v1/upstream/notebook-controller/manager/manager.yaml:47

The Notebooks Helm chart vendors a static rendering, but applications/notebooks-v1/helm/manifests/platform-resources.yaml:2061 still proceeds directly from ISTIO_HOST to CLUSTER_DOMAIN and contains none of these new ConfigMap keys. Helm installations therefore cannot enable subdomain routing, and the kubeflow-notebooks platform parity scenario will fail. Regenerate the chart payload through the synchronization script.

          - name: ISTIO_USE_NOTEBOOK_SUBDOMAINS
            valueFrom:
              configMapKeyRef:
                name: config
                key: ISTIO_USE_NOTEBOOK_SUBDOMAINS

common/istio/istio-install/overlays/oauth2-proxy/kustomization.yaml:6

This overlay feeds the static Helm payloads, but common/istio/helm/manifests/install-oauth2-proxy.yaml and install-gke.yaml contain no EnvoyFilter. Helm installations therefore omit the cookie-stripping security control, and the Helm-to-Kustomize parity scenarios will fail. Run scripts/synchronize-istio-manifests.sh and commit the regenerated payloads.

- envoy-filter-gateway.yaml

proposals/20260705-secure-notebook-setup.md:13

Origin separation does not prevent the browser from sending every API request. Credentialed simple requests and form submissions can still carry the parent-domain session cookie, although the same-origin policy blocks reading responses. The current wording falsely implies that request forgery is eliminated; narrow the claim and retain request-forgery protections on state-changing APIs.

This setup hosts the Kubeflow dashboard and APIs on a separate domain from the notebooks to prevent session hijacking from malicious notebooks through the user’s agent (browser). If an attacker hosts malicious notebooks and convinces a victim to visit the notebook’s URL, the attacker cannot steal session data or make API requests in the victim’s name.

+| `ISTIO_USE_NOTEBOOK_SUBDOMAINS` | Set this value to ``true``.                                                                                                                                          |
+| `ISTIO_HOST_NOTEBOOK`           | Domain template used by Istio to host notebooks (e.g., `${NAMESPACE}-notebook.kubeflow.example.org`). `${NAMESPACE}` will be replaced with the Notebook's namespace (Kubeflow profile). |
+| `ISTIO_HOST_AUTH`               | Host used by Istio for handling authentication callbacks or login flows (e.g., `kubeflow.example.org`).                                                              |
+| `ISTIO_AUTH_PATH`               | Optional, defaults to `/oauth2/`; Can be used to change the base URL path used by Istio for authentication callbacks or login flows (e.g. `/oauth2/`).                                        |


+                  -- Prefix the header so every cookie has an explicit boundary.
+                  local new_cookie = string.gsub(";" .. cookie_header, ";%s*" .. target_cookie .. "=[^;]*", "")
+                  new_cookie = string.gsub(new_cookie, "^;", "")


Copilot AI review requested due to automatic review settings July 15, 2026 00:15

google-oss-prow Bot requested review from juliusvonkohout and kimwnasptd July 15, 2026 00:16

google-oss-prow Bot added the size/L label Jul 15, 2026

Copilot started reviewing on behalf of sven-ernw July 15, 2026 00:16 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

sven-ernw mentioned this pull request Jul 15, 2026

feat: Add support to run notebooks on its own subdomains in Istio kubeflow/notebooks#562

Open

3 tasks

google-oss-prow Bot assigned juliusvonkohout Jul 15, 2026

juliusvonkohout requested a review from Copilot July 15, 2026 10:19

Copilot started reviewing on behalf of juliusvonkohout July 15, 2026 10:20 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

google-oss-prow Bot added the ok-to-test label Jul 16, 2026

sven-ernw requested a review from Copilot July 19, 2026 21:21

Copilot started reviewing on behalf of sven-ernw July 19, 2026 21:21 View session

Copilot AI reviewed Jul 19, 2026

View reviewed changes

Comment thread proposals/20260705-secure-notebook-setup.md Outdated

Comment thread README.md Outdated

Comment thread proposals/20260705-secure-notebook-setup.md Outdated

Comment thread proposals/20260705-secure-notebook-setup.md

sven-ernw requested a review from Copilot July 19, 2026 21:52

Copilot started reviewing on behalf of sven-ernw July 19, 2026 21:53 View session

Copilot AI reviewed Jul 19, 2026

View reviewed changes

Comment thread README.md Outdated

Comment thread proposals/20260705-secure-notebook-setup.md Outdated

Comment thread proposals/20260705-secure-notebook-setup.md Outdated

sven-ernw added 5 commits August 14, 2026 22:45

Address the security issue to prevent session hijacking through a mal…

f47a79b

…icious notebook. Implemented suggestions from Copilot review. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

Address the security issue to prevent session hijacking through a mal…

045834a

…icious notebook. Implemented suggestions from the second Copilot review. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

sven-ernw added 2 commits August 14, 2026 22:45

Address the security issue to prevent session hijacking through a mal…

21ffb19

…icious notebook. Implemented suggestions from the second Copilot review. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

Address the security issue to prevent session hijacking through a mal…

f3cd3e4

…icious notebook. Implemented suggestions from the second Copilot review. Signed-off-by: Lorin Lehawany <llehawany@ernw.de> Signed-off-by: Sven Nobis <snobis@ernw.de>

juliusvonkohout force-pushed the pull-request branch from c524949 to f3cd3e4 Compare August 14, 2026 20:45

juliusvonkohout requested a balanced review from Copilot August 14, 2026 20:46

Copilot started reviewing on behalf of juliusvonkohout August 14, 2026 20:47 View session

Copilot AI reviewed Aug 14, 2026

View reviewed changes

Uh oh!

Conversation

sven-ernw commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✏️ Summary of Changes

📦 Dependencies

🐛 Related Issues

✅ Contributor Checklist

Uh oh!

github-actions Bot commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sven-ernw commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sven-ernw commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thesuperzapper commented Jul 16, 2026

Uh oh!

google-oss-prow Bot commented Jul 19, 2026

Uh oh!

sven-ernw commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sven-ernw commented Jul 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

sven-ernw commented Jul 15, 2026 •

edited

Loading

sven-ernw commented Jul 15, 2026 •

edited

Loading

sven-ernw commented Jul 19, 2026 •

edited

Loading