Skip to content

Document the Captured Surface Control API #39982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

chrisdavidmills
Copy link
Contributor

Description

Chrome 136+ supports the Captured Surface Control API, another extension to the Screen Capture API. See https://chromestatus.com/feature/5092615678066688 for details of the addition.

This PR documents the new features, providing reference docs and a guide.

Motivation

Additional details

Related issues and pull requests

@chrisdavidmills chrisdavidmills requested a review from a team as a code owner June 18, 2025 14:09
@chrisdavidmills chrisdavidmills requested review from wbamberg and removed request for a team June 18, 2025 14:09
@chrisdavidmills chrisdavidmills marked this pull request as draft June 18, 2025 14:09
@github-actions github-actions bot added Content:WebAPI Web API docs size/s [PR only] 6-50 LoC changed labels Jun 18, 2025
@github-actions github-actions bot added Content:HTTP HTTP docs size/l [PR only] 501-1000 LoC changed labels Jun 20, 2025
### Parameters

- `element`
- : A reference to the element whose captured `wheel` events you want to forward to the associated captured display surface.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wheel events of that element are not the thing that is "captured". Rather, the wheel events are forwarded to the "captured" surface. (Same comment throughout.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good catch. I've checked through the docs and edited them to make sure I'm not referring to captured wheel events here, or anywhere else.

- : Thrown when:
- The capturing {{domxref("MediaStream")}} returned by the originating {{domxref("MediaDevices.getDisplayMedia()")}} call is no longer capturing, for example because the associated {{domxref("MediaStreamTrack")}} objects have had {{domxref("MediaStreamTrack.stop", "stop()")}} called on them.
- The application is capturing itself.
- an attempt was made to invoke `forwardWheel()` without transient activation.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is permissible to call forwardWheel() without transient activation, if the relevant permission is already "granted".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added something to the page to cover this:

The forwardWheel method must be invoked via transient activation, in which case the user is asked for permission to scroll the captured page. Specifically, the only events that can successfully invoke it are click and input. If the relevant permission is already "granted", transient activation is not needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is anything we need to say about this API in the Permissions API docs. Permissions API integration is not mentioned anywhere in the spec; just Permissions-Policy.


```js
videoElem.srcObject =
await navigator.mediaDevices.getDisplayMedia(displayMediaOptions);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can omit displayMediaOptions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've thought about this and elected to include the following at the start of each brief code snippet:

// Create controller and start capture
const controller = new CaptureController();
videoElem.srcObject = await navigator.mediaDevices.getDisplayMedia({
  controller,
});

// ...

Just so that we show the context for the controller object then being used.


## Examples

### Basic `getSupportedZoomLevels()` usage

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section title says getSupportedZoomLevels(), but the first API surface discussed is actually the event handler for "zoomlevelchange".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I've cut this down to just focus on the specific bit that uses getSupportedZoomLevels().

incBtn.disabled = true;
}
}
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd might want to mention that decreaseZoomLevel() and increaseZoomLevel() should still be called from a try-block, because the zoom level could be changed asynchronously by an entity other than the application; for example, by the user's direct interaction with the user agent,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I've updated the relevant functions in the guide to use try/catch, and added a note highlighting why you should do this. I've also added the code blocks and note to the reference pages where appropriate (so, this one, and the decreaseZoomLevel() and increaseZoomLevel() pages).


### Basic `resetZoomLevel()` usage

The following snippet adds an event listener to a button so that when it is clicked, the `resetZoom()` function is called. This in turn calls the `resetZoomLevel()` method, resetting the captured surface's zoom level.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably want to explain what "resetting" means in this context. Namely, that it sets the zoom level to 100.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added to this page, and the guide.


When a captured display surface's zoom level changes, a {{domxref("CaptureController.zoomlevelchange_event", "zoomlevelchange")}} event fires on the controller, which can be used to run an event handler such as the following. This writes the updated zoom percentage to an output element of some kind.

```js

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is quite similar to another snippet above which also used the same event handler. (But maybe I am being confused by multiple documents being updated concurrently with roughly the same information.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it is used quite a lot. I've updated the wording on this page to at least make it more relevant to the zoomLevel property.


{{EmbedLiveSample("surface-control-demo", , "500px", , , , "captured-surface-control")}}

EDITORIAL: The demo works, but when you scroll the captured display surface, it crashes the browser tab.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really?! Have you filed a Chromium bug? (If so, please assign it to me.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: https://issues.chromium.org/issues/427295493.

I haven't assigned it to you because I don't think I have the permissions to do so. Mentioning @guidou so that it is handled while you are OOO.

@chrisdavidmills chrisdavidmills marked this pull request as ready for review June 20, 2025 17:53
@chrisdavidmills chrisdavidmills requested review from a team as code owners June 20, 2025 17:53
@chrisdavidmills chrisdavidmills requested review from hamishwillee and dipikabh and removed request for a team June 20, 2025 17:53
chrisdavidmills and others added 3 commits June 24, 2025 10:25
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…x.md

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions github-actions bot added the merge conflicts 🚧 [PR only] label Jun 24, 2025
Copy link
Contributor

This pull request has merge conflicts that must be resolved before it can be merged.

chrisdavidmills and others added 2 commits June 24, 2025 10:27
…x.md

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…x.md

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions github-actions bot added the size/xs [PR only] 0-5 LoC changed label Jun 24, 2025
@@ -28,9 +28,8 @@ getDisplayMedia(options)
- `options` {{optional_inline}}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[mdn-linter] reported by reviewdog 🐶

Suggested change

@github-actions github-actions bot removed the merge conflicts 🚧 [PR only] label Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Content:HTTP HTTP docs Content:WebAPI Web API docs size/l [PR only] 501-1000 LoC changed size/s [PR only] 6-50 LoC changed size/xs [PR only] 0-5 LoC changed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants