From fc9355e26630a74fc2e2f013f43a23c9dc6b9011 Mon Sep 17 00:00:00 2001 From: Jan-Ivar Bruaroey Date: Mon, 4 Aug 2025 20:04:58 -0400 Subject: [PATCH 1/5] Default the generate key frame algorithm to all layers. --- index.bs | 35 ++++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/index.bs b/index.bs index 80eaae5..e30ba38 100644 --- a/index.bs +++ b/index.bs @@ -1003,26 +1003,39 @@ The generate key frame algorithm, given |promise|, |encod in Section 10 of [[!RFC8851]], then reject |promise| with {{TypeError}} and abort these steps. 1. [=In parallel=], run the following steps: - 1. Gather a list of video encoders, named |videoEncoders| from |encoder|, ordered according negotiated RIDs if any. - 1. If |rid| is defined, remove from |videoEncoders| any video encoder that does not match |rid|. - 1. If |rid| is undefined, remove from |videoEncoders| all video encoders except the first one. + 1. Let |videoEncoders| be a list of video encoders from |encoder|, ordered according to their negotiated RIDs if any. + 1. Let |allRids| be the list of rids from |videoEncoders|, using `null` in place of any missing rids. + 1. Assert that |allRids| either [=list/contains=] no `null` values or a single entry that is `null`. + + Note: This algorithm internally uses `null` to identify the single video encoder in unicast. + + 1. If |rid| is not undefined, remove from |videoEncoders| any video encoder that does not match |rid|. + + Note: If no |rid| is passed in, keyframes are generated for all layers. + 1. If |videoEncoders| is empty, [=queue a task=] to reject |promise| with {{NotFoundError}} and abort these steps. - |videoEncoders| is expected to be empty if the corresponding {{RTCRtpSender}} is not active, or the corresponding {{RTCRtpSender}} track is ended. - 1. Let |videoEncoder| be the first encoder in |videoEncoders|. - 1. If |rid| is undefined, set |rid| to the RID value corresponding to |videoEncoder|. - 1. Create a pending key frame task called |task| with |task|.`[[rid]]` set to rid and |task|.`[[promise]]`| set to |promise|. + + Note: |videoEncoders| is expected to be empty if the corresponding {{RTCRtpSender}} encoding is not + {{RTCRtpEncodingParameters/active}}, or the corresponding {{RTCRtpSender}} track is ended. + + 1. Let |rids| be [|rid|] if |rid| is not undefined, or |allRids| otherwise. + 1. Create a pending key frame task called |task| with |task|.`[[rids]]` set to |rids| and |task|.`[[promise]]` set to |promise|. 1. If |encoder|.`[[pendingKeyFrameTasks]]` is undefined, initialize |encoder|.`[[pendingKeyFrameTasks]]` to an empty set. - 1. Let |shouldTriggerKeyFrame| be false if |encoder|.`[[pendingKeyFrameTasks]]` contains a task whose `[[rid]]` - value is equal to |rid|, and true otherwise. + 1. Remove from |videoEncoders| any video encoder whose identifying |rid| (or `null` value if it has no rid) + already [=list/contained|exists=] in any `[[rids]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. 1. Add |task| to |encoder|.`[[pendingKeyFrameTasks]]`. - 1. If |shouldTriggerKeyFrame| is true, instruct |videoEncoder| to generate a key frame for the next provided video frame. + 1. For each |videoEncoder| in |videoEncoders| (if any), instruct |videoEncoder| to generate a key frame for its + next provided video frame. For any {{RTCRtpScriptTransformer}} named |transformer|, the following steps are run just before any |frame| is enqueued in |transformer|.`[[readable]]`: 1. Let |encoder| be |transformer|.`[[encoder]]`. 1. If |encoder| or |encoder|.`[[pendingKeyFrameTasks]]` is undefined, abort these steps. 1. If |frame| is not a video {{RTCEncodedVideoFrameType/"key"}} frame, abort these steps. 1. For each |task| in |encoder|.`[[pendingKeyFrameTasks]]`, run the following steps: - 1. If |frame| was generated by a video encoder identified by |task|.`[[rid]]`, run the following steps: + 1. If |frame| was generated by a video encoder identified by a value + [=list/contained=] in |task|.`[[rids]]`, and this brings to zero the number of + video encoders identified in |task|.`[[rids]]` that still haven't finished + generating its keyframe, then run the following steps: 1. Remove |task| from |encoder|.`[[pendingKeyFrameTasks]]`. 1. Resolve |task|.`[[promise]]` with |frame|'s timestamp. From 60b6c4a698b46b47290cfe1b35fca7f4def4728c Mon Sep 17 00:00:00 2001 From: Jan-Ivar Bruaroey Date: Fri, 19 Sep 2025 14:41:58 -0400 Subject: [PATCH 2/5] Rework algorithm to function sans rids as well --- index.bs | 39 +++++++++++++++------------------------ 1 file changed, 15 insertions(+), 24 deletions(-) diff --git a/index.bs b/index.bs index e30ba38..c538866 100644 --- a/index.bs +++ b/index.bs @@ -53,6 +53,7 @@ spec:webidl; type:dfn; text:resolve 
 spec:streams; type:interface; text:ReadableStream
+spec:infra; type:dfn; text:list
# Introduction # {#introduction} @@ -1003,45 +1004,35 @@ The generate key frame algorithm, given |promise|, |encod in Section 10 of [[!RFC8851]], then reject |promise| with {{TypeError}} and abort these steps. 1. [=In parallel=], run the following steps: - 1. Let |videoEncoders| be a list of video encoders from |encoder|, ordered according to their negotiated RIDs if any. - 1. Let |allRids| be the list of rids from |videoEncoders|, using `null` in place of any missing rids. - 1. Assert that |allRids| either [=list/contains=] no `null` values or a single entry that is `null`. + 1. Let |layers| be a new [=list=] of the layers for this |encoder|, ordered by negotiated encoding index. + 1. Remove from |layers| all layers that are not {{RTCRtpEncodingParameters/active}}, or whose + corresponding {{RTCRtpSender}} track has ended. + 1. If |rid| is not undefined, remove from |layers| all layers whose rid is not |rid|. - Note: This algorithm internally uses `null` to identify the single video encoder in unicast. + Note: If no |rid| is passed in, keyframes are generated for all active layers. - 1. If |rid| is not undefined, remove from |videoEncoders| any video encoder that does not match |rid|. - - Note: If no |rid| is passed in, keyframes are generated for all layers. - - 1. If |videoEncoders| is empty, [=queue a task=] to reject |promise| with {{NotFoundError}} and abort these steps. - - Note: |videoEncoders| is expected to be empty if the corresponding {{RTCRtpSender}} encoding is not - {{RTCRtpEncodingParameters/active}}, or the corresponding {{RTCRtpSender}} track is ended. - - 1. Let |rids| be [|rid|] if |rid| is not undefined, or |allRids| otherwise. - 1. Create a pending key frame task called |task| with |task|.`[[rids]]` set to |rids| and |task|.`[[promise]]` set to |promise|. + 1. If |layers| is now empty, [=queue a task=] to reject |promise| with {{NotFoundError}} and abort these steps. + 1. Create a pending key frame task called |task| with |task|.`[[layers]]` set to |layers| and |task|.`[[promise]]` set to |promise|. 1. If |encoder|.`[[pendingKeyFrameTasks]]` is undefined, initialize |encoder|.`[[pendingKeyFrameTasks]]` to an empty set. - 1. Remove from |videoEncoders| any video encoder whose identifying |rid| (or `null` value if it has no rid) - already [=list/contained|exists=] in any `[[rids]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. + 1. Remove from |layers| all layers already [=list/contained|found=] in any `[[layers]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. 1. Add |task| to |encoder|.`[[pendingKeyFrameTasks]]`. - 1. For each |videoEncoder| in |videoEncoders| (if any), instruct |videoEncoder| to generate a key frame for its - next provided video frame. + 1. For each |layer| in |layers| (if any), instruct |encoder| to generate a key frame for its + next provided video frame to that |layer|. For any {{RTCRtpScriptTransformer}} named |transformer|, the following steps are run just before any |frame| is enqueued in |transformer|.`[[readable]]`: 1. Let |encoder| be |transformer|.`[[encoder]]`. 1. If |encoder| or |encoder|.`[[pendingKeyFrameTasks]]` is undefined, abort these steps. 1. If |frame| is not a video {{RTCEncodedVideoFrameType/"key"}} frame, abort these steps. 1. For each |task| in |encoder|.`[[pendingKeyFrameTasks]]`, run the following steps: - 1. If |frame| was generated by a video encoder identified by a value - [=list/contained=] in |task|.`[[rids]]`, and this brings to zero the number of - video encoders identified in |task|.`[[rids]]` that still haven't finished - generating its keyframe, then run the following steps: + 1. If |frame| was generated for a layer + [=list/contained=] in |task|.`[[layers]]`, remove that layer from |task|.`[[layers]]`. + If this causes |task|.`[[layers]]` to become empty, then run the following steps: 1. Remove |task| from |encoder|.`[[pendingKeyFrameTasks]]`. 1. Resolve |task|.`[[promise]]` with |frame|'s timestamp. By resolving the promises just before enqueuing the corresponding key frame in a {{RTCRtpScriptTransformer}}'s readable, the resolution callbacks of the promises are always executed just before the corresponding key frame is exposed. -If the promise is associated to several rid values, it will be resolved when the first key frame corresponding to one the rid value is enqueued. +If the promise is associated with several layers, it will be resolved once key frames have been enqueued for all of them. The send request key frame algorithm, given |promise| and |depacketizer|, is defined by running these steps: 1. If |depacketizer| is undefined, reject |promise| with {{InvalidStateError}}, abort these steps. From 1bf37d227e7603e7e0a29103c8e82bbefb5af4db Mon Sep 17 00:00:00 2001 From: Jan-Ivar Bruaroey Date: Fri, 19 Sep 2025 14:56:58 -0400 Subject: [PATCH 3/5] s/rid/RID/ in one place --- index.bs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.bs b/index.bs index c538866..82f6723 100644 --- a/index.bs +++ b/index.bs @@ -1007,7 +1007,7 @@ The generate key frame algorithm, given |promise|, |encod 1. Let |layers| be a new [=list=] of the layers for this |encoder|, ordered by negotiated encoding index. 1. Remove from |layers| all layers that are not {{RTCRtpEncodingParameters/active}}, or whose corresponding {{RTCRtpSender}} track has ended. - 1. If |rid| is not undefined, remove from |layers| all layers whose rid is not |rid|. + 1. If |rid| is not undefined, remove from |layers| all layers whose RID is not |rid|. Note: If no |rid| is passed in, keyframes are generated for all active layers. From 7eca34b73762505df70b325e974d3d8d600e415a Mon Sep 17 00:00:00 2001 From: Jan-Ivar Bruaroey Date: Thu, 16 Oct 2025 09:35:47 -0400 Subject: [PATCH 4/5] Fix algo order and resolve promise ahead of first keyframe --- index.bs | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/index.bs b/index.bs index 82f6723..319285f 100644 --- a/index.bs +++ b/index.bs @@ -1012,9 +1012,9 @@ The generate key frame algorithm, given |promise|, |encod Note: If no |rid| is passed in, keyframes are generated for all active layers. 1. If |layers| is now empty, [=queue a task=] to reject |promise| with {{NotFoundError}} and abort these steps. + 1. Remove from |layers| all layers already [=list/contained|found=] in any `[[layers]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. 1. Create a pending key frame task called |task| with |task|.`[[layers]]` set to |layers| and |task|.`[[promise]]` set to |promise|. 1. If |encoder|.`[[pendingKeyFrameTasks]]` is undefined, initialize |encoder|.`[[pendingKeyFrameTasks]]` to an empty set. - 1. Remove from |layers| all layers already [=list/contained|found=] in any `[[layers]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. 1. Add |task| to |encoder|.`[[pendingKeyFrameTasks]]`. 1. For each |layer| in |layers| (if any), instruct |encoder| to generate a key frame for its next provided video frame to that |layer|. @@ -1024,11 +1024,12 @@ For any {{RTCRtpScriptTransformer}} named |transformer|, the following steps are 1. If |encoder| or |encoder|.`[[pendingKeyFrameTasks]]` is undefined, abort these steps. 1. If |frame| is not a video {{RTCEncodedVideoFrameType/"key"}} frame, abort these steps. 1. For each |task| in |encoder|.`[[pendingKeyFrameTasks]]`, run the following steps: - 1. If |frame| was generated for a layer - [=list/contained=] in |task|.`[[layers]]`, remove that layer from |task|.`[[layers]]`. - If this causes |task|.`[[layers]]` to become empty, then run the following steps: - 1. Remove |task| from |encoder|.`[[pendingKeyFrameTasks]]`. - 1. Resolve |task|.`[[promise]]` with |frame|'s timestamp. + 1. If |frame| was generated for a layer [=list/contained=] in |task|.`[[layers]]`, + then run the following steps: + 1. remove that layer from |task|.`[[layers]]`. + 1. If |task|.`[[layers]]` is now empty, then remove |task| from + |encoder|.`[[pendingKeyFrameTasks]]`. + 1. Resolve |task|.`[[promise]]` with undefined. By resolving the promises just before enqueuing the corresponding key frame in a {{RTCRtpScriptTransformer}}'s readable, the resolution callbacks of the promises are always executed just before the corresponding key frame is exposed. From 3704536dc552c10ff60ff3dd67a429af8d89058f Mon Sep 17 00:00:00 2001 From: Jan-Ivar Bruaroey Date: Thu, 16 Oct 2025 09:50:03 -0400 Subject: [PATCH 5/5] use infra consistently --- index.bs | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/index.bs b/index.bs index 24916dd..4f58e68 100644 --- a/index.bs +++ b/index.bs @@ -1030,17 +1030,17 @@ The generate key frame algorithm, given |promise|, |encod these steps. 1. [=In parallel=], run the following steps: 1. Let |layers| be a new [=list=] of the layers for this |encoder|, ordered by negotiated encoding index. - 1. Remove from |layers| all layers that are not {{RTCRtpEncodingParameters/active}}, or whose + 1. [=list/Remove=] from |layers| all layers that are not {{RTCRtpEncodingParameters/active}}, or whose corresponding {{RTCRtpSender}} track has ended. - 1. If |rid| is not undefined, remove from |layers| all layers whose RID is not |rid|. + 1. If |rid| is not undefined, [=list/remove=] from |layers| all layers whose RID is not |rid|. Note: If no |rid| is passed in, keyframes are generated for all active layers. 1. If |layers| is now empty, [=queue a task=] to reject |promise| with {{NotFoundError}} and abort these steps. - 1. Remove from |layers| all layers already [=list/contained|found=] in any `[[layers]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. + 1. [=list/Remove=] from |layers| all layers already [=list/contained|found=] in any `[[layers]]` of any tasks in |encoder|.`[[pendingKeyFrameTasks]]`. 1. Create a pending key frame task called |task| with |task|.`[[layers]]` set to |layers| and |task|.`[[promise]]` set to |promise|. 1. If |encoder|.`[[pendingKeyFrameTasks]]` is undefined, initialize |encoder|.`[[pendingKeyFrameTasks]]` to an empty set. - 1. Add |task| to |encoder|.`[[pendingKeyFrameTasks]]`. + 1. [=set/Append=] |task| to |encoder|.`[[pendingKeyFrameTasks]]`. 1. For each |layer| in |layers| (if any), instruct |encoder| to generate a key frame for its next provided video frame to that |layer|. @@ -1051,10 +1051,10 @@ For any {{RTCRtpScriptTransformer}} named |transformer|, the following steps are 1. For each |task| in |encoder|.`[[pendingKeyFrameTasks]]`, run the following steps: 1. If |frame| was generated for a layer [=list/contained=] in |task|.`[[layers]]`, then run the following steps: - 1. remove that layer from |task|.`[[layers]]`. - 1. If |task|.`[[layers]]` is now empty, then remove |task| from + 1. [=list/Remove=] that layer from |task|.`[[layers]]`. + 1. If |task|.`[[layers]]` is now empty, then [=list/remove=] |task| from |encoder|.`[[pendingKeyFrameTasks]]`. - 1. Resolve |task|.`[[promise]]` with undefined. + 1. [=Resolve=] |task|.`[[promise]]` with undefined. By resolving the promises just before enqueuing the corresponding key frame in a {{RTCRtpScriptTransformer}}'s readable, the resolution callbacks of the promises are always executed just before the corresponding key frame is exposed.