Skip to content

Commit e4c38f0

Browse files
lrhnlrhn
authored
Feature specs for >>> and async-yield (#119)
* Feature specs for `>>>` and async-yield
1 parent 1eb6cff commit e4c38f0

File tree

4 files changed

+302
-0
lines changed

4 files changed

+302
-0
lines changed

‎accepted/future-releases/async-star-behavior/feature-specification.md‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# Behavior of `yield`/`yield*` in `async*` functions and `await for` loops.
2+
3+
**Author**: [[email protected]](mailto:[email protected])
4+
5+
**Version**: 1.0 (2018-11-30)
6+
7+
## Background
8+
See [Issue #121](http://github.com/dart-lang/language/issues/121).
9+
10+
The Dart language specification defines the behavior of `async*` functions,
11+
and `yield` and `yield*` statements in those, as well as the behavior of
12+
`await for` loops.
13+
14+
This specification has not always been precise, and the implemented behavior
15+
has been causing user problems ([34775](https://github.com/dart-lang/sdk/issues/34775),
16+
[22351](https://github.com/dart-lang/sdk/issues/22351),
17+
[35063](https://github.com/dart-lang/sdk/issues/35063),
18+
[25748](https://github.com/dart-lang/sdk/issues/25748)).
19+
20+
The specification was cleaned up prior to the Dart 2 released,
21+
but implementations have not been unified and do not match the documented
22+
behavior.
23+
24+
The goal is that users can predict when an `async*` function will block
25+
at a `yield`, and have it interact seamlessly with `await for` consumption
26+
of the created stream.
27+
Assume that an `await for` loop is iterating over a stream created by an
28+
`async*` function.
29+
The `await for` loop pauses its stream subscription whenever its body does
30+
something asynchronous. That pause should block the `async*` function at
31+
the `yield` statement producing the event that caused the `await for`
32+
loop to enter its body. When the `await for` loop finishes its body,
33+
and resumes the subscription, only then may the `async*` function start
34+
executing code after the `yield`.
35+
If the `await for` loop cancels the iteration (breaking out of the loop in any
36+
way) then the subscription is canceled,
37+
and then the `async*` function should return at the `yield` statement
38+
that produced the event that caused the await loop to enter its body.
39+
The `await for` loop will wait for the cancellation future (the one
40+
returned by `StreamSubscription.cancel`) which may capture any errors
41+
that the `async*` function throws while returning
42+
(typically in `finally` blocks).
43+
44+
That is: Execution of an `async*` function producing a stream,
45+
and an `await for` loop consuming that stream, must occur in *lock step*.
46+
47+
## Feature Specification
48+
49+
The language specification already contains a formal specification of the
50+
behavior.
51+
The following is a non-normative description of the specified behavior.
52+
53+
An `async*` function returns a stream.
54+
Listening on that stream executes the function body linked to the
55+
stream subscription returned by that `listen` call.
56+
57+
A `yield e;` statement is specified such that it must successfully *deliver*
58+
the event to the subscription before continuing.
59+
If the subscription is canceled, delivering succeeds and does nothing,
60+
if the subscription is a paused,
61+
delivery succeeds when the event is accepted and buffered.
62+
Otherwise, deliver is successful after the subscription's event listener
63+
has been invoked with the event object.
64+
65+
After this has happened, the subscription is checked for being
66+
canceled or paused.
67+
If paused, the function is blocked at the `yield` statement until
68+
the subscription is resumed or canceled.
69+
In this case the `yield` is an asynchronous operation (it does not complete
70+
synchronously, but waits for an external event, the resume,
71+
before it continues).
72+
If canceled, including if the cancel happens during a pause,
73+
the `yield` statement acts like a `return;` statement.
74+
75+
A `yield* e;` statement listens on the stream that `e` evaluates to
76+
and forwards all events to this function's subscription.
77+
If the subscription is paused, the pause is forwarded to the yielded stream
78+
If the subscription is canceled, the cancel is forwarded to the yielded stream,
79+
then the `yield*` statement waits for any cancellation future, and finally
80+
it acts like a `return;` statement.
81+
If the yielded stream completes, the yield statement completes normally.
82+
A `yield*` is *always* an asynchronous operation.
83+
84+
In an asynchronous function, an `await for (var event in stream) ...` loop
85+
first listens on the iterated stream, then for each data event, it executes the
86+
body. If the body performs any asynchronous operation (that is,
87+
it does not complete synchronously because it executes any `await`,
88+
`wait for` or `yield*` operation, or it blocks at a `yield`), then
89+
the stream subscription must be paused. It is resumed again when the
90+
body completes normally. If the loop body breaks the loop (by any means,
91+
including throwing or breaking), or if the iterated stream produces an error,
92+
then the loop is broken. Then the subscription is canceled and the cancellation
93+
future is awaited, and then the loop completes in the same way as the body
94+
or by throwing the produced error and its accompanying stack trace.
95+
96+
Notice that there is no requirement that an `async*` implementation must call
97+
the subscription event handler *synchronously*, but if not, then it must
98+
block at the `yield` until the event has been delivered. Since it's possible
99+
to deliver the event synchronously, it's likely that that will be the
100+
implementation, and it's possible that performance may improve due to this.
101+
102+
### Consequences
103+
Implementations currently do not block at a `yield` when the delivery of
104+
the event causes a pause. They simply does not allow a `yield` statement
105+
to act asynchronously. They can *cancel* at a yield if the cancel happened
106+
prior to the `yield`, and can easily be made to respect a cancel happening
107+
during the `yield` event delivery, but they only check for pause *before*
108+
delivering the event, and it requires a rewrite of the Kernel transformer
109+
to change this behavior.
110+
111+
### Example
112+
```dart
113+
Stream<int> computedStream(int n) async* {
114+
for (int i = 0; i < n; i++) {
115+
var value = expensiveComputation(i);
116+
yield value;
117+
}
118+
}
119+
120+
Future<void> consumeValues(Stream<int> values, List<int> log) async {
121+
await for (var value in values) {
122+
if (value < 0) break;
123+
if (value > 100) {
124+
var newValue = await complexReduction(value);
125+
if (newValue < 0) break;
126+
log.add(newValue);
127+
} else {
128+
log.add(value);
129+
}
130+
}
131+
}
132+
133+
void main() async {
134+
var log = <int>[];
135+
await consumeValues(computedStream(25), log);
136+
print(log);
137+
}
138+
```
139+
In this example, the `await for` in the `consumeValues` function should get a chance to abort or pause
140+
(by doing something asynchronous) its stream subscription *before* the next `expensiveComputation` starts.
141+
142+
The current implementation starts the next expensive operation before it checks whether it should
143+
abort.

‎accepted/future-releases/async-star-behavior/implementation-plan.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Implementation Plan for async-star/await-for behavior.
2+
3+
Relevant documents:
4+
- [Feature specification](https://github.com/dart-lang/language/blob/master/accepted/future-releases/async-star-behavior/feature-specification.md)
5+
## Implementation and Release plan
6+
7+
### Phase 0 (Preliminaries)
8+
9+
#### Specification
10+
11+
The language specification already specifies the desired behavior.
12+
13+
#### Tests
14+
15+
The language team adds a set of tests for the new, desired behavior.
16+
See [https://dart-review.googlesource.com/c/sdk/+/85391].
17+
18+
### Phase 1 (Implementation)
19+
20+
Only tools with run-time behavior are affected.
21+
Tools like the analyzer and dart-fmt should not require any change.
22+
23+
The Kernel and the back-ends need to collaborate on this change since the
24+
behavior is an interaction between the Kernel's "continuation" transformer
25+
and classes in the individual back-end libraries (in "async_patch.dart" files).
26+
27+
The new behavior is guarded by the experiments flag `async-yield`,
28+
so to enable the new behavior, the tools need to be passed a flag
29+
like `--enable-experiments=async-yield`.
30+
31+
### Phase 2 (Preparation)
32+
33+
This change is potentially breaking since it changes the interleaving
34+
of asynchronous execution.
35+
Very likely there is no code *depending* on the interleaving, given that it
36+
is un-intuitive and surprising to users, but some users have hit the problem,
37+
and it's unclear whether they have introduced a workaround.
38+
39+
We need to check that Google code and Flutter code is not affected by the
40+
behavior. If it is, the code should be fixed before we release the change.
41+
The only way to check this is to actually run an updated SDK against the code
42+
base.
43+
44+
### Phase 3 (Release)
45+
46+
The feature is released as part of the next stable Dart release.
47+
48+
## Timeline
49+
50+
Completion goals for the phases:
51+
- Phase 0: (TODO)
52+
- Phase 1: (TODO)
53+
- Phase 2: (TODO)
54+
- Phase 3: (TODO)

‎accepted/future-releases/triple-shift-operator/feature-specification.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# The `>>>` Operator
2+
3+
**Author**: [[email protected]](mailto:[email protected])
4+
5+
**Version**: 1.0 (2018-11-30)
6+
7+
## Feature Specification
8+
9+
See [Issue #120](http://github.com/dart-lang/language/issues/120).
10+
11+
The `>>>` operator is reintroduced as a user-implementable operator.
12+
It works exactly as all other user-implementable operators.
13+
It has the same precedence as the `>>` and `<<` operators.
14+
15+
The `int` class implements `>>>` as a logical right-shift operation,
16+
defined as:
17+
```dart
18+
/// Shifts the bits of this integer down by [count] bits, fills with zeros.
19+
///
20+
/// Performs a *logical shift* down of the bits representing this number,
21+
/// which shifts *out* the low [count] bits, shifts the remaining (if any)
22+
/// bits *down* to the least significant bit positions,
23+
/// and shifts *in* zeros as the most signficant bits.
24+
/// This differs from [operator >>] which shifts in copies of the most
25+
/// signficant bit as the new most significant bits.
26+
///
27+
/// The [count] must be non-negative. If [count] is greater than or equal to
28+
/// the number of bits in the representation of the integer, the result is
29+
/// always zero (all bits shifted out).
30+
/// If [count] is greater than zero, the result is always positive.
31+
///
32+
/// For a *non-negative* integes `n` and `k`,
33+
/// `n >>> k` is equivalent to truncating division of `n` by 2<sup>k</sup>,
34+
/// or `n ~/ (1 << k)`.
35+
int operator >>>(int count);
36+
```
37+
38+
## Background
39+
40+
When Dart chose arbitrary size integers as its `int` type, it also removed
41+
the `>>>` operator, not just from `int`, but from the language.
42+
43+
Now that Dart has chosen to use unsigned 64-bit integers as its `int` type,
44+
there is again need for a logical right shift operator on `int`,
45+
and so we reintroduce the `>>>` operator in the language.
46+
47+
This was decided before Dart 2 was released, and `>>>` was put into the
48+
current language specification document, but it was not implemented by
49+
the language tools for Dart 2.0 due to other priorities.

‎accepted/future-releases/triple-shift-operator/implementation-plan.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Implementation Plan for the `>>>` operator.
2+
3+
Relevant documents:
4+
- [Feature specification](https://github.com/dart-lang/language/blob/master/accepted/future-releases/tripple-shift-operator/feature-specification.md)
5+
## Implementation and Release plan
6+
7+
### Phase 0 (Preliminaries)
8+
9+
#### Specification
10+
11+
The language specification already specifies `>>>` as a user-implementable
12+
operator.
13+
14+
The `int` operator will act as documented in the feature specification
15+
when added.
16+
17+
#### Tests
18+
19+
The language team adds a set of tests for the new feature,
20+
both the general implementable operator,
21+
and the specific `int.operator>>>` implementation
22+
23+
The co19 team start creating tests early, such that those tests can be
24+
used during implementation as well.
25+
26+
The language specification is already updated with this feature.
27+
28+
### Phase 1 (Implementation)
29+
30+
All tools implement syntactic support for the `>>>` operator.
31+
The syntax is guarded by the experiments flag `tripple-shift`,
32+
so to enable the syntax, the tools need to be passed a flag
33+
like `--enable-experiments=tripple-shift`.
34+
35+
### Phase 2 (Use)
36+
37+
The library team implements `int.operator>>>`.
38+
This likely needs to be implemented as a branch
39+
which enables the experiments flag by default.
40+
As such, it can only be tested on that branch.
41+
Backends are free to optimize this operation further at any point.
42+
43+
It is possible to delay the `int` operator until a later release,
44+
but it would be a better user experience to get it out as soon as possible.
45+
46+
### Phase 3 (Release)
47+
48+
The feature is released as part of the next stable Dart release.
49+
50+
## Timeline
51+
52+
Completion goals for the phases:
53+
- Phase 0: (TODO)
54+
- Phase 1: (TODO)
55+
- Phase 2: (TODO)
56+
- Phase 3: (TODO)

0 commit comments

Comments
 (0)