Skip to content

Commit

Permalink
semantic streaming (#1293)
Browse files Browse the repository at this point in the history
Add semantic streaming.

- BAML syntax has new @stream.done, @stream.not_null and
@stream.with_state attributes on fields, specifying how the fields
behave during streaming.
 - SAP parser tracks each BamlValue node's completion state.
- Runtime compares node streaming states against streaming behavior,
dropping values where appropriate.
- Client codegen produces modified types to match the invariants
specified by the new attributes, and the respective FFIs target these
new types.

Some tricky things about this PR:

- We have to track more data in the BamlValue responses than we used to.
We now have `ResponseBamlValue` to wrap `BamlValueWithMeta<T>` where `T`
can hold all this extra data.
- `BamlValueWithMeta` has a new `distribute_type_with_meta` method,
which traverses a `BamlValue` in parallel with its `FieldType`. This
traversal is the key thing that allows us to analyze each field of a
nested `BamlValue` with respect to its expected streaming behavior.
`distribute_type_with_meta` is conceptually simple but contains a number
of difficult edge cases, especially when dealing with nested types and
recursive type aliases.
- While the old clients had a strict division between `partial` and
`complete` types, the `@stream.done` annotation allows for the creation
of types that combine parts from `complete` and `partial` codegen'd
modules. The resulting Frankentypes are harder to target from the FFI.
Functions like `cast_to` require visibility into both the complete and
partial modules, and we have to thread `allow_partials` through the
traversal of a BamlValue during serialization, and `allow_partials` may
change when we reach a `done` node in an otherwise partial context.
  • Loading branch information
imalsogreg authored Jan 29, 2025
1 parent 7b85c71 commit e30bdd5
Show file tree
Hide file tree
Showing 127 changed files with 8,441 additions and 3,498 deletions.
163 changes: 157 additions & 6 deletions engine/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit e30bdd5

Please sign in to comment.