semantic streaming

[imalsogreg]

Add semantic streaming.

• BAML syntax has new @stream.done, @stream.not_null and @stream.with_state attributes on fields, specifying how the fields behave during streaming.
• SAP parser tracks each BamlValue node's completion state.
• Runtime compares node streaming states against streaming behavior, dropping values where appropriate.
• Client codegen produces modified types to match the invariants specified by the new attributes, and the respective FFIs target these new types.

Some tricky things about this PR:

• We have to track more data in the BamlValue responses than we used to. We now have ResponseBamlValue to wrap BamlValueWithMeta<T> where T can hold all this extra data.
• BamlValueWithMeta has a new distribute_type_with_meta method, which traverses a BamlValue in parallel with its FieldType. This traversal is the key thing that allows us to analyze each field of a nested BamlValue with respect to its expected streaming behavior. distribute_type_with_meta is conceptually simple but contains a number of difficult edge cases, especially when dealing with nested types and recursive type aliases.
• While the old clients had a strict division between partial and complete types, the @stream.done annotation allows for the creation of types that combine parts from complete and partial codegen'd modules. The resulting Frankentypes are harder to target from the FFI. Functions like cast_to require visibility into both the complete and partial modules, and we have to thread allow_partials through the traversal of a BamlValue during serialization, and allow_partials may change when we reach a done node in an otherwise partial context.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
baml	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 28, 2025 11:15pm

[ellipsis-dev]

⚠️ This PR is too big for Ellipsis, but support for larger PRs is coming soon. If you want us to prioritize this feature, let us know at [email protected]

---

Generated with ❤️ by ellipsis.dev