granit-parser

“YAML is hard. Much more than I had anticipated. If you are exploring dark corners of YAML ... I'm curious to know what it is.”

— Ethiraric

granit-parser is a YAML 1.2 parser in pure Rust with comment and style support, no-std support, and spans for parser events. It accepts many real-world YAML documents from the YAML 1.1 era where the syntax overlaps with YAML 1.2. “Granit” is a correct word in many European languages (English granite).

This crate started as a fork of saphyr-parser that descends from yaml-rust, with influences from libyaml and yaml-cpp. The project has since diverged significantly and is now maintained as an independent project.

Its primary goals are:

• Comment support and StructureStyle information. This is for linting, formatting, and analysis.
• compliance with the yaml-test-suite, including correctness in edge cases
• compatibility with real-world YAML usage
• improved support for formatting, linting, syntax highlight and other comparable projects that need additional metadata
• quickly incorporate the changes we need for the upstream dependency serde-saphyr.

granit-parser 1.0 stabilizes the public API after the experimental 0.0.x releases. The 1.0 migration, including scanner and parser-stack replacements, is documented in CHANGELOG.md.

See releases

Minimal example

Parser::new_from_str returns an iterator of (Event, Span) pairs. The event helpers expose common node metadata, and spans provide byte ranges, source slices, and explicit tag-token starts for tagged nodes:

Comments are emitted as Event::Comment(text, placement). They are presentation metadata for tools such as linters and formatters, not YAML data nodes, so consumers that build YAML values should filter them out. The companion Span for a comment covers the whole source comment, including # and excluding the line break; when parsing from Parser::new_from_str, span.slice(yaml) returns that source comment text.

Applications that do not use comments can disable their capture and emission by setting Options::emit_comments to false or disabling the default comments Cargo feature. The field is always available and defaults to true with the feature enabled and false without it. Setting it to true has no effect without the feature. Comments are still recognized and validated as YAML syntax.

Document starts are emitted as Event::DocumentStart(explicit, version), where version is the optional YamlVersion declared by a preceding %YAML directive for that document.

For editors processing multiple documents, Parser::proceed_next_document() resumes after a syntax error at the next separated --- or valid ... at column zero. Call it after recording the error, then continue reading events normally with original source positions. Its Result<bool, ScanError> is Ok(true) when another document remains (including an empty or malformed document), and Ok(false) at clean EOF. Pending comments and an unread StreamEnd remain available through normal event reads. Looking ahead past comments respects Options::max_buffered_comment_events and can return ErrorKind::TooManyComments when comment emission is enabled. It also skips the current document during successful parsing. Consumers must abandon or mark their unfinished tree nodes; recovery emits no synthetic closing events or error nodes. Anchors remain scoped to their document, and source failures remain terminal. This recovers later documents, not structure within the broken document.

use granit_parser::{options, Event, Parser};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let yaml = r#"
%YAML 1.2
%TAG !example! tag:example.com,2000:
%TAG !core! tag:yaml.org,2002:i
---
items: !shopping
  - milk
  - !example!sliced bread
  - !!str bread
  - !core!nt 1
locations: # Example with composite keys
  [47.3769, 8.5417]: local
  [40.7128, -74.0060]: remote

# JSON-style \uXXXX surrogate pairs:
music: "\uD834\uDD1E\uD83C\uDFB5\uD83C\uDFB6"

favorite: &favorite bread
order: *favorite
unavailable: *missing
dessert: cake
"#;

    let options = options! {
        // Must be enabled in Options to receive anchor names and token spans (default: false).
        emit_anchor_events: true,
        // Must be enabled in Options to report undefined anchors and continue (default: false).
        emit_unresolved_aliases: true,
    };

    for next in Parser::new_from_str_with_options(yaml, options) {
        let (event, span) = next?;

        match &event {
            Event::Anchor(id, name) => {
                // Metadata for the preceding node, with a span covering the &name token.
                println!("anchor: id={id} name={name:?}");
            }
            Event::Alias(id) => println!("alias: target id={id}"),
            Event::UnresolvedAlias(name) => {
                // Still an invalid reference, with a span covering the *name token.
                println!("undefined anchor: {name:?}");
            }
            _ => {}
        }

        if let Some(tag) = event.tag() {
            let tag_start = span
                .tag_start()
                .map(|mark| (mark.line(), mark.col(), mark.byte_offset()));

            if let Some((value, _style)) = event.scalar() {
                println!(
                    "scalar tag: {tag} core-suffix={:?} core-str={} tag_start(line,col,byte)={tag_start:?} for {value:?}",
                    tag.core_suffix(),
                    tag.is_yaml_core_schema_tag("str")
                );
            } else if event.is_node() {
                println!(
                    "node tag: {tag} custom={} tag_start(line,col,byte)={tag_start:?}",
                    tag.is_custom()
                );
            }
        }

        println!(
            "{event:?} bytes={:?} source={:?}",
            span.byte_range(),
            span.slice(yaml)
        );
    }

    Ok(())
}

This prints an event stream like:

StreamStart bytes=Some(0..0) source=Some("")
DocumentStart(true, Some(YamlVersion { major: 1, minor: 2 })) bytes=Some(80..83) source=Some("---")
MappingStart(Block, 0, None) bytes=Some(84..84) source=Some("")
Scalar("items", Plain, 0, None) bytes=Some(84..89) source=Some("items")
node tag: !shopping custom=true tag_start(line,col,byte)=Some((6, 7, Some(91)))
SequenceStart(Block, 0, Some(Tag { handle: "!", suffix: "shopping", original_handle: "!" })) bytes=Some(103..103) source=Some("")
Scalar("milk", Plain, 0, None) bytes=Some(105..109) source=Some("milk")
scalar tag: tag:example.com,2000:sliced core-suffix=None core-str=false tag_start(line,col,byte)=Some((8, 4, Some(114))) for "bread"
Scalar("bread", Plain, 0, Some(Tag { handle: "tag:example.com,2000:", suffix: "sliced", original_handle: "!example!" })) bytes=Some(130..135) source=Some("bread")
scalar tag: tag:yaml.org,2002:str core-suffix=Some("str") core-str=true tag_start(line,col,byte)=Some((9, 4, Some(140))) for "bread"
Scalar("bread", Plain, 0, Some(Tag { handle: "tag:yaml.org,2002:", suffix: "str", original_handle: "!!" })) bytes=Some(146..151) source=Some("bread")
scalar tag: tag:yaml.org,2002:int core-suffix=Some("int") core-str=false tag_start(line,col,byte)=Some((10, 4, Some(156))) for "1"
Scalar("1", Plain, 0, Some(Tag { handle: "tag:yaml.org,2002:i", suffix: "nt", original_handle: "!core!" })) bytes=Some(165..166) source=Some("1")
SequenceEnd bytes=Some(167..167) source=Some("")
Scalar("locations", Plain, 0, None) bytes=Some(167..176) source=Some("locations")
Comment(" Example with composite keys", Right) bytes=Some(178..207) source=Some("# Example with composite keys")
MappingStart(Block, 0, None) bytes=Some(210..210) source=Some("")
SequenceStart(Flow, 0, None) bytes=Some(210..211) source=Some("[")
Scalar("47.3769", Plain, 0, None) bytes=Some(211..218) source=Some("47.3769")
Scalar("8.5417", Plain, 0, None) bytes=Some(220..226) source=Some("8.5417")
SequenceEnd bytes=Some(226..227) source=Some("]")
Scalar("local", Plain, 0, None) bytes=Some(229..234) source=Some("local")
SequenceStart(Flow, 0, None) bytes=Some(237..238) source=Some("[")
Scalar("40.7128", Plain, 0, None) bytes=Some(238..245) source=Some("40.7128")
Scalar("-74.0060", Plain, 0, None) bytes=Some(247..255) source=Some("-74.0060")
SequenceEnd bytes=Some(255..256) source=Some("]")
Scalar("remote", Plain, 0, None) bytes=Some(258..264) source=Some("remote")
Comment(" JSON-style \\uXXXX surrogate pairs:", Above) bytes=Some(266..302) source=Some("# JSON-style \\uXXXX surrogate pairs:")
MappingEnd bytes=Some(303..303) source=Some("")
Scalar("music", Plain, 0, None) bytes=Some(303..308) source=Some("music")
Scalar("𝄞🎵🎶", DoubleQuoted, 0, None) bytes=Some(310..348) source=Some("\"\\uD834\\uDD1E\\uD83C\\uDFB5\\uD83C\\uDFB6\"")
Scalar("favorite", Plain, 0, None) bytes=Some(350..358) source=Some("favorite")
Scalar("bread", Plain, 1, None) bytes=Some(370..375) source=Some("bread")
anchor: id=1 name="favorite"
Anchor(1, "favorite") bytes=Some(360..369) source=Some("&favorite")
Scalar("order", Plain, 0, None) bytes=Some(376..381) source=Some("order")
alias: target id=1
Alias(1) bytes=Some(383..392) source=Some("*favorite")
Scalar("unavailable", Plain, 0, None) bytes=Some(393..404) source=Some("unavailable")
undefined anchor: "missing"
UnresolvedAlias("missing") bytes=Some(406..414) source=Some("*missing")
Scalar("dessert", Plain, 0, None) bytes=Some(415..422) source=Some("dessert")
Scalar("cake", Plain, 0, None) bytes=Some(424..428) source=Some("cake")
MappingEnd bytes=Some(429..429) source=Some("")
DocumentEnd bytes=Some(429..429) source=Some("")
StreamEnd bytes=Some(429..429) source=Some("")

Learning examples

These examples include explanatory comments and assertions for their sample inputs. Run them from the repository root, then read the source alongside the output:

Example	What it teaches	Run
Event walker	Track mapping keys/values, sequence elements, and depth through nested collections, aliases, comments, and document boundaries. Read collection style directly from events.	cargo run --example event_walker
Source positions	Distinguish character indexes, UTF-8 byte offsets, and UTF-16 editor positions; handle Unicode and line endings using the original source.	cargo run --example source_positions
String spans	Convert Span::byte_range() into a required consumer range with explicit errors; preserve raw Unicode and escaped text, distinguish empty ranges from missing offsets.	cargo run --example string_spans
Tooling diagnostics	Match structured errors, inspect declared YAML versions, continue past unresolved aliases explicitly, and enforce a no-directives policy.	cargo run --example tooling_diagnostics

All four also run with --no-default-features, for example:

cargo run --no-default-features --example tooling_diagnostics

The example executables use Rust's standard library; disabling the parser's default features demonstrates that their logic does not depend on comment events or built-in error messages.

Spans emitted by Parser::new_from_str and Parser::new_from_str_with_options always have UTF-8 byte ranges into the original input, including empty ranges for implicit events and EOF. Use Span::byte_range() when storing offsets and Span::slice(source) when only raw text is needed. Keep the same, unchanged source: decoded scalar text and character indexes are not byte offsets. The shared span API returns Option because iterator inputs and manually constructed spans may lack offsets; missing offsets should not silently become zero.

Key differences from saphyr-parser

All changes are intentionally scoped around correctness, compliance, and interoperability. This list summarizes differences done from the time of forking (0.0.6). Saphyr-parser may also have changes in its later releases.

YAML compliance fixes

• Invalid extra closing brackets are rejected

  [ a, b, c ] ]

• Comments no longer truncate multiline scalars

  word1  # comment
  word2

  This is now correctly treated as invalid YAML instead of silently discarding content.

• Reserved directives are ignored

  Previously reported as errors; now handled according to the YAML specification.

Compatibility adjustment

While linters would insist on strict compliance, other applications may prefer full backward compatibility. To balance requirements for all use cases, the compliance is governed by settings in Options.

• Relaxed indentation for flow collection entries and delimiters

  key: [
  1, 2, 3,
  4, 5, 6
  ]

  While not strictly YAML-compliant, under-indented entries and delimiters in flow sequences and mappings are accepted for compatibility with PyYAML, ruamel.yaml, and real-world inputs. The same applies to quoted scalar continuation lines, including in block context. Under-indented implicit mapping keys must keep the key and its : on the same line.

  Set strict_indentation: true to require YAML-compliant indentation, including flow entries, delimiters, and multiline scalar continuations. The default is false.

  use granit_parser::{options, Parser};
  
  let parser = Parser::new_from_str_with_options(
      "key: [\n  1, 2, 3\n  ]\n",
      options! { strict_indentation: true },
  );
  assert!(parser.collect::<Result<Vec<_>, _>>().is_ok());

JSON-style Unicode surrogate pairs

This parser supports explicit handling for JSON-style Unicode surrogate pairs in quoted scalar escape sequences.

• \uXXXX escapes that encode a high surrogate are now required to be followed immediately by a valid low surrogate escape, and both escapes are combined into the corresponding Unicode scalar value.
• Unpaired high surrogates, unpaired low surrogates, and reversed surrogate pairs are now rejected during scanning instead of being treated as generic invalid Unicode escape codes.

Parsing correctness improvements

• Plain scalar continuation fixed

Supports cases like:

hello:
  world: this is a string
    --- still a string

• More helpful error reporting

  Mismatched brackets and quotes now report the position of the opening token instead of the end of file.

Performance

• Zero-copy parsing for &str input

  Uses Cow<'input, str> to avoid unnecessary allocations when parsing from in-memory strings.

• Comment handling If your application does not use comment parsing, set Options::emit_comments = false and disable the comments Cargo feature. The option suppresses capture and emission even if another dependency enables the feature through Cargo feature unification. Disabling the feature compiles out comment handling state when no dependency enables it. Comments can still be present in YAML, they are not emitted.

Internal extensions

• Parser stack support

  Enables features such as !include by exposing additional internal capabilities.

API migration notes

Span is non-exhaustive. Its public fields remain readable and writable, but external code must use constructors instead of struct literals or struct-update syntax. Use the metadata builders or assign individual fields, and include .. when destructuring:

use granit_parser::{Marker, Span};

let start = Marker::new(0, 1, 0);
let end = Marker::new(3, 1, 3);
let span = Span::new(start, end).with_indent(Some(0));
let Span { start: actual_start, .. } = span;
assert_eq!(actual_start, start);

Span::with_tag_start is deprecated and remains available for compatibility. For exact tag locations, use Span::tag_span() on parser-emitted spans; a start marker alone cannot provide the complete token range.

For custom Input implementations, SkipTabs now contains only the input policies Yes and No. Whitespace results use granit_parser::input::WhitespaceResult instead:

• skip_ws_to_eol returns (usize, Result<WhitespaceResult, ErrorKind>).
• skip_ws_to_eol_blanks returns (usize, WhitespaceResult).

Replace SkipTabs::Result(found_tabs, has_valid_yaml_ws) with WhitespaceResult::new(found_tabs, has_valid_yaml_ws). The found_tabs() and has_valid_yaml_ws() accessors remain available on the result. The tuple's character count still reports consumed characters, including when skip_ws_to_eol returns an error. These are breaking API changes intended for the next major release.

Error handling without grepping message text

Granit-parser uses the ErrorKind enum to encode the error kind, and this enum may carry additional metadata. Scanning the English message text for specific substrings is fragile and better to be avoided. See example tooling_diagnostics.rs.

Fallible streaming input

Parser::new_from_iter remains available for genuinely infallible sources. Do not use it for an adapter that turns I/O, decoding, or size-limit failures into EOF: an incomplete YAML prefix may itself be a valid document.

Use Parser::new_from_fallible_iter for a streaming source that can fail. It accepts an Iterator whose items are Result<char, ErrorKind>. The iterator protocol has three distinct outcomes:

• Some(Ok(character)) supplies another character.
• None reports clean physical end-of-input.
• Some(Err(error_kind)) reports a terminal source failure.

The parser returns a ScanError carrying that same ErrorKind and never polls the source again. See the fallible reader example.

The lower-level Scanner is also fallible. Its iterator item is Result<Token, ScanError>, so collecting all tokens keeps scanner failures distinct from clean EOF:

use granit_parser::{Scanner, StrInput};

let tokens = Scanner::new(StrInput::new("key: value\n"))
    .collect::<Result<Vec<_>, _>>()
    .expect("valid YAML should scan");

assert!(!tokens.is_empty());

Security

This crate includes fixes to improve resilience against:

• denial-of-service inputs
• parser hangs
• panic conditions

Like the upstream parser, it does not interpret application-level types, so parsing YAML does not trigger external side effects.

Improved ergonomics

The following ergonomic helpers are available:

• Event::tag
• Event::scalar
• Event::anchor_id
• Event::alias_id
• Event::is_node
• Tag::parts
• Tag::handle
• Tag::suffix
• Tag::original_handle
• Tag::original_parts
• Tag::original
• Tag::core_suffix
• Tag::suffix_in_namespace
• Tag::is_custom
• Tag::is_yaml_core_schema_tag
• Span::slice
• Span::tag_span for the complete raw tag-token range, including URI escapes
• Span::tag_start (you more likely need tag_span)
• ParserStack::push_include

See CHANGELOG.md for details.

Restartable parsing

Some use cases require parsing to continue after an error. granit-parser supports continuing past unresolved aliases (see Event::UnresolvedAlias), and continuing with the next document after a document error using Parser::proceed_next_document.

Anchor events

Anchors were traditionally seen as opaque numeric IDs. The new opt-in Event::Anchor event provides the anchor name if required for advanced processing. The companion Span of this event provides anchor location.

Unsupported features

YAML 1.1 line breaks. YAML 1.1 defines NEL (U+0085), LS (U+2028) and PS (U+2029) as line breaks, this parser uses \n/\r only line-breaking that as expected in YAML 1.2.

Testing

Run the tests from a repository checkout with the official YAML suite initialized:

git submodule update --init tests/yaml-test-suite
cargo test
cargo test --test yaml-test-suite -- ::strict

Strict mode runs every active upstream case without case-specific overrides. The runner reports cases disabled by upstream's own skip: true metadata separately (currently the four ZYU8 directive variants). The relaxed-mode compatibility override for 9C9N does not apply to strict mode.

Missing or empty suite data fails the run. The suite data is not included in the published crate, so running these conformance tests requires a repository checkout. The filesystem-backed suite is not run under Miri isolation.

Tools

The repository includes a few developer tools for inspecting parser output and measuring performance.

Root package binaries:

• dump_events prints the parser event stream for a YAML file.

  cargo run --bin dump_events -- input.yaml

• time_parser measures one full parse and discards the events.

  cargo run --release --bin time_parser -- input.yaml

• run_parser runs repeated parses and reports aggregate timings.

  cargo run --release --bin run_parser -- input.yaml 10

Standalone helper crates:

• walk opens a small REPL for navigating parsed YAML spans.

  cargo run --manifest-path tools/walk/Cargo.toml -- input.yaml

• bench_compare compares benchmark output from multiple parsers.

  cargo bench_compare -- run_bench

• gen_large_yaml generates large YAML inputs for benchmark work.

  cargo gen_large_yaml -- --help

See tools/README.md and tools/bench_compare/README.md for the longer tool notes.

License

Licensed under either:

• Apache License, Version 2.0
• MIT license

At your option.

This project inherits licensing terms from its upstream origins. See the LICENSE file and .licenses/ directory for details.