PROTOCOL.md

The morphological parser conformance adapter protocol

This document specifies, precisely enough that a third party should not need to read any engine's source (or this repo's), what it takes for any morphological/phonological parser to become testable against the fixtures in this repository. The protocol is engine-agnostic: nothing here, or in a fixture's own files, requires any engine's build, layout, or language — only a way to run the engine under test as a command and produce a plain TSV file.

The C# HermitCrab implementation in sillsdev/machine (SIL.Machine.Morphology.HermitCrab) is this suite's founding oracle: every fixture's ground truth was generated by running it, and the batch command of its command-line tool (class BatchCommand) is where this contract's TSV format and signature algorithm were first implemented — but HermitCrab is not a privileged consumer of the protocol, just its first implementer (the Rust port's own CLI independently arrived at the same 3-arg batch contract). Any engine, for any language, implementing section 1 below is equally conformant.

Below, the harness means the program that runs an engine against the fixtures and diffs the result: today sillsdev/machine's conformance runner (hc-conformance) and PanGloss's equivalent. The harness is not part of this repository; this document specifies what it must do so that two harnesses agree.

1. The canonical adapter contract

A conforming engine adapter is a single executable command:

<engine-binary> batch <grammar> <words.txt> <output.tsv> [--start N]

• <grammar> — the engine's grammar input for this fixture. For the reference representation this is grammar.xml (HermitCrabInput XML); see section 6 for engines that cannot consume it directly.
• <words.txt> — one word per line, blank lines ignored; shared verbatim across every engine under test for a given fixture (never engine-specific).
• <output.tsv> — the file the adapter must write its results to, in the format specified in section 2.
• --start N (optional) — 0-based line index to resume at, for crash recovery on long word lists. An adapter that doesn't need resumability may ignore this flag or not implement it; the harness only relies on it for its own optional resume behavior, never as a correctness requirement.

This is exactly what hc-rs batch <grammar.xml> <words.txt> <output.tsv> already does, so the Rust port is adapter-conformant today with zero wrapper code. See section 7 for the one C#-specific wrinkle, and section 8 before implementing an engine from scratch: grammar.xml alone under-specifies the ground truth, because several Morpher host-configuration values that are not part of the grammar file were baked into every fixture's expected.tsv.

2. The output format: a 5-column TSV

Every line of <output.tsv> has exactly 5 tab-separated columns:

idx    word    ms    status    signature

• idx — 0-based index of the word in words.txt (matches the founding oracle's batch loop index).
• word — the literal word string being parsed (echoed back, so the row is self-describing without needing to cross-reference words.txt by line number).
• ms — elapsed wall-clock milliseconds for parsing this one word. Not compared by the harness (see section 3) — timing is informational only and is expected to vary run to run and engine to engine.
• status — one of:
  • ok — the parse attempt completed (regardless of whether it found any analyses — zero analyses is a legitimate ok result, not a failure; see the signature column).
  • SKIPPED — the engine could not attempt the parse at all (C#'s only case today: InvalidShapeException, i.e. the word contains a character outside the grammar's character definition table). A SKIPPED row's signature is always -.
• signature — the order-independent, sorted, semicolon-joined signature of every distinct analysis found (see section 3 for the exact algorithm), or the literal - if zero analyses were found (this is the normal, correct representation of "this word does not parse under this grammar" — a fixture's expect_fail: true words).

The STARTED sentinel line

C#'s BatchCommand writes a sentinel line immediately before attempting each word, so a crashed process can be resumed from the last line written:

idx    word    STARTED

(3 columns, not 5.) This is a crash-recovery convenience for engines that might crash mid-word on a long corpus; it carries no comparison meaning and adapters are not required to emit it (hc-rs deliberately does not — it buffers results and writes the final file in one shot). The harness's TSV parser and comparison logic (section 4) skip any line that doesn't have the 5-column ok/SKIPPED shape, so STARTED lines are silently ignored wherever they appear.

The crash convention: expect_crash

A handful of fixtures (e.g. edge-cases/simultaneous-epenthesis-cascade, whose founding oracle run hit an infinite-loop detector mid-word) pin a crash as their ground truth, not a signature. Such a fixture's expected.tsv contains only a STARTED sentinel line for the word that crashed the oracle — there is no real result row, because the oracle itself died first — and its fixture metadata (words.yaml front matter) carries expect_crash: true to make that explicit rather than leaving the harness to infer it from an empty expected.tsv.

The harness's contract for an expect_crash fixture:

• The engine under test is expected to also crash while loading the grammar or parsing the word (not necessarily the same exception type or message — just fail rather than complete). That counts as a PASS: reproducing the crash IS conforming to the ground truth.
• If the engine crashes but expect_crash is not set, that is a FAIL (an unexpected engine crash), with the engine's diagnostics shown.
• If expect_crash is set but the engine returns normally (produces output instead of crashing), that is a FAIL ("expected crash, engine produced output") — the engine no longer reproduces the bug the fixture pins.
• A harness-level problem (bad --adapter command, the adapter process failing to start, a timeout, or an adapter exiting 0 without producing its output file) is never treated as a crash match, expect_crash or not — it is always a plain failure/error.

Because a crash aborts a fixture's whole word loop, crash fixtures should keep words.txt to a single word: there is nothing after the crash for the harness to compare, so additional words would never actually be exercised.

What expect_crash actually pins, and the hazard that follows from it. For edge-cases/simultaneous-epenthesis-cascade, the crash is InfiniteLoopException, thrown by the oracle's epenthesis synthesis step (EpenthesisSynthesisRewriteSubruleSpec.ApplyRhs) the instant a shape being built during synthesis reaches exactly 256 nodes. This is not one of the host-configuration knobs in section 8 — it is a hardcoded constant, not exposed on Morpher or anywhere else, so there is no value a reimplementing engine can set to opt out of it. Conformance on this fixture therefore requires the engine under test to independently hit some runaway condition on the same input and fail rather than complete — not because failing is correct, but because the founding oracle failed here and the fixture's ground truth is that failure. An engine that instead detects and terminates the same unbounded epenthesis expansion cleanly (e.g. by finding a correct, bounded analysis, or by cycle detection that recovers rather than crashing) is, on its own merits, the more correct engine — and is non-conformant on this fixture per section 4's comparison rules, because "engine returns normally" is scored as a FAIL regardless of whether the output would have been right. Treat this fixture as pinning an implementation limitation of the founding oracle, not a linguistic fact about the grammar it exercises.

3. The signature algorithm

For a word that parses, each distinct result contributes one signature entry:

<morph1.MorphemeId>+<morph2.MorphemeId>+...|<shape-as-regex-string>

• The morpheme-ID chain is built by walking the word's allomorphs in morph order (Word.AllomorphsInMorphOrder), taking each allomorph's owning morpheme's Id, joined with +.
• After the |, the surface shape is rendered via Shape.ToRegexString(word.Stratum.CharacterDefinitionTable, true) — the same character-definition table used to load the grammar, with boundary markers included (the trailing true argument).
• All of a word's distinct-analysis signature strings are then sorted ordinally (StringComparer.Ordinal) and joined with ;. This makes the signature order-independent: two runs that find the same set of analyses in a different internal search order compare equal. It does not dedupe — if an engine's search legitimately produces the same signature string twice (a duplicate, not a distinct analysis), both occurrences appear; a divergence in duplicate count for an otherwise-matching signature set is not automatically a pass, see section 4 for how the harness handles this.
• If zero analyses are found, the signature is the literal string - (not an empty string).

This is a direct transcription of the founding oracle's signature builder (BuildSignature in its batch command); an adapter for a new engine reproduces this string exactly, using that engine's own morpheme IDs and its own surface-shape rendering, for its output to be diffable against expected.tsv.

Morpheme ID / gloss naming discipline: because the signature format compares morpheme names literally, any engine whose grammar representation assigns different IDs to the same morphemes (e.g. a hand-authored non-XML grammar for a fixture, see section 6) must match grammar.xml's IDs exactly, or a real match will register as a false failure.

Multi-character segment representations render individually parenthesized. A character definition whose string representation is more than one character (e.g. a digraph segment like "ch") is wrapped in its own (...) in the shape half of a signature, unconditionally — this comes straight from the oracle's ToRegexString extension, which wraps any representation longer than one character in parentheses and does so regardless of whether the segment is otherwise ambiguous. A word "panucho" in a grammar that defines "ch" as one segment renders as panu(ch)o, not panucho. Easy to miss when hand-deriving an expected signature — check a grammar's character definition table for multi-character representations before trusting a hand-written signature (see languages/suffixing-evidential-adjacency-chain/words.yaml's authoring-discipline note for a case this actually caught).

Guess-stem rendering (Guesser/LexicalGuess, pinned in G1). A guessed root has no distinguishing marker at all in the signature string: Morpher.LexicalGuess builds a throwaway LexEntry whose Id (and Gloss) is set to the literal matched surface substring, so the guessed root's morph-chain component in the signature is that substring itself, indistinguishable by string alone from a real lexical entry that happened to share the same id (e.g. word "foos" with root "foo" unknown but a lexical guess pattern [Any]* in scope, plus a real suffix rule PL="-s", yields signature entries foo+PL|foos and — because the guess pattern is also tried against the zero-affixes candidate — foos|foos; the oracle pins this in a unit test of its own). The legacy batch command calls the two-argument SignatureFormat.ParseOneWord overload, which disables root guessing. The harness self-check uses an explicit morpher.GuessRoots value when present; otherwise it retains the legacy per-word rule: a guess: true parse enables guessing for that word. Settings-aware adapters receive the per-call flag through morpher.json (section 8). When GuessRoots is explicitly present, either true or false, adapter materialization retains all word rows. When it is absent, words carrying a guess: true parse are omitted from the legacy adapter input. A named variant with no adapter-visible words fails rather than passing without a comparison.

4. Comparison semantics (what "pass" means)

The harness diffs an adapter's <output.tsv> against a fixture's checked-in expected.tsv using these rules:

1. Only the word, status, and signature columns are compared. idx and ms are ignored (row identity is keyed by word, not idx, so an adapter is free to reorder rows; ms is pure timing noise).
2. Any line that isn't a well-formed 5-column idx / word / ms / status / signature row (in particular, any STARTED sentinel line) is skipped entirely before comparison.
3. A signature is parsed into its ;-separated set of morph-chain|shape entries and compared as a multiset — reordering the entries within a signature string is not a failure (the algorithm already sorts them, but a hand-edited or alternate-engine expected.tsv reordering is tolerated defensively), but a genuine difference in the entries themselves, or in how many times a given entry appears, is a failure. Within an entry's shape, the members of each bracketed class [...] (and [^...]) are compared as a set, so |b[ae]t equals |b[ea]t. HermitCrab renders class members in character-definition-table insertion order, and FieldWorks fills that table from an unordered collection, so the order carries no meaning (sillsdev/machine#573). Write expected signatures in the order your engine renders; a harness comparing against a differently-ordered engine still passes.
4. A status mismatch (ok vs SKIPPED) is always a failure.
5. A missing or extra word row (present in one file, absent in the other) is a failure for that word.

Anything outside these rules — a real signature difference, a real status difference — is a genuine conformance failure and is reported as such, per word, and rolled up per fixture.

5. Capability profiles

Not every engine implements every grammar-model construct HermitCrab supports (the motivating case is XAmple, SIL's AMPLE-family morphotactic parser, which has no phonological-rule engine at all). Two mechanisms make the suite serve engines of different scope without forking it:

• Engine capability declaration. Whoever invokes the harness against an engine declares that engine's supported capability set, e.g. --capabilities phonology for HermitCrab/hc-rs, or --capabilities "" (empty) for a purely morphotactic engine. Self-check mode (running the live C# oracle in-process) always implies full capability, phonology included.
• Fixture capability requirement. Every fixture's metadata (words.yaml front matter) carries a requires array. phonology means "this fixture's grammar.xml contains at least one PhonologicalRule or MetathesisRule element"; morpher-settings means "this fixture's words.yaml declares a morpher: block" (section 8), which the engine must apply. An empty/absent requires is pure morphotactics, runnable by any conforming engine. The harness mechanically re-derives this list from grammar.xml on every run and FAILs the fixture (a metadata error, not a conformance failure) if the checked-in requires disagrees, so this can never silently drift.

The harness computes, for each fixture, whether the engine's declared capability set is a superset of the fixture's requires list. If not, the fixture is skipped for that engine run — reported as SKIPPED, distinct from both PASS and FAIL, so a capability-limited engine's run is never scored as if it silently passed fixtures outside its scope. New capability values are added the same way phonology was: name the vocabulary word, and mechanically derive it from the fixture's grammar.xml (or other grammar representation) contents, never by hand-guessing from a directory name.

6. Per-engine grammar representation

The adapter contract's first positional argument is the engine's grammar input for the fixture, not necessarily grammar.xml itself. grammar.xml (HermitCrabInput XML) is the default and reference representation — every fixture ships one, and it's what the C# oracle used to generate expected.tsv. An engine that cannot consume that XML directly (XAmple's case: it consumes AMPLE-style control/dictionary files) names an alternate, per-fixture grammar file or directory in its own registration (e.g. a grammar.xample/ directory living alongside grammar.xml in the fixture, anticipating a future XAmple adapter). words.txt and expected.tsv are always shared across every engine tested against a fixture: one word list, one ground truth, regardless of how many different grammar representations exist for it.

7. Aside: the founding oracle's own command-line tool needs a wrapper to conform

The founding oracle's command-line tool in sillsdev/machine (hc.dll) does not implement section 1's contract directly, because of a pre-existing CLI shape it inherited from HermitCrab.NET: grammar loading is a global Program.Main option (-i <grammar.xml>), separate from and prior to dispatching any subcommand, and its batch subcommand only takes 2 positional arguments (<wordlist-file> <output-tsv>), not 3. The normal invocation is:

hc.dll -i grammar.xml -s script.txt

where script.txt contains one line: batch words.txt expected.tsv (paths relative to the fixture directory the tool is run from). This is a real, working invocation shape — every fixture's expected.tsv in this suite was generated exactly this way — it is just not literally section 1's single 3-positional-arg command.

Two consequences:

• The harness's self-check mode does not shell out to hc.dll at all. It calls the Morpher API in-process (loading grammar.xml via XmlLanguageLoader, building a Morpher, and computing the same signature algorithm as the batch command, see section 3) — this sidesteps the CLI-shape mismatch entirely, since self-check mode never needs a CLI contract in the first place.
• If anyone ever wants to point the harness's --adapter flag at hc.dll itself (to test adapter-mode plumbing against a known-good reference, independent of self-check mode), hc.dll needs a one-line wrapper script that (1) writes a temporary script.txt containing batch <words.txt> <output.tsv> with the two paths substituted in, then (2) invokes hc.dll -i <grammar> -s <that temp script>. Not required by any current acceptance criteria (self-check plus --adapter against hc-rs suffices), but straightforward if ever needed. The wrapper script was in sillsdev/machine#480 and is not part of this repository.

8. Host configuration every fixture was generated under

grammar.xml fully determines a grammar's rules, but it does not determine every input to the parse. The oracle's Morpher class has several settable properties that change which analyses are found for the same grammar and word, and every one of this suite's expected.tsv files was generated with Morpher left at its plain constructor defaults unless the fixture's words.yaml declares a morpher: block (see "Overriding a default per fixture" at the end of this section): both the batch command's host (HCContext.Compile) and the harness's self-check runner construct new Morpher(traceManager, language) and never assign any of the properties below. A formalism-correct reimplementation that happens to pick different values for these will diverge from expected.tsv on fixtures that exercise them, with nothing in grammar.xml to explain why. These defaults are pinned executably against the constructor by a unit test in sillsdev/machine (Constructor_DefaultsMatchDocumentedProtocolDefaults).

• MaxStemCount — default 2. Consumed in the oracle's AnalysisCompoundingRule.Apply: once an in-progress analysis's non-head count would reach MaxStemCount, the compounding rule refuses to unapply further, so no candidate with more stems is ever produced. Observable difference: a grammar whose compounding rule can in principle chain three or more stems will only ever be analyzed as a compound of at most MaxStemCount stems under the reference oracle; a reimplementation using a higher value will find compound analyses of words this suite's expected.tsv shows as unparsed or as parsed with fewer stems, and a lower value will miss compound analyses this suite's expected.tsv shows as found.
• DeletionReapplications — default 0. Consumed in the oracle's AnalysisRewriteRule.Apply (its deletion case): a deletion phonological rule always unapplies (recovers a deleted segment) once; it is then permitted to reapply and recover an additional consecutive deletion up to DeletionReapplications more times before the loop stops. At the default 0, exactly one deletion is ever recovered per rule invocation — a surface form that resulted from two or more consecutive applications of the same deletion rule cannot have all of them undone, so the fully-restored underlying form is never reached and that analysis is absent from expected.tsv. Observable difference: raising this value changes the analysis candidate set itself (not just performance) — it can surface additional valid analyses for words with multiple consecutive deletion sites that the default value's expected.tsv shows as not found, or as found via a different (shorter-recovery) analysis path.
• MaxAlternatives — default 0, meaning unlimited. Every guard on this value (AnalysisStratumRule.Apply, RuleCascade, RuleBatch) is written as value > 0 && count > value, so 0 disables the check entirely rather than meaning "zero alternatives." When set to a positive number and exceeded, the analyzer throws MaxAlternativesExceededException — and nothing between there and Morpher.ParseWord/AnalyzeWord catches it, so it propagates out of the call entirely; there is no code path that turns an exceeded limit into a truncated-but-usable result. Observable difference: since every fixture was generated with this disabled, expected.tsv never reflects a truncated search — a reimplementation must not impose an undocumented alternative cap of its own (even one intended purely as a runaway guard) without expecting to diverge on any fixture whose search happens to exceed it, and must not silently truncate where the reference oracle would have thrown or continued.
• MergeEquivalentAnalyses — default true. Consumed in AnalysisStratumRule.Apply (around line 133): while unapplying rules within one stratum, intermediate analyses that end up with identical shapes are merged into one canonical Word, with the others recorded on its Alternatives list instead of being carried forward independently into the next stratum's search. Every merged alternative is expanded back out (Word.ExpandAlternatives) immediately before lexical lookup and synthesis (see the doc comment on Morpher), so this is designed to prune redundant search state rather than to change which analyses are ultimately found. No fixture in this suite is known to depend on MergeEquivalentAnalyses: false producing a different result than true, but the default is documented here because it is still host configuration this suite's expected.tsv was generated under, and a reimplementation that skips the equivalent-shape merge (most will — it is a C#-side performance optimization, not part of the formalism) must still ensure it explores the same eventual analysis set that expansion restores.
• guessRoot — see section 3's "Guess-stem rendering" discussion above. This is a per-call flag rather than a Morpher property. The legacy batch command disables it; the harness supports explicit GuessRoots settings in both self-check and settings-aware adapters.
• LexEntrySelector / RuleSelector — default entry => true / rule => true. These are host-installed predicates that can exclude specific lexical entries or specific rules from participating in analysis/synthesis at all (checked in, e.g., Morpher.LexicalLookup and in each compiled rule's Apply, such as AnalysisCompoundingRule.Apply and AnalysisRewriteRule.Apply above). Neither the batch command / HCContext nor the harness's self-check runner ever installs a narrower predicate, so every fixture's expected.tsv was generated with every lexical entry and every rule in grammar.xml eligible to participate — a reimplementation must not implement any equivalent of a partial-grammar or partial-lexicon mode (even for performance or scoping reasons) without expecting to diverge, since there is no signal in grammar.xml itself that any entry or rule was excluded.

These are the only settable, parse-affecting properties found on Morpher; XmlLanguageLoader (which builds the Language object from grammar.xml) does not read, set, or expose any of them — every one of the above is host-side configuration layered on top of the grammar, never something a grammar file itself can express or override.

Overriding a default per fixture: the morpher: block

A fixture whose ground truth depends on a non-default setting declares it in words.yaml's front matter, keyed by the FieldWorks <ParserParameters><HC> element name that sets it in FieldWorks (HCParser.LoadParser), so the same names a FieldWorks project carries reach every engine:

requires: [phonology, morpher-settings]
morpher:
  DelReapps: 1

Key	Morpher property	Value	Default when absent
DelReapps	DeletionReapplications	whole number, at least 0	0
MaxRoots	MaxStemCount	whole number, at least 1	2
MergeAnalyses	MergeEquivalentAnalyses	true or false	true
MakePartialsFinal	AlwaysEnforceFinalTemplates	true or false	false
GuessRoots	per-call ParseWord(..., guessRoot) flag	true or false	per-word parses[].guess in self-check; otherwise false

Rules enforced by the loader (sillsdev/machine's WordsYamlLoader). The published schema (schema/words.schema.json) checks structural constraints; the loader also checks ordered word-list equality, default-only coverage metadata, unique names, and exact capability equivalence:

• Absent means the defaults above, exactly as before this block existed. An empty block is an error: omit it instead. GuessRoots is an explicit per-call flag and is never applied to a Morpher property; when absent, self-check retains legacy per-word parses[].guess behavior.
• Named run variants are an optional top-level variants sequence. Each entry has a unique name matching [a-z0-9-]+ (default is reserved), a non-empty morpher block, and a complete words sequence with exactly the same ordered word values as the top-level sequence. Its expectations use the same parses, expect_fail, and expect_skip fields. Each run starts from engine constructor defaults and applies only its own settings; a variant never inherits the top-level morpher block. Coverage metadata remains on the default word list. The harness runs the default and every named variant independently and labels variant diagnostics by name.
• The vocabulary is closed. Any other key is an error rather than ignored, because a setting the harness cannot apply must never fall back silently to the default it was meant to override. MaxAlternatives remains unsupported because its limit changes search completion into an error, while MergeMSAs requires FieldWorks loader support; both are hard errors. NotOnClitics, NoDefaultCompounding, AcceptUnspecifiedGraphemes and Strata shape how FieldWorks builds a Language, which grammar.xml already states directly.
• A settings block is a capability requirement. A fixture with a default or variant morpher block must list morpher-settings in requires, and the harness re-derives that from words.yaml the same way it derives phonology from grammar.xml (section 5): an engine that does not declare morpher-settings is reported SKIPPED for that run, never run at defaults. Self-check declares it.
• How an adapter receives the values. Each run is materialized separately and writes its own morpher.json beside grammar.xml (a JSON object of the same keys; {} for a default run with no block), and the harness substitutes its path for a {morpher} placeholder in the --adapter command template. Guessed-root rows are materialized only when GuessRoots is explicit, so an adapter cannot silently appear to support guessing from omitted rows. An adapter that declares morpher-settings must take the placeholder; the harness fails the run, naming the missing placeholder, rather than run it without the settings.
• Counterfactual and coverage gates that rebuild a mutated grammar in a child process (CounterfactualGate, from sillsdev/machine#480) still evaluate every fixture at the defaults; their surface deltas for a fixture with a block are therefore measured under the defaults, not under the block.

edge-cases/deletion-reapplications is the first fixture to use it (DelReapps: 1), observed from the C# oracle at 0, 1 and 2 to show the setting is what moves its expected parses.

9. FieldWorks producibility

Passing this suite proves an engine matches the founding oracle over grammar.xml; it says nothing by itself about whether any real project could have produced that grammar.xml. FieldWorks users never author HermitCrabInput XML directly — their grammars come out of HCLoader.cs (Src/LexText/ParserCore/HCLoader.cs in the FieldWorks repository), which builds a Language straight from a LibLCM project and has a materially narrower reach than the DTD or the engine it feeds. A fixture can therefore be a fully conformant, oracle-verified regression test of the engine and still describe a grammar shape no FieldWorks user will ever see.

The standard for "producible" is the round trip: a real FieldWorks project, built in FieldWorks, goes through HCLoader.cs and yields the fixture's grammar.xml. That the loader source could write a construct is weaker evidence. fieldworks_producible: true is therefore a claim read from the loader until the fixture has a FieldWorks witness (section 10), and only the fixtures with a witness are certified today: 9 of the 134 (all under edge-cases/).

Every fixture's words.yaml front matter carries fieldworks_producible: true, directly after requires. The suite keeps only grammars a real FieldWorks project can produce: true means every construct the fixture's grammar.xml exercises has a known HCLoader code path that can emit it from a real project. A grammar that needs a construct with no such path is rewritten to drop it in favour of an equivalent FieldWorks can produce, or it is deleted; scripts/check_words_schema.py and the consuming engines' gates fail on any fixture that is not true. The loader still parses false (with fieldworks_producible_notes) so a research fixture kept outside this tree can carry the verdict, but nothing in this repository may.

A fixture moves to true by rewriting its grammar and re-running the C# founding oracle, never by relaxing the verdict in place; the rewrite keeps a fixture only if the oracle's rows still hold or change for a reason the rules explain. See docs/engine-only-fixture-retirement.md for the decision and what it removed, and the section "What the XML can say but FieldWorks cannot, so we do not test it" of docs/fieldworks-cannot-say.md for the constructs and the HCLoader.cs lines that rule them out.

The verdict is human-authored, not mechanically re-derived the way requires is. It rests on fieldworks-producibility.tsv (a hand-researched, point-in-time snapshot of HCLoader.cs, from sillsdev/machine#480, not in this repository) plus five usage-level checks no per-row TSV entry can express on its own — three because each depends on where or how an otherwise-producible attribute is used, not merely on whether HCLoader ever writes it at all; a fourth because it depends on whether a fixture's own committed ground truth is sensitive to a value HCLoader can never produce; and a fifth because it depends on what shape the construct an otherwise-producible attribute sits on takes, not on the attribute in isolation:

• A rule reached only through an AffixTemplate Slot is necessarily inflectional, and LoadInflAffixProcessRule never sets MorphologicalOutput's MPR features on an inflectional rule — so a template-slot rule that writes MPR features cannot come from FieldWorks even though the same write is producible on a stratum-level (non-templated) rule.
• requiredMPRFeatures must be identical across every subrule of one rule that is not inflectional: a derivational rule gets a single per-MSA list from FromProdRestrictRC applied to every subrule alike (HCLoader.cs:1017), and an unclassified or clitic rule gets none, so two subrules of the same such rule with different required MPR features cannot come from FieldWorks even though the attribute itself is ordinarily producible. An inflectional rule (an IMoInflAffMsa, which includes every template-slot rule) adds the same kind of per-MSA list (HCLoader.cs:1050) and also each allomorph's own inflection classes (HCLoader.cs:1118, :1145, :1172), so its subrules may differ.
• A MorphemeCoOccurrenceRule/AllomorphCoOccurrenceRule must be type="exclude": HCLoader's two co-occurrence loaders build Exclude constraints only, sourced from LibLCM repositories that are prohibition-only by name, so type="require" has no FieldWorks equivalent regardless of how ordinary the surrounding rule is.
• MorphologicalRuleOrder must not matter to a fixture's committed ground truth: HCLoader hardcodes every Stratum that can hold a rule to Unordered (HCLoader.cs:240,243,387; the Surface stratum at 246 is linear but empty) and can never emit linear for one that holds a rule. The check is NOT "the fixture's grammar.xml declares linear" — HermitCrabInput.dtd:244 defaults morphologicalRuleOrder to linear, so an absent-or-explicit linear declaration is the ordinary case for most fixtures in this suite and flags nothing on its own; testing on the declaration alone would condemn fixtures whose ground truth never actually depends on the ordering. Establish it the way it was originally found: copy the fixture, flip its Stratum to unordered, and re-run the C# founding oracle. Only if some word's found-analysis set changes — a spurious extra parse, a lost one, or a duplicate, never merely a different search order or timing — does the fixture's ground truth actually depend on the ordering, and only then is it HC-engine-only for this reason. What a fixture may declare is narrower than what it may depend on. The order is read for the rules a Stratum lists in its own morphologicalRules and when the stratum applies its affix templates (SynthesisStratumRule and AnalysisStratumRule branch on it in ApplyTemplates; in synthesis an unordered stratum feeds a template's output back through the rules, which can reach a further template, and a linear one applies it once). A linear stratum that lists no morphological rule and holds no active AffixTemplate is producible as written; a stratum that has either must say unordered, which is how HCLoader builds it. sillsdev/machine's FieldWorksProducibleShapeTests enforces this.
• Environments (RequiredEnvironments/ExcludedEnvironments) is producible only where HCLoader writes one, and only as a required environment. It writes a RootAllomorph's own environment (both sides, LoadRootAllomorph); one environment on a circumfix, whose output is a prefix piece, the stem and a suffix piece (LoadCircumfixAffixProcessAllomorph, left from the prefix's context and right from the suffix's); a right-side environment on a lone suffix; and a left-side environment on a lone prefix (LoadFormAffixProcessAllomorph). The other side of a lone prefix or suffix becomes part of its input pattern, not an environment. It never writes an ExcludedEnvironments, an environment on an infix or a reduplication, or one on a multi-action process rule (LoadAffixProcessAllomorph never touches .Environments). A subrule shaped like the circumfix, InsertSegments+CopyFromInput+InsertSegments with one required environment, is therefore producible; HC checks the same environment at each piece of that output.

Three more shapes are checked mechanically because they are plain attribute facts: a rule reached through a Slot must not have an outputPartOfSpeech (LoadInflAffixProcessRule never sets one); a partial stem entry must have no part of speech (HCLoader.cs:757, :817); and a partial rule must be in no slot (:1031) and have no outputPartOfSpeech. sillsdev/machine's FieldWorksProducibleShapeTests fails on a true fixture that brings any of them back.

These are rows of a wider census, fieldworks-writable-shapes.tsv. For each kind of rule, template, slot, stratum, stem name, entry and allomorph it lists every DTD attribute and element and says whether HCLoader.cs writes it (writes), only carries it as a label (metadata), never writes it (never), filters it out (omitted), or writes it where the engine reads nothing (inert); each row names the loader line. a sillsdev/machine test (FieldWorksProducibleShapeTests) fails a true fixture that uses a never member or one the table does not list, and checks what a row cannot say: an empty stem-name region, a required environment on any output shape but a lone prefix, a lone suffix or a circumfix, a subrule RequiredHeadFeatures on a circumfix or an affix process, a cap on an exocompound, and MPR features outside the three loader groups. A new construct goes into the table, with its loader line, before a true fixture may use it.

Because the verdict is authored, not derived, any mechanical checker built against it may only prove a true verdict wrong — by finding a construct in a true fixture's grammar.xml that matches a fieldworks-producibility.tsv "No" row or one of the mechanical checks above — never prove a false verdict wrong. A checker cannot see the absence of an HCLoader code path; it can only see that a construct is used, and that is sufficient to falsify true but never sufficient to certify it, since the research behind a false verdict is exactly the part no checker can redo by reading grammar.xml alone.

10. FieldWorks source witnesses

Section 9's fieldworks_producible: true is a claim about grammar.xml shape; it does not itself prove a real FieldWorks project can be authored and round-tripped to that shape. A fixture MAY carry a fieldworks/ directory alongside grammar.xml/words.yaml, holding the actual LibLCM project the claim is about:

• fieldworks/project.fwdata — a real FieldWorks project, backed out of the fixture's own grammar.xml via LibLCM (not hand-edited), and fieldworks/WritingSystemStore/*.ldml — the writing-system definitions that project needs to open. The .ldml files hold only the elements that define the writing system (identity, layout, characters, delimiters, collations, special); the CLDR locale data LibLCM copies in (dates, numbers, units, display names) is removed because nothing reads it, and a project with the reduced files opens and projects to the same HermitCrab input as the full ones.
• fieldworks/phonology-mutations.yaml — a small, versioned manifest of scripted phoneme-inventory counterfactuals against that same project, for a consumer that wants to probe what a narrower or wider phoneme inventory does to the project's own FieldWorks-side projections without hand-editing a project itself.

Only a fieldworks_producible: true fixture is eligible to carry a fieldworks/ directory — a false fixture has no real project behind it by construction, so there is nothing for one to contain.

This repository never opens, parses, or executes anything under fieldworks/: .fwdata is a FieldWorks/LibLCM format, and nothing in this suite's adapter protocol reads it. It exists for a consumer that already speaks that format (today, PanGloss's xample-projector tool, which stays in PanGloss) to read directly. What this repository DOES enforce is the directory's shape — see scripts/check_fwdata.py — and, since project.fwdata is opaque here, that enforcement is necessarily structural rather than semantic: file names, a schema version, and a hash, not a check that the project actually opens or actually contains what the manifest claims.

A consumer must verify phonology-mutations.yaml's base_sha256 against the actual SHA-256 of fieldworks/project.fwdata before trusting anything else in the manifest — the two files can drift independently (a fixture updated in place, a manifest hand-edited, a partial checkout), and a mutation op naming a guid that isn't even in the project it's about to run against is a worse failure mode than a mismatched hash caught up front.

phonology-mutations.yaml's v1 vocabulary is exactly:

• Two operations: remove_phoneme (guid, assert_representations, require_unreferenced) and remove_all_phonemes (require_unreferenced) — see PanGloss's xample-projector mutate command for what each does to a cloned copy of the project.
• Two expect fields whose only defined value is the literal string same_as_base: xample_projection and hc_analyses. same_as_base means equal up to LibLCM's own hvo renumbering (every M3-dump Id/dst attribute is a raw, per-load-session object handle — bMorphnameIsMsaId in FieldWorks' own XSL transforms — so two projections of two different LibLCM sessions are never byte-identical even when nothing meaningful changed), never byte-equality.
• expect.inferred_segments — the phoneme representations a consumer should expect to no longer be producible after the case's operations run.

A manifest field outside this vocabulary, or an expect.xample_projection/expect.hc_analyses value other than same_as_base, is not yet defined — a consumer encountering one should refuse rather than guess at a meaning this repository has not specified.