One suite of grammars, words and expected parses, so every FieldWorks parser engine can prove it gives the same answers. A linguist's project should parse a word the same way whichever engine runs it. This suite checks that it does.
135 conformance grammars (6 languages, 129 edge cases) · 16 stress grammars · 10 FieldWorks witnesses
A grammar is here only if a real FieldWorks project can produce it. FieldWorks writes less than the grammar format can say, so anything it cannot write stays out. What FieldWorks cannot say, construct by construct.
- Hermit Crab: rule-based and item-and-process. Morphological and phonological rules applied in order, with a C# engine in sillsdev/machine. This suite is written in its format (
grammar.xml,HermitCrabInput.dtd). It needs a complete phonology: every letter of every form and word defined as a segment. - XAmple: FieldWorks' other parser, in the AMPLE family, and a different algorithm (item-and-arrangement). It works on letters and ignores phonological features and rules, so any grammar with neither is in the XAmple subset.
A FieldWorks project switches between the two with one setting (ActiveParser).
| Engine | Phonology it needs | Grammars |
|---|---|---|
| XAmple | None: letters only; it ignores features and rules | 45 conformance + 6 stress, the XAmple subset |
| Hermit Crab (machine) | Complete: every letter defined as a segment | all 135 + 16 |
| PanGloss | None, incomplete or complete | all 135 + 16 |
The suite has no grammar with incomplete phonology yet.
| Project | What it is | What it gets from the suite |
|---|---|---|
| FieldWorks (software.sil.org/fieldworks) | The linguists' desktop app. It stores the grammar and runs either parser. | It does not run the suite; it sets the boundary. Every fixture must be something it can write, and witnesses are real .fwdata projects built from fixtures. |
| machine | The C# Hermit Crab engine, and the reference that produced the expected parses. | hc-conformance runs the whole suite in machine's CI. |
| PanGloss | A Rust parser ported from Hermit Crab. | It runs the same fixtures through its own runner and must match. |
And XAmple? The suite has no XAmple expected answers, so it does not test XAmple. XAmple runs on the subset.
PanGloss's xample-projector turns a FieldWorks witness into both Hermit Crab and XAmple input and drives the real XAmple engine. That is how the two are compared today.
git clone https://github.com/sillsdev/fieldworks-parser-conformance
cd fieldworks-parser-conformance
python -m pip install -r scripts/requirements.txt
python scripts/generate_manifest.py --check
python scripts/check_grammars_md.py
An engine consumes the suite by pinning a commit in conformance.commit and cloning it into an ignored conformance/.
Set HC_CONFORMANCE_SUITE=<path> to use a local copy. Details: docs/consuming.md.
| If you want to | Go to |
|---|---|
| Learn the fixture format | docs/what-it-is.md |
| Browse the grammars | docs/grammars.md |
| Write an engine adapter | PROTOCOL.md |
| Consume the suite | docs/consuming.md |
| Contribute | CONTRIBUTING.md |
| Time an engine | stress/README.md |
| Look up a term | docs/glossary.md |
| See what FieldWorks cannot say | docs/fieldworks-cannot-say.md |
| Trace where results came from | PROVENANCE.md |
| Find every document | docs/README.md |
MIT. See LICENSE.