Skip to content

[parser] Header/prose boundary residues from the #811 lane — six items in three files #843

Description

@s-annam

Six residues left by the #811 parser lane (PR #842), all on the same seam: where a header line ends and prose begins. They live in three files — src/lib/heuristics/line-primitives.ts, src/lib/heuristics/extract/experience-disambiguate.ts, src/lib/heuristics/sections.ts — and share enough reasoning that splitting them into six issues would cost more overhead than the fixes.

All six were surfaced by the adversarial review on #842 and are documented in that PR's ## Adversarial review section. None is a regression from #842 — each is either a pre-existing gap that review made visible, or a deliberately-bounded residue of a fix that landed there.

Two are one-line changes the reviewer already verified.


1. HEADER_CONNECTOR_WORDS omits non-English particles

looksLikeVerbLedScope (line-primitives.ts) classifies a below-anchor line as prose when it leads with an ACTION_VERBS token and carries a lowercase content word. The connector list that excludes function words is English-only, so nobiliary/locative particles read as content words:

"Unified Communications de Mexico"  → prose  (company lost)
"Automated Logic van Nuys"          → prose  (company lost)

Adding de, del, la, van, von, di, da, du, der, y to the reject list is monotone-safe by the list's own design — it only ever makes the predicate more conservative. The reviewer could not construct a likely real name that hits this (it also needs an ACTION_VERBS lead), so this is low-frequency, but the fix is nearly free.

2. The word-strip eats digits off ordinals

Same predicate. The token strip is word.replace(/^[^\p{L}]+|[^\p{L}]+$/gu, ""), which keeps only letters — so 3rd → "rd", 1st → "st", 2nd → "nd", all of which then pass as lowercase content words:

"Managed 1st Choice Health"  → prose

One-character fix, verified by the reviewer against a token battery (budget,, Inc., P&L,, (the, 24/7, e-commerce, O'Brien all unchanged):

const bare = word.replace(/^[^\p{L}\p{N}]+|[^\p{L}\p{N}]+$/gu, "");

Arguably "Won 3rd Place Hackathon" should read as prose, so confirm the intent before changing it.

3. #708's scope-line coverage is narrower than its docblock implies

The content-word rule (chosen over ≥2 lowercase words, which fails on "Secured Lending of the Midwest") means a scope line whose only lowercase words are connectors is not caught. Measured 6 of 12 realistic scope lines missed:

MISSED  "Led the Payments Platform for the Americas"
MISSED  "Owned the Global Risk and Compliance Portfolio"
MISSED  "Managed the EMEA Sales Organization"
MISSED  "Built the Data Platform for Enterprise"
MISSED  "Drove the Cloud Migration across Europe"
CAUGHT  "Led a team of twelve engineers"

This fails closed — missed lines land in team exactly as they did before #708 — so it is a residue, not a regression, and #708's two stated AC shapes are both still caught. The docblock cites one example where the class is roughly half the Title-Cased scope register. Either widen the rule or make the docblock honest about the size.

4. A 3-part trailing location cell surfaces a city as company

experience-disambiguate.ts, delimiter branch:

"Data Analyst, Northwind Retail Co. | Bengaluru, KA, India"
  → company: "Bengaluru, KA"   team: "Northwind Retail Co."   location: "India"

Two-part trailing cells (| Austin, TX, | Bengaluru, India) are strict improvements after #842; this is specific to the 3-part form. HEAD was also wrong here, but differently — it produced an obviously-garbled title, whereas this produces a confidently wrong company, which is worse to read.

5. INSTITUTION_HINTS is un-anchored, so institution-named employers are rejected

sections.ts guard 5 of looksLikeHeaderlessRoleHeader tests INSTITUTION_HINTS (University|College|Institute|School|Academy|Polytechnic) un-anchored over the head, to keep education clusters from opening an experience section. Real employers collide:

"Research Engineer, Stanford University (Sep 2018 - Jun 2021)"
"Content Lead, Khan Academy (Jul 2021 - Present)"
  → cluster recovers 0 roles

The identical cluster at Northwind Systems / Contoso Labs recovers 2. Fail-closed, so this is recall loss, not corruption. #842 corrected the docblock to state the trade honestly but deliberately did not change behaviour — anchoring the institution half the way the degree half is anchored is its own judgement call and wants its own repro. (The degree half was already fixed in #842: DEGREE_RE un-anchored collided with the state codes MA/MD/MS/ME, silently rejecting every Massachusetts/Maryland/Mississippi/Maine role header.)

6. Four fixtures still parse an empty title behind the delimiter branch

#543 fixed the undelimited Title, Company, Location shape and #842 extended the re-split to segment 0 of a 2-segment delimited line. Lines whose first segment carries a comma but which split into 3+ segments are untouched by design (the naive version breaks "Sr. Engineering Manager · Site Lead, Payments Platform · Globex, Toronto").

pdflib-leading-glyph-skills-header.pdf is now fixed. These three, named in #543's own comment thread, were never individually checked and are likely the same class:

  • google-docs-skia-proxy-multiline-bullets-coursework.pdf — role[2]
  • student-projects-activities-singlecol.pdf — role[2]
  • openresume-laverne-word-quartz.pdf — role[1]

Sweep them and confirm before assuming.


Acceptance criteria

  • Items 1 and 2 fixed (both one-line, both pre-verified) with a test pinning each boundary.
  • Item 3 resolved either way — widen the predicate, or correct the docblock to state the real size of the missed class. Do not leave the docblock implying it is one example.
  • Item 5 resolved either way, same standard, with a repro fixture if behaviour changes.
  • Items 4 and 6 investigated; each either fixed with a fixture, or closed with a written reason.
  • No existing corpus baseline moves except where a fix is justified field-by-field.
  • No new knownWrong or corpus-roundtrip.known-failures.json entry. If one seems necessary, measure the alternative first — fix(heuristics): recover dropped roles, titles and skills sections #842 removed one whose "too risky to fix" justification measured false (1 of 60 fixtures, the target, improved).

Notes

Deliberately not in scope, and deliberately not filed separately: #492's deferred experience-no-section defect class. It needs a new DerivedSignals key, which is baked into all 60 .expected.json files and would bump CORPUS_SNAPSHOT_SCHEMA_VERSION. Only worth doing batched with another schema bump — surface it then.

Also out of scope: roleFromSection is still duplicated in 12 extract/ test files (pre-existing; #842 unified two of them). Fold it in opportunistically when touching one of those files.

Refs #811

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingux:parsingUX program: parsing accuracy as the user experiences it

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions