Skip to content

Commit 1bdab5c

Browse files
committed
fix(extract): keep dotted numbers intact when segmenting a source
segment_source split on every `.`, so a version or decimal — 6.8.3, 3.14 — was fractured across segment boundaries and the number-valued answer atom fell out of every quotable span. on a synthetic lookup corpus that dropped ~11% of the ground-truth facts before any budget was applied: the answer simply was not present in any claim to retrieve. a `.` flanked by digits is a decimal/version dot, never a sentence boundary, so the segment regex now keeps it inside the span (sentence-ending periods are unaffected). measured recall ceiling on that corpus rises from 89% to 100% of facts — a cap that bounded every downstream compiler, not just this one.
1 parent e8dd59a commit 1bdab5c

3 files changed

Lines changed: 26 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,15 @@ All notable changes to vouch are documented here. Format follows
7878
this is the selection step the compiler thesis needs: fewer, denser claims
7979
are what move accuracy-per-token against the grep baseline.
8080

81+
### Fixed
82+
- **extraction no longer fractures dotted numbers.** `segment_source` split
83+
on every `.`, so a version or decimal (`6.8.3`, `3.14`) was broken across
84+
segment boundaries and its answer atom fell out of every span — measured at
85+
~11% of the ground-truth facts lost on a synthetic lookup corpus *before any
86+
budget was applied*. a `.` flanked by digits is now kept inside the span
87+
(sentence-ending periods are unaffected), lifting the recall ceiling of the
88+
whole ingest pipeline from 89% to 100% of facts on that corpus.
89+
8190
## [1.5.0] — 2026-07-20
8291

8392
### Added

src/vouch/extract.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,11 @@
2828

2929
# Split on newlines and sentence-ending punctuation. Each match is kept as an
3030
# exact substring of the source so its receipt verifies; only surrounding
31-
# whitespace is stripped (the inner run stays contiguous in the source).
32-
_SEGMENT_RE = re.compile(r"[^\n.!?]+[.!?]?")
31+
# whitespace is stripped (the inner run stays contiguous in the source). A `.`
32+
# flanked by digits is a decimal/version dot ("6.8.3", "3.14"), never a
33+
# sentence boundary — keep it inside the span so the number-valued fact isn't
34+
# fractured out of every claim.
35+
_SEGMENT_RE = re.compile(r"(?:[^\n.!?]|(?<=\d)\.(?=\d))+[.!?]?")
3336

3437
DEFAULT_MIN_CHARS = 16
3538
DEFAULT_MAX_CHARS = 320

tests/test_extract.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,18 @@ def test_segment_source_splits_into_verbatim_spans() -> None:
4040
assert s in text
4141

4242

43+
def test_segment_source_keeps_dotted_numbers_intact() -> None:
44+
# a period between two digits is a decimal/version dot, never a sentence
45+
# end — splitting on it fractures the answer atom out of every span.
46+
text = (
47+
"The deployed build of Harbor Digest is version 6.8.3 today. "
48+
"The measured value of pi is about 3.14 in this document."
49+
)
50+
segs = extract.segment_source(text)
51+
assert any("version 6.8.3 today" in s for s in segs)
52+
assert any("3.14" in s for s in segs)
53+
54+
4355
def test_segment_source_drops_short_noise_and_dupes() -> None:
4456
text = (
4557
"ok. The very same sentence appears here twice in a row now. "

0 commit comments

Comments
 (0)