Vertical CJK writing mode (tategaki) unsupported — reading order shredded (CER 0.96)

[yfedoseev]

Summary

pdf_oxide does not support vertical writing mode (tategaki). Vertical CJK pages extract every glyph correctly but in shredded reading order. On pdf_benches: vertical-cjk-ja CER 0.962, vertical-cjk-zh 0.958 (pypdfium2 ~0.10–0.20). Horizontal CJK (wiki-cat-zh) is fine at 0.059.

GOLD : 縦書きの日本語テキストです。右から左へ列が進みます。
PDFOX: すす縦。。書右きかのら日左本へ語列テがキ進スみトまで

All glyphs present; the Y-band sort reads one horizontal slice across all columns as a "row".

Root cause: the feature is entirely absent

• No vertical reading-order strategy. src/pipeline/reading_order/mod.rs:13-28 ships only StructureTree, Geometric, XYCut, Simple — all horizontal. There is no tategaki/VerticalStrategy.
• No writing-mode detection. No WMode/writing_mode/DW2/W2 handling in the text pipeline. Identity-V is treated identically to Identity-H (just a CID encoding) in src/fonts/font_dict.rs:1895,3324 — never a layout signal.
• ReadingOrderContext (mod.rs:66-95) has no writing-mode field.
• XY-cut is hardwired horizontal (xycut.rs:120-128); row_aware_span_cmp (src/lib.rs:408-424) sorts Y-band then X-ascending.
• These specific PDFs carry no Identity-V/WMode marker — the vertical layout is built by explicit per-glyph positioning (fixed X, descending Y, columns right-to-left), so detection must be geometric.

Proposed fix (deferred — new feature, ~3–5 days)

1. Detection (~1d): vertical mode from Type0 Identity-V/CMap WMode 1 AND a geometric detector (glyph runs sharing X with monotonically decreasing Y; columns ordered by descending X).
2. Strategy (~1–2d): a VerticalStrategy banding by X (right→left) then Y descending within each column — row_aware_span_cmp_rtl (src/lib.rs:438, added for Arabic RTL extraction garbled across text/md/HTML (CER 0.94 vs poppler 0.16) — over-reversal + combining-mark detachment #656/Hebrew RTL extraction word-order reversed across text/md/HTML (CER 0.71 vs pdfbox 0.00) #657) is a close structural template (swap axes, reverse X).
3. Wiring (~1d): writing-mode field on ReadingOrderContext, auto-select, honor in StructureTreeStrategy's single-MCID fallback (structure_tree.rs:23,146). + tests + the two probe docs as fixtures.

A cheap partial win (geometric "is vertical?" + swapped-axis comparator) would likely drop CER from ~0.96 toward ~0.10–0.20 without font/CMap work.

Found via pdf_benches corpus/A_golden/probes/vertical-cjk-{ja,zh}.

[yfedoseev]

Code analysis — vertical CJK (docs/spec/pdf.md):

• Detection + routing already exist and are correct: /WMode parsing src/fonts/cmap.rs:698; predefined Identity-V/*-V src/fonts/font_dict.rs:295; font.wmode set :1159, propagated to spans extractors/text.rs:4746-4755; tategaki dispatched in pipeline/mod.rs:116, text.rs:3727, document.rs:9583-9609. tategaki.rs is reachable + tested.
• Real root cause: (1) span assembly is horizontal-only — flush_tj_buffer builds bbox as width=advance, height=font_size (text.rs:6538-6542) and the merge stage uses same_line=y_diff<1.0, gap along x (text.rs:4147-4167). Vertical glyphs (differing y, same x) never merge into a column → vertical structure shredded before sorting. (2) the column comparator if |ax-bx|<=tol {by y} else {by x} (tategaki.rs:51, dup in text.rs:3754, document.rs:3599) is intransitive → arbitrary sort_by order.
• Spec: §9.7.5.3 (/WMode 0/1), §9.4.3 (vertical displacement w1, /DW2,/W2 §9.7.4.3).
• Fix: make flush_tj_buffer/merge vertical-aware for wmode==1 (accumulate advance into height, width=font_size, "same column" = x_diff<tol, gap along y); replace the band-compare with a transitive key (cluster x-centers → column rank desc, then y desc); dedupe the 3 copies via TategakiStrategy (and add a Tategaki arm to create_strategy).