Proposal: fill flat/scanned forms (no AcroForm fields) via printed-rule detection

[charles-hood]

The gap

Forms v1 (#15) works off Widget annotations, so it covers PDFs with real AcroForm fields. A large share of real-world forms have none: scanned applications, government/HOA forms, and bank paperwork are often just printed rule lines and boxes. Text elements (#17) make it possible to fill these by hand, but the user has to eyeball every position.

Since pdfx already rasterizes pages and runs OCR off the main thread, the missing piece is small: detect the blank answer lines on the rasterized page and surface them as suggested field boxes. Those boxes could feed the existing text-element pipeline, and could also be exposed to the AI assistant as a tool alongside fill_form_field and add_text ("fill in the name line with ...").

Detection algorithm

I have a working Python implementation of this that I use regularly (raster in, PDF-point field boxes out). It is small and has survived contact with real scanned forms. The approach:

1. Rasterize the page (e.g. 2x) and threshold to a dark-pixel mask.
2. For each pixel row, collect every contiguous dark horizontal run above a minimum length. Emit all qualifying runs per row, not just the longest, so a row with two blanks ("City ____ State ____") yields two fields.
3. Bridge interior gaps below ~4 pt so dashed/dotted rules read as one continuous line. Only interior gaps: never extend leading/trailing edges.
4. Gate every bridged run by thinness: measure the dark band height around the candidate row and reject anything that isn't a thin horizontal stroke. This is what keeps a row of ordinary text (a title, a bold label) from bridging into a false rule. It was the main source of false positives before the gate.
5. Cluster adjacent pixel rows (a 1-2 pt rule spans several rows at 2x) into one rule.
6. Classify long full-width runs as table/box borders rather than answer blanks; multi-cell rows from step 2 usually disambiguate the rest.
7. Convert to PDF points, remembering the y-axis flip between image space and PDF user space.

One placement rule matters more than everything else: put the text baseline a few points above the detected rule, so typed text rests on the line instead of striking through it.

Checkbox cells fall out of the same pass (short thin runs forming small squares), though my implementation handles those with a manual calibration grid rather than auto-detection.

What I'm asking

Does this fit your direction for Forms v2? If yes, I'm happy to either:

• port it to TypeScript and submit a PR (it operates on the same rasterized bitmaps your OCR path already produces), or
• write it up in more detail as a spec and share the Python reference implementation, and you fold it in however fits the codebase best.

No hard feelings either way if it's out of scope.

[AlexandrosGounis]

Hi @charles-hood, this is a great suggestion.

Before we proceed, could you please write it up in more detail and share your Python implementation (as a Gist or repository)?

Ideally, we would want to rely less on LLMs to identify form fields and leverage on-device processing (similar to how we do OCR off the main thread). However, even full LLM processing for forms/v2 is perfectly fine for now. We'll have to do some manual calibration, so no worries at all.

Thank you for your time and for improving PDFx with your suggestions.