Extraction of data from table is not accurate

[harrier1290]

Bug

I am trying to extract data from the table from research article and the results are not accurate. I have similar observations with other tables also. Often some values are missing and often some special characters like '=' and '~' are added. Sometimes, number '0' is misinterpreted as letter 'o'.

I tried with tesseractocr also, to rule out the role of EasyOCR engine, but things got worse with tesseract ocr.

Is it known bug in Docling, while extracting the values from tables?
...

Steps to reproduce

Table from paper was used for extracting data https://doi.org/10.1016/j.jpcs.2024.112412
...

Docling version

2.44.0
...

Python version

3.12.3...

Please find the pdf showing the lose of values while extraction of data

[dosubot]

Hi @harrier1290! I'm Dosu and I’m helping the docling team.

Yes, this is a known issue. Table extraction in Docling (including v2.44.0) can result in missing values, misinterpreted characters (like '0' as 'o'), and extra special characters, especially when using OCR engines such as EasyOCR or Tesseract. These problems are widely reported and are mostly due to OCR limitations and the TableFormer model sometimes dropping or merging cell texts during prediction, not a specific Docling bug in the extraction logic itself [source][source].

You can try to improve accuracy by setting do_table_structure=True (default), using table_structure_options={"mode": "accurate", "do_cell_matching": True}, and experimenting with both FAST and ACCURATE modes. Upgrading docling-ibm-models to 3.8.1+ can fix some bugs with multi-line cells [source]. For Markdown export, switch to MarkdownTableSerializer if you have single-column tables, or extract tables directly to Pandas or JSON for better detail preservation.

If you want to debug further, use Docling's visualization options to inspect detected tables and cells on the page image. Ultimately, persistent errors like character confusion or missing values are inherent to OCR and table structure modeling, so custom post-processing may be needed for critical use cases [source].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[harrier1290]

@dosu Thanks for your suggestions. I tried all the options, unfortunately I could not find the solution to my problem. Maybe its inherent to OCR engines.

However, I could not convert to pandas, can you suggest me a way to do it? Saving output as json, doesnot help

[dosubot]

You can convert Docling-extracted tables directly to pandas DataFrames using the docling_core library. If you have a Docling-generated JSON file, load it as a DoclingDocument, then access the tables attribute—each table is a TableItem object with an export_to_dataframe() method. Here’s a typical workflow:

from docling_core.types.doc.document import DoclingDocument
import pandas as pd

# Load your Docling JSON output
doc = DoclingDocument.load_from_json("your_docling_output.json")

# Iterate through tables and convert to pandas DataFrames
for i, table in enumerate(doc.tables):
    df = table.export_to_dataframe()
    print(f"Table {i} as DataFrame:")
    print(df)

This method handles multi-row headers and edge cases like empty tables or missing headers. Make sure you have both docling_core and pandas installed. If you don’t see TableItem or export_to_dataframe(), check that you’re using docling_core ≥2.44.0 and that your JSON was generated by Docling’s pipeline. Let me know if you hit any errors or if your workflow is different!
Reference: docling_core TableItem.export_to_dataframe()

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[harrier1290]

@dosu
I am sorry to share that neither of the solutions helped. Is there any way we can improve the accuracy of text extraction from scanned /pictures of tables?

[harrier1290]

@dosu

Any update for me?

[dosubot]

Here’s the latest on improving table extraction accuracy from scanned or photographed documents in Docling:

Most extraction errors—missing values, misinterpreted characters, and special symbols—are due to OCR engine limitations and the TableFormer model sometimes dropping or merging cell texts during prediction. This is a widely reported issue and not a specific bug in Docling’s extraction logic [source][source].

You can try these practical steps for better results:

• Image preprocessing before OCR can make a big difference. Rescale images to a higher DPI (e.g., 300), convert to grayscale, denoise, and apply adaptive thresholding. Here’s a Python example using Pillow and OpenCV:

  from PIL import Image
  import cv2
  import numpy as np
  
  def rescale_image_to_dpi(input_path, output_path, target_dpi=300):
      img = Image.open(input_path)
      original_dpi = img.info.get("dpi", (72, 72))[0]
      scale_factor = target_dpi / original_dpi
      new_size = (int(img.width * scale_factor), int(img.height * scale_factor))
      rescaled_img = img.resize(new_size, resample=Image.LANCZOS)
      rescaled_img.save(output_path, dpi=(target_dpi, target_dpi))
      return rescaled_img
  
  def preprocess_image_for_ocr(pil_image):
      img = np.array(pil_image)
      gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
      denoised = cv2.bilateralFilter(gray, 9, 75, 75)
      thresh = cv2.adaptiveThreshold(
          denoised, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 25, 15
      )
      return Image.fromarray(thresh)

  [source]

• Set explicit DPI metadata when converting PDFs to images for OCR. For example, with ImageMagick:

  convert -density 300 input.pdf output.png
  

  This helps Tesseract recognize the correct resolution and improves OCR accuracy [source].

• Pipeline options: Try switching between TableFormerMode.ACCURATE and FAST, and toggle do_cell_matching (sometimes setting it to False improves column alignment for complex layouts) [source][source].

• Debugging: Use Docling’s visualization options (--debug-visualize-layout) to inspect detected tables and cells on the page image. This can help you diagnose why tables aren’t recognized or why values are missing [source].

• Custom post-processing: For critical use cases, consider writing a script to compare the raw OCR cell content with the TableFormer output and recover dropped lines or missing values [source].

There’s ongoing work to improve table extraction, including leveraging word-level bounding boxes and synthetic training data, but for now, image quality and preprocessing are the most effective ways to boost accuracy. Let me know if you want more details on any of these steps!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}