Skip to content

feat: Excel IO — from_excel/2 and to_excel/2 - #23

Merged
cigrainger merged 2 commits into
mainfrom
cg/excel-io
Mar 24, 2026
Merged

cigrainger merged 2 commits into
mainfrom
cg/excel-io

Conversation

@cigrainger

Copy link
Copy Markdown
Contributor

Summary

Add Excel (.xlsx) read and write support via DuckDB's read_xlsx and COPY TO XLSX.

Usage

# Read
df = Dux.from_excel("sales.xlsx")
df = Dux.from_excel("data.xlsx", sheet: "Q1 2024", range: "A1:F100")
df = Dux.from_excel("messy.xlsx", all_varchar: true)

# Write
Dux.from_list([%{name: "Alice", age: 30}])
|> Dux.to_excel("/tmp/output.xlsx")

Safe defaults for messy spreadsheets

DuckDB infers Excel column types from the first data row only. A NULL or empty cell in row 1 causes the column to be inferred as DOUBLE, which fails when later rows contain strings. Two safety nets enabled by default:

  • ignore_errors: true — type-cast failures become NULL instead of raising
  • empty_as_varchar: true — empty/null cells infer as VARCHAR instead of DOUBLE

Together these handle the most common "messy spreadsheet" scenarios. Users who want strict type checking can pass ignore_errors: false, empty_as_varchar: false. Users who want full control use all_varchar: true.

Uses Keyword.validate! for option checking — typos like headers: instead of header: raise immediately.

Test plan

Happy path (3):

  • Read basic xlsx with column names
  • Read with all_varchar: true
  • Read → filter pipeline composition

Happy path — writing (3):

  • Write basic xlsx
  • Write → read round-trip with types
  • Pipeline before write

Sad path (2):

  • Non-existent file raises
  • Invalid write path raises

Adversarial (5):

  • Special characters survive round-trip
  • NULL in first row handled by ignore_errors (no crash)
  • NULL in first row + all_varchar: true preserves all data
  • ignore_errors: false + empty_as_varchar: false raises on type mismatch
  • Empty dataframe writes valid xlsx
  • 50-column wide table round-trip

Scale (1):

  • 1000-row round-trip with SUM verification

🤖 Generated with Claude Code

cigrainger and others added 2 commits March 24, 2026 14:15
Add from_excel/2 (read_xlsx with header/sheet/range/all_varchar opts)
and to_excel/2 (COPY TO XLSX with header). Requires DuckDB 1.5+.

from_excel defaults to header: true so column names are read from the
first row. to_excel always writes headers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add from_excel/2 (read_xlsx) and to_excel/2 (COPY TO XLSX).

Defaults to ignore_errors: true and empty_as_varchar: true for safe
handling of messy spreadsheets where type inference from the first row
would fail. Uses Keyword.validate! for option checking.

Options: sheet, range, header (default true), all_varchar,
ignore_errors (default true), empty_as_varchar (default true),
stop_at_empty.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cigrainger
cigrainger merged commit 3fa3297 into main Mar 24, 2026
5 checks passed
@cigrainger
cigrainger deleted the cg/excel-io branch March 24, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant