Support pyarrow.date32 ingest

[mauropagano]

1. Describe your new request in detail

Support ingesting pyarrow.date32 column type.
This would be really useful because dates without a time component are read as such by Arrow.
Right now the only workaround is to explicitly cast the table before passing it to oracledb

import oracledb
import pyarrow as pa
import datetime
  
a = pa.array([datetime.date(2025, 11, 1)], type=pa.date32())
t = pa.Table.from_arrays([a], names=["a"])
# pyarrow.Table
# a: date32[day]
# ----
# a: [[2025-11-01]]
conn = oracledb.connect("...")                                                                                                                                                                                                                                                                                                                                   
cursor = conn.cursor()
cursor.executemany("insert into insert_pq (start_month) values (:1)", t)
# oracledb.exceptions.NotSupportedError: DPY-3032: conversion from Arrow format "tdD" to Oracle Database is not supported

2. Give supporting information about tools and operating systems. Give relevant product version numbers
   Not applicable

[cjbj]

@mauropagano how's other testing with direct binding of dataframes to executemany() going?

[mauropagano]

Pretty great!!

I don't have a 100% apple to apple yet (small things like this one force me to diverge a bit) but "bytes on disk to bytes in Oracle" I'm seeing between 2x and 4x in elapsed time, depending on input files, mostly because I can do a lot less massaging on inputs before passing data to oracledb

Biggest gain in pure oracledb is CPU/memory, elapsed seem to be around 30-50% but that's a little harder to measure. Because of reduced resource, I can parallelize more on the client side, more or less linearly reducing overall elapsed (assuming backend can keep up).

[cjbj]

@mauropagano that's great to hear.

@anthony-tuininga is working on Direct Path Loading for the next release (due soonish), which adds another bump in speed for large data sets. As long as the DB restrictions on Direct Path Loads aren't an issue for the schema, it will be very useful. We need to do a bit more testing before making a development build available on GitHub.

cjones@mac:~/p/directpath$ python createcsv.py 500000

cjones@mac:~/p/directpath$ python loadcsv.py 100000
Loading with executemany()
Wrote in 5 batches of size 100000
Total elapsed time: 4774.4 ms
500000 valid rows were inserted

Direct Path Load
Wrote in 5 batches of size 100000
Total elapsed time: 2811.8 ms
500000 rows were inserted

cjones@mac:~/p/directpath$ python createcsv.py 5000000

cjones@mac:~/p/directpath$ python loadcsv.py 1000000
Loading with executemany()
Wrote in 5 batches of size 1000000
Total elapsed time: 50217.1 ms
5000000 valid rows were inserted

Direct Path Load
Wrote in 5 batches of size 1000000
Total elapsed time: 19164.1 ms
5000000 rows were inserted

[mauropagano]

Oh nice! That is VERY exciting and would probably be the default way to go in my libraries.

That would make concurrent inserts on the client side (that can't do DPL since you need a table level lock, unless you use extended partition syntax, I believe) appealing in even less cases, love it!!

Thank you for all your work!

[anthony-tuininga]

I have pushed a patch that should implement this enhancement and have initated a build from which you can download pre-built development wheels once it completes. You can also build from source if you prefer. If you can test your scenario and confirm the patch works as expected, that would be appreciated!

[mauropagano]

That was really quick!! :-)

Out of curiosity, is DPL close enough that will make into the next release (where I guess this fix as well will go)?
Asking so I can plan the next round of changes to my libraries.

[mauropagano]

Btw patch worked great, thanks so much for the quick turnaround!

[cjbj]

@mauropagano While DPL is a goal for 3.4, I'm not committing to anything because it needs some tweaking, and our testers haven't pounded on it yet. Note that it is a Thin mode -only feature.

[mauropagano]

Completely understand! and no problem at all on the thin mode, at least for me that's not a deal breaker