Background
DatabricksStatement.SetStatementProperties currently sets IntervalTypesAsArrow = false unconditionally, meaning the Thrift server returns interval values as strings rather than native Arrow interval types.
JDBC (DatabricksThriftServiceClient.java) enables IntervalTypesAsArrow = true for protocol V5+ servers alongside the other advanced Arrow types.
Work
- Implement client-side handling for native Arrow interval types (analogous to how
ComplexTypeSerializingStream handles ARRAY/MAP/STRUCT).
- Once handling is in place, set
IntervalTypesAsArrow = advancedArrowTypes in FeatureVersionNegotiator-gated block in DatabricksStatement.SetStatementProperties (same pattern as DecimalAsArrow / ComplexTypesAsArrow).
- Add E2E test coverage for interval columns under both Thrift and SEA paths.
References
csharp/src/DatabricksStatement.cs — SetStatementProperties (currently hardcodes IntervalTypesAsArrow = false)csharp/src/FeatureVersionNegotiator.cs — SupportsAdvancedArrowTypes (V5+)- JDBC:
DatabricksThriftServiceClient.java lines 245–251
Background
DatabricksStatement.SetStatementPropertiescurrently setsIntervalTypesAsArrow = falseunconditionally, meaning the Thrift server returns interval values as strings rather than native Arrow interval types.JDBC (
DatabricksThriftServiceClient.java) enablesIntervalTypesAsArrow = truefor protocol V5+ servers alongside the other advanced Arrow types.Work
ComplexTypeSerializingStreamhandles ARRAY/MAP/STRUCT).IntervalTypesAsArrow = advancedArrowTypesinFeatureVersionNegotiator-gated block inDatabricksStatement.SetStatementProperties(same pattern asDecimalAsArrow/ComplexTypesAsArrow).References
csharp/src/DatabricksStatement.cs—SetStatementProperties(currently hardcodesIntervalTypesAsArrow = false)csharp/src/FeatureVersionNegotiator.cs—SupportsAdvancedArrowTypes(V5+)DatabricksThriftServiceClient.javalines 245–251