Skip to content

[chore] Implement datatype conversion for all types in arrow.rs#81

Merged
luoyuxia merged 6 commits into
apache:mainfrom
beryllw:pr-to_arrow_type
Dec 12, 2025
Merged

[chore] Implement datatype conversion for all types in arrow.rs#81
luoyuxia merged 6 commits into
apache:mainfrom
beryllw:pr-to_arrow_type

Conversation

@beryllw
Copy link
Copy Markdown
Contributor

@beryllw beryllw commented Dec 8, 2025

Purpose

Linked issue: close #78

Brief change log

Tests

API and Format

Documentation

Comment thread crates/fluss/src/record/arrow.rs Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements Arrow datatype conversions for previously unimplemented Fluss types (Decimal, Time, Timestamp, TimestampLTz, Bytes, Binary) in the to_arrow_type function. This completes the type mapping functionality needed for Arrow serialization/deserialization, addressing issue #78.

Key changes:

  • Adds Decimal128 conversion with precision/scale mapping
  • Implements Time/Timestamp type conversions with precision-based TimeUnit selection
  • Adds Binary and Bytes type mappings to Arrow binary types

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs Outdated
Comment thread crates/fluss/src/record/arrow.rs
Copy link
Copy Markdown
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@beryllw Thanks for the pr. Left minor comments.

Comment thread crates/fluss/src/record/arrow.rs Outdated
.expect("length exceeds i32::MAX"),
),
DataType::Array(arrow_type) => ArrowDataType::List(
arrow_schema::Field::new_list_field(to_arrow_type(arrow_type.get_element_type()), true)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
consider nullable of arrow_type

Comment thread crates/fluss/src/record/arrow.rs Outdated
let key_type = to_arrow_type(map_type.key_type());
let value_type = to_arrow_type(map_type.value_type());
let entry_fields = vec![
arrow_schema::Field::new("key", key_type, false),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dito

@luoyuxia
Copy link
Copy Markdown
Contributor

@beryllw I append a small fix. Will merge if ci pass

@luoyuxia luoyuxia merged commit 3037192 into apache:main Dec 12, 2025
13 checks passed
@beryllw
Copy link
Copy Markdown
Contributor Author

beryllw commented Dec 13, 2025

I append a small fix.

Thanks for fixing the issue I overlooked.

@beryllw beryllw deleted the pr-to_arrow_type branch April 29, 2026 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement datatype conversion for all types in arrow.rs

3 participants