GH-48204: [C++][Parquet] Fix Column Reader & Writer logic to enable Parquet DB support on s390x

[Vishwanatha-HD]

Rationale for this change

This PR is intended to enable Parquet DB support on Big-endian (s390x) systems. The fix in this PR fixes the column reader & writer logic. Column Reader & Writer are the main part of most of the parquet & arrow-parquet testcases.

What changes are included in this PR?

The fix includes changes to following files:
cpp/src/parquet/column_reader.cc
cpp/src/parquet/column_writer.cc
cpp/src/parquet/column_writer.h

Are these changes tested?

Yes. The changes are tested on s390x arch to make sure things are working fine. The fix is also tested on x86 arch, to make sure there is no new regression introduced.

Are there any user-facing changes?

No

GitHub main Issue link: #48151

• GitHub Issue: [C++][Parquet] Fix Column Reader & Writer logic to enable Parquet DB support on Big-Endian (s390x) systems #48204

[github-actions]

⚠️ GitHub issue #48204 has been automatically assigned in GitHub to PR creator.

[kou]

Do we need this #if?

It seems that the below (*impala_timestamp).value[2] = static_cast<uint32_t>(julian_days); does the same thing.

[kou]

Can we use the following instead of #if?

auto last_day_nanos = last_day_units * NanosecondsPerUnit;
auto last_day_nanos_little_endian = ::arrow::bit_util::ToLittleEndian(last_day_nanos);
std::memcpy(impala_timestamp, &last_day_nanos_little_endian, sizeof(int64_t));

[kou]

Could you share implementation as much as possible something like:

template <>
struct SerializeFunctor<::parquet::FLBAType, ::arrow::HalfFloatType> {
  Status Serialize(const ::arrow::HalfFloatArray& array, ArrowWriteContext*, FLBA* out) {
#if ARROW_LITTLE_ENDIAN
    return SerializeLittleEndianValues(array.raw_values(), out);
#else
    const uint16_t* values = array.raw_values();
    const int64_t length = array.length();
    converted_values_.resize(length);
    for (int64_t i = 0; i < length; ++i) {
      // We don't need IsValid() here. Non valid values are just ignored in SerializeLittleEndianValues().
      converted_values_[i] = ::arrow::bit_util::ToLittleEndian(values[i]);
    }
    return SerializeLittleEndianValues(converted_values_.data(), out);
#endif
  }

 private:
  Status SerializeLittleEndianValues(const uint16_t* values, FLBA* out) {
    if (array.null_count() == 0) {
      for (int64_t i = 0; i < array.length(); ++i) {
        out[i] = ToFLBA(&values[i]);
      }
    } else {
      for (int64_t i = 0; i < array.length(); ++i) {
        out[i] = array.IsValid(i) ? ToFLBA(&values[i]) : FLBA{};
      }
    }
    return Status::OK();
  }

  FLBA ToFLBA(const uint16_t* value_ptr) const {
    return FLBA{reinterpret_cast<const uint8_t*>(value_ptr)};
  }

#if !ARROW_LITTLE_ENDIAN
  std::vector<uint16_t> converted_values_;  
#endif
};

[kou]

@pitrou You added - 4 in #6848. Do you think that we need - 4 with big endian too?