Skip to content

gpfdist: set default compression level to 3 #1195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

gpfdist: set default compression level to 3 #1195

wants to merge 1 commit into from
+1 −1

Conversation

yjhjstz
Copy link
Member

@yjhjstz yjhjstz commented Jun 26, 2025 •
edited
Loading

🔍 Test Configuration

  • Test tool: wrk
  • Test parameters: 4 threads, 10 connections, 15s duration

📊 Complete Performance Comparison Table

Configuration RPS Relative Baseline Gain Avg Latency P50 Latency P90 Latency P99 Latency Transfer Rate
No Compression (Baseline) 12,151.35 0.00% 572.99μs 563.00μs 657.00μs 748.00μs 2.19MB
Compression Level 1 12,169.58 +0.15% 583.22μs 559.00μs 654.00μs 773.00μs 2.20MB
Compression Level 2 12,237.98 +0.71% 565.10μs 562.00μs 652.00μs 719.00μs 2.21MB
Compression Level 3 12,254.67 +0.85% 571.59μs 559.00μs 652.00μs 725.00μs 2.21MB
Compression Level 4 12,248.22 +0.80% 573.53μs 556.00μs 651.00μs 748.00μs 2.21MB
Compression Level 5 12,067.48 -0.69% 580.70μs 566.00μs 661.00μs 754.00μs 2.18MB
Compression Level 6 12,185.62 +0.28% 575.21μs 558.00μs 653.00μs 753.00μs 2.20MB
Compression Level 7 12,182.25 +0.25% 574.92μs 563.00μs 654.00μs 734.00μs 2.20MB
Compression Level 8 12,177.17 +0.21% 575.67μs 566.00μs 652.00μs 718.00μs 2.20MB
Compression Level 9 11,741.52 -3.37% 595.62μs 586.00μs 684.00μs 815.00μs 2.12MB

🔎 Key Findings

  1. Optimal Level: Level 3 shows best overall performance (+0.85% RPS)
  2. Diminishing Returns: Higher levels (7-9) show reduced benefits
  3. Performance Drop: Level 9 shows significant degradation (-3.37% RPS)
  4. Latency Impact: Compression generally maintains or improves latency until level 5+

Fixes #ISSUE_Number

What does this PR do?

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Breaking Changes

Test Plan

  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions

my-ship-it
my-ship-it approved these changes Jul 1, 2025
View reviewed changes
Copy link
Contributor

@my-ship-it my-ship-it left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yjhjstz @my-ship-it
gpfdist: set default compression level to 3
8d1959f 
The previous default compression level of 1 for gpfdist provides only
minimal data reduction while still incurring CPU overhead for compression.

This commit changes the default level to 3. This value represents a
better general-purpose trade-off, offering a more significant
reduction in network traffic for a modest increase in CPU cost.
This should improve performance for most network-bound ETL workloads
that use gpfdist with compression enabled.
@my-ship-it my-ship-it force-pushed the yjh/opt_gpfdist_compress branch from 961e7cf to 8d1959f Compare July 1, 2025 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@my-ship-it my-ship-it my-ship-it approved these changes

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yjhjstz @my-ship-it