PHOENIX-7731: Add phoenix-hbase-compat module for HBase 2.6.4 #2291

sanjeet006py · 2025-09-24T18:18:32Z

JIRA: PHOENIX-7731
The PR has added a new phoenix-hbase-compat module for HBase 2.6.4. Link to the Phoenix dev mailing list thread with alignment on creating new phoenix-hbase-compat module.
The new phoenix-hbase-compat-2.6.4 module is same as phoenix-hbase-compat-2.6.0 except for class: CompatIndexHalfStoreFileReader where new API in 2.6.4 is being used: StoreFileInfo.createStoreFileInfoForHFile().

… memstore and block count read from FS

virajjasani · 2025-10-13T00:35:40Z

@sanjeet006py at high level, the changes look good but I think once we have hbase 2.6 release, we should create separate PR for the new module without including phoenix metrics changes.
Then the separate changes for metrics will make it even better for review!

sanjeet006py · 2025-11-18T05:47:14Z

PR validation build: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/view/change-requests/job/PR-2291/

virajjasani · 2025-11-18T06:12:06Z

PR validation build: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/view/change-requests/job/PR-2291/

@sanjeet006py this build will still use 2.5.4 by default right? you can make temporary commit with applying <name>!hbase.profile</name> to latest 2.6.4 and remove it from 2.5.4 and let one full build run? once the results look good, you can revert the commit and we can merge it.

The high level changes look good so far.

stoty · 2025-11-18T08:51:38Z

pom.xml

+      </activation>
+      <properties>
+        <hbase.profile>2.6.3</hbase.profile>
+        <hadoop.version>3.4.1</hadoop.version>


Why did you add this ?
While 2.6.3 is built with 3.4.1, in my tests it ran fine with 3.4.2.
We generally want the latest Hadoop that the Hbase version works with (and can run the tests)

When I initially started on PR then Phoenix by default was using hadoop 3.4.1 so, I also used same for pinning hadoop version for HBase 2.6.3. But I see now HBase & Phoenix are using 3.4.2 so will update the same. Thanks

You should just remove the line and fall back to the default 3.4.2

stoty

+1 LGTM

Don't forget to revert the default profile before committing.

virajjasani · 2025-11-24T05:23:15Z

@sanjeet006py we still don't have full build results with 2.6 profile

sanjeet006py · 2025-11-24T05:32:29Z

@sanjeet006py we still don't have full build results with 2.6 profile

Yes, one of my changes in the PR broke regions splits for local indexes. And, Phoenix version has been updated to 5.4.0 so, that also broke PR's build as I didn't sync latest from master.

virajjasani · 2025-11-24T05:35:19Z

My bad, i thought 2.6 default profile was changed to 2.5. I misunderstood with 5.4 change

…or, checkstyle error is false -ve

virajjasani · 2025-11-24T13:26:33Z

pom.xml

+    <module>phoenix-client-parent</module>
+    <module>phoenix-client-parent/phoenix-client-embedded</module>
+    <module>phoenix-client-parent/phoenix-client-lite</module>
+    <module>phoenix-server</module>
+    <module>phoenix-mapreduce-byo-shaded-hbase</module>
+    <module>phoenix-assembly</module>


Curious, we need this change for checkstyle? It seems we should have had these modules defined here right? @stoty wdyt?

These shouldn't be listed here.
Adding these here will disable the build optimizations that enable quicker builds by skipping the shading steps.

See the comment above.

Actually, I am seeing Javadoc failure after patch is applied saying above extra modules listed under shade-and-assembly profile for hbase-2.6 are not found. I thought adding them here might help me get past that but doesn't seem to be case. So, reverting this change. I will be posting a summary of unexpected failures I am seeing and would need help on that. Thanks

sanjeet006py · 2025-11-24T14:46:41Z

@stoty @virajjasani I am seeing following issue with HBase 2.6 profile (using HBase 2.6.4):

Error related to Javadoc status check failure:

Mon 24 Nov 2025 05:50:43 AM UTC
cd /home/jenkins/jenkins-home/workspace/enix-PreCommit-GitHub-PR_PR-2291@2/yetus-general-check/src/phoenix-assembly
mvn --batch-mode -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/enix-PreCommit-GitHub-PR_PR-2291@2/yetus-m2/phoenix-master-patch-1 --threads=1 -DPhoenixPatchProcess -Dskip.code-coverage clean javadoc:javadoc -DskipTests=true
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: linux
[INFO] os.detected.arch: x86_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 5.15
[INFO] os.detected.version.major: 5
[INFO] os.detected.version.minor: 15
[INFO] os.detected.release: ubuntu
[INFO] os.detected.release.version: 20.04
[INFO] os.detected.release.like.ubuntu: true
[INFO] os.detected.release.like.debian: true
[INFO] os.detected.classifier: linux-x86_64
[INFO] 
[INFO] ----------------< org.apache.phoenix:phoenix-assembly >-----------------
[INFO] Building Phoenix Assembly 5.4.0-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[WARNING] The POM for org.apache.phoenix:phoenix-server-hbase-2.6:jar:5.4.0-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.phoenix:phoenix-mapreduce-byo-shaded-hbase-hbase-2.6:jar:5.4.0-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.phoenix:phoenix-client-embedded-hbase-2.6:jar:5.4.0-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.phoenix:phoenix-client-lite-hbase-2.6:jar:5.4.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.814 s
[INFO] Finished at: 2025-11-24T05:50:48Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project phoenix-assembly: Could not resolve dependencies for project org.apache.phoenix:phoenix-assembly:pom:5.4.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.phoenix:phoenix-server-hbase-2.6:jar:5.4.0-SNAPSHOT, org.apache.phoenix:phoenix-mapreduce-byo-shaded-hbase-hbase-2.6:jar:5.4.0-SNAPSHOT, org.apache.phoenix:phoenix-client-embedded-hbase-2.6:jar:5.4.0-SNAPSHOT, org.apache.phoenix:phoenix-client-lite-hbase-2.6:jar:5.4.0-SNAPSHOT: Could not find artifact org.apache.phoenix:phoenix-server-hbase-2.6:jar:5.4.0-SNAPSHOT -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

The exact maven command is working fine locally for me. Any suggestions on how to fix this?

stoty · 2025-11-24T15:04:59Z

Ignore yetus errors if it works locally.

Yetus is known to have problems when the projexct structure changes

sanjeet006py · 2025-11-24T19:38:37Z

All tests have succeeded with HBase 2.6.4 so, switching back to HBase 2.5 as default profile.

haridsv

LGTM

Sanjeet Malhotra added 12 commits September 11, 2025 01:07

Consume new metrics added in HBase for bytes read from cache vs FS vs…

396e826

… memstore and block count read from FS

Account for cross RS scans from Unconvered indexex and read repairs

5cf01f2

Add tests for different query plans

8b362b8

Added tests for different query plans

89226cf

Added tests for different query plans

55644e9

Prepare to create PR

d2bbca3

Prepare to create PR

7c485d1

Make it backward compatible

4d4f05b

Add phoenix-hbase-compat-2.6.4

440fecf

Merge branch 'master' into new-hbase-metrics

d923638

Fix 2.6.4 compat module build

dee0a5f

Fix modules list with new phoenix-hbase-compat-2.6.4

56f80f0

sanjeet006py requested review from haridsv, tkhurana and virajjasani October 8, 2025 18:50

Sanjeet Malhotra added 4 commits November 13, 2025 15:14

Merge remote-tracking branch 'apache/master' into new-hbase-metrics

64150af

PHOENIX-7731: Add phoenix-hbase-compat module for HBase 2.6.4

9fa7de0

Remove extra changes

7e580ba

Remove extra changes

fe437d3

sanjeet006py changed the title ~~PHOENIX-7704 Integrate HBase's new Scan Latency Metrics~~ PHOENIX-7731: Add phoenix-hbase-compat module for HBase 2.6.4 Nov 18, 2025

sanjeet006py marked this pull request as ready for review November 18, 2025 05:34

virajjasani requested a review from stoty November 18, 2025 06:10

stoty requested changes Nov 18, 2025

View reviewed changes

Sanjeet Malhotra added 3 commits November 18, 2025 16:59

Address Viraj's comment

295abf3

Address Stoty's comments

88a51fb

Fix 2.6.3 profile build

6a5b8a5

stoty approved these changes Nov 19, 2025

View reviewed changes

Sanjeet Malhotra added 3 commits November 24, 2025 00:31

Fix local index splits for MutableIndexSplitReverseScanIT

5a3c952

Merge remote-tracking branch 'apache/master' into new-hbase-metrics

309fc99

Roll POM versions to 5.4

248b2dc

Fix javadoc error, spotbug error is expected if consturtcor throw err…

537038a

…or, checkstyle error is false -ve

virajjasani reviewed Nov 24, 2025

View reviewed changes

Remove extra modules

bacf75b

Switch back to default HBase 2.5 profile

f7df85c

haridsv approved these changes Nov 25, 2025

View reviewed changes

PHOENIX-7731: Add phoenix-hbase-compat module for HBase 2.6.4 #2291

Are you sure you want to change the base?

PHOENIX-7731: Add phoenix-hbase-compat module for HBase 2.6.4 #2291

Conversation

sanjeet006py commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

virajjasani commented Oct 13, 2025

Uh oh!

sanjeet006py commented Nov 18, 2025

Uh oh!

virajjasani commented Nov 18, 2025

Uh oh!

stoty Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

sanjeet006py Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stoty Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stoty left a comment

Choose a reason for hiding this comment

Uh oh!

virajjasani commented Nov 24, 2025

Uh oh!

sanjeet006py commented Nov 24, 2025

Uh oh!

virajjasani commented Nov 24, 2025

Uh oh!

virajjasani Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

stoty Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

stoty Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

sanjeet006py Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjeet006py commented Nov 24, 2025

Uh oh!

stoty commented Nov 24, 2025

Uh oh!

sanjeet006py commented Nov 24, 2025

Uh oh!

haridsv left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sanjeet006py commented Sep 24, 2025 •

edited

Loading

sanjeet006py Nov 18, 2025 •

edited

Loading

stoty Nov 18, 2025 •

edited

Loading

sanjeet006py Nov 24, 2025 •

edited

Loading