Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building failure with Hadoop 3.3.1 #14

Open
jerrychenhf opened this issue Feb 16, 2023 · 3 comments
Open

Building failure with Hadoop 3.3.1 #14

jerrychenhf opened this issue Feb 16, 2023 · 3 comments

Comments

@jerrychenhf
Copy link

When building with the following command:
mvn clean package -Pdist -Dhadoop.plat.version=3.3.1 -Dhadoop.version=3.3.1 -Dmaven.test.skip=true

The build failures.

  1. First the duplicated class error for OBSClientFactory. There are two java files (OBSClientFactory.java and ObsClientFactory.java) which have the definition of the same interface.
  2. When delelete ObsClientFactory.java and the compile goes on but a few compiling errors for StringUtils and other functions from commons. Seems that the jar is not available using hadoop 3.3.1 dependency. Hadoop 3.1.1 is correct.

image

@jerrychenhf
Copy link
Author

It is very strange why there is an issue like duplicate class error since the project is releasing.

@jerrychenhf
Copy link
Author

The thing for fix and compile:

  1. rm hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/ObsClientFactory.java
  2. Replace apache.commons.lang import with apache.commons.lang3 because Hadoop 3.3.1 is depending commons.lang3 instead of commons.lang.
    sed -i "s/apache.commons.lang/apache.commons.lang3/g" grep "apache.commons.lang" -rl ./

@kiwik
Copy link

kiwik commented Feb 17, 2023

Hi @aheartfulguy, could you help to address this issue? When @jerrychenhf and I trying to add Huawei Cloud support into CloudTik, we face it, thanks.

oap-project/cloudtik#1011 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants