protoc-gen-grpc-java can't be built with gradle and protobuf 22+

[strophy]

I'm trying to package grpc-java 1.66.0 for Alpine Linux. I have modified compiler/build.gradle as follows:

cppCompiler.executable = 'aarch64-alpine-linux-musl-c++'
linker.executable = 'aarch64-alpine-linux-musl-c++'

I'm using the following steps to run the build:

apk add openjdk21-jre-headless protobuf-dev nss
wget https://github.com/grpc/grpc-java/archive/v1.66.0.tar.gz
tar xzf v1.66.0.tar.gz 
cd grpc-java-1.66.0/compiler
export GRADLE_USER_HOME="$srcdir"/.gradle
export CXXFLAGS="--std=c++17"
export LDFLAGS="-L/usr/lib"
../gradlew --no-daemon --parallel --info java_pluginExecutable -PskipAndroid=true

The build seems to succeed, but linking fails with:

> Task :grpc-compiler:compileJava_pluginExecutableJava_pluginCpp UP-TO-DATE
Found all include files for ':grpc-compiler:compileJava_pluginExecutableJava_pluginCpp'
Caching disabled for task ':grpc-compiler:compileJava_pluginExecutableJava_pluginCpp' because:
  Build cache is disabled
Skipping task ':grpc-compiler:compileJava_pluginExecutableJava_pluginCpp' as it is up-to-date.
Resolve mutations for :grpc-compiler:linkJava_pluginExecutable (Thread[#74,Execution worker Thread 5,5,main]) started.
:grpc-compiler:linkJava_pluginExecutable (Thread[#74,Execution worker Thread 5,5,main]) started.

> Task :grpc-compiler:linkJava_pluginExecutable FAILED
Caching disabled for task ':grpc-compiler:linkJava_pluginExecutable' because:
  Build cache is disabled
  Not made cacheable, yet
Task ':grpc-compiler:linkJava_pluginExecutable' is not up-to-date because:
  Task has failed previously.
See file:///home/builder/package/grpc-java-1.66.0/compiler/build/tmp/linkJava_pluginExecutable/output.txt for all output for linkJava_pluginExecutable.
Starting process 'command '/usr/bin/g++''. Working directory: /home/builder/package/grpc-java-1.66.0/compiler Command: /usr/bin/g++ @/home/builder/package/grpc-java-1.66.0/compiler/build/tmp/linkJava_pluginExecutable/options.txt -m64
Successfully started process 'command '/usr/bin/g++''
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lprotoc: No such file or directory
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lprotobuf: No such file or directory
collect2: error: ld returned 1 exit status

Finished linkJava_pluginExecutable, see full log file:///home/builder/package/grpc-java-1.66.0/compiler/build/tmp/linkJava_pluginExecutable/output.txt.

FAILURE: Build failed with an exception.

But the libraries installed by protobuf-dev exist:

/home/builder/package/grpc-java-1.66.0/compiler # ls -lha /usr/lib/libproto*
lrwxrwxrwx 1 root root   22 Aug 13 21:18 /usr/lib/libprotobuf-lite.so -> libprotobuf-lite.so.24
lrwxrwxrwx 1 root root   26 Aug 13 21:18 /usr/lib/libprotobuf-lite.so.24 -> libprotobuf-lite.so.24.4.0
-rwxr-xr-x 1 root root 487K Jul 22 17:26 /usr/lib/libprotobuf-lite.so.24.4.0
lrwxrwxrwx 1 root root   17 Aug 13 21:18 /usr/lib/libprotobuf.so -> libprotobuf.so.24
lrwxrwxrwx 1 root root   21 Aug 13 21:18 /usr/lib/libprotobuf.so.24 -> libprotobuf.so.24.4.0
-rwxr-xr-x 1 root root 2.7M Jul 22 17:26 /usr/lib/libprotobuf.so.24.4.0
lrwxrwxrwx 1 root root   15 Aug 13 21:18 /usr/lib/libprotoc.so -> libprotoc.so.24
lrwxrwxrwx 1 root root   19 Aug 13 21:18 /usr/lib/libprotoc.so.24 -> libprotoc.so.24.4.0
-rwxr-xr-x 1 root root 2.4M Jul 22 17:26 /usr/lib/libprotoc.so.24.4.0

Can anyone help me resolve this linking issue?

[strophy]

After a bit more head scratching, I came across this previous attempt: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/54120

According to https://github.com/grpc/grpc-java/blob/v1.52.1/compiler/build.gradle#L112 it seems grpc-java wants to be linked with static versions of libprotoc and libprotobuf.

Since Alpine now builds protobuf with cmake instead of make as shown in the compile environment setup steps described here, static linking is not an option any longer. Alpine also won't accept a PR that vendors static protobuf together with grpc-java, so my question is now how can I modify build.gradle to dynamically link against the `/usr/lib/libproto*.so files?

I tried modifying the linker.args a few times like this:

linker.args "-lprotoc", "-lprotobuf", "-labsl_raw_hash_set", "-labsl_hash", "-labsl_base", "-labsl_raw_logging_internal", "-lpthread", "-s",
// linker.args "-lprotoc", "-lprotobuf", "-labsl_raw_hash_set", "-labsl_hash", "-labsl_base", "-lpthread", "-s",
// linker.args "-lprotoc", "-lprotobuf", "-labsl_raw_hash_set", "-labsl_base", "-lpthread", "-s"
// linker.args "-lprotoc", "-lprotobuf", "-labsl_raw_hash_set", "-lpthread", "-s"
// linker.args "-lprotoc", "-lprotobuf", "-lpthread", "-s",
// linker.args "-Wl,-Bstatic", "-lprotoc", "-lprotobuf", "-static-libgcc",
// "-Wl,-Bdynamic", "-pthread", "-s"

Which ultimately results in the error:

> Task :grpc-compiler:linkJava_pluginExecutable FAILED
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/builder/package/grpc-java-1.66.0/compiler/build/objs/java_plugin/java_pluginCpp/cmkx8cb9wgxcy6i9dx7v2pqn9/java_generator.o: in function `java_grpc_generator::MessageFullJavaName(google::protobuf::Descriptor const*)':
java_generator.cpp:(.text+0x5d0): undefined reference to `google::protobuf::compiler::java::ClassName[abi:cxx11](google::protobuf::Descriptor const*)'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/builder/package/grpc-java-1.66.0/compiler/build/objs/java_plugin/java_pluginCpp/cmkx8cb9wgxcy6i9dx7v2pqn9/java_generator.o: in function `java_grpc_generator::PrintGetServiceDescriptorMethod(google::protobuf::ServiceDescriptor const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, google::protobuf::io::Printer*, java_grpc_generator::ProtoFlavor)':
java_generator.cpp:(.text+0x5a15): undefined reference to `google::protobuf::compiler::java::ClassName[abi:cxx11](google::protobuf::FileDescriptor const*)'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/builder/package/grpc-java-1.66.0/compiler/build/objs/java_plugin/java_pluginCpp/cmkx8cb9wgxcy6i9dx7v2pqn9/java_generator.o: in function `java_grpc_generator::ServiceJavaPackage[abi:cxx11](google::protobuf::FileDescriptor const*)':
java_generator.cpp:(.text+0x8b00): undefined reference to `google::protobuf::compiler::java::ClassName[abi:cxx11](google::protobuf::FileDescriptor const*)'
collect2: error: ld returned 1 exit status


FAILURE: Build failed with an exception

Is this an issue with different compiler versions or CXXFLAGS specifying C++ standards being applied at the time Cmake is building protobuf? It is trying to link against protobuf 24.4 which would seem to satisfy the requirement for protobuf 21.7+. The change requiring static linking was made 9 years ago when Gradle 2.3 was in use, is dynamic linking still impossible today?

[ejona86]

You might want to glance at #10048 . We haven't been able to update to any absl-based protobuf release for our official binaries. Our Bazel build is working with the newer protobuf.

Since Alpine now builds protobuf with cmake instead of make as shown in the compile environment setup steps described here, static linking is not an option any longer.

Static linking is definitely still an option. But overall absl is the issue and makes our build a PITA. I got Linux building locally by hard-coding the huge list of -l (as protobuf was broken at the time). Later I tried a bit to use pkg-config, but Windows was in a bad situation then; pkgconf makes things better but we haven't gotten back to it. We are seriously considering swapping the protoc-gen-grpc-java build exclusively to Bazel as the Gradle build just doesn't seem like it will work well any more. And we could then follow protobuf and provide fully-static binaries built on modern distros.

[cho-m]

Was looking at this as trying to build protoc-gen-grpc-java in Homebrew for usage on Apple ARM.

The latter part of issue appears to be on Protobuf side due to not exposing symbols. Opened a PR that worked for me protocolbuffers/protobuf#19363

• Before:

  ❯ nm -gU libprotoc.28.3.0.dylib | grep 'java.*ClassName'
  
  ❯ echo $?
  1

• After:

  ❯ nm -gU libprotoc.28.3.0.dylib | grep 'java.*ClassName'
  0000000000131fb8 T __ZN6google8protobuf8compiler4java13FileClassNameEPKNS0_14FileDescriptorE
  000000000013197c T __ZN6google8protobuf8compiler4java9ClassNameEPKNS0_10DescriptorE
  0000000000131a30 T __ZN6google8protobuf8compiler4java9ClassNameEPKNS0_14EnumDescriptorE
  0000000000131b98 T __ZN6google8protobuf8compiler4java9ClassNameEPKNS0_14FileDescriptorE
  0000000000131ae4 T __ZN6google8protobuf8compiler4java9ClassNameEPKNS0_17ServiceDescriptorE
  
  ❯ c++filt __ZN6google8protobuf8compiler4java9ClassNameEPKNS0_10DescriptorE
  google::protobuf::compiler::java::ClassName(google::protobuf::Descriptor const*)

pkgconf/pkg-config is probably simplest way to handle extra flags in current build setup.