Issue
I use GCP client libraries to implement pub/sub model in my spring-boot application. For authenticating i'm using GOOGLE_APPLICATION_CREDENTIALS path env variable. It works fine with other versions of JDK/JRE, But it fails with segmentation Error with below mentioned jdk/jre
Environment details
Java version:
openjdk version "1.8.0_322"
OpenJDK Runtime Environment (Zulu 8.60.0.22-SA-linux-musl-x64) (build 1.8.0_322-b06)
OpenJDK 64-Bit Server VM (Zulu 8.60.0.22-SA-linux-musl-x64) (build 25.322-b06, mixed mode)
Log:
# A fatal error has been detected by the Java Runtime Environment:
# SIGSEGV (0xb) at pc=0x0000000000003fd6, pid=1, tid=0x00007f99a14fcb38
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.60.0.22-SA-linux-musl-x64) (8.0_322-b06) (build 1.8.0_322-b06)
#
# Java VM: OpenJDK 64-Bit Server VM (25.322-b06 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C 0x0000000000003fd6
#
# Core dump written. Default location: //core or core.1
#
# An error report file with more information is saved as:
# /tmp/hs_err_pid1.log
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j java.lang.ClassLoader$NativeLibrary.load(Ljava/lang/String;Z)V+0
j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+328
j java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+92
j java.lang.Runtime.load0(Ljava/lang/Class;Ljava/lang/String;)V+57
j java.lang.System.load(Ljava/lang/String;)V+7
j io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(Ljava/lang/String;Z)V+5
v ~StubRoutines::call_stub
J 2066 sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x00007f5cad99bdf7 [0x00007f5cad99bd80+0x77]
J 2065 C1 sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (104 bytes) @ 0x00007f5cad9a2a8c [0x00007f5cad9a1900+0x118c]
J 1974 C1 sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (10 bytes) @ 0x00007f5cad961784 [0x00007f5cad961680+0x104]
J 2084 C1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x00007f5cad9a3e8c [0x00007f5cad9a3aa0+0x3ec]
j io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader$1.run()Ljava/lang/Object;+53
v ~StubRoutines::call_stub
J 1349 java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object; (0 bytes) @ 0x00007f5cad764f4f [0x00007f5cad764f00+0x4f]
j io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(Ljava/lang/Class;Ljava/lang/String;Z)V+10
j io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(Ljava/lang/ClassLoader;Ljava/lang/String;Z)V+15
j io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(Ljava/lang/String;Ljava/lang/ClassLoader;)V+359
j io.grpc.netty.shaded.io.netty.channel.epoll.Native.loadNativeLibrary()V+60
j io.grpc.netty.shaded.io.netty.channel.epoll.Native.<clinit>()V+76
v ~StubRoutines::call_stub
j io.grpc.netty.shaded.io.netty.channel.epoll.Epoll.<clinit>()V+28
v ~StubRoutines::call_stub
J 993 java.lang.Class.forName0(Ljava/lang/String;ZLjava/lang/ClassLoader;Ljava/lang/Class;)Ljava/lang/Class; (0 bytes) @ 0x00007f5cad6995fa [0x00007f5cad699580+0x7a]
J 1952 C1 java.lang.Class.forName(Ljava/lang/String;)Ljava/lang/Class; (15 bytes) @ 0x00007f5cad948d4c [0x00007f5cad948ba0+0x1ac]
j io.grpc.netty.shaded.io.grpc.netty.Utils.isEpollAvailable()Z+3
j io.grpc.netty.shaded.io.grpc.netty.Utils.<clinit>()V+144
v ~StubRoutines::call_stub
j io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.<clinit>()V+16
v ~StubRoutines::call_stub
j io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress(Ljava/lang/String;I)Lio/grpc/netty/shaded/io/grpc/netty/NettyChannelBuilder;+2
j io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress(Ljava/lang/String;I)Lio/grpc/ManagedChannelBuilder;+3
j io.grpc.ManagedChannelBuilder.forAddress(Ljava/lang/String;I)Lio/grpc/ManagedChannelBuilder;+5
j com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel()Lio/grpc/ManagedChannel;+285
j com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$$Lambda$596.createSingleChannel()Lio/grpc/ManagedChannel;+4
j com.google.api.gax.grpc.ChannelPool.<init>(Lcom/google/api/gax/grpc/ChannelPoolSettings;Lcom/google/api/gax/grpc/ChannelFactory;Ljava/util/concurrent/ScheduledExecutorService;)V+71
j com.google.api.gax.grpc.ChannelPool.create(Lcom/google/api/gax/grpc/ChannelPoolSettings;Lcom/google/api/gax/grpc/ChannelFactory;)Lcom/google/api/gax/grpc/ChannelPool;+9
j com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel()Lcom/google/api/gax/rpc/TransportChannel;+10
j com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel()Lcom/google/api/gax/rpc/TransportChannel;+35
j com.google.api.gax.rpc.ClientContext.create(Lcom/google/api/gax/rpc/StubSettings;)Lcom/google/api/gax/rpc/ClientContext;+179
j com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(Lcom/google/cloud/pubsub/v1/stub/SubscriberStubSettings;)Lcom/google/cloud/pubsub/v1/stub/GrpcSubscriberStub;+6
j com.google.cloud.pubsub.v1.Subscriber.doStart()V+16
j com.google.api.core.AbstractApiService$InnerService.doStart()V+4
j com.google.common.util.concurrent.AbstractService.startAsync()Lcom/google/common/util/concurrent/Service;+33
j com.google.api.core.AbstractApiService.startAsync()Lcom/google/api/core/ApiService;+4
j com.google.cloud.pubsub.v1.Subscriber.startAsync()Lcom/google/api/core/ApiService;+1
Dependencies:
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>libraries-bom</artifactId>
<version>25.1.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
</dependency>
<dependencies>
And also i wanted to know is there any other way to authenticate other than using path env variable? Can I use spring.cloud.gcp.credentials.location=file:{location}
with GCP client libraries, instead of env variable?
Solution
As mentioned by @Juraj Martinka, it was problem with underlying google library io.grpc.netty.shaded
. It seems Netty does not support Alpine since Netty depends on glibc but Alpine does not have it, it has musl libc instead.
The issue disappears if you disable Netty's native support or if you use an image that has glibc, e.g.:
azul/zulu-openjdk-alpine:11-jre
: Alpine-based, no glibc -> does not work
azul/zulu-openjdk:11
: Ubuntu-based, has glibc -> works
Using -Dio.grpc.netty.shaded.io.netty.transport.noNative=true
avoids the segfault
example:
java -jar -D-Dio.grpc.netty.shaded.io.netty.transport.noNative=true app.jar
The other workaround is, using grpc-netty
instead of grpc-netty-shaded
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<exclusions>
<exclusion>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty</artifactId>
</dependency>
Reference Links: Link 1, Link 2
Answered By - Nitish
Answer Checked By - Pedro (JavaFixing Volunteer)