Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

QBM is down with the JVM crash #4572

duclong.ng ·

Hello@robinshen,
We are upgrading JAVA11 for our QBM and encountered a crash. The java version we use are Zulu11 and OpenJDK 11 from Azul (Zing11).
Please check if Zulu11 and Zing11 run stably on QB10.

error log:

jvm 1    | main    | 2024/05/16 12:36:11.873 | #
jvm 1    | main    | 2024/05/16 12:36:11.874 | # A fatal error has been detected by the Java Runtime Environment:
jvm 1    | main    | 2024/05/16 12:36:11.874 | #
jvm 1    | main    | 2024/05/16 12:36:11.874 | #  SIGSEGV (0xb) at pc=0x00007f5ccae85018, pid=426572, tid=431737
jvm 1    | main    | 2024/05/16 12:36:11.874 | #
jvm 1    | main    | 2024/05/16 12:36:11.874 | # JRE version: OpenJDK Runtime Environment Zulu11.70+15-CA (11.0.22+7) (build 11.0.22+7-LTS)
jvm 1    | main    | 2024/05/16 12:36:11.874 | # Java VM: OpenJDK 64-Bit Server VM Zulu11.70+15-CA (11.0.22+7-LTS, mixed mode, tiered, shenandoah gc, linux-amd64)
jvm 1    | main    | 2024/05/16 12:36:11.874 | # Problematic frame:
jvm 1    | main    | 2024/05/16 12:36:11.874 | # J 37253% c2 sun.net.www.http.KeepAliveCache.run()V java.base@11.0.22 (286 bytes) @ 0x00007f5ccae85018 [0x00007f5ccae83100+0x0000000000001f18]
jvm 1    | main    | 2024/05/16 12:36:11.874 | #
jvm 1    | main    | 2024/05/16 12:36:11.874 | # Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /root/Quickbuild/quickbuild_new/bin/core.426572)
jvm 1    | main    | 2024/05/16 12:36:11.874 | #
jvm 1    | main    | 2024/05/16 12:36:11.874 | # An error report file with more information is saved as:
jvm 1    | main    | 2024/05/16 12:36:11.874 | #
  • replies 7
  • views 55
  • stars 1
robinshen ADMIN ·

We tested with Oracle JDK 11 and Eclipse OpenJDK 11 (temurin). However other distributions should also work. Can you please run it again to see if it can start? JDK does crash sometimes (very rarely) at my side, and I believe it is a JDK bug.

duclong.ng ·

Can you test with Zulu11? Initially the service can still start normally. Crash occurs after a few days of operation.
On another server running QB12 we got errors with OpenJDK11 and now running fine with Zulu11.

steveluo ADMIN ·

Hi@duclong.ng

I have tested with Zulu11 and QB12 for a half day by running some simple builds. My environment is:

JDK: OpenJDK 64-Bit Server VM Zulu11.72+19-CA (build 11.0.23+9-LTS, mixed mode)
OS: Ubuntu 22.04.4 LTS
Linux 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Also, I have tested OpenJDK 11 (installed by apt) below without any problems:

OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1, mixed mode, sharing)

So, would you please tell me what is your operating system? And which OpenJDK you are running when got errors?

duclong.ng ·

Hello@steveluo

All my system running in Ubuntu 22.04:
QB12 + OpenJDK 11 -> crash
QB12 + Zulu11.70.15-ca-jdk11.0.22 -> ok
QB10 + OpenJDK 11, Zulu11, Zing11 -> crash
QB10 + java-8-openjdk-shennandoah -> ok

steveluo ADMIN ·

Hi@duclong.ng

When you say "crash", is that mean you are not able to start the server or the server crashed after running some time with the error log you mentioned before? If the server crashes when starting, any log there to be shown?

And I also want to know if you installed any 3rd-party plugins for your QuickBuild?

As I'm not able to reproduce the issue, is it possible for you to test with a new clean installation (a new machine with a new installed Ubuntu with new installed QuickBuild) instead of using your old instance?

duclong.ng ·

@steveluo

Yes, we do use plugins from other parties. but I can't share it here.
"the server crashed after running some time with the error log you mentioned before" -> Yes, that's right.

It seems like it would be very difficult for you to reproduce the error we encountered. Thank you for your help.

Could you look at the error logs I sent? Do you have any suggestions for us to continue our investigation?

steveluo ADMIN ·

Hi@duclong.ng

Please refer to below page for more about the crash from JDK:

From your log, we can know this is a compiled error and caused by code sun.net.www.http.KeepAliveCache which belongs to JDK.

You may also open the crash report (/root/Quickbuild/quickbuild_new/bin/core.426572) for more details.

I think you may try other JDK to see if the issue persists. From your previous post, I want to know do you encounter the same crash for below cases?

QB12 + OpenJDK 11 -> crash
QB10 + OpenJDK 11, Zulu11, Zing11 -> crash

Also, you may disable the 3rd plugins for a while to see if the crash persists.