Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using GDAL in java cause JVM crash #11544

Closed
johnwick67943532 opened this issue Dec 25, 2024 · 19 comments
Closed

using GDAL in java cause JVM crash #11544

johnwick67943532 opened this issue Dec 25, 2024 · 19 comments

Comments

@johnwick67943532
Copy link

johnwick67943532 commented Dec 25, 2024

What is the bug?

simple run Dataset dataset = gdal.Open(filePath, gdalconst.GA_ReadOnly);
Cause:

# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffcb4b63080, pid=16096, tid=16992
#
# JRE version: OpenJDK Runtime Environment Corretto-21.0.5.11.1 (21.0.5+11) (build 21.0.5+11-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-21.0.5.11.1 (21.0.5+11-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C  [msvcp140.dll+0x13080]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\CODE\spd_proj_main\web-back\hs_err_pid16096.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/corretto/corretto-21/issues/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

[error occurred during error reporting (), id 0xc0000005, EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffcb4b63080]
Disconnected from the target VM, address: '127.0.0.1:10152', transport: 'socket'

Steps to reproduce the issue

as above

Versions and provenance

win10, gdal 3.6.1, jvm21

Additional context

hs_err_pid16096.log

@rouault
Copy link
Member

rouault commented Dec 26, 2024

Are you sure your runtime environment is sane ? I see in the hs_err_pid a mix of Visual Studio runtimes:

0x00007ffcef6d0000 - 0x00007ffcef6ee000 	C:\Users\huashi01\.jdks\corretto-21.0.5\bin\VCRUNTIME140.dll
0x00007ffceeda0000 - 0x00007ffceedac000 	C:\Users\huashi01\.jdks\corretto-21.0.5\bin\vcruntime140_1.dll
0x00007ffcb4b50000 - 0x00007ffcb4bdd000 	C:\Users\huashi01\.jdks\corretto-21.0.5\bin\msvcp140.dll
0x00007ffcb5150000 - 0x00007ffcb516e000 	C:\Users\huashi01\.conda\envs\gis_old\Library\bin\vcruntime140.dll
0x00007ffc803e0000 - 0x00007ffc8046d000 	C:\Users\huashi01\.conda\envs\gis_old\Library\bin\msvcp140.dll

And does your gdalalljni.dll matches the gdal.dll version ?

0x00007ffc80330000 - 0x00007ffc803d8000 	C:\Users\huashi01\.conda\envs\gis_old\Library\bin\gdalalljni.dll
0x00007ffc684f0000 - 0x00007ffc6993a000 	C:\Program Files\GDAL\gdal.dll

@rouault
Copy link
Member

rouault commented Jan 6, 2025

@johnwick67943532 Ping w.r.t my question in #11544 (comment)

@johnwick67943532
Copy link
Author

johnwick67943532 commented Jan 9, 2025

Thanks alot to caring about this!
I will try another way to fix this by using ubuntu and docker contianer with gdal .

@dansiviter
Copy link

dansiviter commented Jan 10, 2025

I'm getting a similar issue with:

  • GDAL 3.10.0 installed using winget install GISInternals.GDAL using Java Bindings via Maven,
  • Java Temurin 21.0.4+7-LTS
  • OS: Windows 11 24H2
  • Code: https://github.com/OSGeo/gdal/blob/master/swig/java/apps/ogr2ogr_new.java
  • Args: -nln foo1 -lco FID=gid PG:dbname='postgres' host='localhost' user='postgres' port='5432' password='mysecretpassword' C:\Users\DANIEL~1\Downloads\test.kml

Error

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ff86cb82f58, pid=25244, tid=9280
#
# JRE version: OpenJDK Runtime Environment Temurin-21.0.4+7 (21.0.4+7) (build 21.0.4+7-LTS)
# Java VM: OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (21.0.4+7-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C  [msvcp140.dll+0x12f58]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\DanielSiviter\Develop\amld-api\hs_err_pid25244.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

hs_err_pid25244.log

@jratike80
Copy link
Collaborator

Have you tested with older Java versions?

@dansiviter
Copy link

@jratike80 Alas, no. I just don't have time.

@jratike80
Copy link
Collaborator

We can wait 😀

@XiaoYang943
Copy link

I'm getting a similar issue with:

  • gdal 3.10.0
  • JRE version: Java(TM) SE Runtime Environment (21.0.5+9) (build 21.0.5+9-LTS-239)
  • Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0.5+9-LTS-239, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
  • code: Dataset dataset = gdal.Open(filePath, gdalconst.GA_ReadOnly);

@jratike80
Copy link
Collaborator

Does it work with JRE 17? I know almost nothing about Java and JNI but Java 21 introduced the new FFM API. JNI is still supported but perhaps something has changed?

But by looking at the error it looks like the error is not in GDAL but in some Microsoft Visual C++ Redistributable file:

# Problematic frame:
# C  [msvcp140.dll+0x12f58]

@XiaoYang943
Copy link

Does it work with JRE 17? I know almost nothing about Java and JNI but Java 21 introduced the new FFM API. JNI is still supported but perhaps something has changed?

But by looking at the error it looks like the error is not in GDAL but in some Microsoft Visual C++ Redistributable file:

# Problematic frame:
# C  [msvcp140.dll+0x12f58]

i am not try it

@jratike80
Copy link
Collaborator

Hey, it may be this!
https://forums.oracle.com/ords/apexds/post/microsoft-stl-redistributable-found-in-jdk-is-out-of-date-5618
But it should have been fixed for Adoptium
adoptium/temurin-build#3887

@xydwhaha
Copy link

Hey, it may be this! https://forums.oracle.com/ords/apexds/post/microsoft-stl-redistributable-found-in-jdk-is-out-of-date-5618 But it should have been fixed for Adoptium adoptium/temurin-build#3887

yes,you are right ,I delete msvcp140.dll in JDKS/BIN , I solved this problem.

@XiaoYang943
Copy link

Hey, it may be this! https://forums.oracle.com/ords/apexds/post/microsoft-stl-redistributable-found-in-jdk-is-out-of-date-5618 But it should have been fixed for Adoptium adoptium/temurin-build#3887

yes,you are right ,I delete msvcp140.dll in JDKS/BIN , I solved this problem.

nice bro,it is effective.

@sxa
Copy link

sxa commented Jan 13, 2025

To be clear, if you're using Temurin please use the latest version - 21.0.5 should not have this issue as per the release notes as that is when the issue was fixed (I see a reference to the older Temurin 21.0.4 above which will have this problem.

Other java distributions (Such as the one @XiaoYang943 is using - I'm not sure where that is from but it's not Temurin) may still have this problem in 21.0.5, in which case removing the dll from the JDK directory is a reasonable fix until an update is available from your java distribution.

Have you tested with older Java versions?

Since it is a known issue with older MS redistributables being shipped with the JDK, older versions will likely show the same issue.

@rouault
Copy link
Member

rouault commented Jan 13, 2025

We were actually facing the same issue two months ago when Microsoft updates their Windows github action worker with a new version of the compiler that used the new version of the runtime but the Java version they deployed still used the old DLL.

Closing as not a GDAL bug

@rouault rouault closed this as completed Jan 13, 2025
@dansiviter
Copy link

I'd argue it still is a GDAL issue. At best the native code should guard against it, at worst the documentation needs to be updated to state which versions of Java are compatible. The vast majority of Java builds (Corretto, Oracle, etc.,) will still fail and people will bemoan GDAL and loose a lot of time.

FYI I've tried Temurin-21.0.5+11 (build 21.0.5+11-LTS) and does not appear to cause issues, however I'm still trying to understand the API so not had a completely successful run.

@johnwick67943532
Copy link
Author

johnwick67943532 commented Jan 14, 2025

Well guys, i changed codebase stack into python with conda. just 3 command finish GDAL lib :
simple, elegent, independent from dll hell.

conda create -n gis python=3.9
conda activate gis
conda install -c conda-forge gdal

Other info:
And i have heard some good tools call uv might also do something similar https://github.com/astral-sh/uv

@rouault
Copy link
Member

rouault commented Jan 14, 2025

I'd argue it still is a GDAL issue. At best the native code should guard against it, at worst the documentation needs to be updated to state which versions of Java are compatible. The vast majority of Java builds (Corretto, Oracle, etc.,) will still fail and people will bemoan GDAL and loose a lot of time.

you're welcome to submit a pull request to our documentation

@johnwick67943532
Copy link
Author

johnwick67943532 commented Jan 16, 2025

I'd argue it still is a GDAL issue. At best the native code should guard against it, at worst the documentation needs to be updated to state which versions of Java are compatible. The vast majority of Java builds (Corretto, Oracle, etc.,) will still fail and people will bemoan GDAL and loose a lot of time.我认为这仍然是一个 GDAL 问题。最好的情况是本机代码应该防止它,最坏的情况是需要更新文档以说明哪些版本的 Java 兼容。绝大多数 Java 版本(Corretto、Oracle 等)仍然会失败,人们会哀叹 GDAL 并浪费大量时间。

FYI I've tried Temurin-21.0.5+11 (build 21.0.5+11-LTS) and does not appear to cause issues, however I'm still trying to understand the API so not had a completely successful run.仅供参考,我已经尝试了 Temurin-21.0.5+11(构建 21.0.5+11-LTS),似乎没有引起问题,但我仍在努力理解 API,因此没有完全成功运行。

use python to free from this issue

#11544 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants