Full test run crashes -- memory leaks

[dnvasilev]

Crashes in the middle of full test run (./gradlew cAT)

08-30 10:38:32.021 10881 14074 F libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0xfffffff4 in tid 14074 (GLThread 929)
08-30 10:38:32.362 14097 14097 F DEBUG : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-30 10:38:32.363 14097 14097 F DEBUG : Build fingerprint: 'Verizon/heroqltevzw/heroqltevzw:7.0/NRD90M/G930VVRU4BQC1:user/release-keys'
08-30 10:38:32.363 14097 14097 F DEBUG : Revision: '11'
08-30 10:38:32.363 14097 14097 F DEBUG : ABI: 'arm'
08-30 10:38:32.363 14097 14097 F DEBUG : pid: 10881, tid: 14074, name: GLThread 929 >>> org.gearvrf.tester <<<
08-30 10:38:32.364 14097 14097 F DEBUG : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xfffffff4
08-30 10:38:32.364 14097 14097 F DEBUG : r0 c8003274 r1 00000006 r2 c8003294 r3 00000000
08-30 10:38:32.364 14097 14097 F DEBUG : r4 c8003274 r5 cb070a10 r6 00000000 r7 c8003294
08-30 10:38:32.364 14097 14097 F DEBUG : r8 cc587e00 r9 c6679bfc sl c855e800 fp d1924fc4
08-30 10:38:32.364 14097 14097 F DEBUG : ip f1f04948 sp c6679b30 lr d18c46b7 pc d18aee8e cpsr 600f0030
08-30 10:38:32.371 14097 14097 F DEBUG :
08-30 10:38:32.371 14097 14097 F DEBUG : backtrace:
08-30 10:38:32.371 14097 14097 F DEBUG : #00 pc 00090e8e /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::Light::render(int, int)+69)
08-30 10:38:32.371 14097 14097 F DEBUG : #1 pc 000a66b3 /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::CustomShader::render(gvr::RenderState*, gvr::RenderData*, gvr::Material*)+830)
08-30 10:38:32.371 14097 14097 F DEBUG : #2 pc 0008e3d9 /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::GLRenderer::renderMaterialShader(gvr::RenderState&, gvr::RenderData*, gvr::Material*)+1424)
08-30 10:38:32.372 14097 14097 F DEBUG : #3 pc 0008cd8d /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::GLRenderer::renderMesh(gvr::RenderState&, gvr::RenderData*)+92)
08-30 10:38:32.372 14097 14097 F DEBUG : #4 pc 0008f23f /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::Renderer::renderRenderData(gvr::RenderState&, gvr::RenderData*)+40)
08-30 10:38:32.372 14097 14097 F DEBUG : #5 pc 0008bfad /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::BatchManager::renderBatches(gvr::RenderState&)+404)
08-30 10:38:32.372 14097 14097 F DEBUG : #6 pc 0008d2e9 /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::GLRenderer::renderCamera(gvr::Scene*, gvr::Camera*, int, int, int, int, int, gvr::ShaderManager*, gvr::PostEffectShaderManager*, gvr::RenderTexture*, gvr::RenderTexture*, bool)+520)
08-30 10:38:32.373 14097 14097 F DEBUG : #7 pc 0008cd19 /data/app/org.gearvrf.tester-2/lib/arm/libgvrf.so (gvr::GLRenderer::renderCamera(gvr::Scene*, gvr::Camera*, gvr::ShaderManager*, gvr::PostEffectShaderManager*, gvr::RenderTexture*, gvr::RenderTexture*, bool)+108)
08-30 10:38:32.373 14097 14097 F DEBUG : #8 pc 005ba965 /data/app/org.gearvrf.tester-2/oat/arm/base.odex (offset 0x53e000)

[dnvasilev]

Running framework-tests from Android Studio: 50/90 completed, crashes at ShadowTests/directLightAtFrontCastsShadow

[liaxim]

How reproducible are these crashes? If you just run the ShadowTests for example?

[dnvasilev]

If I run only ShadowTests, directLightAtFrontCastsShadow does not crash.
I don't have a lot of statistics at that point: it takes a long time to run a complete test, and in addition gradle and Android Studio take a lot of time to detect that a test has failed (on failing AS run, 49 minutes for a single subtest)

[liaxim]

It is likely that we are leaking stuff again. It was one of the reasons of similar crashes in the past.

[dnvasilev]

Problem is more destructive on Adreno. On Adreno, I have 4/4 full run failures, and on Mali only 1/4

[liaxim]

Yep. A lot of Java memory is being leaked and some native. The EGL track is fine this time around. Eventually I see an OutOfMemory exception coming from Java.

[dnvasilev]

Sometimes we can make it: just now G930V completed all tests under Android Studio

[liaxim]

Fixed by Samsung/GearVRf#1569 and mainly by #211 in the maint branch. Cherry-picking to master will be straightforward but I expect some additional work.

[liaxim]

Master: all renderdata, light and scene objects are leaked due to impossible to close global references from the native side.