Repository navigation
Make Android inference lifecycle-, coroutine-, and memory-pressure-safe - #5469
Open
maziyarpanahi wants to merge 1 commit into
Open
maziyarpanahi wants to merge 1 commit into
maziyarpanahi wants to merge 1 commit into
Conversation
…essure-safe Add OpenMedRuntime, a lifecycle-owned Android runtime with an explicit IDLE/LOADING/READY/BUSY/DRAINING/RELEASED state machine. Operations hold leases on the session they started on, loading is single-flight, and close, model replacement, memory pressure, and backgrounding release a native session only after its final lease. Cancelled or revoked work never publishes a late result; refusals raise value-free typed errors with retry guidance. Trim-memory and host lifecycle signals arrive through injectable sources, including a ComponentCallbacks2 adapter, without an androidx.lifecycle dependency. The facade and ONNX classifier gain reference-counted close guards and stage cancellation checkpoints, and ModelCache leases keep in-use models out of eviction and trimming. The React Native module now owns a host scope cancelled in invalidate(), rejects with value-free messages, and builds a fresh metadata map per span.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3647.
Adds
OpenMedRuntime(android/openmedkit/.../OpenMedRuntime.kt), a lifecycle-owned Android runtime with an explicit IDLE, LOADING, READY, BUSY, DRAINING, RELEASED state machine and a value-freestatusflow. Every operation holds a lease on the session it started on. Loading is single-flight behind aMutex, so concurrent first use opens one session.close(),replaceModel, memory pressure and backgrounding close a native session only after its final lease is released. Cancelled callers getCancellationException; revoked, refused or stale work raisesOpenMedRuntimeExceptionwith a stable value-free reason code and retry guidance, and never publishes a late result. Trim-memory and host lifecycle signals come in through injectable seams:OpenMedLifecycleSource, aComponentCallbacks2adapter registered on the application context, and a documentedProcessLifecycleOwnersnippet, so core takes noandroidx.lifecycledependency. The default policy releases idle sessions at LOW pressure, also stops admission at CRITICAL for a cool-down, refuses new work while backgrounded, and bounds consecutive reloads; each reacquisition re-runs the factory, which re-verifies the cached bundle.The
OpenMedKitfacade and the ONNXOnnxTokenClassifiernow have reference-counted close guards (util/LeaseGate.kt) and cancellation checkpoints before tokenization, after native inference, and after decoding, merging and policy application.ModelCachegains leases, soevictand budget trimming skip a model that an open session uses (OpenMedSessionFactory.fromCache). The React Native module now owns anOpenMedHostScopethatinvalidate()cancels and releases (it also unregisters trim-memory callbacks). It rejects promises with value-free messages plususerInfo.reason/retry, and builds a fresh metadata map for every span: the old sharedWritableMapwas consumed by the firstputMap. The docs section is indocs/android-integration.md.Breaking:
OpenMedKitcalls made afterclose()now throwOpenMedRuntimeException(RUNTIME_CLOSED)instead of reaching the closed classifier, andclose()defers the native release until in-flight calls finish.ModelCache.evictreturnsfalsefor a leased model. React Native rejections no longer forward the underlying exception message; readerror.userInfo.reasoninstead.Not covered: emulator and real-device runs for the Android matrix, including peak-memory, thermal-class and leak-check evidence. Interrupting an in-flight native ONNX Runtime run is also out of scope, so the cancellation bound is one native window run. The React Native Android module is only compile-checked, against local stand-ins for the React Native APIs it uses, because the repo has no React Native Android build yet. #4478 moves that file, so one of the two PRs will need a rebase.
Validation:
cd android && ./gradlew :openmedkit:testDebugUnitTest(140 tests, 0 failures; 33 new, including virtual-time cancel/close/swap/trim race and stress tests), RN module compiled with Kotlin 1.8.22 against React Native API stubs (pass),pytest tests/ -q -m "not slow and not fuzz" --ignore=tests/unit/training/synthetic/test_dicom_overlays.py(27921 passed, 271 skipped, 298 deselected, 11356 subtests passed; the ignored file needspydicom, which the shared venv lacks),ruff check .andruff format --check .(pass),scripts/security/check_effect_paths.py --check(pass),mypy(pass).kotlinx-coroutines-testwas already a test dependency, so there are no new dependencies and no lockfile changes. Release AAR size: 565,571 to 655,245 bytes (+89,674), which is over the 589,824-byte limit inandroid/gradle/budgets.properties. I left the limit unchanged, soverifyAndroidBudgetswill fail until it is revisited. The strict MkDocs build is left to CI.