Shared memory between isolates

[themisir]

I'm currently maintaining hive package - somewhat widely used key-value database on flutter projects. For a few years one of the biggest issues of hive was multi-isolate / parallelism support. Because users might need to access or write data on background services like work manager, alarm manager or FCM notification handler, etc. But because Dart doesn't have a concept of shared memory I don't think if it's possible to implement ReadWriteLock for database to make sure there's no concurrent writes to the database so the data won't break randomly. Due to that users have to deal with isar/hive#263 issue which in the end lead to some of them abandoning hive and having to use libraries that uses native APIs like shared preferences which is not comfortable for storing complex data structures.

Why doesn't ports isn't enough ​

Currently the ports has to be explicitly transferred to background isolates as entrypoint argument. That limits user experience so they either has to pass Hive's port or their own communication port to their isolate argument. Or some libraries might even doesn't support passing ports to background isolates. Also it creates dependency of background isolates from main isolate - main isolate has to create ReceivePort and pass its send port to background isolate but in some cases there might not be a "main isolate" - eg: the background isolate is created for alarm service when the app is not running, so there's no main/UI isolate.

Also as far as I know, there's no mechanism of port discovery in dart isolates. For example we could tag created ports with unique keys that isolates could retrieve them using the keys they created with. If this is possible in hive I could create a port identified with database name for each database file and when opening the same database on different isolate I can check if there's a port taged "hive-mydbfile" I could use that port as ReadWriteLock's communication layer.

PS: I've checked dart API docs for isolates, send and receive ports and I have not found any clue about identifying Ports like I said in previous paragraph. If there's already a feature for doing that I would appreciate to learn more about it.

Shared memory

Another solution would be sharing memory between isolates. If this would be possible I can just implement a locking mechanism on shared memory and use it for syncing read/write calls to database files. But referring on similar issues I don't think dart team willing to implement this feature anytime soon, rather they want to focus on improving already implemented features.

References:

• Read write lock - IBM
• ReadWriteLock class - C# docs

I'm currently exploring other ways of solving this issue on hive, but again I think having one of those features might help to solve another issues

[Levi-Lesches]

There is some discussion of threads and shared memory at #333 (it's not my expertise though, so you decide if this is a duplicate).

[themisir]

There is some discussion of threads and shared memory at #333 (it's not my expertise though, so you decide if this is a duplicate).

Thanks. Yes its partially related. I'm closing this in favor of #333.