MicroPython compatibility

[mhsmith]

Work in progress for making Toga sufficiently compatible with MicroPython to work with Invent.

PR Checklist:

• All new features have been tested
• All new features have been documented
• I have read the CONTRIBUTING.md file
• I will abide by the code of conduct

[mhsmith]

OK, MicroPython is now able to import all the Toga modules that would be required to support Invent's current features. Actually testing those modules on MicroPython can wait until a future PR.

Main changes:

• Added a CI job which builds MicroPython for Linux with the same configuration as PyScript, and verifies that it can import the necessary modules.

• Added a toga.compat package which monkey-patches the stdlib to add the functions that Toga uses but MicroPython doesn't provide. This package is only imported when not running on CPython. In the future, most of this should be upstreamed to micropython-lib.

• Changed many of app.py's imports to be on-demand, so if an app doesn't use a feature, its module is never imported.

• Removed dependencies:

  • Third-party importlib_metadata package – replaced with a wrapper function that emulates the Python 3.10+ entry_points API on older Python versions.
  • email.message.Message – the only method we call is get, so replaced with a dict.
  • weakref.WeakValueDictionary – #2517 already added code to clean up the WidgetRegistry when a window is closed, so the weakref functionality is no longer needed. Replaced with a dict.

• Syntax limitations:

  • MicroPython doesn't support / syntax for positional-only arguments, which Toga only used in Protocol classes. There's an alternative syntax of a leading __ on the argument name, but that would harm readability of the documentation. And as we discussed a few months ago, the Protocols exist primarily for the documentation, and we're not currently attempting to support their use with type checkers. So I think removing the / syntax is acceptable.
  • MicroPython doesn't support [] syntax for generic types, e.g. Sequence[int]. This doesn't matter in type annotations because they aren't evaluated at runtime, but it does matter when the [] expression is used as one of the bases of a class declaration. I couldn't find a workaround which allowed both the [] form and the bare form to be used on the same class, so I settled on the following, based on Toga's most common usage:
    • collections.abc classes can only be used in the [] form.
    • typing classes can only be used in the bare form.

[HalfWhitt]

I've tried various search terms, to no avail. What's Invent?

[mhsmith]

https://github.com/invent-framework/invent

It's a PyScript-based GUI framework created by one of our colleagues at Anaconda. How exactly we'll be working with them is TBD, but getting at least a subset of Toga to run on MicroPython is a prerequisite.

[HalfWhitt]

Huh, interesting. Seems like it's currently an orthogonal alternative to what Toga does. I guess it might potentially use the Toga web backend instead of its current internal web implementation? Or Toga's web backend might use Invent?

[mhsmith]

There may still be two separate backends, since toga-web provides a more traditional app structure with menus, toolbars and dialogs, while Invent is focused on more web-based idioms. But hopefully at least the widget and layout APIs can be shared between them.

[freakboy3742]

A couple of Qs inline, but largely looks good.

My biggest concern is the extent to which we're going to degrade the non-MicroPython developer experience to satisfy the constraints of MicroPython - I've flagged a couple of them; the / usage in the protocol definitions is another.

[freakboy3742]

Idle thought - should the compat module name be based on sys.implementation.name in some way? I don't have any idea which other implementations we'd be concerned about, but at the very least it would be clear from the namespace that toga.compat.micropython is a micropython compatibility shim.

[mhsmith]

Since there won't be any other implementations in the foreseeable future, let's cross that bridge when we come to it. I see this compat package as a temporary stopgap anyway – virtually all of it should eventually be upstreamed to micropython-lib.

[freakboy3742]

We need to be careful that this doesn't introduce leaks elsewhere. This was a WeakValueDict historically to ensure that there wasn't a reference loop between the ObjC and Python objects; that might not be an issue any more given the work done on the window/widget lifecycle, plus the in-flight Rubicon reference handling changes - but we should confirm this is true.

[mhsmith]

The registry was made a WeakValueDictionary in #2066 because at that time it stored references to all widgets which existed. But then #2320 and #2517 tightened that up to include only widgets which are attached to a window which has not yet been closed.

That means that if a widget is in the registry, it must also be reachable via a chain of strong references from App.windows. So I don't believe the weakref functionality is being used anymore – widgets are being removed from the registry before their weakrefs expire.

[freakboy3742]

I suspect you're correct here - and the memory leak tests in the testbed should confirm this.

[freakboy3742]

I get why this has been done, but I don't love it as an approach - it feels a bit too much like the properties of one particular medium making the ergonomics for every other medium materially worse. At this "proof of concept" stage I can live with it; but as a long term solution, I'd rather see a dummy DocumentSet implementation that is used when the platform doesn't support documents.

[freakboy3742]

Similar to DocumentSet - I don't love it, but I can live with this for now.

[mhsmith]

Are you OK to merge this now? Even if we're not actively using MicroPython yet, it would be useful to have the CI job to make sure we maintain this initial level of compatibility.