Use PyPy for pure python code #20
Comments
|
It's MIT licensed, so that shouldn't be a concern. Seems like it would save a lot of work. |
|
Definitely support this but from my brief look at PyPy stdlibs it seemed like there was a fair bit of weirdness going on with some of the PyPy code, e.g. https://bitbucket.org/pypy/pypy/src/61294de6f130/pypy/module/itertools/?at=default I have no problem begging, borrowing and stealing what we have to to get the stdlib functional. |
|
Yeah, that is strange. Definitely optimized for PyPy. That module in particular depends on PyPy's jit package. |
|
Another option is https://github.com/pybee/ouroboros |
|
Interesting I did not know about that project! I'm a little unclear about the origin of the libraries. Seems like some of it's from PyPy. I'm concerned about the licensing. It looks like a non-standard license in addition to PSF: https://github.com/pybee/ouroboros/blob/master/LICENSE Does anybody know if that project includes novel code or is it all from other places? |
|
@trotterdylan It looks like most of it is from PyPy/Python, so you can just grab the standard licensed files. |
|
@trotterdylan You should ask @freakboy3742. He's the mastermind behind BeeWare (pybee) |
|
You can certainly use PyPy code as a starting point - but "Just Use PyPy" isn't a complete solution, because there will always be system native components. For example, you can't implement a socket library without being able to access sockets. I've got similar needs to Grumpy on my projects pybee/batavia and pybee/voc, which is why I started pybee/ouroboros. Most of the code that is already there has originated from either PyPy or CPython. I'd be delighted to collaborate on Ouroboros with anyone from the Grumpy team (and any other Python implementations, including PyPy) who shares the need of having a Python standard library that isn't derived from CPython. |
|
Thanks for sharing, Russel. A collaboration sounds really interesting. A common pure-Python subset of the standard library that could be shared by all the different interpreters would make implementing Python runtimes a lot less of a chore. It reminds me a bit of the old STLPort project that was so useful when every C++ compiler had its own STL quirks and limitations. Things get really interesting when you start to think about how to pare down the native integrations to get platform-specific features working. Like, if a runtime implements and exposes a POSIX interface to Python then most stuff should "just work". Another area for investigation is how to enable optimization. ISTM that the design of many of the CPython libraries was dictated by what needed to be written in C. The re library is the obvious example here. If there was a more generic and flexible way to specify what ends up being implemented natively, then maybe it'd be easier for many runtimes to share the same standard library. Anyway, happy to chat about this more. It'd be really cool if Grumpy could just use Ouroboros more or less as-is. Ideally the Grumpy repo would not include the stdlib or it would include Ouroboros wholesale rather than piecemeal from CPython and PyPy. |
|
@alanjds There are also times where it's entirely possible to implement a module in Python, but a native implementation may give a significant speed boost (e.g., Decimal was a pure Python module in Python 3.4, but was turned into a C module in Python 3.5 for performance reasons). Getting (and documenting) a clear definition of the "native" API surface is a key piece of the puzzle; as is defining a compliance test suite to confirm that implementations are compliant. And I agree that, ideally, Grumpy, VOC, or any other implementation wouldn't include a copy of Ouroboros, but rather "link" to it in some meaningful way - that's what both VOC and Batavia currently do; they require an Ouroboros checkout as part of their build process, and "compile" the Ouroboros sources to the "native" representation needed for their own operation. If you've got time to hook up for a chat, I'd love to discuss the potential for collaboration here. Shoot me an email and we can coordinate details. |
|
Hi, @freakboy3742. I got puzzled that you replied here, not on google/grumpy... then I realized the mailbomb I triggered with this multiple issue imports :/ Sorry for that. About the collaboration: Sure. Will mail you as soon as I got to migrate all this issues/comments to grumpyhome/grumpy repo. (See: #8) |
google#60 opened by @santagada on 6 Jan 2017
On the current Readme it states:
"But there are also a number of libraries in CPython that are C extension modules that need to be rewritten. This includes re, itertools and others."
I think that at least itertools is implemented in app level python on PyPy, why not just use them instead of rewriting modules?
The text was updated successfully, but these errors were encountered: