grumpy python blender?

General Forums Blender and CG Discussions
#1

so we have

that fixes c extensions to run on go
so grumpy python can call them.

grumpy py is multithreaded and has no gil.

its a jit python compiler calling go bindings

Grumpy: Go running Python!

Wednesday, January 4, 2017

Google runs millions of lines of Python code. The front-end server that drives youtube.com and YouTube’s APIs is primarily written in Python, and it serves millions of requests per second! YouTube’s front-end runs on CPython 2.7, so we’ve put a ton of work into improving the runtime and adapting our application to work optimally within it. These efforts have borne a lot of fruit over the years, but we always run up against the same issue: it’s very difficult to make concurrent workloads perform well on CPython.

To solve this problem, we investigated a number of other Python runtimes. Each had trade-offs and none solved the concurrency problem without introducing other issues.
MeatGrinder.png
So we asked ourselves a crazy question: What if we were to implement an alternative runtime optimized for real-time serving? Once we started going down the rabbit hole, Go seemed like an obvious choice of platform since its operational characteristics align well with our use case (e.g. lightweight threads). We wanted first class language interoperability and Go’s powerful runtime type reflection system made this straightforward. Python in Go felt very natural, and so Grumpy was born.

Grumpy is an experimental Python runtime for Go. It translates Python code into Go programs, and those transpiled programs run seamlessly within the Go runtime. We needed to support a large existing Python codebase, so it was important to have a high degree of compatibility with CPython (quirks and all). The goal is for Grumpy to be a drop-in replacement runtime for any pure-Python project.

#2

bumpitronomus

could blender c bindings be called by go with this and then grumpy call go?

#3

This statement is the most interesting.

The goal is for Grumpy to be a drop-in replacement runtime for any pure-Python project.

But then again, Blender is not a pure Python project (and the language is only used as an API for game logic, the UI, and addons) so it may actually be more complicated than just ‘dropping it in as a replacement’. I won’t even try to explain details, as I’m sure Campbell can do it far better :slight_smile:

#4

I don’t think this is interesting for Blender, Python is used as a scripting language and relies on Python’s dynamic features. It can’t work as a drop-in replacement.

This is mainly interesting for people that have made the (popular but arguably misguided) decision to implement their web backend in Python, somehow need better performance, but don’t have CPython extension modules as hard dependencies. I doubt that’s a big audience.

CPython extension modules are very important (Blender’s Python API itself is an extension module), lots of popular libraries and library bindings are implemented in this way. It’s a major reason why faster alternative runtimes (such as PyPy) aren’t more popular.

#5

the link above states that you can import go libs

it also has a link to a app that converts c/c++ bindings in assembly to go assembly (so you can than import the lib)

#6

Grumpy is Python2.x only, further c2goasm doesn’t seem very mature (limit of 14 args).

In practice most of the hassles with these kinds of projects ends up being getting all the libraries to correctly link and load correctly.

#7

That doesn’t mean CPython extension modules magically work. The CPython (python.org) Python interpreter has a C API that alternative implementations have to either emulate (performance hit) or ignore (extension modules don’t work). That’s one reason to prefer pure Python modules, but lots of important libraries are not pure Python.