Running background jobs with Asyncio

Coding Python Support
1

EDIT: this solution is terribly dated and perhaps wrong, I didn’t check. I just fixed some formatting here and the thread appeared on top again, so sorry – I didn’t mean to bump.

Over the last few days I’ve been coding a script that silently runs in the background and only from time to time, it decides to download some objects from the Internet. It was a job assignment, details are not important. You will face the same issue if you make any interactive network communication in Blender.

There seems to be a recommended approach to this in Python, and it is the async/await scheme. Asynchronous download of a webpage may look like this:


import asyncio
import aiohttp

async def fetch_page(url):
    with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            content = await response.read()
            # do something with the content

task = asyncio.ensure_future(fetch_page("http://blender.org"))
asyncio.get_event_loop().run_until_complete(task)

The beauty of Asyncio is that multiple such requests can be handled in parallel, although there are no threads involved.

If you would run exactly this code in Blender, it would block the user interface until all pages are downloaded. To make it work, we need a bit of magic – namely:

  • Tell Blender to go through our task list several times a second
  • In each go, handle web pages that have just been downloaded

A nice solution to (1) is to have a modal operator and set a timer to it, with period much below a second. Like this:

wm.event_timer_add(0.01, context.window)

The trick in (2) is to let Asyncio just once read through the list of pending tasks, and return as soon as possible. This surprising snippet does just that:


loop = asyncio.get_event_loop()
loop.stop()
loop.run_forever()

I learnt this elegant idea from the Blender Cloud plugin. It is actually a documented feature, but a bit hard to find: despite the loop is stopped already before starting, it will visit each task once.

All combined together into a handy package:


import asyncio
import bpy

timer = None

class AsyncLoop(bpy.types.Operator):
    bl_idname = "asyncio.loop"
    bl_label = "Runs the asyncio main loop"
    command = bpy.props.EnumProperty(name="Command",
        description="Command being issued to the asyncio loop",
        default='TOGGLE', items=[
            ('START', "Start", "Start the loop"),
            ('STOP', "Stop", "Stop the loop"),
            ('TOGGLE', "Toggle", "Toggle the loop state")
        ])
    period = bpy.props.FloatProperty(name="Period",
        description="Time between two asyncio beats",
        default=0.01, subtype="UNSIGNED", unit="TIME")

    def execute(self, context):
        return self.invoke(context, None)

    def invoke(self, context, event):
        global timer
        wm = context.window_manager
       if timer and self.command in ('STOP', 'TOGGLE'):
            wm.event_timer_remove(timer)
            timer = None            
            return {'FINISHED'}
        elif not timer and self.command in ('START', 'TOGGLE'):
            wm.modal_handler_add(self)
            timer = wm.event_timer_add(self.period, context.window)
            return {'RUNNING_MODAL'}
        else</b>:
            return {'CANCELLED'}

    def modal(self, context, event):
        global timer
        if not timer:
            return {'FINISHED'}
        elif event.type != 'TIMER':
            return {'PASS_THROUGH'}
        else:
            loop = asyncio.get_event_loop()
            loop.stop()
            loop.run_forever()
            return {'RUNNING_MODAL'}

def register():
    bpy.utils.register_class(AsyncLoop)

def unregister():
    bpy.utils.unregister_class(AsyncLoop)

if __name__ == "__main__":
    register()

The asynchronous web download then looks much like above, just with an extra operator call:


asyncio.ensure_future(fetch_page("http://blender.org"))
bpy.ops.asyncio.loop()

The difference is that the former code did exit as soon as the page was downloaded, whereas this will silently loop forever. Fortunately, you can call bpy.ops.asyncio.loop(command=‘STOP’) as soon as the page is downloaded, and get the same result.

btw, I marked this thread as Solved because I am just happy how I solved the task :slight_smile:
Another note: the aiohttp library is great, but has to be installed. By default, Asyncio only allows you to read/write files and raw TCP/IP connections.

2 Likes