Use-able loops 'within' bge python

Hello everyone,
I have been experimenting with some of the urllib stuff lately and much of this code contains loops. I was wondering if anybody has been able to have loops running ‘along side’ the bge. I don’t actually need any data back from the specific functions that I am calling and so if it had to run on a separate ‘process’ or ‘thread’ that would be fine. From the looks of things, it would seem that I want to use something like the multiprocessing module or maybe the concurrent futures. But I haven’t had much success.

Any help is greatly appreciated! :yes:

EDIT:
I suppose that being able to ‘run’ a python script outside of the bge would work as well, but I would need to know how to launch a separate program from within bge. I will look into it though.

Sure you can run other Python threads.

What exactly do you need to loop through?

Well currently I am downloading a file from google drive. This is successfully done using:

from urllib.request import urlretrieve
from bge import logic


path = logic.expandPath('//')
file = 'DLTest.blend'
url = 'https://docs.google.com/uc?authuser=0&id=0B1A3fiPFsaZMc3hQZE5KVFRzWFU&export=download'


urlretrieve(url, path+file)

It would seem that there is a loop somewhere in urlretrieve. As long as the file is downloading blender is “locked up” and appears to have stopped responding. So I want to be able to run this code on a separate thread.

I do not know much about that. But I suggest to look if there is a non-blocking way to run that.
If not … a thread is your friend.

To be honest I would place such activities into a separate “updater” application. I my opinion a game should not do internet activities if it is not part of the game (e.g. multiplayer, chat etc.). But that is just my personal taste.

Some games do use this for loading files from internet servers if they are user-provided maps, but in this case I suspect that isn’t necessary. I would agree, it’s a nicer user experience and more error-proof to have a separate, offline application do this for you :slight_smile:

Loops themselves aren’t what causes Python scripts to “Block”. It is only when they are doing a lot of something which takes a large amount of time, or when they don’t exit (in the case of while loops which can never evaluate to False).

In programming, some operations are known to be blocking - that is, they take a finite amount of time and cannot be checked at a later time, they have to perform the operation, to the detriment of other code. Socket IO (like file IO) is blocking. However, socket’s can be asked to perform requests in a non-blocking fashion, but the urllib module doesn’t implement this for http requests.

You can either use something like Eventlet or Twisted, which enable you to perform non-blocking requests, or create a thread which consumes tasks from a Queue and puts the results into an output queue.

I wrote a simpler interface to the Thread module class which makes this quite easy:
https://github.com/agoose77/PyAuthServer/blob/master/bge_network/threads.py

To use this yourself, you’d want to subclass SafeThread (it handles cases where the thread ends before the BGE disposes data)

Example file
loader.blend (498 KB)

I don’t see how it’s simpler. I think it needlessly complicates matters in this context, as your example clearly shows: You have 3 files, containing ~200 lines of code, for something that could be done in ~20:


from bge import logic
from urllib.request import urlretrieve
from threading import Thread

path = logic.expandPath("//DLTest.blend")
url = 'https://docs.google.com/uc?authuser=0&id=0B1A3fiPFsaZMc3hQZE5KVFRzWFU&export=download'

def load(cont):
    own = cont.owner
    own["download"] = Thread(target=urlretrieve, args=(url, path))
    own["download"].start()
    print("Loading ...")

def handle_loaded_data(cont):
    own = cont.owner
    download_finished = not own["download"].is_alive()
    if download_finished and not "done" in own:
        print("Downloaded {} successfully!".format(url))
        logic.LibLoad(path, "Scene", async=True)
        own["done"] = True

However, our examples are not identical.

  • The proxy.py file is there for good reason. It prevents the BGE from throwing an exception when the embedded player is exit and a thread object is stored in a BGE object. If you care to remove it, you’ll reduce the amount of code considerably.
  • Both get_result and add_task are aliases / simple wrappers that could be removed if one desired.
  • set_default is a convenience function, which handles the case when the one function is called before the other. If you use a controller marked for execution then this can be removed
  • I have included a simple callback to handle the loaded file, meaning that you do not need to change the execution code to add new functionality.
  • My threads.py module is designed with the premise that you will have a consumer thread that works on data you supply using queues. If you have other tasks which require threading, then you can either write a consumer subclass of thread, or make a generic consumer which receives a callback and data, and runs the callback on the data. The point being that the extra code is there to make things easier, and readable.

I believe in keeping things simple where it makes sense, but I think that there is a tradeoff between aggressive simplification and readability and extensibility

Yea I did eventually get it working on my own, though it was horrendously sloppy. I have tried Goran’s example (Agoose’s was giving me errors) and it worked perfectly! I did remove the LibLoad line but that is simply because I do not intend to immediately load the .blend file. Thank you guys!
:yes:!

Kendrick, did you try the file? The script was an older iteration, and doesn’t include the additional imports.

It doesn’t make sense in this case?

For examples, offered to others on this forum, I think it makes sense to prioritize simplicity/readability.

If/when people actually encounter those potential problems, then it would be appropriate to introduce structures that address them, but until that time, extra code (that doesn’t really need to be there, for the context in question) will just serve to confuse.

Ah, I had tried using your threads module (the github thing). I didn’t even actually see that you attached an example :o. Yes when I use your example file it works as well. :slight_smile: