[Dev] How would you like to see threaded LibLoading work?

The two main ways of interfacing with threaded LibLoading that I see are callbacks or some kind of future object. Possible example code which each could be as follows:

future object


def do_something():
  print("Scene loaded!")

future = logic.LibLoad('foo.blend', 'Scene', async=True)

# Later
if future.is_loaded:
  do_something()

callback


def do_something():
  print("Scene loaded!")

logic.LibLoad('foo.blend', 'Scene', async=True, async_callback=do_something)

One advantage to callbacks is that LibLoad() will never return anything as opposed to sometimes returning an object if certain keywords are used. This helps keep things consistent. Also, there have been some plans to add other callbacks to the BGE, so a callback for LibLoad() keeps things consistent there too.

However, with future objects, they allow a clear path for future extensions. For example, it could be possible to add some sort of percentage counter to the future object (I have yet to evaluate the feasibility of this though). I guess optional arguments could be added to the callback for things like percentage:


def load_level(percent):
  if percent == 100:
    print("Level loaded!")
  print("Level is %.0f%% loaded...")

logic.LibLoad('level.blend', 'Scene', async=True, async_callback=load_level)

And if the callback doesn’t have an argument, it would only be called when the libload is done.

And one last thought, as a potential shortcut, if async_callback is specified and async is not, should async default to True?

Feedback would be much appreciated!

Cheers,
Moguri

PS
Overall, I’m leaning more toward a callback myself.

What does async do? And I think I’d like callbacks too, since it’s easier to read than objects. You should be able to expand the callback functionality by just looking for additional arguments from the function.

async enables asynchronous loading (i.e., starting a new thread for the loading). Another option for callbacks is to just have an object that you give the callback with various bits of information. Then, if you want to add more information, you can just add it to the callback. For example:


def load_level(info):
  if info.percent == 100:
    print(info.library_name, "loaded in", info.time_taken, "seconds")
  else:
    print("%s is %.0f%% loaded..." % (info.library_name, info.percent)

Would the callback be called asynchonusly, at the start of the logic loop or at the end of the logic loop?

I’d rather the future object, mostly because it fits in better with what I already have. I can deal with the loaded libs when I want to and callbacks tend to be harder to debug because the stack trace becomes less helpful.

The callback would be called when the scene is merged (at the beginning of the frame before any other logic). This could potentially get shifted around a bit.

If you’re creating a wrapper around LibLoad and callbacks were being used, it would be possible to create your own future object:


# Stored "globally" somewhere
libload_futures = {}

class LibLoadFuture:
  def __init__(self, level):
    self.level = level
    self.percent = 0
    self.is_done = False

def ll_callback(info):
  future = libload_futures[info.library_path]
  future.percent = info.percent
  future.is_done = info.percent == 100
  # Or make a method on LibLoadFuture that takes an info object

def MyLibLoad(level):
  logic.LibLoad(level, 'Scene', async=True, async_callback=ll_callback)
  future =  LibLoadFuture(level)
  libload_futures[level] = future
  return future

#some other function
level_future = None
def main():
  if not level_future:
    level_future = MyLibLoad(level)

  if level_future.is_done:
    do_something()

While I’m not saying this is the nicest solution, it does show how callbacks could be used to better fit a framework that would work better with future objects.

However, you do make a good point about callbacks sometimes being troublesome to debug. I would like to keep this interface somewhat simple since the BGE community isn’t exactly filled with Python experts.

it’s a good idea
and as you work on libload you could ensure that we can load a group?

Do it like threads:

Return a reference to the running LibLoad-Thread. It can provide information regarding the current status like LoadingProgress. This enables any code to check if LibLoad is ready or not.
e.g.


def startLoading(cont):
  myLoaderObject = cont.owner
  libLoader = logic.LibLoad(...)
  myLoaderObject["libLoader"] = libLoader
...

def displayPercentage(cont):
  myLoaderObject = cont.owner.parent
  libLoader = myLoaderObject["libLoader"]
  cont.owner.text = libLoader.getLoadedPercentage()+" %"
  cont.owner["percentage"] = libLoader.getLoadedPercentage()
...

def showDone(cont):
  myLoaderObject = cont.owner.parent
  libLoader = myLoaderObject["libLoader"]
  if libLoader.done:
    cont.owner.text = "loaded!"

If you do not care the loading status you do not need to store this reference.
You might want to enhance the module GameLogic to hold all running LibLoad-Threads.
e.g.


libLoads = bge.logic.libLoadThreads
print(libLoads)

You still can add optional callbacks. This would be possible even when LibLoad is already running.
I think they will not always be necessary as loaded game objects will run as soon as the are merged into the scene.

How are you implementing the merge? Do new game objects start as soon as they are loaded or do they start as soon as the last game object is loaded? Maybe a configuration option?

Just my thoughts
Monster

if both here robust, i prefer the more short

important is which all argument recurrent in the function here in the first place(in playAction() to change to play mode to loop mode is pretty complcated!)

Could you use a similar method to preload any resources, not just scenes?

I like the future object better for aesthetic reasons. But I’m not voting because I don’t quite understand the functional advantages of one over the other.

Callback gets my vote.

Moguri,

I think what you call “future” is usually called a “loader”. It manages the loading and merging of the data. Additional it provides the current load status (status provider).

I got the term “future” from this Wikipedia article and this Python object.

Future here!

It would be really helpful if people said why they prefer one format over the other. At the moment I’m still leaning callback, but with a 50/50 split, I’d like to hear more arguments for futures.

For callbacks:

  • Consistent with BGE API (e.g., render callbacks)
  • Doesn’t require maintenance (set them up and they get called when needed)
  • Less new code in the codebase (easier to review, implement, etc)

For future objects:

  • Easier to debug/read
  • Similar to Python’s threading API

future objects:

  • fits better into the logic scheme (e.g You can keep a reference to the loader at a property. You can check the loader, which is taken from the property, for the current status)
  • You might enable the user to cancel and/or hold the loading (accessible via loader reference)
  • You can have multiple loaders at the same time which are distinguishable from each other (or you restrict the implementation to one loader a the time)

callbacks:

  • the called code is separate from the game logic (no current controller/no current object/no current scene) and the logic’s execution time (which means you can’t tell when the code is executed). I guess you will get a hard time to synchronize this with the game loop.
  • Render callbacks seems not to be the best guide (as the name says they are for render not for logic)

I’d prefer future objects, purely because of the programmatic advantages of returning loading data.

BTW. If you redesign the LibLoad already. You might want change the API to the match the BGE naming conventions. It is a bit strange that a few functions start with upper case while all other start with lower case. (Strange enough that there is a mix of camel-case and underline identifiers ;))

The existing names could be marked as deprecated.

Maybe LibLoad is a nice function for a new actuator. Then it would be easy to set an callback in the gui, as you would normally do for module controllers. For future objects you have to poll the status by hand, I think in the mosts cases a simple callback on finish is more than enough.

+1 for renaming those functions. The whole logic module is a little bit messy…

greetings, moerdn

I’d prefer not to have an Actuator. In my current game, I have one logic brick, and one controller :wink: And i’d like to stay that way!
Besides, polling it is normal anyway.