[Dev] How would you like to see threaded LibLoading work?

Making this available via GUI seems to be a good idea. The status could be automatically returned into a property similar to the ActionActuators frameProperty.

Even without the access to the loader the newly added objects could announce themselves after loading. Simply send a message triggered by an AlwaysSensor :). Indeed this does not help if the objects are halfway loaded :stuck_out_tongue:

@Agoose77: If something is available via Actuator does not mean it is not available via Python.

I really like the idea of possibly having a loading bar to show progress. I canā€™t see how you could accomplish that with a callback.

The callback would return the percentage of data loaded, which you would use the scale the loading bar.

Future object. For the same reasons already stated by Monster.

Woah! Cmon, really? 1 logic brick, thats really impossible .

I think he meant 1 sensor and 1 controller. Itā€™s possible now, actually - looping through objects, performing their logic for them, animating with Python - most everything that can be done with a logic brick has a Python equivalent.

Which reminds me that it would be cool if 2D filters could be pushed to Python.

I donā€™t mind too much either way; both can do what we need, and both can be wrapped to behave like the other. But I think Iā€™m leaning towards the future object. The ability to cancel would be nice, which would seem to be more natural with a future. And as for the return type, simply always return a future. The only difference would be that for a synchronous load, future.progress = 100% by the time it is returned.

A callback sounds good if it can monitor progress. We can always wrap it in our own object.
Then againā€¦ if a future object can support cancelling, I would prefer that.
/Undecided

I think I would provide callbacks for LibLoad. Just being able to load an object and have it run a function when its done would be much easier than having to check the future object to see if it has completed, and then run a function. Callbacks just seem more intuitive in this way for me.

Iā€™m not sure how it would work, but having a callback function that returns the progress of the loading file would be awesome, but this is not essential. Loading a file without that awkward pause will be more than enough for this. I also agree that if your poking around in the source code, to maybe try and unify the variable names so they follow the same naming convention over the entire API.

ASync loading will be a fantastic addition to the bge. With the project Iā€™m working on at the moment, I was just about to ā€˜get usedā€™ to the pause when libloading files, but hopefully this development has come along and solved that problem nicely :slight_smile:

Do you have any updates on this moguri?

Oh, and Iā€™m also in favor of unifying the variable names.


For callbacks:
* Consistent with BGE API (e.g., render callbacks)
* Doesn't require maintenance (set them up and they get called when needed)
* Less new code in the codebase (easier to review, implement, etc) #<<<<<<< thats is matter!

For future objects:
* Easier to debug/read        #<<<<<<<<<(this should be a option usenless)
* Similar to Python's threading API

callback again!

PS: to import ALSO the script togheder to the obj (as the brick do) change something callback or future?

Could anybody show me an working example of callbacks with bge logic?
There are so many fans of callbacks. Maybe Iā€™m missing something.

With the rather mixed response to future objects versus callbacks, Iā€™m thinking it might be best to actually implement both. My thoughts are a future object that can also be used to register callbacks. To start, the future object will have the following properties and methods:


progress -- estimate of how much of the load is done (will just be 0 and 100 to start)
libraryName -- the name of the library being loaded (the first arg to LibLoad)
timeTaken -- the time it took to load the library
onProgress -- callback to be used when the progress of the load updates
onFinished -- callback to be used when the loading is done

By having LibLoad return this object, it would still be simple (although maybe not entirely obvious) to just use the callbacks and not mess with the future object much. So instead of:


bge.logic.LibLoad('mylib.blend', 'Scene', async, async_callback=llcb)

you could do:


bge.logic.LibLoad('mylib.blend', 'Scene', async=True).onFinished=llcb

@Monster
On of the main reasons I like them is I can set them and forget them. In other words, I donā€™t have to waste time constantly checking in on things. This is one of the nice things about event driven programming, which callbacks facilitate.

Hi,

I know about callbacks and I know they are heavily used in a lot of frameworks. As Java RCP-Developer Iā€™m more used to listeners rather than callbacks. Both systems are quite nice and encapsulate ā€œuser logicā€ from the core processing.

I wondered about the statement ā€œConsistent with BGE API (e.g., render callbacks)ā€. As far as I see render callbacks belong to Blender not to the BGE respectivly the BGE logic.

Currently I canā€™t see callbacks somewhere at the current BGE logic. That was the reason why I asked where to find them.

Because of that I think you need to add a complete new concept on top of or beside the existing event systems (SCA). Iā€™m worried that this is a bit out of scope of the topic ā€œThreaded Loadingā€. I fully understand your question what we would like to get.

Somehow I have the impression some people see callbacks as the ā€œmagical super weaponā€ against logic bricks (which seems to be not that super ;)).

One of the problem I see with callbacks is, they do not have the usual context of the BGE Python code. E.g. current controller, current object, current scene. The context would be essential as the user would try to get the information from the callback into the BGE logic.

It is also a timing issue.
When is a callback called? (At sensors evaluation? At controller execution? At actuator running? What is the order? )
With multiple scenes running the scenes are processed one after the other. When does the callback trigger? (I guess loading belongs to a specific scene).
This is important as the user usually needs to know when the code is executed (before or after other logic).

Callbacks completely bypass the SCA (as they do not fit into the SCA event-system). This means they will not be available for non-programmers.

The progress bar feature is mainly asked by noobs (I guess because advanced BGE users know it is currently not possible). Beginners are usually not really hot to add code (if they even know what a callback is).

Anyway, if moguri can add both then Iā€™m happy about it.

In near future a developer should be able to wrap your future object with an LoadingActuator. The benefits are:

  • easy to use GUI (useful for beginners and experts)
  • perfectly fitting into the SCA-scheme
  • user defined activation
  • auto-deactivation after loading -> which means you can use an actuator sensor
  • (optional) similar to the ActionActuator it can write the estimated percentage into a property -> Progressbar!
  • as usually the actuator can still be configured and activated via Python controller.
  • Python purists still do not need to use them as it is a wrapper.
    Disadvantages:
  • someone has to implement is :wink:

I do not think we need this as logic block right now, but we should keep this in mind and in the design.
Iā€™m not sure if the ActionActuator is even possible with a callback.

Thanks for reading
Monster

To be honest, the main reason people want asynchronous loading of libraries, is so that their game doensā€™t freeze for such long loading durations. However, by using a callback, which is called on_finish (or something like that) then you cannot check the status of loading. Thus, you would still have to display the loading screen during that time, the only difference is that now your loading screen could have some moving images.
If you were to use a future object, the ability to read an attribute that returned the percentage or data progress, then you could go one step further and have a moving, realtime progress bar.
With the aforementioned feature, it would be much harder to find an advantage of any other engine to the BGE.
I thought, at first, that Monster was asking for an Actuator - only loading feature. However, if he believes that a solution that incorporates both would be good, then I am in agreement with him. The ability to load and set a property to the progress of a loading library would greatly open up new oppertunities to non-programmers.

why not use multiprocessing which is the only module that can do true threading and take advantage of multiple cores ā€¦

Hi Moguri, the future + callback design looks good. Nice work!

@Agoose77 - You could have a loading bar with a callback. Moguriā€™s example already shows that it would allow you to see how much of a percentage an object or scene is loaded - you could set the loading bar scaling in the middle of that (instead of printing out the percentage of the object loaded like in Moguriā€™s example).

@Monster - I donā€™t really see the problem with callbacks. You would have to use Python to use callbacks or even have and set up a loading bar, so it seems clear to me that only programmers would have access to them. I donā€™t see how you would make a loading bar with only logic bricks - donā€™t you have to use Python to set the actual scaling variable (can you use a property sensor to set a property to obj.scaling.x, for example)? Youā€™re right that it doesnā€™t quite fit into the SCA, but one could assume that it would run as an ā€˜invisibleā€™ Python controller.

And I like the idea of both. I think a future object that allows you to use callbacks allows you to use what you want and gives you the capabilities and advantages of both. Honestly, since either one allows you to do the same thing, it doesnā€™t really matter (neither is worse than the other), so either way, asynchronous loading would be great to have.

Moguriā€™s example shows the callback function gets called when loading is done = 100%.

Exactly that is one of the problems I see ;). We should clearly avoid solutions that are accessible by Python only. The advantage of the BGE is you do not need to use Python but you can.

With the suggested LoadingActuator it would be quite simple:

  • Set up the LoadingActuator with an progress property ā€œprogressā€ as float.
  • Set up a ActionActuator with property mode playing property ā€œprogressā€
  • Activate the ActionActuator on property change of ā€œprogressā€
  • Activate the LoadingActuator whenever you need

the ActionActuator simply plays whatever you need for a progress bar (e.g. scale) see the HealthBar-tutorial.

Both solutions demand a synchronized value transfer (percentage).

BTW. I suggest to normalize the percentage (0.0 ā€¦ 1.0) rather than (0 ā€¦ 100)

Or you could just use an Action for scale etc. Itā€™s been done for health bars etcā€¦
Iā€™d much prefer an object that i could poll when needed, rather than calling a callback every percentage / tick.
I also agree, that it shouldnā€™t be a Python only solution, when it would be possible to integrate into an actuator.