Spawnable thread or process objects

Hello blenderheads!

I’d like to dynamicaly create objects with threading functions!

There’s a main object whic is a controller tt feeds data into the other process objects. the process objects have infinite threaded functions, that simultaneously process and return data to the main object.
The problem is that as soon as I create one object, the evil while loop stops te entire process, when it should run on its own process…
Here’s an example of my attempts:


import multiprocessing, threading
from multiprocessing import Process
from threading import Thread


class Controller(Thread):
    def __init__(self):
        self.child= []
        self.serial = 0
        
        
        
        for aa in range(10):
            exec("self.Thread"+str(aa)+"= Thread")                  # create an unique threads?
            exec("self.Thread"+str(aa)+".__init__(self)")
            


    def deploy(self):
        # Generating objects
        for q in range(10):
            exec("Child"+str(self.serial)+"= child(Cont)")          # Add an object
            self.child.append([eval("Child"+str(self.serial)), 0])      # Add a reference to the object in the controller
            #exec("self.Thread"+str(self.serial)+".start(Cont)")# Initiating threads
            self.serial+=1
    def run(self):
        self.deploy()
        
        for a in range(10):
            b= self.child[a]
                
            exec("self.Thread"+str(a)+"(target= (b[0]).run(Cont))")       
        while 1:
            print( self.child[0][1], self.child[1][1], self.child[2][1], self.child[3][1], self.child[4][1], self.child[5][1], self.child[6][1], self.child[7][1], self.child[8][1], self.child[9][1]) # Printin operations results from the mai object


class child:
    def __init__(self, cont):
        
        self.serial= cont.serial
        self.num= self.serial
    def count(self, cont):     # Supposedly continuous method, out of the while loop
        self.num+=1
        cont.child[self.serial][1]= self.num
        #print(self.num)
    def run(self, cont):
        self.count(cont)      # Run the objec's fnction


Cont= Controller()    # Initiate controlelr object
Cont.run()
        




Edit: Cleaned up he code a bit…

Use Thread.start() instead of Thread.run(). run() just executes the thread without forking a new process.

Hmm, will see!
Thank you Mobious! Most likel, I’m structuring the whole thing in a not so clever way! Bad design?

At least it is very hard to read. What are you trying to achieve?

Parallel processing is a large resource for hard to discover problems. Better avoid it unless you are sure the benefits are stronger then the drawbacks ;).

I actually don’t really have any experience in using multiple threads in video game design, and I’m not sure how common it is to do so or when it is appropriate. Why are you trying to use multiple threads in the first place?

And yes, your code is a bit hard to read. Read over this Python style guide to get a good idea of what’s good practice.

Ah, I see!
This is the basis of my network setup!
There’s a controller object that stores the whole data, and there are satellite objects that distribute and process the data(the servelets).
Normally I make so each server waits it’s turn to run. The controller at it’s max capacity should have 100 servelets running simple operations and handling 256 users.

In the script, the Controller object dynamically adds new threads on initialization. Then it sets the child object as a thread target. The threads should continuously running in parallel! But having a while loop on any of the objects causes the entire setup to block.

I want to have independent objects running and feeding the controller, I can dynamically create different applications and run them on different python environments, but will still have to use a file or sockets to connect with the controller…

“While loops” are Evil!

For network you need just one parellel prosess. I think this is already provided by the Python library. You have the option to check the communicaton queue in a non-blocking manner when you need it. I do not think you need a lot of parallel processes. Mostlikely it will be slower than singlethreaded or doublethreaded.

I think agoose can you tell more about that.

I believe Monster is talking about the select.select function. It allows you to input a group of sockets and determine which ones can be read, written to, or have an error. It is a blocking command that will return when one of the sockets meets the criteria.

Well, my setup is supposed to dynamically add servers as the number of clients increases. Each server is basically a socket used to comunicate with the clients.
Because of the number o sockets receiving data, the solution I found is to thread them. Before I know is a bottlenck, I ned o make it work.
Going back to basics, I realised, threads work better in a procedural environment rather than an OOP environement.

So, yeah, let’s see how it goes!

The sample code that I pointed you toward handles multiple clients connect at the same time. The sample clients just don’t stay connected long enough for multiple clients to really connect at the same time. The part where the server makes an ‘accept()’ call is where a new client is spawned and tracked in a ServerClient object.

Because of the number o sockets receiving data, the solution I found is to thread them. Before I know is a bottlenck, I ned o make it work.

The most efficient use of the computer’s resources is to spawn the correct number of threads for your system. If you have a single core then running 4 threads might actually be a performance hit. If you have a quad-core system then running 4 receive threads would generally increase your performance.

is there a way blender can check how many threads or resources a computer has?

is there a way to setup a auto LOD in the video card? move as many processes to threads as we can?

I know some problems have to be linear, but for the most part, many different pieces @ work build the scene every frame,
Can a outside program edit a .blend while it is in the ram?

The sample code that I pointed you toward handles multiple clients connect at the same time. The sample clients just don’t stay connected long enough for multiple clients to really connect at the same time. The part where the server makes an ‘accept()’ call is where a new client is spawned and tracked in a ServerClient object.

I saw it, however, since you use a BGE server… also it’s incompatible with my game’s data structure!
Also, I’m running the server on a Remote server, it will be running almost nothing more. Besides, the server operations are as simple as possible to prevent lagging. So far, I was able to spawn 128 threads without any sort of problems, but I still have to communicate wit the threads to know how it will go!

128 threads = 128 possible sources of problems when sharing data :wink:

what about having a “game” that is the server, that has no textures, and only physics etc, and have everyone input effect the scene, and they just get back (your here, that is there, this changed etc)

that way everyone is getting the same data, the only thing that is different is where there input control socket links to each player on the server, then there “client” has open GL Etc? and prettyness

that way people can’t hack, and lag is not a issue for cheating with lag-mods, etc, because no client side data effects the server,
unless you want it to,

128 threads = 128 possible sources of problems when sharing data

That’s why I keep it simple!
The thread mission is to listen to the clients it holds(serves) and set that data to the controller, which does the heavy work… See the threads as sightly smart sockets…

is there a way blender can check how many threads or resources a computer has?

Normally, IMHO, threads are parallel operations, handled by the processor(s). Depending on the CPU architecture, threads will be handled differently, resulting in high performance for some, and lack of it for others. If you can use system functions on the BGE, you might get the CPU/GPU specs, maybe!?
BTW, why do you ask? Normally a game has minimum requirements, to avoid such questions…

what about having a “game” that is the server, that has no textures, and only physics etc, and have everyone input effect the scene, and they just get back (your here, that is there, this changed etc)

that way everyone is getting the same data, the only thing that is different is where there input control socket links to each player on the server, then there “client” has open GL Etc? and prettyness

that way people can’t hack, and lag is not a issue for cheating with lag-mods, etc, because no client side data effects the server,
unless you want it to,

Yeah, that’s a basic online game for you…
In my case, there’s no physics in the server side, it’s pure python server. All users in the same level receive the same data…

what’s the reason behind your questions?

Hey. So, in my version, I just made the thread controller a basic object that has a list of thread objects. That list gets added onto when you add a new thread object, and running the controller’s start() function starts all of the threads (run() is just the overridden function; the start() function queues the thread up to run alongside the main, BGE / Blender Python thread, as far as I can tell). Note that I’m a super noob at this, so I’m not sure if this is the way it’s 100% supposed to work, but it seems to be working alright. The code could be cleaned up a bit, too…

Anyway, run it, and notice that two threads will be running. After five seconds, five more threads will be added into the list and started (really could be part of the add() function, but whatever). After ten seconds, the threads should basically all end. They might stick around for a little while (up to half of a second) because of the time.sleep() function within their run functions. Anyway, if you stop the BGE before the 10 seconds are up, the threads will continue to run in the console inside of Blender! So, be sure to start Blender from a console if you need to so that you can stop the threads (you could also use Task Manager, I would assume).

ThreadingTest.blend (467 KB)

That perfectly fine if your server is running on a dedicated machine that is not under Blender. The client code still shows one method to do socket communication without blocking the game engine. There is no reason why the single BGE thread could not have multiple sockets open and doing work at the same time.

However, if you don’t want to mess with working the socket programming into the BGE game-loop then a solution has already been suggested. You need to “start()” your thread instead of just calling the “run()” method yourself. That will cause it to run in a separate thread from the BGE and won’t block up the game.

As I’m very interested in threading, I would like to see more about this subject, as well. Your implementation is very interesting. Why is the controller a thread, when it spawns other threads? Wouldn’t it make sense to just make the controller a simple object that houses other threads (and collects data from them as necessary)?

Yes, the controller is not a hread. This is what I learned:

  1. Start all threads from the main loop!
  2. If you start an object with loops with a thread, the thread will take over the main loop.
  3. The same happens if you spawn a thread from within another or from a running object.
  4. Using dynamic process is more troublesome, they don’t end on their own ad seem to muliply like a plague (a least in my setup). Stopping them via task manager causes sysem instability and he processes can’t be run ill you restart the PC!

So instead, I spawned as many threads as necessary , like openning all gates to the controller. Besides that, I have login and registration objects running to save users to the database and add tem to the controller list.
The users will fill in the “seats” in the socket objects and when the limit s reached, the next ones will use the next in line.
In the controller, users within the same level, receive the same data.

I’ll now go back to the BGE and start creating the world in order to add checkpoints.

Thanks a lot for your assistance!

Edit:
One more question: How do I make a program run in the background?
I tried to start and minimize a program , but even tat doesn’t work. I used : os.startfile(file).minimized()

wow, ok, :slight_smile:

anyway, is there a easy way to see how many threads a system has from python?
the UI?
I was genuinely curious, and my on topic stuff is gone as well…

Yea you can run threading.active_count() to see how many threads you have started from the main thread.