python multi-threading in Blender


(smn) #1

Hey everyone,

its been a long-long time since i’ve posted anything Blender related on any forum really. Used to hang out @ blender.nl as mono but that’s ages ago.

Anyway, I’m currently involved in a school project (HvU for you dutchies) and we’re looking at the possibilities of developing a multiuser game in blender. I’ve written a server that is able to deal with any client as long as it speaks XML, and so I’ve written a little Python class for Blender that opens a socket connection to the server and talks XML over it. It’s not even close to being complete. But that’s not the problem, the problem is -> how does one program threaded Python scripts that don’t lock up Blender while they are running…

I’ve got quite a bit of Java programming knowledge but Python is relatively new for me. Any help would be greatly appreciated!

Here’s the script, I hope this forum keeps it formatted ok.

thanks in advance, smn.



# simon de haan
# TestClient.py

import socket
from select import select
import sys
import xml.parsers.expat
import time
import threading

class Client(threading.Thread):

	def __init__(self,host,port,username,passwd,channel,terminator='\0'):

		## create the thread
		threading.Thread.__init__(self)
		
		## create the parser 
		self._parser = xml.parsers.expat.ParserCreate()
		self._parser.StartElementHandler  = self.handleStartElement
		self._parser.EndElementHandler    = self.handleEndElement
		self._parser.CharacterDataHandler = self.handleCharData

		## store all data in variables
		self._host = host
		self._port = port
		self._username = username
		self._passwd = passwd
		self._channel = channel
		self._terminator = terminator
		self._connected = 0
	
	def run(self):
		while 1:
			time.sleep(1)

	## XML functions
        def handleCharData(self, data): 
        	print data
        
        def handleStartElement(self, name, attrs): 
        	pass
        
        def handleEndElement(self, name): 
        	pass

	# server connection functions

	def connect(self):
		
		self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
		try:
			self._sock.connect((self._host, self._port))
		except socket.error, e:
			raise error(e)
			return 0
		
		## if we've gotten here the connection is fine, lets report :p
		self.report("connection setup ok")
		self._connected = 1
		
	def disconnect(self):
		self._sock.close()
		self._sock = None
		self._connected = 0
		self.report("disconnected ok")

	def process(self,timeout):
		ready_for_read, ready_for_write, err = select([self._sock],[],[],timeout)
		if err:
			self.report("select returned an err")
		for s in ready_for_read:
			if s == self._sock:
				self.read()
				return True
		return False

	def disconnected(self):
		self.report("network disconnection")
		pass


		
	
	# communication functions

	def login(self):
		self.write('<LOGIN nick="%s" pass="%s" channel="%s" />' % (self._username, self._passwd, self._channel) )
	
	def read(self):
		data = u''
		data_in = self._sock.recv(1024)
		while data_in:
		    data = data + data_in
		    if len(data_in) != 1024:
			break
		    data_in = self._sock.recv(1024)
		self.report("got data %s" % data )
		self._parser.Parse(data)
		return data

	def write(self,s=''):
		self.report("sending %s" % s)
		try:
			self._sock.send (s+self._terminator)
		except:
			self.disconnected()
	

	# debug functions
	def report(self,s):
		print "Client: "+s
		
c = Client("localhost",6543,"username","passwd","Messenger")
c.connect()
c.login()
c.start()

while 1:
	time.sleep(1)
	c.write("<PING/>")




(saluk) #2

Ah, this problem. I’m in the middle of writing my own multiuser game in blender. (I wonder how many others are?)

You need to run it in a thread, but as far as I can tell, it still locks blender up from quitting. So you can’t quit blender. But it will at least run.

Fairly good documentation is at the python site:
http://www.python.org/doc/current/lib/module-threading.html

What I do is run the read/write in a thread, and the while at the bottom you should use a timer for. Ping should only be every once in a while.


(smn) #3

hey thanks for the reply,

the <PING/> is just really a useless tag, it was just for me to test the connection and to see when things seriously locked up. It could have been <BALDMONKEYSRULE/> for all I care…

I know I need to run it in a thread, that’s the whole problem. I’ve got the thing threaded, in the interpreter this works fine I can run two instances at a time… BUT… in blender it locks up, I was just wondering if anyone’s got some golden-tip to help me out here…

How about you? What are you using to communicate between your client & server ? And have you fixed this problem in blender or are you stuck at the same thing ?

thanks anyway! smn.


(theeth) #4

you could probably use the spawnl function in the os module, as it can spawn independant threads if given a true value as the first argument.

Martin


(saluk) #5

Nope, I have everything working. Each client can control a box and move it around. It’s pretty tight. Took ages to get it to work though.

Which thread system are you using? I didn’t ever have luck getting the Thread module to work right in blender, but using the threading module seems to work.

I use while loops in my code, so that’s not the problem. Are you SURE it’s crashing blender? I can’t figure out how to make blender CLOSE once a threads been started (even if you stop it) but blender still executes. Double check by having something constantly rotate in the game window. If it continues rotating, then blender has not really crashed.

Um, actually, looking over your code again, I think I found your problem:
Blender can not run when python is being executed. Since you use an infinite while loop in your code, it never returns blender to normal execution. Either put this code in a thread (bad) or just get rid of the while loop and have it run every frame.


(smn) #6

theeth: thanks a lot! I’ll check the spawnl function.

saluk: using the threading module as I am doing now does not crash Blender, it just doesn’t give the Blender GUI any CPU time (probably an incorrect description…). Uhm when the script is running it communicates fine with the blender module but the GUI doesn’t react to any mouse activity at all… I’ll try setting up the spinning cube & test it with the script. Thanks for the help so far anyway :slight_smile:

smn.


(saluk) #7

Well see, if you have any while loops that run outside of a thread, that will lock up the blender interface, and it appears as though the while loop at the bottom of your script could be the culprit.


(smn) #8

ah. I see. But the strange thing is that if I put the stuff of the while loop (the <PING/>) in the def run() It’ll send it once and then just sorta hang which is really weird. Anyway I’m going the IBC 2002 today in Amsterdam so I won’t be able to play with any code, I’ll post a reply later of my results. thanks again! smn.


(Akhar) #9

I strongly suggest you take a look at stackless python and micro-threads, for your threading problem. it helps by being lighter on system ressources and faster as a whole.
www.stackless.org


(saluk) #10

Wouldn’t that be stackless.com?

Thanks for the link, looks interesting.


(smn) #11

hey everyone, thanks for all the help. The project I’m involved in has decided to go on in Shockwave3D, havoc & Plasma/3DMax. And I’m fine with it really we’ve checked out all the pro’s and con’s & have come to this combination. Thanks for all the help anyway, maybe someday as a personal project I’ll try to hook up blender to my lil’ XML server.


(saluk) #12

Sorry to see you go:(

And yet, this is one less person my multiuser blender server has to go up against:)

Good luck with shockwave.


(smn) #13

oh I’m not going don’t worry. I enjoy messing around with blender, just for this project the game engine doesn’t seem ready yet for what we’re planning for…

I’ll be back, mwuhahahaha

are you using your own game server btw (developed it yourself) or are you using one of the already available ones?


(saluk) #14

It’s my own server, my dad and I are developing it together.

Still having some problems with lag issues, but that’s more to do with our python than blender.


(smn) #15

that’s pretty cool, developed it in python huh?

are you using any of the asynchat, asyncore, SocketServer modules or are you building it straight up from the low level sockets?

I built mine in java with a built in python interpreter that allows me to program plugins in python. Its pretty neat, don’t have any serious lag problems and its pretty fast. We’re getting ready to rebuild the whole thing from scratch though, the way I implemented the python interpreter is causing some serious problems and that needs to be fixed. Maybe someday I can run a demo and I’ll post it here.


(saluk) #16

Sounds awesome. No, we’re going straight from the ground up. Fixed the server last night while also realising some of the “lag” issues are due to the in-game messaging system we’re using to keep tracki of incoming messages. It should all be taken care of tonight, and I can release a demo of some sort in the next few days. I can only hope:)

Then we can start working on NEW features.