"Server-based?" Networking

I am currently trying to work on a simple game to test out some networking and learn sockets. Currently the setup is practically the exact same as the bgmc networking template (wish I had found this before I wrote the code – would have saved so much time). It works: the client connects, and creates a player entity that represents it on every other client that is at the same position/orientation.

Now, what I struggle to understand is how to have objects that aren’t determined by clients:
The game is based on trying to avoid obstacles (just avoiding swinging pendulums and stuff), and every client basically needs to be “synced to the exact same time” that every obstacle is. The problem there, however, is that the client can’t determine this, because then there would be multiple entities of the obstacles all at different timing at the same place. How do I go about making it so the server determines the positions.orientations of those objects on the map?

This probably has an obvious answer. My mind just isn’t in the right mood right now :wink:

Thanks :slight_smile:

Here’s the bgmc_net.py code:


from socket import AF_INET, SOCK_DGRAM, socket
from pickle import dumps, loads
import urllib.request as urlReq
from time import time
   
def client(controller):
    from bge import logic
    scene = logic.getCurrentScene()
    objects = scene.objects
    ob = controller.owner
    
    if not hasattr(logic, 'user_id'):
        logic.remote_users = {}
        logic.memory = {}
        logic.user_id = time()
        logic.client = socket(AF_INET, SOCK_DGRAM)
        logic.client.setblocking(0)   


    player = objects['Player']
    data = [logic.user_id, player.worldPosition[:], player.getAxisVect((0,1,0))[:],player.getAxisVect((0,0,1))[:]]
    packet = dumps(data)


    logic.client.sendto(packet, (ob['server'], 35107))
    
    try:
        received_packet = logic.client.recv(1024)
        logic.memory = loads(received_packet)
    except:
        pass
    
    active = []
    for user_id in logic.memory:
        user_pos, yVect, zVect = logic.memory[user_id]
        
        if user_id in logic.remote_users:
            logic.remote_users[user_id].worldPosition = user_pos
            logic.remote_users[user_id].alignAxisToVect(yVect, 1)
            logic.remote_users[user_id].alignAxisToVect(zVect, 2)
        else:
            ob.worldPosition = user_pos
            new_user = scene.addObject('RemoteUser', ob)
            new_user.worldPosition = user_pos
            new_user.alignAxisToVect(yVect, 1)
            new_user.alignAxisToVect(zVect, 2)
            logic.remote_users[user_id] = new_user
        
        active.append(user_id)
    
    for uid in [uid for uid in logic.remote_users if uid not in active]:
        logic.remote_users[uid].endObject()
        del logic.remote_users[uid]
    
class netServer:
    def __init__(self):
        print('''
        [ Simple Python Server - BGMC Edition ]
        
        Make sure that port 35107 is forwarded
        and players will be able to join at:
                ''', str(urlReq.urlopen("http://icanhazip.com/").read()).split("'")[1][:-2], '
')
        
        self.memory = {}
        self.history = {}


        self.socket = socket(AF_INET, SOCK_DGRAM)
        self.socket.bind(('', 35107))
        self.socket.setblocking(0)
        self.serve()
    
    def serve(self):
        while True:
            try:
                packet, address = self.socket.recvfrom(256)
                data = loads(packet)
                
                user_id = data[0]
                user_sd = data[1:]
                
                if not user_id in self.memory: print('User %s joined' %user_id)
                self.memory[user_id] = user_sd
                self.history[user_id] = time()
                
                data = {_id:self.memory[_id] for _id in self.memory if _id != user_id}
                packet = dumps(data)
                self.socket.sendto(packet, address)
            except:
                pass


            inactive = [ uid for uid in self.history if time() - self.history[uid] > 1]
            
            for uid in inactive:
                del self.memory[uid]
                del self.history[uid]
                print('User %s left' %uid)
            
if __name__ == '__main__':
    bgmcServer = netServer()

Mine’s slightly different, as I wanted the server and client to be separate, but basically works on the same concept

So , how do we put it to a test ? What do we need and what are missing ? Complicated to set it up ? You know we have to test over the internet not just over local network.

I’m basically trying to find out how to determine the location/orientation of objects according to the server (for map objects) rather than the clients (for player positions etc.).

You have to simply rotation to be represented as an offset from North direction. The entire rotation be a single INT from 0 to 360 (only Z axis) or from -360 to +360 or from 0 to 720 degrees. I mean , there is no way you can add 64 players on a map if you have to update an entire 4x4 matrix each frame for each player and do it 60 times per second.

4x4 x 64players x 60fps = 61440 floats/sec

I have to look into it cause I don’t know how to do it either.

Try to do a local test first , with the print function , to see how many objects can you update and how often.

You create a channel for server information, either by making fake server users or establishing a more robust system. Eventually you will need more than just positions and rotations, and it will require more complex systems. I’ve written a library for this, check the support forum

Sent from my Nexus 5 using Tapatalk

This seems to print the local rotation in Z axis. Can anybody guess how do we print out or extract the rotation in Z axis out of that messy matrix ?

print(obj.localOrientation.row[0][0])
print(obj.localOrientation[0].xyz)
print(obj.worldTransform)
print(obj.localTransform)

I still don’t know how the matrix works and which one is the Z axis.

print(obj.worldTransform.col[3][2])

That seems to show the translation in world z axis.

When do we get to test the Tunnel Runner over the internet ?

@blenderer2012
MrPutuLips is asking about networking. Unless you have something to add on the subject, please stop posting

@MrPutuLips
I went through the exact same process about a week or two ago.
The BGMC networking script is great and simple, but the only thing that gets transmitted across the server is the player objects position and rotation.
The magic line that generates the packet sent is line 20. Anything in the list created in line 20 gets transmitted across the network. Then, at the other end, it has to be decoded. That is dealt with using what I consider a rather hacky solution. The packet data is dumped into the bge.logic module as the variable bge.logic.memory then the for loop on line 32 goes through stored list of players, and reads from the stored packet.

So, how to hijack your own data to be transmitted?
First you’ve got to insert it into the packet in line 20.
Then you’ve got to write some code to extract the data from the packet in bge.logic.memory. Because the bge.logic module is shared across the whole game, any script that imports bge will have access to this data.

What data should you send?
For swinging things that have to be aligned perfectly, I’d send a couple of things:

  1. rotation of the pendulum
  2. The velocity of the pendulum (angular)
  3. the time it was at that rotation

Then on the client receiving the information, you can apply some high-school physics and extrapolate the position of the pendulum forwards, and the put it there.

Bear in mind that if you have 50 swinging things, bandwidth is going to be a huge issue here. You may also run into problems with different peoples pendulums setting the pendulum’s position differently, so it may jump a bit. To offset both of these, have only one person sending the data on the pendulums. I can’t think of a good way of deciding who does this over the network at the moment. I’m sure there is an easy solution.

what about using a sequencer for the pendulums? that way only 1 object needs updated each frame.

always----------and-----------action property based - “sequence”
_________________----------copy property sequence from sequencer

and feed each pendulum the sequence by feeding a host, and having them copy?

that way they can all be different using there own action but remain in sync?

The only thing I hate about having yet another client to fake it, is that it introduces even more data to be processed and sent. I suppose if it’s on the same PC, connecting to localhost, it shouldn’t cause much of a hassle.
I have used your networking addon before and it looks great, I just don’t understand most of it because it’s so glorious in length and stature, which is why I am trying to understand things from the ground-up. I didn’t understand how to do this in your addon either, unfortunately.

Another pretty hacky solution. Using the loop to directly control the attributes of the pendulum should actually work, though it would kind of annoying to have to write an extra line of code for each pendulum.

Sending the data attributes only once seems like a good idea (maybe once every 60 seconds or something) in case of a dropped packet.

Yeah I guess I’ll have to do some client-side optimizations for keeping everything in sync. I can only imagine how terrible it would be if there was any packet loss.


Well I think this is what I’m gonna try for now:

  1. Have the client include the server
  2. Define if the client should be a host or not.
  3. Send map data only from “host” client
  4. Receive map data only from host client on “client”.
  5. Calculate how the objects should move, rather than waiting for the next packets.

Thanks.Will post back with results.

I am not talking about my addon, there’s a pure python library that gives powerful control over what I’d sent, and when

Sent from my Nexus 5 using Tapatalk

Yeah I guess I’ll have to do some client-side optimizations for keeping everything in sync. I can only imagine how terrible it would be if there was any packet loss.

just have the sequence march on if it’s not changed,

property interval min/max(end frame-1)---------and add 1 to property
if property=end frame---------and-----------property=0

Was doing a bit trying to fake it by having another “fake” client, and it kind of works. It’s hell of a pain to have to edit parts of the map in two different clients though. Going to be trying directly created packets from the server with the % at which the pendulums/obstacles are at in their loop. Should be easy enough (I hope).

An idea I had is if it is possible to entirely sync the game to the time that the server is at. Basically, on connection to the server, get the time variable and then “fast forward” to that time. Would this be a good method? To me it seems like it would be quite difficult to get it to sync in a short amount of time.

This can be done in a variety of ways. Unity has RPC calls that are remembered and replayed when new clients connect. This isn’t a bad system, but perhaps a little clunky. The way that I, and Unreal solve this issue is to maintain a sense of state that belongs to a network object. This state is its attributes and RPC calls. If the attributes have been changed since the start of the game, the server will send the changes the first time the object is created for a new client. You want to have a more abstracted sense of state that can be modified by the system without changing core replication code. I solve this by defining a system which uses handlers of data types to read or write data to bytes, and separate the interface for handlers (so they can be added later on). The base network object supports the addition of attributes in the class definition, and acts as an interface for the network system. The point of all of this separation is that the developer should ultimately need to write very little network code. The only code needed to be written may be the authentication information.

This way, a map becomes a network object with physics, (a position etc) and some other features that may be of use. If a door is opened, it would be a network object with physics that has a state attribute. When this is changed it could trigger a door open animation.
The biggest problem that networking involves is how to make it as complementary (least invasive) as possible - so that your code isn’t fully aware of what happens behind the scenes.

Okay, so it’s been a little while since I was working on this. Wasn’t in the mood for some intense thinking :stuck_out_tongue:

I decided to go for a less realistic approach for the networking and have a pretty decent method to align some obstacles:

  1. Server keeps time
  2. Server sends time to clients
  3. Client uses math.cos() to determine the location of the obstacle (this can be done for rotation too). (this way it varies between -1 and 1)

Problem I am encountering: I changed the server so it sends the packet data including the time of the server, then the client receives it, but the problem is that, because it is a while loop, that it isn’t actually ticking the logic, therefore the time just sits at 0…

Is there a way to keep the loop and logic running?

Thanks
EDIT: Sorry to make this post so long, here are the scripts just to clarify things a bit (still using that bgmc networking template, but a bit edited):
Server.py


from bge import logic
from socket import AF_INET, SOCK_DGRAM, socket
from pickle import dumps, loads
from time import time


class server:
    def __init__(self):
        print("Server Running. Port 45245")
        
        self.memory = {}
        self.history = {}


        self.socket = socket(AF_INET, SOCK_DGRAM)
        self.socket.bind(('', 45245))
        self.socket.setblocking(0)
        self.serve()
    
    def serve(self):
        own = logic.getCurrentController().owner
        print(time())
        print(own["timer"])
        while True:
            try:
                packet, address = self.socket.recvfrom(256)
                data = loads(packet)
                
                user_id = data[0]
                user_sd = data[1:]
                
                if not user_id in self.memory:
                    print('User %s joined' %user_id)
                    
                self.memory[user_id] = user_sd
                self.history[user_id] = time()
                
                data = {_id:self.memory[_id] for _id in self.memory if _id != user_id}
                packet = dumps(data, own["timer"])
                self.socket.sendto(packet, address)
    
            except:
                pass
    
            inactive = [ uid for uid in self.history if time() - self.history[uid] > 1]
            
            for uid in inactive:
                del self.memory[uid]
                del self.history[uid]
                print('User %s left' %uid)

Client.py


from bge import logic
from socket import AF_INET, SOCK_DGRAM, socket
from pickle import dumps, loads
from time import time


def client(controller):
    scene = logic.getCurrentScene()
    objects = scene.objects
    own = controller.owner
    
    if "server" in logic.globalDict:
        if not hasattr(logic, "user_id"):
            logic.remote_users = {}
            logic.memory = {}
            logic.user_id = logic.globalDict["username"]
            logic.client = socket(AF_INET, SOCK_DGRAM)
            logic.client.setblocking(0)
    
        player = objects["Player"]
        data = [logic.user_id, player.worldPosition[:], player.getAxisVect((0,1,0))[:]]
        packet = dumps(data)
    
        if own["ticker"] > 1/20:
            own["ticker"] = 0
            logic.client.sendto(packet, (str(logic.globalDict["server"]), 45245))
        
        try:
            received_packet = logic.client.recv(256)
            logic.memory = loads(received_packet)
        
        except:
            pass
        
        active = []
        for user_id in logic.memory:
            user_pos, yVect, systime = logic.memory[user_id]
            
            own["timer"] = systime
            
            if user_id in logic.remote_users:
                logic.remote_users[user_id].worldPosition = user_pos
                logic.remote_users[user_id].alignAxisToVect(yVect, 1)
            else:
                own.worldPosition = user_pos
                new_user = scene.addObject("RemoteUser", own)
                new_user.worldPosition = user_pos
                new_user.alignAxisToVect(yVect, 1)
                new_user.children[0]["Text"] = user_id
                logic.remote_users[user_id] = new_user
            
            active.append(user_id)
        
        for uid in [uid for uid in logic.remote_users if uid not in active]:
            logic.remote_users[uid].endObject()
            del logic.remote_users[uid]
            
        print(own["timer"])

The server doesn’t interact with the actual blend, right?
So you can just dump it in it’s own thread. Have a look at the multiprocessing module (orthis resource is also great)

That’s not necessarily the best solution, nor the reason you face your problem. If you dedicate a thread to the network code you still must handle the data and BGE data interaction, which would require some form of messaging system.

You simply need to break when you don’t receive any data (change pass to break)

Follow up:
I see you mention “server”, which indeed doesn’t interact with BGE data.

Well I am trying to get it to interact with the blend. I tried to get it to multithread after reading that but it just keeps creating more threads. I think I did something wrong. I’ll see if I can get it to work.

Is there a way to do it without a while loop? That seems like the simplest solution if possible.
Also, break doesn’t work because the server just ends and restarts rather than keeping a connection. Do you mean for the client?

You simply need to ensure that the initialisation code is run once and the rest of the code is run from a module controller.

You cannot directly use that code with multithreading, it is not safe to modify BGE data from different threads.

I mean something like this


from bge import logic
from socket import AF_INET, SOCK_DGRAM, socket, error as SOCKET_ERROR
from pickle import dumps, loads
from time import time




class server:
    def __init__(self):
        print("Server Running. Port 45245")
        
        self.memory = {}
        self.history = {}


        self.socket = socket(AF_INET, SOCK_DGRAM)
        self.socket.bind(('', 45245))
        self.socket.setblocking(0)
    
    def serve(self, own): 


        while True:
            try:
                packet, address = self.socket.recvfrom(256)
                data = loads(packet)
                
                user_id = data[0]
                user_sd = data[1:]
                
                if not user_id in self.memory:
                    print('User %s joined' %user_id)
                    
                self.memory[user_id] = user_sd
                self.history[user_id] = time()
                
                data = {_id:self.memory[_id] for _id in self.memory if _id != user_id}
                packet = dumps(data, own["timer"])
                self.socket.sendto(packet, address)
    
            except SOCKET_ERROR as err:
                break
    
        inactive = [ uid for uid in self.history if time() - self.history[uid] > 1]
        
        for uid in inactive:
            del self.memory[uid]
            del self.history[uid]
            print('User %s left' %uid)
            
def run(cont):
    own = cont.owner
    
    try:
        serv = own['server']
    except KeyError:
        serv = own['server'] = server()
    
    serv.serve(own)            

Why run the server through Blender at all? Shutting down threads that are waiting on blocking networking calls is a real pain through Blender. Much easier to run the code through a Python interpreter instead.