[BGMC29] - Online Retro Zombie Shooter - wkk.py (delayed)

upbge

(wkk.py) #1

Ok folks,

Still having commitment issues, as usual, but here is my attempt at a setup for a game, hopefully my entry to the BGMC #29 :slight_smile:

Here is the repo:

And here is a link to the latest published artifact of the thing:

DISCLAIMER: Right now there is only the base for an auto-installing game, no gameplay yet, sadly…

Everytime I push a new commit, it should update the archive, hopefully.

The goal was to be able to publish a game that would install dependencies as required, to allow for more flexibility on the dev side (such as being able to use most packages from PyPI).

I am starting to be happy with how it turned out, now I would be curious to know if everything works on everyone’s machine?

I have a few ideas about what to actually do for this BGMC, nothing really set in stone. Just want it to be networked somehow, but the details will come with the actual mini-game ideas. I already made asyncio run in the BGE, so there’s that :wink:


UPDATE 23-02-2019:

Finally set up the game server:
https://bgmc29-server.herokuapp.com/

You can get the sources for it here:
https://gitlab.com/marechal-p/bgmc29-server

Now to finish the server logic…


(BluePrintRandom) #2

image


(wkk.py) #3

@aWeirdOwl just tried and he had issues with the Python Component auto-run being off.

So, I don’t really know how to set this to on exactly, but it should be somewhere in UPBGE’s preferences.


(wkk.py) #4

We are coming closer to the deadline, and I am still short a game. Maybe I just don’t understand BGMCs.

But I haven’t been slacking, my goal was to make something online with the excuse of making it for this BGMC, turns out I suck at finding game ideas and commiting to it.

Speaking of commit, here is a repo to the game server I did setup:
https://gitlab.com/marechal-p/bgmc29-server

The actual server can be accessed here:
https://bgmc29-server.herokuapp.com/

I wrote the server in TypeScript, because I simply find the way to work with it amazing, and a super productive language, once you are a bit familiar with it, writing any server is a piece of cake.

For technical details, I first wanted to use UDP for communication in my game, and while this is still a possibility if I make the game clients connect in peer to peer via hole punching, I won’t be doing that because it is an amazing pain to implement correctly (but it is doable given enough time).

So I went with WebSockets, which are built on top of TCP, but still have this nice message-based flow.
(this is also the sole protocol game-able that Heroku seems to be supporting…)

The server currently doesn’t do much, but if you have NodeJS installed on your machine, you could write a small script to try it out already:

// test.js
const WebSocket = require('ws')
const socket = new WebSocket('https://bgmc29-server.herokuapp.com/', {
    'bgmc-username': 'blender-artists'
})
socket.on('message', message => console.log(message))
socket.on('close', () => console.log('closed.'))

And then node test.js.

Ninja-edit: Despite the fact that I wrote a client example in Javascript, the protocol is the same when using Python, and thanks to the websocket package, it should work as well in the BGE with my setup!

Samurai-edit: Updated the test code following the new improvements of the server, namely the player username is passed in the protocol upgrade header.

Next to do is finalizing the game server logic, and actually making a game :thinking:


(wkk.py) #5

Ok, so I kept on working on the server, I have a friend trying his hand on the zombie AI, but I think I will end in overtime for this BGMC.

Biggest issue encountered so far: Struct management in NodeJS.
It is a big pain in the butt, as I expected it to be as easy as Python’s struct module.
Boy was I wrong.

But this got figured out, not the best way ever, but at least I have something working as of this commit.

Will keep puking code until I get something worth sharing again :slight_smile:


(wkk.py) #6

More updates:

Finally improved the protocol, I can now send the data to synchronize the different entities, although the server currently only serve the position of the players.

On the BGE I am now able to open a websocket connection to said server, and I can also parse the structs, which wasn’t as straightforward as I initially thought it would be, but still much easier than on NodeJS.

TL;DR:

  • Server sends every player position/rotation/velocity
  • Client can parse every entity position/rotation/velocity

To do (short-term):

  • Add a type on an entity (player/zombie?)
  • Have clients identify their own entity to skip it
  • Merge the patch adding zombies (any game is cooler with zombies)

I don’t think I’ll finish on time, but oh well :frowning:
Maybe I’ll be able to finish before midnight in Montreal, we’ll see!

Note: Here is how I had to parse the binary data on the Python client:
https://gitlab.com/marechal-p/bgmc29-retro/blob/master/retro/protocol.py


(Nicholas_A) #7

so are you saying we can finally make a global online game?


(wkk.py) #8

When you say global, what do you mean?

In my case, you all and me as well will be able to play together, for sure :slight_smile:


(Nicholas_A) #9

ay thats crazy! can’t wait. Did you do that udp hole punching thing or whatever?


(wkk.py) #10

No, would’ve taken too much time, which I am already short off :wink:

In this case there is a NodeJS server managing one unique game session for everyone.


(Nicholas_A) #11

so what is this server? a website or is this peer to peer? or what


(wkk.py) #12

It is a web server indeed, but I am using websockets, which are a protocol update, from an HTTP request.

I wish I could’ve used UDP, but websockets are good enough for this, and they are also the only realtime protocol available/supported on Heroku :slight_smile:


(Nicholas_A) #13

keep up the good work. i cant wait to make a game with this


(wkk.py) #14

Ok, I think I am officially out of time :slight_smile:

I will keep progressing during the next week, although I will rather go slower, too much code for me!

So far, what I’ve been able to do:

  • Identify each client so that he can ignore itself
  • Server decides where you spawn
  • Entities are correctly broadcasted, instantiated with a type

I was mostly having troubles with clients uploading their position, maybe it is the fatigue?

Once clients actually update their position, I will need to progress on the following:

  • Player input
  • Zombies

But right now, sleep :wink:


(wkk.py) #15

Time for an update: The client finally uploads its position to the server!

The reason for my struggle was the NodeJS library to manage binary structures (sigh), illustrating once again that if a tool is whacky or undocumented, you are going to lose a lot of time.


Breakdown of the issue:

I am using TypeScript, which is some kind a Javascript with static typing and several “fixes” to make it more intuitive to use (if you don’t know what I mean, try writing OOP in JS…)

But the npm package struct that I use to create and parse binary structures is written in JS, not TS, which means that I don’t have any static helper for the methods. Thankfully, in TypeScript you can write yourself a small typing file to describe what the third party module will do. Basically you can add type information to an existing pure JS library, for you to use in TS.

The package being not very well documented, I had to guess a bunch of things, which I did wrong of course. But I had written my wrong assumptions down in the typings already, so my tools when writing code as well as the actual TypeScript compiler never complained, as they assume type information is always right… In my case it was not :frowning:

So it is this kind of issue where you have to put everything in question until you figure out what goes wrong silently, lot of console.log and print and sadness to figure out if either Python or NodeJS was the issue (of course it was node, duh).


TL;DR: When using non-ergonomic tools, be prepared to lose time uselessly fighting.
(although the library does what it is supposed to do… It is a pain, but it works)

What was finished:

  • Clients upload their position to the server
  • Server broadcasts every entity position

Where I loosed a lot of time again:

  • Infamous struct parsing in NodeJS
  • Trying to make a mist fade effect, turns out UPBGE won’t hear any of it :frowning:
    (took 15min to write to effect, 2h to debug for nothing… actual debugging would be using gdb, but not today, other priorities sadly).
  • Found a bug when on Windows where the assets aren’t correctly loaded (path issue).
    (UPBGE seems to interpret r'.\blends\player.blend' as r'C:\blends\player.blend' somehow on Windows, even though the actual cwd is the game folder, what the duck?)

Still gotta work on user input now, then zombies still.
But priority to bugs, so first the fix for Windows, then player input, then we’ll see :slight_smile:


(Cortu) #16

I would be interested in knowing how you are preventing tcp from blocking game tics. I have looked threw your code, and noticed that you are using asyncio. My experience with that module is negligible. A tcp packet may take 30 to 600 ms to complete, or about is 2 to 30 game tics. This means you have to rerun your scripts while the packet is still downloading. Dose asyncio allow you to do this? If not, how are you handling the issue? Networking is one of the main things I have not quite got the grasp of yet.


(wkk.py) #17

There are multiple ways to go about this.

Usually, people would use worker threads, and have their server run there, and maybe even dispatch request handling in yet another threaded worker pool. It works but it is meh at best, with all the issues from thread parallelism (lock and joy).

Another way would be to handle sockets using an I/O selector, which is a system object that can tell you if a resource can be used (readable/writable state). So you would register your sockets to this central selector, ask it the state of the file descriptors, and perform your operation on sockets that have data, so the call to recv(buffer) won’t block, because you know there is data buffered, ready to be read.

The selectors module from Python standard library does that, and it is what I was using before.

Now comes asyncio, the idea is that you will have an “event loop” provided by the framework, and this loop will be your main application main loop, and will block everything until the thing stops somehow. In my case I run the loop in bursts, so that it only blocks for half the frame time, and then lets Blender/Python continue, then back to blocking on the following frame, etc…

The event loop job in this case is to monitor I/O operations (mostly network I/O), but also bundles a coroutine/task scheduler: You can basically write asynchronous code that will pause itself while waiting for something else, example:

    @asynchronous
    async def mainloop(self):
        ws = self.websocket
        while not ws.closed:

            try:
                message = await ws.recv()
            except exceptions.ConnectionClosed:
                print('connection closed.')
                bge.logic.endGame()
                break

            type, raw = parse_message(message)
            handler = self.HANDLERS.get(type, None)
            if handler is None:
                print(f'Unknown message type: {type}')
                continue

            handler(self, raw)

            # [...]

From: https://gitlab.com/marechal-p/bgmc29-retro/blob/master/retro/network/message_handler.py#L46

In the above function (which should be registered in the asyncio event loop to run), once we will execute the await ws.recv() line, if data is not yet ready, the event-loop will pause the execution of the function, and look for someone else to be executed, like some other coroutine that would be ready to run (data arrived, some future resolved, etc…).

Event loops are a wonderful way of writing asynchronous applications in my opinion, so having asyncio run in the BGE is really a big step forward to me.

Only issue is that asyncio makes the developer having to know a lot of things, while if you look at how NodeJS works: most people don’t even know there is an event loop running, yet everyone is able to use it.


TL;DR: I just run my network operations inside asyncio's event loop, which I run in bursts for half a frame.


(wkk.py) #18

To detail a bit more how this specifically works on asyncio: Until the packet arrives, the event loop will just keep looping and polling the low-level resources to see if something is ready. No logic code is ran in parallel, just asyncio doing its thing, and executing your callbacks/resuming coroutine execution until the next thing to await. Everything happens in only one thread, we just do nothing when there is nothing to do and skip to the next frames.


(Fred/K.S) #19

Pretty interesting I’m curious to see how this one’s going to turn out later on.

Fred/K.S :sunglasses:


(Cortu) #20

Ok, I think I got it working. Let me guess why it dose. I have tried, and read about, various ways to have a process run across multiple ticks. For the most part this has been pretty unsuccessful. Perhaps the reason why is that the processing was happening above Bge’s ticks. The key thing seems to be that asyncio.get_event_loop() is some sort of Singleton. Pythons internal singletons appear to operate below the tick. For instance, when you import a module, it only gets imported once. This allows you to store information in the module between ticks. Sys.excepthook seems to operate below the tick as well. Regardless, if this actually works in an actual game, then it will be a very big deal indeed.
I personally may not try it soon tho. I’ve decided that the next time I tackle networking, I will do it with a server that I have root access to. It will be a while before I have the resources to secure such a computer, and I have my mind on a few other projects any ways.
If this works in an actual game then it is almost certainly the way to do it. Thank you for taking the time to explain it to me! :grin:

import bge

if 'frameTime' in bge.logic.globalDict:
    bge.logic.globalDict['frameTime'] += 1
else:
    bge.logic.globalDict['frameTime'] = 0
print('\n-'+str(bge.logic.globalDict['frameTime'])+': tick-')


from asyncio import get_event_loop, sleep
from random import randint

event_loop = get_event_loop()

async def longTask(taskId, secs):
    print(str(taskId)+': sleep for '+str(secs)+' seconds.')
    message = await sleep(secs)
    print(str(taskId)+': slept for '+str(secs)+' seconds.')

event_loop.create_task( longTask(bge.logic.globalDict['frameTime'], randint(1,10)) )
event_loop.run_until_complete(sleep(0))

Occasionally I would get the following error, but it doesn’t seem to effect game operation. It probably has something to do with the python executable not being full reset when using the Embedded Player.

Task was destroyed but it is pending!
task: <Task pending coro=<longTask() running at Text:22> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f4b43e64408>()]>>