Managing a large blender game

Game Engine Game Engine Resources
1

Question: What happens when you have a large blender file with dozens of scenes, hundreds of models and a few dozen scripts.
Answer:
Chaos

It is clear that there must be a better way to manage things.
Well, this unfortunately isn’t a “This is the ideal way to do it” Rather it is just “Heres one way, there may be a better one” So here goes.

1) Plan for it to be big from the beginning.
It is a lot of work to split up a single large blend into multiple small ones. So when you start a game, I recommend planning for it to grow really big. Even if it doesn’t get so big as to being unwieldy, it still helps with updating the game once it’s finished (ie patches).

2) Make what can be separate separate.
Texture files can be packed, but then say you want to edit one, you have to un-pack it, edit it, and then replace the old one. If it’s separate than you just have to edit it and hit the reload button. The same goes for sounds and such. If you’re worried about people stealing your textures, well, what use is a UVtexture without the right model?

3) Split that blend.
If you are using a system like dropbox or another syncing system, then people get annoyed at every few minutes a new 20mb blend file updating, while you are only changing a small part of it. So instead, split the blend file logically and link them together. Then when you update the blend it only has to sync maybe 3 or 4 mb.
More info when I cover filesystems

4) Separate out the scripts.
Make the scripts external, as long scripts don’t work well in blenders text editor, due to not being able to “roll” up chunks of code, leading to long periods of scrolling up and down to check the name of a function at the bottom.

5) Standardize formats:
For all images use format X, for all sounds use format Y etc. This helps make sure that everyone can read and edit them. It also makes it look more professional. How many high quality games have a bizarre combination of .tiff, .tga, .png, .jpg all for images. At the most you’ll want two image types, one with transparency, and one without

5) Get a decent filing system going.
Everything in one folder is just as bad for tidiness as having it all in the blend. So get a decent system going. Not also have good folder names, bu also have good file names.

6) Communicate all this to your team members
If this is a team project, then make sure everyone else knows about it. A good way is a flow-chart, like the one attached below.

Screenshot.jpg647×700 97.5 KB

7) Stick with it.
Once you have a system in place, keep using it.

If you have any thoughts or spot any mistakes, post below and I will update it

2

I totally agree. I can’t name all of the projects I’ve quit after putting months of my time into…just because everything was so unorganized and I didn’t want to go back and sort everything. Thanks for the advice! I’m definitely going to use the flow-chart idea.

3

If you use separate files you get following benefits:

  • changes to the game result in a few changed files
  • repositories prefer a lot of small files rather than a few large files (when using binaries) [e.g. Subversion, git, dropbox)
  • easier to work in time as it avoids conflicts (working at the same file at the same time)
  • easier to patch (less files to replace)
  • it forces you to encapsulate separate units of the game = better design
  • it improves re-usability (you can simply copy a file to another game)
  • improves readability of the game structure/design
  • easier to implement different protections for each file (licences)
    edit:
  • easier to find duplicates

Disadvanages:

  • multiple files to distribute
  • hard to integrate all files into one
  • harder to add [techn.] protection (encryption etc.)

From my experience it is really important to establish the file structure at the beginning. It is really hard to change it later. Blender does not support re-factoring very much.
The Outliner can be a great help but it still is limited.

4

Really nice thread. But seperating files might get quite problems with materials and textures. So as to avoid huge RAM usage you need to use if possible the same materials and textures for many different objects. So if you split them you also need to import materials and textures to the seperate blends. That could get really complex.

Best would be to have a blend with all the materials and textures so you could just import the scene from the blend to any file. But I don’t know if that would be a good idea. Then you need some sort of ghost object for creating each material.

5

Good thread, keep those hints tips and advice coming!

6

I agree, good Thread!
Still there is one Point not handled:

  • Storing data from the game (save game)

Where and how?

Use a file folder/file system?
Use the globalDict and store it into a .bgeconf file ? There will be a problem saving multiple files, because the .bgeconf can not be renamed, means it has to be the name of the blend ( i might be wrong )
Use sql? Wich one ?

I tried all of them, and in a way i guess sqlite3 is my favorite.
This could make it possible to store data on a server, allows to create unlimited Worlds etc.
Any suggestions?

7

You can use pickle.
To save, you dump a dictionary in the text file.

Example:
Save


import pickle   

#The information you want to store 
data = {'Level': '5','Gold':100}
   
# Write to the file 
file = open('/optionalPath/save1.data', 'wb')
pickle.dump(data, file) # dump the object to a file 
file.close()

Load


import pickle

# Read back from the storage 
file = open('/optionalPath/save1.data', 'rb') 

storedData = pickle.load(file) # load the object from the file 
print(storedData)
8

I couldn’t agree more. It is more effort from the start, but all of the largest games seem to need this strategy. What’s more my poor computer cannot handle updating 50-100 Megabytes every time I move an object, or adjust a texture.

9

I do not really agree with the statement of the Image file type.

Yes, be consistent. The arguments in post #1 are valid.
No, use the format that fits most. But remain consistent withing the usage.

*.jpg

  • with the content changing compression it is good for images with high noise
    Examples: sky, background, photographs

*.png, *.tga

  • is good if the content needs to be preserved as it is
    examples: bitmap text, geometric shapes, straight lines, slow gradients.
  • transparency
    examples: bitmap text, plants

…

10

import pickle #The information you want to store data = {‘Level’: ‘5’,‘Gold’:100} # Write to the file file = open(‘/optionalPath/save1.data’, ‘wb’) pickle.dump(data, file) # dump the object to a file file.close()

Allright, another way to do it.

But how would you go on with, for example the following Scenario:
(concrete in my Projekt)
A Multiplayer setup, where you have hundreds of Planets, wich have dozens of Sectors, with dozens of Objects ??

How to manage, that all Player can load and save their values and also need to get the Values of the other players?

I’m working on a very adult software(15 Years of developing) at the moment for Client-Administration, wich is similiar to this approach. We use MSSQL. It is fast and almost unlimited of possibilities.
Related to Point 1, i want to plan it in advance, because later it will be almost impossible to port to another system.
I might ask; are there really alternatives to SQL for huge setups?
( i’m new to these things, so any expirienced ideas are welcome)

11

This depends on the requirements you have:

  • store multiple data
  • store a subset of data
  • read multiple data
  • read a subset of data
  • distributed access to shared data
  • available offline to each user?
  • distributed data storage? or central data storage?
  • Ensured data integrity?
  • User authentication?
    etc.

A DBMS sounds reasonable for such a system. But it does not prevent you to implement your own solution. Even a file system would meet the requirements. The benefit of a DBMS is clearly: It is designed for such requirements.

Plan in advance is always good.
Your solution should not depend on a specific system. It should depend on a specific design. That means you can replace the system later without affecting the other parts of the game.

SQL is “just” an agreement to standardize the access to an relational DBMS. This allows to replace an sqlite with MySQL. Unfortunately there are several dialects of SQL and each DBMS has some very specific enhancements. Which can make the replacement really hard.

For an independent data storage you might want to have a look at the three tier architecture.
E.g. you can establish a webservice as data storage. The clients are using the webservice to write and read their data. The webservice uses whatever you implemented (DBMS, Filesystem wheatever).

Such things do not really depend on the size of the project. They depend on the requirements and how to solve them. Even a small application can have a large database, while a large application just writes a few bits of data into a file.