Dienstag, 26. März 2013

[BGE, Qt5] Coding Blender Game Engine

The Blender Game Engine lacks features. But as it's Open Source you can improve it as you like.

I thought so myself and tried to do it. Here is a short log on what I did.

While working with BGE I found some points that could need improvement or would be nice to have implemented. Here is a little list I came up with:

- irgendwie müsste man auch Qt GUI Elemente in die GameEngine rendern und nutzen können.
- dyn. add&rem. lights

- custom glsl material node ?

Just some improvements that are floating through my mind:
→ multithreading
→ easy (especially on the documentation part) usage of dynamic libraries and low-level engine improvements written in other languages (particularly C++)
→ easy usage of non-python scripts/modules, like C++, Java, C#, ... (e.g. create a dyn. lib. and use it via C++-Script-/Module-Controller)
→ OpenGL 3+4 support
→ integration of a high-level shading language like surface-shaders from Unity3D or OpenShadingLanguage
→ code-completion in editor (e.g. bge python and shader stuff)
→ independent game window so you can tweak stuff in Blender
→ ability to tweak variables while game is running
→ shop integration for plugins and assets
→ integration of a version control system, version controlable blend-files
→ more detailed profiling
→ better lighting (I know it's being worked on in candy/harmony)
→ ability to export/deploy to different platforms and not just to yourself
→ better/easier system it insert and keep track of project-assets and prefabs/linked objects (also those that are not in the blend file but in project path)
→ hardware particle system (I heard that someone is also working on that)
→ special game screen layouts
→ game logic sets ( putting different SCAs in a set/group, loading and saving those, adding sets on different objects, having different sets on an object,,, e.g. loading a first person set on your main-char-object)
→ same 3D-view when I switch screen layouts
→ easy system to do shadow- and lightmapping for e.g. all statics objects in scene
→ easy/simple GUI system [seen that around here]
→ easy/simple multiplayer system (if not integrated then e.g. a simple system to talk to e.g. RakNet) [yeah, here is already flying something around]
→ (deployment of an HTML5 version)

Additions to the wish-list:
→ Controller to call binary modules written in C++/Java/C#/…
→ flag to module-controller to start module asynchronous/threaded
→ adding a module-controller that doesn't already exist will create that including code for bound in- and outlets
→ "Next" / "Follow-Up" Actuator & Sensor - combination
e.g. || S: OnCollision → C: CheckConditions → A: Next & S: Next + S: OnKey[W] → C: AND → A: Motion ||
the Actuator should have a flag to say if the follow-up should be evaluated in the same or the next round
→ "State"-Sensor: reacting on entity:: creation, initialization, awake, update, sleep, deinitialization, destruction(, …?)
→ Option for S,C,A to select on which object/objects to act on (like having a control-object to control many equal object, you don't wanna put bricks on all of them, you could put logic on the controller and then on options tell on which objects/groups to act/react on,
e.g. a group of fireflies, S:OnHit(any of group FireFlies) → C: And → A: Glow(the one) | or A: Glow(all) | or A: Death(pony) )

[I know that most of those stuff is already doable via SCAs and Python. But having it a bit cleaner and packed together seems nice.
Also I don't mind having different Sensors being able to do the same stuff. As long as it's easier to use/understand or more consistent.]

→ grouping of SCAs
→ selection of SCAs → drag & drop to other object → option to move or copy over
→ saving of SCAs into & creation of SCAs from code-files

→ ability to pause game → continue normally or do just one step
→ ability to change properties (especially when paused)
→ visualize "flow" of commands through SCAs ("which why what when")
→ profiling of SCAs
→ profiling of modules

- UV scrolling
- Texture nodes
- OSL (Open Shading Language)
- timer in shader nodes
- Particle system (mokazon)
- Hive node GUI
- change vars while running
- OpenGL 3/4
- Prefabs
- VCS support for blend
- PureData for sound effects?
- C++ modules to call from nodes
- dynamic/procedural textures
- Threading
- group logic-bricks (hive)
- profiling
- VBOs
- HW Skinning
- Team-Development - Connected via network, working on one file(?)

Currently the shader program is part of BL_Shader. And it is capsuled pretty good. So BL_Shader should be exchangeable. So some "setShader" would be nice inside KX_BlenderMaterial.

2) RAS_IPolyMaterial just has one texture. BL_Material has access to more. Same goes for KX_BlenderMaterial. As KX_BM is a RAS_IPM there might be a way to query the the texturename with a virtual method.

 3) Create a state that disables the output of those messages.

My last works with BGE mostly had something to do with shader:
[BGE+GLSL] Finding interest points
[BGE + GLSL] Py GL Rant
[BGE + GLSL] GLSL Shader Repository Addon

That's why I started digging around in the C++ BGE shader code.
To get into the code, and because I like to work with the new features of C++11, I refactored the code in BL_Shader.
Nothing special.
Replacing plain pointers with shared pointer.
Using constant variables and expressions instead of defines.
Strings and not char pointer.
Arrays, templates and 'move'ment against void pointer.

I like it much better in this way.
You can find the code here in the cpp11 branch:

The first real coding was the integration of a shader manager.
Having just some objects around it might not matter so much. But using lots of objects with a custom shader on each this might be helpful.
Currently a shader is placed on a material. Many objects with all having the same material will also have the same shader on it. But if you have many objects with unique materials but each using the same shader, each material will have its own shader.
10k objects with the same material → one shader
10k objects with different materials and same shader → 10k shader

Additional nice little feature: Every shader needs to be compiled and linked. And every time you do that you'll get some nice message lines saying that it worked, or not.  Getting 10k messages isn't nice.

That's why I made a little shader manager.
It is rather simple:
Each shader-program gets a unique name and is stored in a map. When you create a shader-program you can give it a name. If it is already taken then that one is returned, else it will give you a new shader-program. (A shader-program is a combination of vertex and fragment shader.)
The map contains a shared pointer to the shader-program. When you remove a shader from the material the shared pointer will be decreased automatically.
Normally closing the game engine would also remove all shader. But using the in Blender embedded game engine the shader manager will stay and the shader in it too. So I included a check if the reference counter got to a minimum, meaning no object is using the shader anymore, then the shader would be removed.
Still one little thing bothers me a bit:
if one object has a shader and is the only object having this shader and I remove it, the shader gets removed. When I want to used the shader on an object again, it needs to be recreated. A possible way around could be to "store" the shader in another material.
With the shader manager I also put methods to python to query e.g. 'useShader', 'hasSource' and 'hasUniform'.

From OpenGL 3.2 geometry shader are a core part. I have a graphics card that can use those. So my next move was to make geometry shader available in BGE. That in itself isn't that hard. Just like with vertex and fragment shader you create the shader object, load the code into it and link it to the shader program. Because I dislike duplicated code I made a little refactoring first. But as I tested my changes it didn't seem to work. Up to this point I still don't know why. I guess it's BGE and its buggy OpenGL code.
Just to check if my drivers and my code are OK I created a little OpenGL sample with Qt5:
Qt5 OpenGL GLSL Source Code

Vertex shader:
attribute vec4 qt_Vertex;
attribute vec4 qt_Color;

uniform mat4 qt_ModelViewProjectionMatrix;

void main(void)
    gl_Position = qt_ModelViewProjectionMatrix * qt_Vertex;
Geometry shader:
 #version 330

layout(triangles) in;
layout(triangle_strip, max_vertices = 6) out;

uniform float time;

void main(void)

    gl_Position = vec4(abs(time),0,0,1);
    gl_Position = vec4(0,time,0,1);
    gl_Position = vec4(0,0,time,1);

    gl_Position = vec4(-abs(time),0,0,1);
    gl_Position = vec4(0,-time,0,1);
    gl_Position = vec4(0,0,time,1);

Fragment shader:
void main(void)
    gl_FragColor = vec4(1,0,0,1);
Simple as that.
And as you can see, it works.
(Coding Qt5 is a nice thing. But you currently are mostly on your own. Tutorials aren't yet ported over (it seems). And to make geometry shader work I needed to use the older QGLShader classes instead of the new QOpenGLShader classes. The extra bits and pieces should arrive with Qt5.1 and 5.2. At least that is what the man sad:
QtDD12 - OpenGL with Qt 5 - Dr. Sean Harmer
QtDD12 - Modern Shader-based OpenGL Techniques - Dr. Sean Harmer

But in Blender I get this:
You see the missing cube?
That one has an active geometry shader.
The console output says: All is fine:
source/gameengine/Ketsji/BL_Shader.cpp:295:---- Vertex Shader Error ----
source/gameengine/Ketsji/BL_Shader.cpp:296:Vertex shader was successfully compiled to run on hardware.

source/gameengine/Ketsji/BL_Shader.cpp:295:---- Geometry Shader Error ----
source/gameengine/Ketsji/BL_Shader.cpp:296:Geometry shader was successfully compiled to run on hardware.

source/gameengine/Ketsji/BL_Shader.cpp:295:---- Fragment Shader Error ----
source/gameengine/Ketsji/BL_Shader.cpp:296:Fragment shader was successfully compiled to run on hardware.

source/gameengine/Ketsji/BL_Shader.cpp:363:---- GLSL Program ----
source/gameengine/Ketsji/BL_Shader.cpp:364:Vertex shader(s) linked, fragment shader(s) linked, geometry shader(s) linked.
First time I tried the geometry shader in BGE it just crashed.

Digging through code I saw a place saying: "Don't use displaylist with VBO"
As I thought BGE would use VBOs I looked in the property panel and found displaylists enabled. So with that on it wouldn't use VBOs.
(As far as I know VBOs should be currently the best way to get your vertex data to your graphics card: http://www.opengl.org/wiki/Vertex_Specification#Vertex_Buffer_Object)
But in the storage-section of the property-panel there is no VBO to choose from.
Just because someone disabled them –.–

Oh men!

OK. Uncommenting was easy. I re-enabled VBOs. But that didn't help. The object still didn't show up.
I don't know. Without any error messages it is really hard to figure out.
Because it is working with VBOs and Qt5 I would say that there is an implementation problem inside BGE.

If OpenGL is working better within Qt5 I thought about moving the BGE somehow in to Qt5. The first thing I wanted to try was to show a Qt window when pressing play and having both windows, the Blender one and the Qt one, show the game. "No problem" I thought. BGE uses GLEW for OpenGL access currently. To render to different windows I would need to switch the context. There is a special GLEW_MX version for that (http://glew.sourceforge.net/advanced.html).
So the first step was to replace GLEW with GLEW_MX.
Next step: integrate Qt5.
I found a spot where I thought it would be a good place to start the Qt window. I got it open. And closed. And closing Blender gave me a crash somewhere in the window handling. Even with that error I've put in the Qt window a simple OpenGL context and tried to show that.
Didn't work.
Qt tells you that you can't mix GLEW and Qt-OpengGL functions.
And my system hang up sometimes when trying to start the game.

So I stopped.

One way to make this work could be to put the blenderplayer in Qt.
But the code isn't nice.
I mean the code of BGE, Blender and the glue between.
I have nothing against C and C++. But please don't mix…
Finding the code entry where BGE starts was a heck of a search. You know that starting a C or C++ program has always something to do with a  main function. You'll find some. And one is the one for Blender and BGE. And it calls sweet C functions. I didn't find the right entry into BGE here.
There is the game start button in the properties panel. And you'll find some lines of code for that in a python file. But as it looks like it doesn't start the BGE. A suspicious looking string will get you to a C function. Still not starting anything.
Per luck I used a debugger, started the game and did a break in the debugger. There I saw a function history that lead me to the start of the BGE.

I'm not someone that wants classes and object orientation anywhere. But having easy traceable and capsuled code is nicer then this extern C mess.
The game engine itself is written in C++. It's easier to jump around and find what you are looking for. But the code is also a mix of C and C++. That makes it smell not so good. Having clean C++, especially with C++11, would be very nice.
And a documentation. Please document your code :-(
Walking around the code it wasn't that hard to follow the concepts. But all in all documented code looks much nicer.

As I said above the glue code is also mind bowing. For Qt and GLEW_MX I needed to change some CMake files. But what the heck is that?
They made macros for like everything. Changing GLEW to GLEW_MX was the easy part here. Just search for the string in all cmake files and replace it. But for Qt I needed to put in the finding and includes and linking and such. As I couldn't figure out where to really put it I just placed it where the corresponding executable would be build.

A Qt-OpenGL powered BGE would be really nice. I'd like to do things like this:

With a package like Qt it should be much easier to develop and extend a big application like Blender and BGE.
Not just easier access to OpenGL and shader. Also a very nice way to put GUI into your game. And probably instant portability to the major systems: Mac, Windows, Linux, Android and iOS (last two should arrive in Qt in the near future).

The funny thing about all this is:
You can still create great stuff with BGE.
Just look through some projects here:

or search on http://www.google.com/videohp for Blender GE or BGE or game.

But also the engine needs an upgrade to 2013.
Look at http://devmaster.net/devdb/engines and you will see lots of open source game engines that perform much better then BGE.

Here are some:

The big winning points BGE currently have are:
→ very easy to use
→ nice integration with Blender (even when you can't take all things over 1 to 1)

I'd really like to see and help BGE grow…