custom glsl shader for videotexture

Game Engine Game Engine Support and Discussion
1

Hi all,

I’m having a hard time writing a custom glsl shader to deal with greenscreen on incoming video from a live camera, and it’s working out ok… the thing is it only works on multitexture mode…
when my fragment shader compiles in GLSL mode, the videotexture simply doesn’t refresh, and is stuck at the actual image texture that should be replaced by the bge.texture module…

if I remove the shader, it refreshes the texture to video, but it won’t allow me do do any changes on it…

I know bge.texture has a built in greenscreen filter, but it’s not enough to get good results, since it does not allow feathering or despilling…

is there a specific order for the scripts? is the script that configures and refreshes the bge.texture messing up my shader, or the shader overriding the changes in the videotexture? can I access the video in the shader itself? so far I only got access to the texture slot of the texture to be replaced, and it works fine in multitexture… but it would be really great to get it to work in GLSL mode as well…

Any help would be appreciated! Thanks
Nuno Estanqueiro

2

Can you provide a blend-sample please? Makes it much easier to help.

Do you have a vertex shader too?
Can you play a video without using shaders?

You did what they say here:
http://www.blender.org/documentation/blender_python_api_2_66_release/bge.texture.html

you need to call this function every frame to ensure update of the texture.

logic.video.refresh(True)

3

Thank you for your reply, I’ve made a simple example showing the problem with some very basic color changing function to the texture… Also note that The video part is all setup correctly, and I assure you I’ve read that documentation several times… I even got blenders internal cromakey filter to work in GLSL… all ok if you don’t use your own shaders on top of the video texture… and that is where I’m standing right now…

you can get my example file at https://www.wetransfer.com/downloads/9d3c3d33c03a975eb6efecc10144baf620130305154242/80f517dfad26a83ca22530511fa63a1920130305154242/8d6509

Maybe I’m adressing the texture the wrong way in the shader, or maybe the code has to move somewhere else… anyone used to working with shaders to help?

Thank you once again
Nuno Estanqueiro

4

Looks good.

And I see your problem…

The Shader works and the movie works too.
Just the combination doesn’t.

I guess there is a problem with transferring the video-texture over to the GPU.

I tried rapidly switching between two textures and that worked.
So setSampler is ok.
Just the VideoTexture isn’t pushed correctly in the right texture slot.

If no BGE guru says anything different, you might wanna post a bug report on that.

5

I think the same… somehow the fragment shader is overriding the video transfer… my question is about the setSampler, because for me… that’s the guilty line on the shader… I guess I’m gonna have to dig into the code to try and understand how this transfer is made… maybe the guys developing the harmony branch know best how this is actually working, and I think they are implementig tools to work around this, such as FBO’s and render passes… I’m going to look into the GPU module to see if there are any other options that could help me on my task…

hopefully others may know something about this they can share…
Thanks
Nuno Estanqueiro

6

Ok, I figured it out, although it’s kind of strange… first one has to apply the shader, and then aply the script to change the content of the texture, although theoreticly the shader as already changed the image… any way… it now works…
cheers
Nuno Estanqueiro

7

Ah!
A strange thing. And I totally forgot about it although I had the same solution in another project.

Thanks for the follow-up!