How to use GPU module to render camera to rectangle?

Game Engine Game Engine Support and Discussion
1

I used render-to-texture onto a plane parented in front of my main Camera.

How to do the same directly with GPU module?

Some examples from
https://upbge.org/docs/latest/api/gpu.html

work well with a code like this (which draws a simple rectangle):

import bpy, bge
import gpu
from gpu_extras.batch import batch_for_shader

vertices = (
    (100, 100), (300, 100),
    (100, 200), (300, 200))

indices = (
    (0, 1, 2), (2, 1, 3))

shader = gpu.shader.from_builtin('2D_UNIFORM_COLOR')
batch = batch_for_shader(shader, 'TRIS', {"pos": vertices}, indices=indices)


def draw():
    shader.bind()
    shader.uniform_float("color", (0, 0.5, 0.5, 1.0))
    batch.draw(shader)


def postDraw():
    
    scene = bge.logic.getCurrentScene()
    scene.post_draw = [draw]
2

Maybe you should consider using a 2D Filter instead, with the filter manager you can easily set textures, and filters work by literally drawing a quad with the texture filtered.

3

Thank you, this sounds like an idea. But I’m too dumb.
I suppose one would render to texture and then use this as input into the Filter.
A bit like here (which doesn’t work anymore in UPBGE 0.3):

But I have no clue how to set the texture even with the provided info. E.g. here the tex in my pseudo code:

in vec4 bgl_TexCoord;
#define gl_TexCoord glTexCoord
vec4 glTexCoord[4] = vec4[](bgl_TexCoord,bgl_TexCoord,bgl_TexCoord,bgl_TexCoord);
out vec4 fragColor;
#define gl_FragColor fragColor

uniform sampler2D bgl_RenderedTexture;
uniform sampler2D tex;
vec2 texcoord = vec2(gl_TexCoord[0]).st;

void main(void)
{    
      
    vec4 dif = texture2D(bgl_RenderedTexture, texcoord);
    vec4 cam2 = texture2D(tex, texcoord);
    
    gl_FragColor = mix(dif, cam2, 0.5);
     
}
4

You need to use the bind code from the dynamic texture with setTexture, along with the name of the sampler. This code is for setting the filter, you could put it after creating the dynamic texture.

bindCode = texture.bindId # texture is the dynamic texture

shaderText = """in vec4 bgl_TexCoord;
out vec4 fragColor;

uniform sampler2D bgl_RenderedTexture;
uniform sampler2D tex;

vec2 texcoord = bgl_TexCoord.st;

void main(void)
{
    vec4 dif = texture2D(bgl_RenderedTexture, texcoord);
    vec4 cam2 = texture2D(tex, texcoord);
    
    gl_FragColor = mix(dif, cam2, 0.5);
}"""

passNumber = 0

filter = scene.filterManager.addFilter(passNumber, logic.RAS_2DFILTER_CUSTOMFILTER, shaderText)

filter.setTexture(1, bindCode, 'tex')
1 Like
5

Thank you!
This now works basically! :grinning:
(edit: replaced previous file that somehow was corrupt, and blurred and stopped the rendering)

The final goal is to have a square area in the top right corner only filled with the render result of the 2nd cam. If this is at all possible?

UPBGE 0.3
rtt_test_with_filter.blend (897.1 KB)

import bge
from bge import texture

scene = bge.logic.getCurrentScene()
cont = bge.logic.getCurrentController()
own = cont.owner

rendercam = scene.objects["rendercam"]
renderplane = scene.objects["renderplane"]
if not 'init' in own:
    
    own['init'] = True
    bge.tex = texture.Texture(renderplane, 0, 0)
    bge.tex.source = texture.ImageRender(scene, rendercam)
    
    bindCode = bge.tex.bindId # texture is the dynamic texture

    shaderText = """in vec4 bgl_TexCoord;
    out vec4 fragColor;

    uniform sampler2D bgl_RenderedTexture;
    uniform sampler2D tex;

    vec2 texcoord = bgl_TexCoord.st;

    void main(void)
    {
        vec4 dif = texture2D(bgl_RenderedTexture, texcoord);
        vec4 cam2 = texture2D(tex, texcoord);
        
        gl_FragColor = mix(dif, cam2, 0.5);
        
    }"""

    passNumber = 0

    filter = scene.filterManager.addFilter(passNumber, bge.logic.RAS_2DFILTER_CUSTOMFILTER, shaderText)

    filter.setTexture(1, bindCode, 'tex')   

#def refresh():
bge.tex.refresh(True)
6

You can modify the texture coordinate and use to sample the texture and create a mask, it’s possible to pass the position and size of the square/rectangle (in pixels) as a vec4 uniform.
When testing, I kept getting a black texture, turns out it when scaling down the texture, it was trying to access the mipmaps, which appear to be enabled but not set. To avoid this, I used .capsize on the source to limit the size to that of the rectangle.

    # draw square/rectangle, in pixels
    rect = [25, 15, 250, 250] # left, bottom, width, height
...
    bge.tex.source.capsize = rect[2:4]
...
    filter.setUniform4f('rect', *rect)

rtt_test_with_filter1.blend (835.9 KB)

2 Likes
7

Thank you sooo much!!!
I tried to get my hands on something like this but this is way over my top!
I would have never been able to create such a solution!
Thank you !
(PS: I would mark it as solution but it’s another solution to the asked question.)

This way has 2 benefits:

  • Much better image qality than using a plane in front of the camera.
  • The possibility to adjust the impact on the FPS by skipping frames on the Texture.refresh.

I also tried using setViewports on the rendercam which works nicely but I didn’t find a way to skip frames.

Here you can see the much better quality of the upper (filter)-rectangle opposed to the ugly rendering on the plane beneath it (which also washes out depending on the background)

The only advantage of the plane is that it can be cropped to a square easily

2 Likes
8

If you want a square aspect ratio, I shared a code that let you set the aspect ratio for a camera.

1 Like
9

Wonderful, works like a charm!!
Is it also possible to have this with orthographic projection?

10

I added the code for orthographic projection.

1 Like
11

How great is this? Thank you so much!! It works perfectly!

12

Sadly though, Apple Silicon computers cannot use GLSL-based shaders that are rendered onto an object. Due to a bug or lack of a proper shading pipeline for GLSL object-based rendering, it will just output an unsupported message, and show nothing but a blank texture. I do know that the upcoming Metal support will likely solve this problem. Postprocessing shaders (2D Filters) do not have this problem as they are drawn using the GPU and also through something else, and are able to bypass this through a different way, possibly through the screen.