# Live Scene Capture to Texture

**URL:** <https://blenderartists.org/t/live-scene-capture-to-texture/1380153>\
**Category:** Blender and CG Discussions\
**Tags:** scripting, code\
**Created:** [May 16, 2022, 9:33am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153 "2022-05-16T09:33:17Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kolupsy](https://blenderartists.org/user_avatar/blenderartists.org/kolupsy/32/753133_2.png) [@Kolupsy](https://blenderartists.org/u/Kolupsy)\
**Post date:** [May 16, 2022, 9:33am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/1 "2022-05-16T09:33:17Z")

</div>

**LIVE SCENE CAPTURE → IMAGE TEXTURE**

Lets talk abour rendering your scene in realtime and outputting the result into a texture. This is a common technique used in games for portal effects and whatnot but it is not an essential feature to have in offline rendering software like Blender. There still seems to be demand for this technique though. This post has been prompted by the frequent requests to break this effect down on a twitter post:

[https://twitter.com/kolupsy/status/1525811584718741505?s=20&t=2eDLtjrgVcKw-TmsQXDutA](https://twitter.com/kolupsy/status/1525811584718741505?s=20&t=2eDLtjrgVcKw-TmsQXDutA)

To take on this effect we make use of a hidden but very potent part of the Blender API: the GPU module. You can find out more about it [here](https://docs.blender.org/api/current/gpu.html?highlight=gpu#module-gpu).

If you look at the many examples posted there you will find snippets that generate images from offscreen rendering and ones that capture the scene view from a camera and display it as a little window on your UI. By combining both techniques, we can achieve the “live texture” effect.

Below is the code that I used in the twitter post. Before running it you need to make sure to have a camera object called ‘LiveCam’ in your scene or else the script wont work properly.

```python
import bpy, gpu, numpy as np

RES = 512
offscreen = gpu.types.GPUOffScreen( RES, RES )

CAMERA = bpy.data.objects['LiveCam']
LIVETEX = bpy.data.images.new( 'LiveTexture', RES, RES, alpha = True )
LIVETEX.pack( )
LIVETEX.use_fake_user = True
LIVETEX.colorspace_settings.name = 'Linear'

def draw( ):
    context = bpy.context
    scene = context.scene
    
    vm = CAMERA.matrix_world.inverted( )
    pm = CAMERA.calc_matrix_camera( context.evaluated_depsgraph_get( ), x = RES, y = RES )
    
    offscreen.draw_view3d( scene, context.view_layer, context.space_data, context.region, vm, pm )
    
    gpu.state.depth_mask_set( False )
    buffer = np.array( offscreen.texture_color.read( ), dtype = 'float32' ).flatten( order = 'F' )
    buffer = np.divide( buffer, 255 ) 
    LIVETEX.pixels.foreach_set( buffer )

bpy.types.SpaceView3D.draw_handler_add( draw, ( ), 'WINDOW', 'POST_PIXEL' )

```

Now we will get a new image in our blend file called ‘LiveTexture’ as long as their is not already an image of the same name in the same file. This image can be used like any other image as a texture for the shader graph, Geometry Nodes ect…

**Limitations:**

1. This will only work in the viewport, material preview or Eevee render. It will not capture Cycles renders. You can modify the script to render a cycles image on every frame update but this is not recommended for obvious performance reasons
2. Speaking of performance. It is not great. Now this is not exactly a hack but I am sure Blender was not built with “texture streaming” in mind. Large captures with a lot of samples will tank preview performance fairly quickly.
3. The script above does not handle On-Render updates, they can be implemented using [app handlers](https://docs.blender.org/api/current/bpy.app.handlers.html?highlight=handler#module-bpy.app.handlers) altough I have not tested it myself yet.
4. The capture will always include the entire visible UI. You can toggle of overlays if you want to capture only the relevant parts of the scene.

**Tips:**

1. Use 1 or 0 Eevee viewport samples. Each viewport sample contributes to the performance load of the capture so keeping it low will help with performance. 0 samples allowes you to have unlimited samples in your scene but the capture will still only use 1 (very useful).
2. Disable viewport denoising. This will make your viewport less slushy and clean up your capture.
3. Change the resolution in the script to your liking. It currently is set up for a 1:1 ratio but you can modifiy the script to have any resolution you want. Just keep it low. 256x256 can be heavy enough. 512x512 has worked great too in simple scenes. Just try to find the limit for your scene.
4. If you want to iteratively make changes to the script, either make sure to reopen Blender with every update of the script or implement script updating yourself.

**Final remarks:**  
I personally dont find this technique too useful, it is a bit janky especially in its current state. BUT I will gladly listen to what you have to say about it and why it is ACTUALLY useful. I would be open to expand on this idea given a good enough reason to do so.  
If you decide to do something cool with it, then I am very glad to have helped. Be sure to tag me on twitter so that I can see what cool things you were able to do with it. My twitter handle is **@kolupsy**. You can also follow me on twitter where I am most active on social media and often post funky stuff like this.

[https://twitter.com/kolupsy/status/1525924648168333315?s=20&t=2eDLtjrgVcKw-TmsQXDutA](https://twitter.com/kolupsy/status/1525924648168333315?s=20&t=2eDLtjrgVcKw-TmsQXDutA)

---

<div class="post-metadata">

**Author:** ![msamilg](https://blenderartists.org/user_avatar/blenderartists.org/msamilg/32/619655_2.png) [@msamilg](https://blenderartists.org/u/msamilg)\
**Post date:** [May 16, 2022, 9:59am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/2 "2022-05-16T09:59:21Z")

</div>

Great one, Kolupsy! I wasn’t even aware of the gpu module until today. Saw this yesterday on twitter and have been waiting for a breakdown. 😀

---

<div class="post-metadata">

**Author:** ![tomjelfs](https://blenderartists.org/user_avatar/blenderartists.org/tomjelfs/32/927818_2.png) [@tomjelfs](https://blenderartists.org/u/tomjelfs)\
**Post date:** [May 16, 2022, 10:22am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/3 "2022-05-16T10:22:38Z")

</div>

As someone who is using Blender as a pseudo game engine/live virtual scene for a kind of V-tubing this interests me greatly, I know my next question is probably a straight no however I’ve not got time to read it all right now, but can you bring a source in from outside of Blender?

---

<div class="post-metadata">

**Author:** ![PTCreative](https://blenderartists.org/user_avatar/blenderartists.org/ptcreative/32/1326655_2.png) [@PTCreative](https://blenderartists.org/u/PTCreative)\
**Post date:** [May 17, 2022, 1:23pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/4 "2022-05-17T13:23:15Z")

</div>

Do you mean a real camera source, such as webcam? if so yes that is possible but you’ll need to install some external packages.

---

<div class="post-metadata">

**Author:** ![softyoda](https://blenderartists.org/user_avatar/blenderartists.org/softyoda/32/543396_2.png) [@softyoda](https://blenderartists.org/u/softyoda)\
**Post date:** [May 17, 2022, 1:23pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/5 "2022-05-17T13:23:24Z")

</div>

Hi, I really wanted this feature from a long time, I’ve seen many [thread](https://blenderartists.org/t/create-texture-dynamically-from-camera-view-possible/1275834) or [feature requests](https://blender.community/c/rightclickselect/rgdbbc/?sorting=hot) about this.

So sad, it is still only a viewport gpu drawcall and not something you can actually use… I wish you create an addon to integrate it as it is really needed. I had to pass to Unreal Engine only to get render target.

---

<div class="post-metadata">

**Author:** ![PTCreative](https://blenderartists.org/user_avatar/blenderartists.org/ptcreative/32/1326655_2.png) [@PTCreative](https://blenderartists.org/u/PTCreative)\
**Post date:** [May 17, 2022, 2:13pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/6 "2022-05-17T14:13:43Z")

</div>

It is usable, not a gpu overlay. You can delete the draw handler bit and use:

```auto
for _area in context.screen.areas:
        if _area.type == 'VIEW_3D':
            space_data = _area.spaces[0]
            for _region in _area.regions:
                if _region.type == 'WINDOW':
                    region = _region 
                    break

```

and…  
offscreen.draw\_view3d( scene, context.view\_layer, space\_data, region, vm, pm )

then call draw() as you wish. The handler is just for realtime updates. Because its being essentially a ‘render texture’ you can then use that data for anything. I know Unity uses it for some special reflective shaders, what would you use it for?

---

<div class="post-metadata">

**Author:** ![tomjelfs](https://blenderartists.org/user_avatar/blenderartists.org/tomjelfs/32/927818_2.png) [@tomjelfs](https://blenderartists.org/u/tomjelfs)\
**Post date:** [May 18, 2022, 12:05pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/7 "2022-05-18T12:05:33Z")

</div>

Actually I was thinking a virtual camera like to have a desktop or browser.

---

<div class="post-metadata">

**Author:** ![Kolupsy](https://blenderartists.org/user_avatar/blenderartists.org/kolupsy/32/753133_2.png) [@Kolupsy](https://blenderartists.org/u/Kolupsy)\
**Post date:** [May 18, 2022, 2:28pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/8 "2022-05-18T14:28:00Z")

</div>

```python
import bpy, gpu, numpy as np
from gpu_extras.presets import draw_texture_2d

RES = 256
offscreen = gpu.types.GPUOffScreen( RES, RES )

CAMERA = bpy.data.objects['LiveCam']
if not 'LiveTexture' in bpy.data.images.keys( ):
    bpy.data.images.new( 'LiveTexture', RES, RES, alpha = True )
LIVETEX = bpy.data.images['LiveTexture']
LIVETEX.pack( )
LIVETEX.use_fake_user = True
LIVETEX.colorspace_settings.name = 'Linear'

def draw( ):
    
    context = bpy.context
    scene = context.scene
    vm = CAMERA.matrix_world.inverted( )
    pm = CAMERA.calc_matrix_camera( context.evaluated_depsgraph_get( ), x = RES, y = RES )
    
    original_overlays = context.space_data.overlay.show_overlays
    context.space_data.overlay.show_overlays = False
    offscreen.draw_view3d( scene, context.view_layer, context.space_data, context.region, vm, pm )
    context.space_data.overlay.show_overlays = original_overlays
    
    gpu.state.depth_mask_set( False )
    buffer = np.array( offscreen.texture_color.read( ), dtype = 'float32' ).flatten( order = 'F' )
    buffer = np.divide( buffer, 255 ) 
    LIVETEX.pixels.foreach_set( buffer )
    draw_texture_2d(offscreen.texture_color, (10, 10), RES/4, RES/4 )

class LiveCapture_OT_capture( bpy.types.Operator ):
    bl_idname = 'livecapture.capture'
    bl_label = 'Live Capture'

    def execute( self, context ):
        self.report({'WARNING'}, 'Operator has no execution. Use as modal.')
        return{'CANCELLED'}
    
    def invoke( self, context, event ):
        self.report( {'INFO'}, 'Start realtime update.' )
        self._handler = bpy.types.SpaceView3D.draw_handler_add( draw, ( ), 'WINDOW', 'POST_PIXEL' )
        context.window_manager.modal_handler_add( self )
        return{'RUNNING_MODAL'}
    
    def modal(self, context, event ):
        if event.type == 'ESC':
            return self.finish( context )
        return{'PASS_THROUGH'}
    
    def finish( self, context ):
        bpy.types.SpaceView3D.draw_handler_remove( self._handler, 'WINDOW' )
        self.report( {'INFO'}, 'Stopped realtime update.' )
        return{'FINISHED'}

bpy.utils.register_class( LiveCapture_OT_capture )

```

Small update to the script. I wrapped the realtime update function in a modal operator that can be called from the Search menu. It will also force the overlays to be invisible. You can remove that if you dont want it. Its also still very context sensitive so keep that in mind. Cancel by pressing escape. You can see when the modal is active when there is a little preview of whats happening appearing in the 3d view lower-left corner.

---

<div class="post-metadata">

**Author:** ![PTCreative](https://blenderartists.org/user_avatar/blenderartists.org/ptcreative/32/1326655_2.png) [@PTCreative](https://blenderartists.org/u/PTCreative)\
**Post date:** [May 19, 2022, 2:11am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/9 "2022-05-19T02:11:49Z")

</div>

Like this? [https://otrealms.gumroad.com/l/cam2blender](https://otrealms.gumroad.com/l/cam2blender) .Its the same whether virtual or real camera. You need to use the virtual camera plugin if using OBS. Plus OpenCV installed in Blender. Also works if you want to green screen yourself into Blender! I knew it wasn’t hard to make but never saw any reason to do it. I can make a separate topic with instructions when I have time.

---

<div class="post-metadata">

**Author:** ![tomjelfs](https://blenderartists.org/user_avatar/blenderartists.org/tomjelfs/32/927818_2.png) [@tomjelfs](https://blenderartists.org/u/tomjelfs)\
**Post date:** [May 19, 2022, 12:11pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/10 "2022-05-19T12:11:17Z")

</div>

Amazing, so cool, I’m going to try and implement this into my setup any advice you could give would be great as I hit confusion mode when I started looking into OpenCV.

---

<div class="post-metadata">

**Author:** ![xcoxc](https://blenderartists.org/letter_avatar_proxy/v4/letter/x/bc8723/32.png) [@xcoxc](https://blenderartists.org/u/xcoxc)\
**Post date:** [May 19, 2022, 7:08pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/11 "2022-05-19T19:08:41Z")

</div>

is this not working for me because i havent got a gpu? is there a way to do this without a gpu? thanks

---

<div class="post-metadata">

**Author:** ![Michael\_Knubben](https://blenderartists.org/user_avatar/blenderartists.org/michael_knubben/32/10600_2.png) [@Michael\_Knubben](https://blenderartists.org/u/Michael_Knubben)\
**Post date:** [May 20, 2022, 11:12am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/12 "2022-05-20T11:12:55Z")

</div>

> [@Kolupsy](#):
>
> I will gladly listen to what you have to say about it and why it is ACTUALLY useful. I would be open to expand on this idea given a good enough reason to do so.

I’ve wanted something like this for ages, for using in live, responsive visuals in EEVEE. I’ve done a lot of fun stuff with UV warping and such, and always found myself missing a render-to-texture. One usecase: hall of mirrors-type effects. Particularly those with a colour-shift applied, as in 70’s video effects, but the technique’s more broadly applicable too!

---

<div class="post-metadata">

**Author:** ![claus](https://blenderartists.org/user_avatar/blenderartists.org/claus/32/868691_2.png) [@claus](https://blenderartists.org/u/claus)\
**Post date:** [July 29, 2022, 8:56am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/13 "2022-07-29T08:56:44Z")

</div>

> [@Kolupsy](#):
>
> This will only work in the viewport, material preview or Eevee render. It will not capture Cycles renders. You can modify the script to render a cycles image on every frame update but this is not recommended for obvious performance reasons

Let’s say I have a great gpu and a simple scene. How would one modify the script to work with Cycles?

---

<div class="post-metadata">

**Author:** ![Yohigo](https://blenderartists.org/user_avatar/blenderartists.org/yohigo/32/865917_2.png) [@Yohigo](https://blenderartists.org/u/Yohigo)\
**Post date:** [November 5, 2022, 5:58am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/14 "2022-11-05T05:58:08Z")

</div>

I’ve tried to test your script and I can’t get it to work, I don’t know what I’m doing wrong. Can you make a step by step video?

Thank you

---

<div class="post-metadata">

**Author:** ![Kolupsy](https://blenderartists.org/user_avatar/blenderartists.org/kolupsy/32/753133_2.png) [@Kolupsy](https://blenderartists.org/u/Kolupsy)\
**Post date:** [November 5, 2022, 7:30am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/15 "2022-11-05T07:30:39Z")

</div>

It is sadly not a performance question. The type of scene capture only works using the viewport or Eevee engine.

---

<div class="post-metadata">

**Author:** ![Yohigo](https://blenderartists.org/user_avatar/blenderartists.org/yohigo/32/865917_2.png) [@Yohigo](https://blenderartists.org/u/Yohigo)\
**Post date:** [November 5, 2022, 1:51pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/16 "2022-11-05T13:51:13Z")

</div>

![image](https://blenderartists.org/uploads/default/original/4X/a/6/c/a6cecea902d8a33b1b9a63208765d17fd7dd6783.jpeg)

What is wrong in my configuration?

---

<div class="post-metadata">

**Author:** ![AlphaChannel](https://blenderartists.org/user_avatar/blenderartists.org/alphachannel/32/1432111_2.png) [@AlphaChannel](https://blenderartists.org/u/AlphaChannel)\
**Post date:** [December 25, 2022, 4:40pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/18 "2022-12-25T16:40:07Z")

</div>

![image](https://blenderartists.org/uploads/default/original/4X/3/d/e/3de0caf56cc4d6b542d06bc3f3a7c7c62fe4c927.jpeg)

This makes the old VJ in me very happy, thanks for this!

Best (VJ) Christmas present ever.

---

<div class="post-metadata">

**Author:** ![Michaelll](https://blenderartists.org/user_avatar/blenderartists.org/michaelll/32/739599_2.png) [@Michaelll](https://blenderartists.org/u/Michaelll)\
**Post date:** [January 29, 2023, 1:43pm UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/19 "2023-01-29T13:43:17Z")

</div>

Greetings! You have done a very useful thing, thank you. But I don’t understand how to use this script. Could you briefly explain? I have a scene with a camera (LiveCam) plane with a created texture saved in the project folder and a Suzanne object that needs to be rendered. I copied your script into a new text document in upbge but nothing happens when I run it. I don’t understand what to do…

---

<div class="post-metadata">

**Author:** ![BobMeta](https://blenderartists.org/letter_avatar_proxy/v4/letter/b/ecc23a/32.png) [@BobMeta](https://blenderartists.org/u/BobMeta)\
**Post date:** [January 31, 2023, 10:07am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/20 "2023-01-31T10:07:23Z")

</div>

[/quote]

 ![Screenshot 2023-01-31 at 11.02.23](https://blenderartists.org/uploads/default/original/4X/2/7/2/272511f727ce372bacf40a8d9c9aa3408e356caa.png)  
Run the script, open the blender file as shown in the top green arrow. Select the “livetexture” image and drag it to your shading workspace and use it as any normal texture.

---

<div class="post-metadata">

**Author:** ![BobMeta](https://blenderartists.org/letter_avatar_proxy/v4/letter/b/ecc23a/32.png) [@BobMeta](https://blenderartists.org/u/BobMeta)\
**Post date:** [January 31, 2023, 10:10am UTC](https://blenderartists.org/t/live-scene-capture-to-texture/1380153/21 "2023-01-31T10:10:13Z")

</div>

Hi, can anyone instruct me as to how to change the aspect ratio of the captured image? I want to make a widescreen tv feed.

Cheers.

[Next page](https://blenderartists.org/t/live-scene-capture-to-texture/1380153.md?page=2)