Compositing cache

Hi everyone!

This is a failed attempt at creating some sort of caching for the compositor.

This is what I attempted to do:

On frame change
  If there is a cache image
    Uncheck Use Nodes
    Load image in Viewer Node
  else
    Check Use Nodes
    Save Viewer Node image as cache

On scene change
  If nodes has changed
    Delete cache

This is what I have

import bpy, os.path
from bpy.app.handlers import persistent

path= bpy.path.abspath(bpy.context.scene.render.filepath);
 
@persistent   
def postFrameChange(scene):
    tmp= path + "cache_" + str(bpy.context.scene.frame_current) + bpy.context.scene.render.file_extension
    if os.path.exists(tmp):
        bpy.context.scene.use_nodes = False
        bpy.data.images['Viewer Node'].filepath = tmp
        bpy.data.images['Viewer Node'].update()
    else:
        bpy.context.scene.use_nodes = True
        bpy.data.images['Viewer Node'].save_render(filepath=tmp)
        
def postSceneChange(scene):
    for fname in os.listdir(path):
        if fname.startswith("cache"):
            os.remove(os.path.join(path, fname))
        
bpy.app.handlers.frame_change_post.append(postFrameChange)
#bpy.app.handlers.scene_update_post.append(postSceneChange)

Notice that I commented out the scene_update_post since it would delete the cache on frame change even.

To use:

  • Setup a simple composite with a input movieclip connected to the output viewer.
  • Run the script.
  • Change the frame.
  • Check your output directory.

What works:

  • Knows if there is a cache and unchecks “Use Nodes”.
  • Saves Viewer Node image on frame change (if the change is not too fast or on playback).

What doesn’t work:

  • Deletion of cache works but uses scene_update_post which gets fired too often (have to find out if nodes have changed).
  • Wont’ update the Viewer Node image with cache.
  • Doesn’t use the 3d render but only 2d composite.

Things in my way:

  • Composite doesn’t happen on playback(alt-a).
  • 3d view won’t render on frame change.
  • The approach of loading each cache image one at a time may be too slow for realtime playback, instead it should add a image sequence called “cache” and update this when needed.

Right now i’m kind of stuck with the above problems…

Any ideas/help on this would be greatly appreciated.

Thanks!

Hi again,

Here is another, slightly different, approach at compositing cache:

import bpy, time, os.path
 
class nodePanel(bpy.types.Panel):
    bl_space_type = "NODE_EDITOR"
    bl_region_type = "UI"
    bl_label = "Cache to Disk"
 
    def draw(self, context):
        scn = bpy.context.scene
        layout = self.layout
        row = layout.row()
        col = row.column()
        col.operator( "bpt.cache" )
        col.operator( "bpt.del_cache" )
 
 
class cacheOperator(bpy.types.Operator): 
    bl_idname = "bpt.cache"
    bl_label = "Cache to Disk"
 
    def invoke(self, context, event ):
        bpy.ops.bpt.del_cache()
        print("Caching")
        path = bpy.path.abspath(bpy.context.scene.render.filepath)
        fileExt = bpy.context.scene.render.file_extension
        startFrame = bpy.context.scene.frame_start
        endFrame = bpy.context.scene.frame_end
        for curFrame in range( startFrame, endFrame):
            bpy.context.scene.frame_current = curFrame
            tmpName = "cache_" + str(curFrame) + fileExt
            tmpPath= path + tmpName
            print("Caching frame " + str(curFrame))
            bpy.data.images['Viewer Node'].save_render(filepath=tmpPath)
            
        if("Disk Cache" in bpy.data.images):
            bpy.data.images["Disk Cache"].filepath = path + "cache_" + str(startFrame)  + fileExt
            bpy.data.images["Disk Cache"].reload()  
        else:
            bpy.data.images.load(tmpPath);
            bpy.data.images[tmpName].name = "Disk Cache"
            bpy.data.images["Disk Cache"].source = "SEQUENCE"
            #bpy.data.images["Disk Cache"].frame_duration = endFrame READONLY??
            bpy.data.images["Disk Cache"].frame_start= startFrame
                
        return {'FINISHED'}
        
class delCacheOperator(bpy.types.Operator):
    bl_idname = "bpt.del_cache"
    bl_label = "Delete Cache"
 
    def invoke(self, context, event ):
        print("Deleting Cache")
        path = bpy.path.abspath(bpy.context.scene.render.filepath)
        if("Disk Cache" in bpy.data.images):
            bpy.data.images["Disk Cache"].filepath = ""
            
        for fname in os.listdir(path):
            if fname.startswith("cache"):
                os.remove(os.path.join(path, fname))
        return {'FINISHED'}        
 
def register():
    bpy.utils.register_class( nodePanel )
    bpy.utils.register_class( cacheOperator )
    bpy.utils.register_class( delCacheOperator ) 
 
def unregister():
    bpy.utils.register_class( nodePanel )
    bpy.utils.register_class( cacheOperator )
    bpy.utils.register_class( delCacheOperator )
 
if __name__ == "__main__":
    register()

Lacks comments atm but should be pretty self explanatory.

What it suppose to do:
Creates 2 buttons in the compositor N-space(press the N key to see them). One is called “Cache to Disk” which when pressed first deletes the current cache then loops from start to end frame saving the output from the Viewer Node on each frame. When its done saving the images it loads them as a UV image sequence called “Disk Cache”.

The other button is called “Delete Cache” which deletes the current cache.

Current problems:

  • When looping the frames the Viewer Node doesn’t update (saves the same frame for every frame) need to wait until composite is done for each frame how??.
  • Can’t find out how to set the frame_duration(Readonly) of the image sequence.
  • Can’t find out how to check the “Auto Refresh”(.use_auto_refresh) of the image sequence.
  • Can’t seem to call my cache delete operator before caching - error “wm_operator_invoke: invalid operator call ‘BPT_OT_del_cache’”.
  • Can’t find out how to make the “Disk Cache” image active in the image editor when caching is done.

/ Peddie

Hi again,

After tinkering some more with this I have come to the conclusion that this would be much better/easier if there were an event/callback for when the composite is finished and some way of modifying the viewer node via Python… It would actually make it possible to do a fairly good compositor cache, although the best would of course be to have it coded in the source…

Are there any planes for more events/callbacks?

/peddie

Hey Peddie pretty awesome idea! But not many think a cache on compositor output is so great, what with frame change events and all. Would be terrific to scrub in some renders to a cache like the tracker does.

Thanks 3pointEdit!

This just seems weird to me, fine if working with stills but for animation a cache is a must! I wouldn’t have survived without After Effects ram preview…

Imagine cacheing 4K animations from Mango! I hear also that Smoke eats ram for cache like no other program. Perhaps it is not a flexible solution memory wise?