Access 3D Screen Buffer or save as file?

[Loolarge]

Hi all,

i would like to know if or how i can access or render the 3d screen.

I would like to write an "Onion skining"-script that displays character poses from certain frames in the background.

[forTe]

You can probably do it with a View3d Draw space handler:

http://www.blender.org/documentation...ed-module.html

[Loolarge]

What i ment was: How do i capture the OpenGL viewport?

[jesterKing]

I don't think you need to capture the viewport, instead use the BGL and Draw modules to do direct OpenGL drawing into the 3d viewport.

Concerning the onion skinning, I assume you are aware of the "ghost" option for armature visualisation?

http://wiki.blender.org/index.php/Ma...ects#Options_3
http://en.wikibooks.org/wiki/Blender.../Armature/pose

/Nathan

[Loolarge]

Hi Jesterking, thanks for your interest. I am aware of the onion skinning for armature in blender. But i think you cannot use it for an actual character mesh. A friend showed me this plugin for maya, where at desired frames the OpenGL screen is simlpy stored as a buffer once and displayed in the background instead of re-evaluating the whole Rig over and over. I love it. It feels more like traditional animation to work like that.

Here is the link to the maya plugin on HighEnd3d. Unfortunately the video does not work anymore though.

Here is quick demo video

[jesterKing]

Originally Posted by Loolarge Here is quick demo video

That looks pretty cool indeed. But AFAIK there is currently no way to get such a buffer through the Python API. I'm thinking that it should be possible to add though.

/Nathan

[Loolarge]

Great, that would be awesome!

[Michael_S]

Hello,

i have not seen the demo or video, but...

You can add dynamic ghosting to the 3d window quite easily - though this may not be exactly what you're after:

#SPACEHANDLER.VIEW3D.DRAW
from Blender import BGL
BGL.glAccum(BGL.GL_MULT, 0.25) # strength of ghosting
BGL.glAccum(BGL.GL_ACCUM, 0.75)
# the factors 0.25 and 0.75 control the strength of ghosting (must add to 1.0)
BGL.glAccum(BGL.GL_RETURN, 1.0)

copy the code into blender and name the text something like "ghosting.py"
In the 3D window header menu you can then select View -> Space handler scripts -> ghosting.py
You may want to do a proper initialisation. Also check for mousebuttons or changes to the viewmatrix etc. to detect if the user has changed the view and disable ghosting then (it's pretty much confusing this way).

As said - this may not be what you need exactly. You could also grab a single frame (a keyframe for example) by creating a buffer using BGL.Buffer(...) and do a screencopy with BGL.glReadPixels(...) after the keyframe was drawn on screen. Then you can bind the buffer to a texture and render the textures of previous keyframes using alphablending - would give good control over the grabbing (only keyframes) and blending operation.
In any case: BGL may be your friend...

Michael

[jesterKing]

That might actually work, yes. Hadn't thought of that yet.

/Nathan

[Loolarge]

Hi Michael_S,

thank you very much for the hints! I will try that. With your code snippet i get some funky letters and memory artefacts, but i figured that you can use glClear(GL_ACCUM_BUFFER_BIT) to fix it.

OpenGL is new to me, but i think it will be fun to figure things out. I am confident that i will get it done, since you pointed it all out allready.

Thanks again

[cesio]

Originally Posted by Loolarge Hi all, i would like to know if or how i can access or render the 3d screen. I would like to write an "Onion skining"-script that displays character poses from certain frames in the background.

Look these scripts http://cesio.arg.googlepages.com/otto4.blend
i think is hard to implement Onion Skin in 3d view with python.
Need better access to deformed meshes.

[Loolarge]

I think i am stuck. I wanted to read a block from the screen and display it somewhere else. Nothing happens though. I am not quite shure about what types and formats to feed into the BGL commands. And i also am not shure how to attach the buffer to an Image object afterwards.

#SPACEHANDLER.VIEW3D.DRAW import Blender from Blender import BGL, Registry buffer = BGL.Buffer(BGL.GL_FLOAT, [128,128]) BGL.glReadPixels( 0,0,128,128, BGL.GL_RGB, BGL.GL_BITMAP, buffer ) BGL.glDrawPixels( 128, 128, BGL.GL_RGB, BGL.GL_BITMAP, buffer ) BGL.glBitmap( 128, 128, 0,0, 50,50, buffer)

[Michael_S]

ok - try this

#SPACEHANDLER.VIEW3D.DRAW
from Blender import BGL
 
dim_x = 64
dim_y = 64
readpos_x = 0
readpos_y = 0
writepos_x = 100
writepos_y = 100
 
scissor = BGL.Buffer(BGL.GL_INT, 4)
BGL.glGetIntegerv(BGL.GL_SCISSOR_BOX, scissor)
 
buffer = BGL.Buffer(BGL.GL_FLOAT, 3 * dim_x * dim_y)
 
BGL.glReadPixels(scissor[0] + readpos_x, scissor[1] + readpos_y, dim_x, dim_y, BGL.GL_RGB, BGL.GL_FLOAT, buffer)
 
BGL.glRasterPos2i(writepos_x, writepos_y)
BGL.glDrawPixels(dim_x, dim_y, BGL.GL_RGB, BGL.GL_FLOAT, buffer)

I used the scissorbox to get the screencoordinates for the lower left corner of the 3D-window. You could also use a Blender-api call (can't remember which) or query the opengl viewport. The result is used as offset for the ReadPixels command. For drawing, you need to set the startposition with glRasterPos - which is already relative to the 3D-window, so no offset is needed here.

Michael

[Crouch]

I had a try at writing an "onion skinning" script.
It consists of two parts: a normal python script with a gui and a spacehandler script.

GUI

import Blender
from Blender import *
from Blender.BGL import *

# Get window coordinates
def getCoords():
    global xmin, ymin, width, height
    
    windows = Window.GetScreenInfo()
    for w in windows:
        if w['type'] == Window.Types.VIEW3D:
            xmin, ymin, xmax, ymax = w['vertices']
            width = xmax-xmin
            height = ymax-ymin

# Get z and colour buffers
def getBuffers():
    global zbuf, colbuf
    
    zbuf = Buffer(GL_FLOAT, [width*height])
    glReadPixels(xmin, ymin, width, height, GL_DEPTH_COMPONENT, GL_FLOAT, zbuf)
    colbuf = Buffer(GL_FLOAT, [width*height, 4])
    glReadPixels(xmin, ymin, width, height, GL_RGBA, GL_FLOAT, colbuf)

# Get image and saved data
def getImgData():
    global img, frames, curframe, view, offset
    
    try:
        img = Image.Get("OnionSkinning")
        frames = [f for f in img.properties['OnionSkinning']['frames']]
        view = [q for q in img.properties['OnionSkinning']['view']]
        offset = [o for o in img.properties['OnionSkinning']['offset']]
    
    except:
        img = Image.New("OnionSkinning", width, height, 32)
        frames = []
        view = Window.GetViewQuat()
        offset = Window.GetViewOffset()
    curframe = Blender.Get('curframe')

# Draw new image
def drawImage():
    frames.append(curframe)
    newfac = 1.0/len(frames)
    oldfac = 1.0 - newfac
    
    for y in range(height):
        for x in range(width):
            r_old, g_old, b_old, a_old = img.getPixelF(x,y)
            stack = int(round(a_old*(len(frames)-1)))
            i = y*width + x
            z = zbuf[i]
            if z != 1:
                r, g, b, a = colbuf[i]
                if newfac != 1:
                    r = (r_old*stack + r)/float(stack+1)
                    g = (g_old*stack + g)/float(stack+1)
                    b = (r_old*stack + b)/float(stack+1)
                img.setPixelF(x,y,(r,g,b,1.0))
            else:
                if newfac == 1:
                    a_old = 0.0
                img.setPixelF(x,y,(r_old,g_old,b_old,a_old))

# Save data along with image
def saveData():
    # Save data as ID properties
    img.properties['OnionSkinning'] = {}
    img.properties['OnionSkinning']['frames'] = frames
    img.properties['OnionSkinning']['size'] = [width, height]
    img.properties['OnionSkinning']['view'] = Window.GetViewQuat()
    img.properties['OnionSkinning']['offset'] = Window.GetViewOffset()
        
    # Free old image from memory
    img.glFree()

# Test if an onion skin has already been created
def onionExist():
    try:
        img = Image.Get("OnionSkinning")
        return True
    except:
        choice = Draw.PupMenu("No onion skin present yet%t|Onionize!|Cancel")
        if choice == 1:
            addFrame()
        return False

# Test if the current view matches that of the onion skin
def testView():
    getImgData()
    if len(frames)==0:
        return True
    if Window.GetViewQuat() != view or Window.GetViewOffset() != offset:
        choice = Draw.PupMenu("Current view doesn't match onion skin%t|Match view|Cancel")
        if choice == 1:
            resetView()
        else:
            return False
    return True

#########################

# Add the current frame to the image
def addFrame():
    getCoords()
    
    # Error checking
    if not testView():
        return
    if curframe in frames:
        choice = Draw.PupMenu("Frame already skinned%t|Update onion skin|Cancel")
        if choice == 1:
            updateOnion(True)
        else:
            return
    
    getBuffers()
    drawImage()
    saveData()
    Blender.Redraw()

# Reset the view to the one used by the onion skin
def resetView():
    if not onionExist():
        return
    
    getImgData()
    Window.SetViewQuat(view)
    Window.SetViewOffset(offset)
    
    Blender.Redraw()

# Update the onion skin
def updateOnion(true_update):
    global width, height, frames
    
    # Error checking
    if not onionExist():
        return
    if not true_update:
        if not testView():
            return
    else:
        getImgData()
    
    # Clear image
    width, height = img.size
    all_frames = frames
    for y in range(height):
        for x in range(width):
            img.setPixelF(x,y,(0.0,0.0,0.0,0.0))
    frames = []
    saveData()
    
    # Add the old frames
    original_frame = Blender.Get('curframe')
    for f in all_frames:
        Blender.Set('curframe', f)
        Blender.Redraw()
        addFrame()
    Blender.Set('curframe', original_frame)
    Blender.Redraw()

# Remove the current frame from the onion skin
def removeOnion(removeAll):
    global frames
    
    # Error checking
    if not onionExist():
        return
    if not testView():
        return
    if curframe not in frames:
        Draw.PupMenu("Error%t|Current frame not in onion skin")
        return

    if removeAll:
        frames = []
    else:
        frames.remove(curframe)
    img.properties['OnionSkinning']['frames'] = frames
    updateOnion(False)
    
#########################

# Draw the Graphical User Interface
def drawGui():
    but_add = Draw.PushButton("Onionize!", 1, 5, 135, 80, 20, "Add the current frame to the onion skin")
    but_view = Draw.PushButton("Go to view", 2, 5, 100, 80, 20, "Go to the 3D view with the onion skin")
    but_refresh = Draw.PushButton("Update", 3, 5, 80, 80, 20, "Update the onion skin to the current situation")
    but_del = Draw.PushButton("Delete", 4, 5, 60, 80, 20, "Remove the current frame from the onion skin")
    but_delall = Draw.PushButton("Delete all", 5, 5, 40, 80, 20, "Remove all frames from the onion skin")
    but_quit = Draw.PushButton("Quit", 99, 5, 5, 80, 20, "Exit this script")

# Mouse and keyboard processing
def event(evt, val):
    if evt in [Draw.QKEY, Draw.XKEY, Draw.ESCKEY]:
        Draw.Exit()

# Button processing
def bevent(evt):
    if evt == 1:
        addFrame()
    elif evt == 2:
        resetView()
    elif evt == 3:
        updateOnion(True)
    elif evt == 4:
        removeOnion(False)
    elif evt == 5:
        removeOnion(True)
    elif evt == 99:
        Draw.Exit()

Draw.Register(drawGui, event, bevent)

Spacehandler

#SPACEHANDLER.VIEW3D.DRAW
from Blender import Image, Window
from Blender.BGL import *

# User settings
alpha = 0.3

# Draw the background 
def drawImage():
    # Load image and data
    img.glLoad()
    width, height = [i for i in img.properties['OnionSkinning']['size']]
    z = 1.0
    
    # Get the scale of the openGL view matrix
    viewmatrix = Buffer(GL_FLOAT, 16)
    glGetFloatv(GL_MODELVIEW_MATRIX, viewmatrix)
    f = 1/viewmatrix[0]
    width*=f
    height*=f
    
    # Set openGL settings and bind image
    glEnable(GL_TEXTURE_2D)
    glEnable(GL_BLEND)
    glColor4f(1.0, 1.0, 1.0, alpha)
    glBindTexture(GL_TEXTURE_2D, img.getBindCode())
    
    # Draw the plane with the texture
    glBegin(GL_POLYGON)
    glTexCoord2i(0, 0)
    glVertex3f(0.0, 0.0, z)
    glTexCoord2i(1, 0)
    glVertex3f(0.0+width, 0.0, z)
    glTexCoord2i(1, 1)
    glVertex3f(0.0+width, 0.0+height, z)
    glTexCoord2i(0, 1)
    glVertex3f(0.0, 0.0+height, z)
    glEnd()
    
    # Being tidy
    glDisable(GL_BLEND)
    glDisable(GL_TEXTURE_2D)

# Check if the image can be drawn
def main():
    global img
    
    try:
        img = Image.Get("OnionSkinning")
    except:
        return
    
    if Window.GetViewQuat() == [q for q in img.properties['OnionSkinning']['view']]:
        drawImage()

main()

Copy and paste the two scripts into two different text files in the blender text window.
Load the spacehandler in the 3d view. Run the gui in the text window.
Tomorrow I'll post some more instructions on how to use it, and I will try to improve the speed of creating the onion skin, but now it's time to get some sleep.

[Loolarge]

Hi Crouch, thank you very much! This is awesome. Unfortunately i only see the screen getting brighter but nothing in the background. I will look at it again tomorrow.

Thanks everybody for the help, i am astonished! Blender rules.

[Loolarge]

Ok, i figured out that you have to set the OnionSkin Image as background image manually and set the camera to the exact resolution of the 3D View. Creating the buffers takes long time, but it is working.

[Crouch]

Creating the buffers does indeed take some time, especially when the 3d-view window is large, but unfortunately I can't think of a method to increase the speed.
It shouldn't be necessary to set the image as background image manually, or to adjust the camera. It sounds like you didn't activate the space handler.

Full explanation on setting everything up:
1 - go to python text-editor
2 - text --> open --> # select the spacehandler script on your harddisk
3 - text --> open --> # select the gui script on your harddisk
4 - go to the 3d view and press CTRL+UParrow (to enlarge the window)
5 - View --> Space Handlers Script --> Draw: onionskin_spacehandle
6 - CTRL+DOWNarrow (to get the window back to its normal size)
7 - go to the text editor and press ALT+P
Steps 4 and 6 are only necessary to be able to see the option in step 3, because it often happens that it falls off screen otherwise.

Below you can find a demo-video of the script in action and links to download the script files.



Script files:
Spacehandler
GUI

EDIT: a new version was written. It's located a bit further in this thread.

[Loolarge]

Hi Crouch. Thanks again for the script, it is awesome even now. But still when i do it like you described, the image is not displayed, the 3D screen just gets brighter. Maybe it is a thing with my gfx card. I am using a winxp laptop with a geforce card. What are you using?

Can someone else please confirm if it is working well for him?

[Crouch]

I'm on a winxp desktop using a geforce card, so that resembles your situation. Next week I'll go to a friend of mine and test it on his computer, to see if it works there.
If I find the source of the problem and/or a solution, I'll post here again.

[blenderificus]

this looks like such a very useful script, thank you for your hard work. I plan on testing it soon, OSX, 2.48a, 10.5.6.

EDIT: The spacehandler loads fine, as did the script(interface) execute properly, however upon referencing your vimeo tutorial, and the instruction on the thread above, I was not able to get the onion skinning appear for me on my system, it produced no console errors that i noticed. The script would add and delete frames without error, but I just didn't see the other onion skinned frames in the 3d view no matter how many i added. Looks like a useful script, however doesnt seem to work for me on OSX with the specs i listed above. hopefully soon I can use it :-)