want to make a Facial Motion Capture script

Coding Python Support
1

I want to make a marker-based Facial Motion Capture script for Blender 2.49 (will possibly port to 2.6 when its out).

I’m going to use Python Imaging Library to choose a color from a recorded video and apply contrast to get the approximate marker positions, then get the marker positions each frame, make empties for markers in Blender and apply the motion data to empties, smooth the results. Later bones, etc could inherit the motion from empties with constraints or if parented.

I would like to be able to make the script work in realtime. Record a video from camera and send the frames directly to Blender and show it as background image in the 3d view or at least in another window and allow users to record animations in realtime.
Firstly, I couldn’t find any way to get frames from cameras with PIL, do you guys know of any other library? Also can you send and view the video inside Blender in realtime?

2

You might want to look into integral images, mean shift methods and normalized cross correlation.
Those are used to track features/patterns in an image. I am not sure just tracking a color would give the wanted result, you´ll just get jumpy points as the color would change too much during a video to track it properly.
Pattern matchmoving also makes it easier to calibrate, users would be able to do nothing at all, if you require red dots on the forehead, green dots on the cheeks and blue dots around the mouth, you can detect them automatically.

Do you only want to track the face markers in 2D or vectorcalculate their spatial positions as well?

3

I dont know that much.

I am not sure just tracking a color would give the wanted result, you´ll just get jumpy points as the color would change too much during a video to track it properly.
Why would it change?
heres a random image from the internet: angry-pixel.blogspot.com/2010/04/bedroom-mocap.html
heres a quick edit with gimp with choosing a color and applying contrast:
i54.tinypic.com/33tplkx.jpg
I dont think its too bad, with better camera and lightning you might get better results.

Do you only want to track the face markers in 2D or vectorcalculate their spatial positions as well?
I was just thinking 2d.

4

Have you seen this? Looks cool, although I got stuck at importing the bvh file into 2.5x.

5

That looks like a commercial application with limited free version.
I want to make my own open source script

6

Never even thought of posting something in the Blender site for anyone who might be after this. I’ve developed a very (very very) rought system for use in a small film team. It’s raw, but it works, and it’s open source. You will need a reasonable video editing suite to use for picking out the points (as an alpha key from chroma etc.) and formatting the video so that the PIL can read it quickly.

I’d like to post links to where the script is sourced but it seems that the forum system won’t let me. I’ve got them listed with a demo of it’s function:

I’m still developing the system, and would love any feedback you have!