Open source Video SDK

Does anybody know of a descent video sdk that’ll actually let you anylize/manipulate the image, rather than just sticking it on a surface?
The only one I’ve found so far is the Open Computer Vision from intel, which from what I can tell, is partially tied into special instructions on intel’s chips.

I noticed a shortage of open mo-cap programs, and decided to investigate the possibility of rectifying the situation. It may be over my head (my programming is self-taught, so there’s certain gaps in my knowlege) but I figure it can’t hurt to at least take a look at it…