Mediapipe is impressive. It is much faster than OpenPose and Wrnch and usually self-reports 100% accuracy, even for hands. It even runs on smartphones. For an example of python programming with Mediapipe see: https://www.youtube.com/watch?v=EgjwKM3KzGU
A good start on a Mediapipe plugin for Blender is at https://github.com/zonkosoft/BlendyPose This plugin worked well with Blender 2.92 but has a problem with changes made in 2.93. The developer is asking for contributions to the code.
It seems that with adaptation this code, which calculates x, y, and z bone information, could take the landmark data from BlendyPose and convert it into a .bvh file for easy use in Blender.
Thanks for sharing the link @Aurosutru
Today was released a new version with a better mediapipe data import and even a menu to make the loading of video easier.
As a bonus, there is also a vroid rigged model and remap files to use with rokoko retargeting solution or autorigpro.
Thanks for your work Carlos! I am wondering, is your work open sourced? I am currently starting some research for an “avialable for everyone” solution to capture signs for sign language and I find your work very interesting. I’d be happy to contribute if I can.
Thanks for the words.
It is open source.
The main code is in this link
The other links here on this post, are from some tests I’m doing before adding to mocap_import (the full addon)
I never worked with somebody helping on the code, and to be honest I dont know how to deal with some other things, like, I receive donations for the addon on gumroad and patreon, and I dont think its fair for me to receive for somebody else works and neither I have an idea of how to split…
I didnt think somebody would want to be part of it
Nice to hear it’s on github! I am not interested in money so don’t worry about that. This is some research I am doing, so it makes sense to me to contribute to work already being done instead of reinventing the wheel. Not that I am an expert in animation anyway.
I’ll be experimenting with your plugins for a few days and then I’ll try to contribute if I can. Thanks again!
Thanks a lot!!!.
And also if you have suggestion, or if you have som idea to implement i can give some directions and codes to help on what you want to archieve.
This is a great achievement. Am not using rokoko or autorigpro, but will look into open source tools to transfer the animation from the vroid rig to a MakeHuman rig.
Do the 4-5 BlendyPose code lines for smoothing look useful?
Those 4 lines of code helped a lot.
It gave me better depth position and it showed me an very simple way to put the animation in the right axis (flipping 2 axis)
Can the output be made smoother? Significant jerking is seen in the Michele video you posted and in results from my test input videos, though the video motions themselves are smooth. Can the smoothing code found in BlendyPose be incorporated? It works well there.
The smoothing options in mediapipe seem to work well here: https://github.com/zonkosoft/BlendyPose This plugin has been updated to work with newer Blender versions.
What are you interested in working on? It would be nice to have full body or at least upper body tracking using mediapipe. Lots of face tracking options are already showing up on the Internet.
My research is done in Universidad Pompeu Fabra for the european SignOn project (https://signon-project.eu/). Currently in early stages, but the idea is to ease the communication between Deaf, hard of hearing and hearing people.
Haven’t seen any tuts for blendypose, but there is a new tutorial released today for the mediapipe solution being developed by close2animation. You can checkout and subscribe to that YouTube channel.
