I created a free app for iphone X and a blender add-on to import facial data into blender!!
app: (https://apps.apple.com/app/id1494579713)
add-on: (https://github.com/pjardin/iphoneCaptureAddon)
blendshapes: (https://developer.apple.com/documentation/arkit/arfaceanchor/blendshapelocation)
Please try it out and tell me how i could make it better! I would love if this became a tool for all blender artists! : )
I’ll have to try it out when my PC gets fixed. Will you expand it work with the whole body, like they have in animation studios? I would love that.
I will! That will be version 2! : ) It is truly insane how advance the facial tracking and body tracking is on the iPhone.
Is this going to be tracking a person or is this something that you can use to make animations or video games?
It will track a person and will record the full body tracking. One file would be the physical location of the person, another file would be the motion recording, the movements of the person.
so you will be able to record:
Amazing, what language are they specking?
Dude this is incredible, and I’m stoked. I wish I could try it out, but I’m still on an iPhone 6s, which was fine for me until now. I’m going to start saving for an iPhone XR because I’ve been looking into this motion capture stuff a LOT and finding out what my best solutions for it would be. I’ve been thinking about recording motion captures into iclone and then editing them, and then exporting those animations into blender as either BVH or FBX files.
Would it be possible for you to have an export as BVH or FBX with your mocap software? Blender is a nightmare for editing animations so it’s not ideal for me to have the animations be in blender only.
GREAT work dude. I think you could charge money for this software. iclone has a facial mocap app for 400$ that I’ve been thinking of buying, and yours is very competitive with it! I can’t wait for your full body mocap because I was going to buy a Kinect v2 to do that, but I’d MUCH rather be able to do it on my iPhone!!! I’d be HAPPY to pay you for the app. A Kinect costs 100$ + 50$ for the USB connector. If you had FBX and/or BVH export directly from the iPhone, you could get away with charging 100$ for the app.
Thanks for the compliment! I am currently working on version 2 of the app which will have full body capture! The data that is being saved from the iPhone is a json file, so it should be easy to convert to BVH or FBX files : ).
My current business model is to have BMC, blender motion capture, free and have fun $1.99 face apps. For example, i am currently working on a dragon ball face app, where you can be dragon ball characters and message other people with your favorite characters! : )
Hmm, unfortunately I wouldn’t have the know how to convert a json file to BVH or FBX, as I’m not a programmer and don’t know the behind the scenes of how any of it works. If that was a feature I have no doubt it would command a higher price and could be a business opportunity as I guarantee I’m not the only one who would be interested in such capability, but you got to follow your heart! I hope the DBZ face apps make you boatloads of money 
edit:
I just found this video on editing animations before bringing them into iclone so perhaps I will be able to accomplish this with your app. I just need to save up to buy an iPhone XR and hope you’ve finished it by then!
Dude nice!!! Great work!!! I have an iPhone 6, otherwise I’d try it out. I’m looking into getting an iPhone XR to take advantage of ARKIT3, but not many apps utilize it yet!
Yea, it looks really cool.
I was considering scrapping my LG smartphone for an iphone, since I use a Mac Mini everyday. This may be the motivation I need.
So this would be a substitute for Rhubarb lipsync on Makehuman characters?
Joshea
Pretty much, the app allows facial capture and full body motion capture.
I am currently working on the next version which will allow easy comparability with make human.
Hi, @PascalJardin
I’m trying the BMC app but having some trouble. When I create the rig through the addon the armature appears wrong, the rotations are weird, some axis is inverted. I can recognize the joints structures but the joints connections bones are pointing backward…
Also, the app is emailing a file without extension. I just discovered that I can just add .json to the file and it works but very weird, the MoCap result is not what I see when capturing…
I’m using iPhone 11 with iOS 13.5.1