Blender Rhubarb Lipsync is an addon for Blender integrating Rhubarb Lip Sync to automatically generate mouth-shape keyframes from a pose library.
Rhubarb Lip Sync is a command-line tool that automatically creates 2D mouth animation from voice recordings. You can use it for characters in computer games, in animated cartoons, or in any other project that requires animating mouths based on existing recordings.
Unfortunately, I don’t have Linux installed at the moment, so I can’t give it a go, but they do have a Linux release at https://github.com/DanielSWolf/rhubarb-lip-sync/releases
Error message 
RuntimeError: class OBJECT_OT_rhubarb_lipsync, function modal: incompatible return value , str(, Function.result expected a set, not a NoneType)
location: :-1
Thanks for the report. Is there any more info you can give me? A screenshot showing the Rhubarb lipsync panel or even a .blend file would be helpful.
Thank you,
I solved the problem. it was file format troube. I was trying with mp3.
Two more tutorials!
One where we use different object for the 9 mouth shapes - one where we use shapekeys.
The Blend files can be downloaded here: https://www.blendswap.com/blends/view/92431
The 5th and final episode of the Rhubarb and Blender pentogy (like a trilogy, but with 5) of tutorials. This time we animate textures - with virtual Lego!
00:00 - Intro
00:24 - Downloading and installing
01:14 - Separating the mouth
05:42 - UV Projecting the mouth
08:38 - Rigging the UV project
10:21 - Setting up the pose Library
12:29 - Setting up Rhubarb Lipsync
12:45 - Setting up audio playback
14:56 - Running Rhubarb
15:40 - Constant Interpolation
Links:
Rhubarb lipsinc: https://github.com/DanielSWolf/rhubarb-lip-sync
Rhubarb Lipsinc Download: https://github.com/DanielSWolf/rhubarb-lip-sync/releases
Rhubarb Lipsync for Blender: https://github.com/scaredyfish/blender-rhubarb-lipsync
Minifig Model: https://www.blendswap.com/blends/view/69499
(other CGI minifigs are available)
Mouths Texture: https://i0.wp.com/yogyog.org/wp-content/uploads/2018/10/mouths.png
Hi!
First of all I want to thank you, thank you!
I am a person who is making games with Blender + UE4. And, thanks to you, it seems that the production is going to be progressing.
I have a question.
I am not an English speaker, so I would like to switch the recognizer from pocketSphinx to phonetic.
How should I do?
It will be very helpful if you tell me.
Hi nanasu,
I don’t know, sorry. You should probably contact Daniel S Wolf about this - he’s the creator of Rhubarb Lip Sync. I just made the addon that integrates it with Blender.
He’s on Twitter @RhubarbLipSync and email [email protected] or you can submit an issue on the github https://github.com/DanielSWolf/rhubarb-lip-sync
Thank you for your quick reply.
In Daniel S Wolf’s Github page, it was described that it can be switched with Command-line options.
Therefore, I was convinced that it would be controlled by python script in Blender.
It seems that it was misplaced, because I do not have programming knowledge.
I’m sorry, and thanks for the wonderful addon!
I wasn’t aware of that - it may be a new option I hadn’t seen before. I’ll look at adding support for that in the addon.
Rhubarb is built for English… I would suggest you try importing the audio files into Rhubarb without scripts and seeing how well it does. At a guess, I’d say if you import, say, German into it (German has nearly exactly the same sounds as English) it should work OK, but it would struggle with Chinese.
Also - how are you rigging the mouths? Are you using shape keys? Though shape keys DO import into unreal via FBX I’m not sure if shape key animation data does.
Thanks for the advice, I studied a bit from that and tried it without script but it failed.
In the first place designation of the recognizer on the command line is not good.
I looked at help, but -r was not mentioned.
It was written in github…
About this, I’m thinking of asking DanielSWolf.
Being able to import animation of shape key to UE4 is okay as my friend confirmed it.
Thanks for your concern.
I tried to contact DanielSWolf about I could not change the recognizer.
Then, I found out that I misunderstood.
It seems that it became possible to change the recognizer after being upgraded to 1.9.0
Version 1.9.0 seems to be released soon. However, I really apologized for bothering you due to my misunderstanding.
I will be looking forward to it until 1.9.0 comes.
