Lip sync plans for Blender...

Hi, I just thought you may like to know that there are plans for lip synch in Blender:

http://wiki.blender.org/index.php/Requests/lipSync

There is a .pdf about it here:

http://marc.gardent.free.fr/files/Lipsync-Documentation.pdf

It is just in planning stages and will probably take a while to do so don’t get too excited yet!

Koba

No, we don’t believe it, it must be some kind of april fools!

p.s. I’ve read about this yesterday at dev forums so it is real :wink:

Good point. I shouldn’t have posted today.

If you don’t believe me, here is the thread on blender.org that was started on the 27th:

http://www.blender.org/forum/viewtopic.php?t=11038&highlight=

See, not April fools!

Koba

Blecchh! Where’s the fun in that?

its a fairly good idea (even if it were an april fools joke)

Automated lip sync. Sort of like the Poser or MakeHuman of animation. Great for quick background stuff, but crap for anything that requires any sort of care or organic attention.

Very interesting. One plea, though: No arbitrarily fixed set of phoneme shapes! I have been using Papagayo and the LipSynchro script, and not only is the user limited with the set design of 8(?) phoneme-to-shape directors, but tweaking is also more clumsy if extra Shape IPOs have to be set manually; in fact, it kinda makes the whole lipsync feature redundant.

So please let the number of phoneme/viseme Shapes be user-defined. Beyond that, I wholeheartedly support the effort. And after all, those disliking the feature are still free to do by hand :wink:

Um, doesn’t it say “deadline in may”? Don’t mind waiting for quality, but it seems to be a deadline set by their university, and not jus tthemselves…

Um, doesn’t it say “deadline in may”? Don’t mind waiting for quality, but it seems to be a deadline set by their university, and not jus tthemselves…

You are right. I was worried that people would jump on this thread and start asking for builds and patches. Thought it was best to tone it down.

The deadline is in may but that doesn’t mean it will be done or done well. The chaps doing it seem to think it is ambitious themselves. That said, I wish them luck!

Koba

I definitely second that. Also, they are advised to look at the LipSynchro script for inspiration! Are you a part of the team, or just well-informed, Koba?

Just well informed. :wink:

Read the .pdf, it details their project well.

Koba

Very interesting and exciting

I agree with BlackBoe… I think it would be perfect to make a movie like monsterHouse… where everything is mechanic and automated… but where everything losses the “art”(?) of making animation…

I don’t understand people waiting for this kind of stuff while they can’t make a simple bone animation …??? No offend

Same things happens with other soft’s… WHy you need tools like CharacterStudio to make steps to your character automatically…? C’mon, guys!!!

I’m gonna see in a couple of years someone asking for a button that make the whole animation for you…

To the people saying this is important and necessary… try to make mancandy speak with stuff like that without loose personality…

I think blender needs a loooooot of improvements on tools that already has…

Examples???
Improve lattices ( modelled lattices , following the shape you need…) Hooks…( deformations… parenting… this things aren’t working properly right now)… crazyspace vertex (not solved ) bones parented to a vertex o similar… etc etc and a large etc…

I think sometime we should stop requesting features and request improvements as I said before… don’t you think…? :wink:

I agree with bataraza and BlackBoe and AndyD. Focus on enhancement of old stuff maybe more appropriate than automatization and robotization of crative processes. I wish Blender would never meet he aproach of MakeHuman where actualy lack of art, crativity and imagination is quite obvious.

But I understand it may be interested for someone who wants fast, easy and prefabricated products.

Wow it sounds like the last feature that basse has been waiting for before he goes pro…

Looks very interesting, good luck.

I for one welcome new automation tools like this, for a noobs development! (Like myself!) sometimes it’s more important to finish a project than to be painstakingly artistically brilliant over every detail. That all comes later on. I just want to make stuff talk!

And I’m sure the results will look pretty good…or be tweakable.

This project looks interesting, and if it can streamline some of the process for lip sync, I think it would be great to have in Blender.

Some things will always need to be manually adjusted, but why start from scratch every time if some of the steps can be automated?

Maybe they have no desire to make a simple … or any other animation?

For some applications, e.g. simulations, or other animations where the overall story / situaion is the goal, the specific way a character talks or walks or whatever else action, may not be important.

So why would that be a bad thing?

Again, it depends on what the goal is. If someone is more interested in getting the final result, and is not interested in the actual process, a “make movie” application, is just what they would be looking for. There are a number of those programs starting to appear now. We even have a Blender version in the works with Didu’s “Movie Maker” project.

Those features are obviously important … to you. Auto-lip sync, or tools to help with lip sync is important to others.

For instance, right now I’m playing around with a relatively simple script that will add dialogue text to the timeline, so that I don’t have to enter it manually every time.

If you happen to program those or get someone to improve them, I’m certainly not going to complain and say they should’nt be in the program …unless it slows it down or affects something else :slight_smile:

I don’t understand the mindset of dismissing / bashing a good idea that will provide a useful tool.

This was not a feature request, and is not being produced by Blender developers, it’s an outside university project.

Mike

I agree with Mike_S.

Sometimes it seems that people view any progress as cheating. It is all about enabling expressiveness of the artist. And frankly, I am so tired of hearing that Makehuman is cheating. It is a tool and a great one at that. If you feel you are cheating:

  • Heavily modify a Makehuman mesh after it is created. Has Makehuman stopped you from doing that? Does Makehuman instill a fear of edit mode? How exactly is Makehuman anything other than a tool to realise your artistic vision? Such a lame argument.

  • Go make your own Makehuman targets and stop complaining. Now you realise that Makehuman is simply a mesh morphing program with a couple of presets - ever use the Blender presets, icosphere for example? Just another tool, in the same way that subsurf is a tool in Blender. Is subsurface cheating? After all you loose artistic control over every little polygon you mesh is “pretending” to have!

  • Create your art and state very clearly that you used Makehuman. What is wrong with that?

My artistic freedom has been increased manyfold since the last release of Makehuman. Coupled with Blender I can create humanoids to my satisfaction in a reasonable timeframe with a reasonable effort. I can now focus on my composition, lighting and texturing instead of just modelling. With many, many characters, my current project is more than just a still or five seconds of animation and would simply be impossible without Makehuman. It has enabled me as an artist.

Back to the cheating issue - some people spend ages creating astounding images in MS paint. If I rendered a shiny sphere in Blender and claimed that I had drawn it pixel for pixel in MS Paint, I would be both cheating and lying. Similarly, I would not try passing a Makehuman model as my own model created from scratch. So you state the tools you used and the final artwork and let other people judge. Allowing people to express themselves to any level quicker and with less effort is progress, not cheating. Art is about vision and dedication - are movies “cheating” because they use special effects? It is the impact of the final product that counts.

There are images that used Makehuman in the Blender.org gallery. Go tell those artists that their integrity has been compromised by using began life as a Blender script. I hope you don’t use Python scripts in Blender written by other people because it is “cheating”.

I’m gonna see in a couple of years someone asking for a button that make the whole animation for you…
What kind of button will be able to see inside your head and with a single click specify everything in your vision. Even if one day it does take a single click to draw out something in your mind, there is still effort. Effort in imagining the scene, the timings, the lighting the details. Their would still be art in such a world because not all people’s minds are the same.

Sorry for the rant but I’ve heard this nonsense for years now. The same argument applies to adding lipsync. If you believe any major CG movie (eg. from Pixar) was done without lip sync tools, you must be out of your mind. If tools make your life easier as an artist, use them and to hell with “cheating”.

Koba

EDIT> Just for the record, I probably wouldn’t use Makehuman exclusively for a single still render where the character is the main focus of the piece. That said, I may use a base mesh for guidance and prototyping in Makehuman.

Id be happy with something that would just be able to highlight the phonenomes (im sure thats spelled wrong) in a loaded wav file…that would be great as for a whole system im not sure how that would work out, I have a feeling all your characters will have the same uniform expressions, but who knows if its done right it could be useful, anything that helps increase the speed of the work flow is a good thing, as long as you dont have to sacrifice quality.

That’s what papagao does :

http://www.lostmarble.com/papagayo/index.shtml

As for any automation, it can be just a starting point, which you can tweak afterword, as the output of any automatoin is going to be ipo curves driving eithe bones or shape keys.

Mike

Holy crap. I never meant to transform it into a huge discussion. Nor did I say it was BAD, or cheating. I just said it almost definitely wouldn’t be worth half the same quality as a properly done hand-animated piece. For all you people that want to make of movie of rigidly chatterboxing MH models, that’s fine with me, it’ll make the rest of us look better. :stuck_out_tongue: