Help with shots and scene setup in blender for animated films

There’s a AI front end called ComfyUI (https://github.com/comfyanonymous/ComfyUI) which while not super one-click artist friendly, due to it’s extensive node setup and addition plugin system, I suspect that one could put together a setup which would generate storyboards.
As in not just a thumbnail image, but a full storyboard with a sequence and shot number, any dialogue as text, etc all based on specific inputs and formatted in a storyboard type panel, even based on local training data of said concept art.

At least that’s my feeling at this stage. It’s just that even to play and test all of this sort of local AI, usually requires installing a bunch of libraries, git, python, etc, etc and I just don’t want to mess around with all that on my current main PC which is a fairly clean build, running smoothly and used for all my day to day stuff.

Hence needed a second PC that I can totally mess around with and mess up, find out what works, what doesn’t, crash it, reformat and re-build as need be and it doesn’t matter if it’s broken for a week or two.

I have thoughts of using the same PC for AI code generation, as in basic Blender python scripts to automate some production tasks and finally a very good text to speech AI, for the character voices. As luck would have (or I’m just slow and take too long to get anything done), I’m not yet really at the stage I need most of that yet.
So it can wait till near the end of the year, which has the advantage that a lot of new PC hardware should be out by then and of course AI moves at such a fast pace at the moment that chances are much better options will also be out to explore in 6 months time.

1 Like

Everybody’s different, but here’s what I do . . . My workflow is built entirely around the video editor, as though this was “a real movie.”

I start by creating the set to scale. (How high is that wall? How long?) The set is at this point un-decorated. Then characters: once again, simple geometric shapes, but “to scale.” Basic armature: just enough to get by.

Next, the props. Each one is at this point a “bounding box.” The finished prop will just fit inside of it. But it hasn’t been built yet. None of it has. But, “that stand-in thing” is of exactly the right size / dimensions.

And, everything(!) – sets, actors, props, files, scenes, cameras – has "a meaningful name."

All of these are in a library of individual files, designed from the start to be “linked.” In due time, you’ll work on each linked file to replace its contents with “the real thing.” But, you can defer(!) that for a very long time.

I then try to "start shooting." A shot-file consists of a base scene – linked to the background set and all needed assets – which contains some [single …] bit of action. Then, “shot scenes” are linked to that within the same file. Each one has a (named …) camera (and camera rig), a particular output filename, and a meaningful scene-name. I then “shoot” each one. The shot begins slightly before the action ends, and ends slightly after, to give me a “tail” on both ends for editing purposes.

I simply use the Workbench renderer (for speed), and “stamp” each frame with the following metadata: filename, scene name, camera name, and frame number. This allows me to instantly see, in the editor, exactly where each frame came from.

Now, I take all this “footage” to a video editor of my choice. Could be Blender’s, could be any one of several good open-source products, or (as in my case) a familiar commercial product.

:point_right: I have done everything up to this point "to get to here." In the editor, looking at “footage,” and using my imagination.

As you work with the editor – which is where you will spend most of your time – you begin to get a feel for the timing, pace, and rhythm of “the movie.” Very importantly: every shot – “rough” though it now is – actually corresponds exactly to the “real, final” version of the same shot. The point is that you can shoot it all “fast and easy.” Doesn’t matter if you actually use the shot, in the end. But you can make very detailed decisions, “using just this.”

Each shot corresponds to one action. If the character’s planned to go through several moves, they’re still shot “one at a time.” (Recall: base scene contains the action using all-linked asset, then linked camera scenes in a single file contain the shots. If I want to choreograph the entire move, I can freely do so.) This gives me leeway during edit.

“If I was a real director, directing real actors on the set, what would I direct them to do?” Then, if I was a real cameraman/cinematographer, using real equipment (booms, dollies, trucks …), “how might I shoot it?”

I actually try to get as close as possible to “final cut” at this point. (And, believe it or not, you can actually do it.) Then, once I know what “the movie” is actually going to need, I can begin working on the various linked files, one at a time. Simply re-render the appropriate stuff after each change, and “magically” the movie begins to appear on your editing screen. When you get to “final render,” you now know precisely what material you actually need.


I also keep a daily “Captain’s Log.” What did I do today? What unanswered-question do I have? (I’ll back-fill the answer when I find it.) What’s next? What’s the URL/hyperlink of the cool video that I just found? And, what was I thinking as I was looking for it? Just write it down and never change what you wrote. Because, your memory isn’t as good as you think it is. (I was going to add a nice closing comment here, but I forgot …) :wink:

4 Likes

Thank you for sharing your approach! I really like your “Movie first; Assets later” approach. The way you are using blender’s Link to assets, even when they are still just a bounding box is inspirational! :smiley:

I do this on a shot-by-shot (sequence) basis. The main question I’m asking/answering with this step is: does this animation WORK (timing, plausible physics) ?
The Viewport render modes still frame-drop, so you can get close to real-time, but not actually. So as I work out the animation for just this sequence, I have to test preview it – this is where workbench shines – super-fast renders (materials and final-lighting ignored). Render quick. Assemble quick. Play video to preview. If good enough, switch back to Cycles and invest the time in the “real” render. As I complete each shot sequence (as a folder of frames for that shot, and iterative videos of just that shot), I then add this next shot to the projectName.assemble.blend file in the Video-Sequencer (import image sequence) after the preceding shot. Slowly the finished video gets longer and longer by each shot/sequence added.

1 Like

If you very carefully focus on “scale” from the very beginning, the “very rough shots” that you are working with actually aren’t “rough.” They are very much exact. This is “a digital computer,” after all . . .

Also: “that bounding box” will actually become(!) “that prop.” Which is why you very-carefully plan the file-linking system, even when there’s nothing (yet) to “link to.” The project [data …] structure must(!) be carefully laid out, even though most of the content isn’t there yet. Because: “you are going to live with that structure throughout.”

I zeroed in on this approach for one fairly-simple reason … “I can’t draw.” (My nephew can easily “just crank out” a fully-colored scene from a graphic novel … so, I hate him. :wink: ) Any discussion of “storyboards” is therefore “out of the picture”: I cannot, for the life of me, “draw a picture of” what the final shot is supposed to look like.

But, I can proceed forward in this way: it’s “previz” but but more than that. Defer every consideration as long as possible, while very carefully preserving the visual accuracy of everything that you are looking at. Race for “the movie.” Then, back-fill the rest of it “one issue at a time.”

4 Likes

My next question, regarding building the movie (initially) with bounding-box assets (linked, to be supplied later) is:

do you add a text-object to all the bounding-boxes to label them with their “meaningful name” (or at least some initials or abbreviations) ?

(how do you tell all those bounding-boxes apart in the previz images?)

I will often add simple colors to them, but yes, I use the “display the name” option when necessary, so that the bounding-shape displays its meaningful-name in the viewport. (As noted: the file-name, scene-name, camera name and frame number [metadata …] are also “stamped” into every frame.)

Usually, though, there aren’t very many assets in any particular shot, and I already know what they are. And, the “bounding shapes” usually aren’t cubes: their look sort-of also suggests what they are. (Stand-in geometry of any sort works too, as long as it is of the correct size and form factor and you can hammer it together quickly.)

The key thing is that, since they are all to scale, as are the sets and the actors, and the cameras and lenses are real, and so-on, what I’m looking at isn’t “just ‘previz.’” It is going to, over time, turn into(!) the final version of the shot.

If it makes it all the way through the editing process to “final cut.” Which many shots don’t. When I’m “on set,” I’ll shoot lots of different ways. After all, at this point, “film is basically free(!), and you don’t have to pay the actors Union scale.” :smiley:

1 Like

Which is a very important point that may not have been made clear enough.

Outside of how one does storyboards, animatics, etc etc. Keeping ALL modelling, scene setup, etc to the same scale is vital.

For the most part this means REAL WORLD SCALE. Ideally any ‘professional’ is working in real world scale, so any drop in asset pack, etc would just match up to anything you also do in real world scale.

So to anyone just starting out and reading through this, don’t be a 3D noob and make your pot plant 30m tall, while your human character is 65cm tall.

Outside of having a very good and specific reason to use some other scale, everything should be REAL WORLD SCALE.

I agree. One of the nice additions to Blender was a variety of “real-world scale systems.” Both “feet and inches” and metric. (In addition to the ubiquitous “Blender Units (BUs),” which have no real-world corollary.)

It’s okay to buy a 100’ real(!) tape measure and a loose-leaf notebook and, with a helpful assistant, go to (!) some real-world place that is similar to what you are now thinking of modeling, and actually measure the thing. (Or maybe you use a “laser tape” these days … much less conspicuous.)

One thing that you notice constantly(!) being used on a real-world set is: a tape measure.

And – if you are creating a purely-imaginary world, still … “pretend!”

1 Like