A flat granular Blender pipeline thought. Keen for feedback!

Sorry for the double post…also posted this in general discussion. Realised its somewhat a “technical support” question.

So recently I’ve been pondering about creating flat and highly adaptable pipelines in Blender. This is sort’ve pre-static-overides coming in 2.8, though that could also be incorporated as well.

So one good and bad thing about Blender’s linking a rigged asset in is that its highly protective yay! But also not very versatile, boo…in that only armatures really are editable (without using static-overides).

One of the great things about it thought is that its highly granular (link in 1 material from within a file).

So…my thought is, in pipeline could you basically have a massive library of materials, mesh (with applicable vertex groups) and armatures. That get assembled each env and shot. All that is “passed” down pipeline is a custom configuration of how the assembly looks (like a scene description, could be json file or whatever). So each stage down the pipe post asset creation is just reassemblies of this more granular aspects of an asset.

I haven’t run any tests yet just a conceptual thought experiment at the moment. I’m not entirely sure whether a linked mesh and a linked(or appended) armature can be combined. Is the modifier stack open for additions on a mesh?

This would maintain the integrity of data to quite a high level and would also mean you have an ascii file scene description that could be edited in a text or custom editor.

Thoughts? Feedback much appreciated!

What you are describing is basically what Katana and Clarisse are doing. Scene assembly from base elements with all kinds of possible scene/shot/whatnot based overrides. I think for bigger or more complicated productions it is very valuable, question is, how it would fit with Blender.

I’m giving it a test so will report back how it goes. I can see how it all SHOULD theoretically work…but whether blender will cry at me or not…we’ll see.

Simple tests have worked so far!

Loosely based on these diagrams.

A reasonably flat pipeline.

Documents ill be updating as i go:

Hi mafster,

I’m really interested in what you are aiming for. I’ve been working lately on developing pipeline tools and reached a similar design. In particular, I also have this rig/surfacing assembly to solve. The main element that you are missing imho is the versioning.

Also, I’d make a difference between vertex_groups/shape_keys related to the setup (skinning/rigging) and those defined by the modeling step (less common).

Anyway, I’d be happy to help with the implementation as well. :wink:

Some tests so far, mostly about linking and versionning:

archiviste-pipe02
Asset list loaded from asset database.

archiviste-pipe03
Importing asset groups (in most cases, there is only one group per asset).

archiviste-pipe04
Link editor, with update button to switch to the last version (should be an enum menu).

1 Like

@Mink Cheers for the reply! Yeah actually i do have versioning, its part of a separate system that isn’t blender specific (as we plan on integrating with houdini further down the track). So there’s kind’ve like a global versioning system that manages resources.

So with this setup of yours, my understanding is the mesh and rig assemble separately? Is this addon open source?

Can you programmatically update vertex_groups to linked mesh data? I wouldn’t think so without localising the mesh? Though i suppose you could always reimport a new mesh and swap it out…

I am on the lookout for further dev’rs and i have a number of supporting packages which i have put out on github and will be putting another one soon. I’m a heavy qt user. Are you familiar with pyqt? I have been able to successfully use pyqt in blender in early tests (pre 2.8)

The addon is totally WIP at the moment, I’ve just started a couples of days ago.

I don’t have any magical solution for the assembly: I was thinking about some automated processing appending the rigging asset scene and merging interesting data from the modeling scene. My asset importer then links to this automatically generated blend file. But I’m willing to look for a cleaner solution.

I’m very familiar with qt (more C++ than Python, but the API looks the same), though I’ve never used it in a Blender environment. There must be no difference with regular Qt.

I’m curious about this Houdini integration: do you then merge data back to Blender for rendering with Cycles?

1 Like

I’ve done a working prototype of the surfacing/rigging merge today. It is split into an operator for doing it manually and a script meant for batch processing (calling Blender in non interactive mode).

TransferMaterialsOperator.py

BlenderSurfRigMerge.py

This assumes that the rigging and surfacing scene both contain the same mesh topologies, exported into groups. For now, it only transfers material slots and each polygon’s material index. But it is quite easy to generalize to other stuff such as merging extra UVs and vertex groups for instance.

interesting, and does this work if the mesh to transfer data to is linked in? I suppose one can update to add 2.8 static overrides which would allow to override even in a linked scenario.

bpy.ops.object.material_slot_remove() as in this command. Would it work if the to_obj is linked? (and the following edits to to_obj

The from_obj can be linked, in which case the material slots of to_obj will link to the materials as defined in the surfacing file, which is convenient. But to_obj cannot be a linked object, otherwise we could not alter its material slots.

The way I use it is:

  1. Modeling step, producing some asset-mod.blend
  2. Parallel Surfacing and Rigging, each appending the content of asset-mod.blend and producing respectively an asset-surf.blend and an asset-rig.blend.
  3. A fully automatic step merges the surf and rig files into an asset-merged.blend. This is done using blender -b -P BlenderSurfRigMerge.py.

Of course, all of this with some versionning support.

Blender 2.8’s material overrides might work on linked objects (they should as far as I understand it), but what if the surfacing also adds extra data that you want to merge?

I have to admit that I misinterpreted you original post: I though you were working with blender “pre 2.8” meaning “2.79”, not “2.8 alpha”. I am still working with the 2.79, but I’ll be switching to 2.8 when it’ll be ready for production.

yeah we took the plunge…even some client work on 2.8 :grimacing: little scary but…just so many benefits already just from eevee…
Bit tied up at the moment but later today i will absorb what you wrote. Lets keep the convo going! cheers