Links & Library Overrides are still glitchy after 4 years of development

General Forums Blender Development Discussion
303

We use about 3 tiers usually.

Character Rig for example.

And as a note we have very strict guides for this. And this basic guide was sussed out discused, visual graphs made and agreed before linking was used.

The artist is free to update all that they feel necessary to the original mesh object. These "name"v.000 versions live completely in another folder named WIP.

And the model file (no rig) always has the same extention. “Character_Name _MDL”.

The linked (_MDL) file always has a copy of itself in the WIP folder.

For example the latest .031 file is always the same as the _MDL file.

If changes are made .032 now is the same as the _MDL file. And so on.

This ensures that working versions are never ovewritten and debugging link breaks is easy. And the chain can be restored easily to the last working version.

This is all done manually!

No script.

The current linked file also has strict naming conventions in it. Strict guides that all artists must follow to structure and name collections within the file. So the _MDL file is always structured exactly the same from asset to asset.

And so it goes through the whole pipeline.

Each tier has only one logical override.

The _MDL file is linked into a rig file with its own location and also WIP folder.

It gets the name _RIG

The Character object(s) have an override so that they can be rigged.

This final _RIG file is then linked into an Animation File and now the rig has an override so it can be animated.

And so it goes through all of the assets.

The RIG file also has a proxi model for the animator in the case the original mesh is too heavy for animation.

And this ends the linking process as far as that goes.

The original RIG files - not the animator files - are linked into another file where scene layout and lighting is done.

Animations (actions) are simply apended into that file. The rigs have another override (from the ogiginal _RIG file not the animators file) so that actions can be stored in that file and used on the same rig.

Shot cameras are then set up in that file. Lighting is set and animations are applied where needed.

One extra step (and production note) is that we always link Collections.

So in the example of a character that is needed in one file to perform several shots and different animations, a new collection is created in that file with the linked RIG collection inside it.

That collection is then duplicated as many times as needed with the necessary rig settings of the original animation and hand props needed for that shot or scene.

Therefore one file has now all of the envioronment ojects and props, as well as characters linked.

Not linked in that file are Camera, Lights, Actions.

If there are 20 shots in that scene, they all live in the same file so that any updates to characters,rigs and environment objects only need to be managed one time. ANd of course versions are made .001 etc. of that layout scene.

There is a constant back and forth iteration between the rigging team and the animator team, as well as the model team, and even texture team.

These changes must be carefully monitored up the chain.

Animations might need to be redone to tweek for the rig changes, model changes and so on.

But this is all manageble.

But it requres a structure and a discipline to follow that structure as well as a knowledge of what to expect and what to know won’t work.

4 Likes
304

Hello,

Thanks for the couple of last responses , I found them quite interesting !
You said that in a rigging .blend you link/override the models from another .blend…
In that case how do you do skinning and shape keys without breaking the links ?

I see how this can be helpful to clearly separate the tasks so if you need to update the model you don’t have to fiddle with the rig’s stuff.
Beside that is there any other benefits ? does this allows you to work in parallel , or do other things that wouldn’t be possible if you had only one character.blend without any linking before animation ?

3 Likes
305

That’s sounds somewhat like the overall folder/file/object naming system I’ve setup for myself.

Mines a little more simple, like the rigging is in the model file, so it all just gets linked in one go (yes as a collection, only ever link a collection).

But of course, it’s just me, so no reason to split all that up to allow different people to work on it.

3 Likes
306

In the case of adding shape keys which is usually an exception for our projects, those rig and model assets simply live at the same level.

There is no good reason to separate them, and link the model. In fact there is almost zero benefit in that generally speaking anyway.

So it could be since the last time I checked in and got details on final decisions, they might be doing this as the norm now.

Edit:

And considering this:

It could be we made the decision only to append at this level.

It’s been a while since I checked in on the tech stuff as it has been running smoothly for over two years now. So I will have to go back and get that answer.

There is a project we are gearing up for where the face rig is all shape keys.

So that and with any assets that gets shapes we use the following workflow:

Model still lives on it’s own level.

However it is appended and not linked into the same rig file.

In both cases the model file also gets UV and materials updates and are either propagated up the chain automatically through linking or manually updated with appending.

This is where I enjoy the Blender rigging workflow much more because it is far more forgiving with model updates to a rig file.

EDIT: I do want to circle back to this because I need to open our servers and look at the files as well as talk with my team heads and find what the current pipeline actually looks like.

I am doing this all from memory and that is not a good idea at my age…ha ha ha.

2 Likes
307

Ok, so I confirmed the obvious. We made the decision early on to keep the models and rig files separate.

Modelers work on the models, UV materials and so on. The structure inside the file is always the same.

The riggers append and update as necessary the updated model into the rig file.

They work closely with the modelers on this.

Modelers provide a proxy model until all retopo is done and often animators use the proxy model even when the final is finished for rendering.

So the file linked into the layout scenes and animation scenes is this rig file.

For materials, I outlined the workflow here:

3 Likes
308

Ok thanks for explaining !
In that case, say you change the model or the materials, you need also to wait for the rigger to provide the final asset …

Anyway I don’t think there is a perfect workflow there, what you do keep things organised and straightforward, but even a small modification requires time and communication between departments…

What is possible to do in maya is to make links between models and rig, so small modifications from model could propagate without further action from rigging, but it’s likely to break if there is some topological changes for instance…

Finally, what I tend to do by having everything in the same file that goes from department to department, so a modification in modeling is made on the asset that is used for animation right away…
This allow fast modification and iterations, but it works better when people are generalists and modelers aren’t scared to fiddle with some bones or something else… To me it’s the easiest way of working but that probably won’t work on big teams…

Thanks for sharing your workflow it’s pretty interesting to see how studios do things !

3 Likes
309

Yes you are right.

If I was working alone - or when I was - the asset would move through the production line and simply advance in stages. Once the rig was added it would stay and that version of the model and file would continue as it advanced.

If adjustments needed to be made they were made on the file that had the last updates, rig uv etc.

Originally I tried to keep a similar workflow when I first started the studio.

But after a number of years it just wasn’t working and people would update the file and put it in the wrong place.

And overall it was just not practical.

Breaking up the production process into levels not only helps organize the production and keep linking simple, it also affords parallel production.

It is pretty much impossible to manage a team that is working on different versions of the same file.

But it is also more efficient not to have to wait for something to be finished to begin working on the next level.

Expected in the workflow is that we build a proxy lightweight file for the animators with a working rig.

And it would not be unusual for us to build this version from a concept first maybe even before sculpting and other details or retopo is done.

Or the first thing done from a sculpt is the proxy model for the rigging and animation team while they wait for the modeling team to deliver updates, final retopo and UV.

Meanwhile the texture team is working on the textures while the animation team is still working on scenes.

By no small miracle all these assets make it to the finish line so lighting and rendering can be done.

But…. Nothing stopping us from blocking out lighting first from even the proxy models and camera first.

Probably the biggest challenge with a team is keeping everyone busy not waiting around.

Then in those cases where that is impossible there is always some fixing that needs done, research and prep for the next scenes and so on.

Also I always have more than one production ongoing at the same time. This also helps efficiency.

Currently I am directing the lighting team on Thrive. Pushing them on all the final adjustments and rendering.

When they are done with that they will start on Yin Yang while the animators are still finishing up and the environment team is prepping scenes etc.

Then I have the next project to cycle through with the animation team when they are finished.

And so on.

Then we have client work that interrupts all of this… ha ha ha!

3 Likes
310

I notice you said append, since I assume the model would be as local data. But even then, if that model needs to be updated in the rig file, any replacement would wipe out things like vertex groups and shapekeys, etc as that data is replaced and likely even different (if any new polygons where actually added or removed in the model).

Or is only the overall control rig setup at first with automatic weights and any final weight painting/shapekeys, etc not done till the final model is signed off and locked down?

2 Likes
311

Yeah so there is no one solution fits all. I am just outlining the basic structure.

Every change has to be evaluated on a case by case basis.

Just weight paints alone, those would have to be transferred to a new mesh. It is a trivial problem. That does happen a lot.

But it depends. The rigger might just hand that file back to the modeler to continue with updates to the model, then work on something else in the meantime.

Changing a model with shapes of course would require handing that off to the modeler first.

No one is free to work independently without consulting the ramifications of what they are doing.

No production protocol can replace team coordination and communication.

And in general a working pipeline on a production is agreed beforehand. Or even some assets might get special treatment due to specific requirements.

But the general structure is as I have outlined. And then we take up the edge cases as they present themselves, but we do try and plan for the variety of problems beforehand.

We talk. We plan.

That’s the key.

But the main thing is to plan for things in a given sequence. Fixing problems always disrupts the flow one way or another.

So there is that.

3 Likes
312

Thank you for delivering concentrated deep production experience, it is so interesting to read!

So the point here is not if Maya is better or if Blender is inferior. It’s just a different architecture.

True, this way the purpose and possibilities of any software is predefined by its system design and architecture.
Different system design realizations contain different unavoidable tradeoffs (like flexibility vs manageability which naturally contradicts each other), so indeed none of them are perfect.

1 Like
313

I am glad to hear that the developers are working on improving Links and Library Overrides. I have been working on building an animation pipeline for a game project last year with a team and I had so many issues as a result of how fundamentally flawed the current system is.

The fact that you can break links for objects by changing the name in the Outliner is such a serious design flaw. I wasted so much time trying to fix animation files because of me having renamed Armature to Skeleton, which resulted in a chain reaction of making the animations incompatible with the Unreal Engine pipeline. I couldn’t just change the name back without having to re-link everything while making sure that the animation data wasn’t lost in the process.

Then there are the dependency issues where linking or appending something causes you to always import everything even remotely attached to that object. In many cases you want this, but in other cases you don’t. Having to clean up the scene of unnecessary stuff is just unnecessary work instead of just giving you the option when linking/appending.

Finally, there is the general workflow of linking and being able to override and reload missing links. I had to teach multiple animators on how to work with the system and it is not intuitive or user friendly. Too many settings that aren’t well documented and they are all over the place as to make it impossible to find for a newcomer to Blender unless you follow instructions or look up a video tutorial on how to locate everything.

I really hope the new system addresses these concerns. The current system does work, but it is far from ideal. If only the system took the best aspects from Maya and Blender regarding asset linking, it would be great.

10 Likes
314
Long quote listing the usability topics discussed

Most of the meeting was aimed at the current usability issues of the Asset Browser.
.

Usability

Andy explained his view on the current limitations of Blender’s asset system.

Andy’s Wishes

  • Enable Asset Shelf in object mode, for set dressing (like with the ‘Asset Shelf’ Python template script),
  • and for other things the Asset Browser supports, like materials.

Asset Browser (& Shelf)

  • Missing options for editing multiple assets at once. Lower prio is tags, higher prio is everything else.
  • and also for drag & drop of multiple assets
  • with options for “one location”, “row”, “grid”, etc.
  • Could be in the Redo Panel, or in the toolbar/header of the asset shelf. The latter is preferred, for discoverability.
  • Link vs. Instance a collection: would be nice to be able to choose on a per-drag basis, but currently hidden in redo panel, not that accessible.
  • Having these as a popover in the header would be really good.

Snapping

  • Dragging in objects respects scene snapping, is great.
  • Only works for objects, and not for collections (it’s not snapping the instancing empties).
  • Doesn’t have to use the bounding box of the collection, as that’s not (yet) stored in the asset. Could just snap the empty itself.
  • Snapping to the 3D cursor would be great too, might not be necessary as an option for dragging into the scene, though (see below).

Materials

  • Wish: Link to the scene without assigning, for assignment later.
  • Could be in context menu (see below).

Controversial option: context menu for more things

  • Right-click on asset, “Open Blend File” is super cool.
  • Want: “Instance to Scene” at 3D Cursor, “Link/Append to Scene”, etc.
  • Want: “Execute Script” (link to scene, execute, discard). Would maybe need “trust scripts” option per asset library.

Header

  • Define options for dragging into/onto things (next to append/link/follow prefs)
  • Sybren’s wish: show what “Follow Prefs” means. That’s the default setting, and to understand that default you have to know what the default preference is.
  • Context menu items can ignore that setting, and be more specific / independent.

Andy’s Issues

  • Big issue: Long names of assets are a problem in the grid view, even in N-panel. Issue in AssetBrowser and AssetShelf.
  • Possible solution: column view, small icons with full name behind it (like file browser column layout, but bigger icons than that, like half the size of the default asset browser)
  • Possible, less desirable option: multi-line names under the icons. macOS does 2 lines max.
  • Wish: show in the tooltip which blend file the hovered asset comes from.

Editing from Asset Shelf?

  • Rik Schutte (via Andy): organisation, catalogs of poses? Would be nice if that could happen from the asset shelf.
  • Sybren: automatic switching of the shown catalog(s) in the asset shelf would also be nice. That way it can be set up to only show the poses for the selected character(s), automatically switching when you select another character.

Asset Browser in Object Mode

  • What does it show? Everything?
  • Nice default: move primitives to assets, show those by default.

[…]

Here’s trying to give some exposure to the fact the devs do not actually always disregard anything related to usability, UI or UX, despite what some users seem eager to believe.

greetings, Kologe

2 Likes
315

The wish list looks impressive.

Unfortunately, this part seems to be following exactly the same path that led to the unfinished UX that is the Extensions Platform.

…see above. Guess why.

316

What do you mean ?
Task from your link is open. It is 5 days old.
It mentions the list posted by @Kologe as a priority.
It refers to less detailed, 7 months old, closed tasks as an early draft.

So, basically, both links are about same project. And both are proving an advancement, a precision of wanted design.

317

I’m looking at the extensions UX, the lead on extensions, the author of the task, and concluding that this feels familiar.

318

That meeting and the task are about downloading assets from extensions platform and custom repositories like for add-ons.

What is the issue about being coherent with what we have for add-ons and themes ?
If you have objections about design and mock ups ; now is the moment to formulate them precisely.

319

The UX for what was built for add-ons and extensions still needs work and improvement. I’m not awarding points just for being consistent.

I believe such foundational decisions have likely been made already. And I’ve no interest in playing another round of “why are you hiding buttons?”

320

Yes. That is a legitimate complaint.
But, in that case, asking for consistency is having a point to change Repositories popover into a sub-panel in Extensions tab.
There is no reason to have a directly available readable panel for Assets Libraries and a popover for Extensions Repositories.

Being forced to open Repositories popover, then click on + button, to have choose between add a remote or local one. That is slow.
And being forced to redo those steps in case of error, clicking again on a - button giving again two choices, is annoying.
Indeed, redoing the same mistake for 7 choices, about Asset Libraries, is not great.

The list template with +/- buttons was made for quick addition/removal of simple items.
Being not sufficient to manage the list, the down arrow popup was created.
But, that is becoming ridiculous, to have lists showing a minimum of 5 rows of items (so, space for a column of 5 buttons) and hiding 5 operators in 2 buttons.
Plus, that is for a panel that will be wide to show long url paths.
There could be a row a dozen of buttons.

321

I’m back to Blender after a 4 month sojourn.

Currently on 4.1. Any recent developments on Overrides? Or same old?

1 Like
322

There has been movement, but not exactly because of user complaints.

The Blender Foundation wanted to use the asset system to ship assets with Blender (which currently includes hair modifiers and will soon include particle functionality), only to find there were too many issues in their own system to make use of it properly. If even the BF can’t properly make use of their own tools, then that is about as effective as it gets in getting the message across.

2 Likes