Links & Library Overrides are still glitchy after 4 years of development

[Conan_The_Blendarian (I Came I Saw I Blended)]

tl;dr: Blender can deliver AAA animations, but bugs and library overrides are serious problems which are not on BF’s radar.

In the middle of production, spending far too much time fighting bugs and overrides.

Don’t hold your breath for overrides though:

Here’s a recent article posted on Blender.org telling the 2024 plans for Blender: Compositor, EEVEE Next, Grease Pencil, Brush Assets, Cycles, USD, Animation 20XX, and after that: Blender Apps, Layered Texturing, Vulkan. https://www.blender.org/development/projects-to-look-forward-in-2024/

Not a peep about code quality or Overrides. And although Animation 20XX is in the list, not a peep about overrides in there either: https://code.blender.org/2022/11/the-future-of-character-animation-rigging/ (The issues covered by “Animation 20XX” are not the same issues those raised in this thread.)

As for code quality, there are posts on the Guano thread discussing how bug fixes aren’t getting through because BF isn’t dedicating sufficient resource for code reviews, in favor of more cool new toys. Also a suggestion that BF freezes development for a release to clear out the backlog. https://blenderartists.org/t/what-is-the-most-bat-guano-insane-thing-in-blender-that-you-cant-believe-they-havent-fixed-yet

Good! Now if only BF was listening…

And reviewing last year’s Annual BF Report found this gem:

Bastien Montagne: ‘Bastien finished the library override system with a usability overhaul, while the old proxies were finally removed’

https://download.blender.org/foundation/Blender-Foundation-Annual-Report-2022-v2.pdf

Finished? No. I don’t think so, but BF think it is.

@sozap

Eventually it will work for blender too, but it’s just a long road to refactor everything.

Will it though? There is zero indication of that.

@sozap

I think it’s a bit naive to think we can take better decisions, it’s like commenting a sport match on TV, we criticize the players on the field because we are passionate but in the meantime it’s obvious that we are completely unable to do what they do.

That’s a flawed analogy. Using Blender is not a spectator sport. They make a complex tool they expect others to use. Engineering or product development is a better analogy: https://i.pinimg.com/originals/f1/9c/77/f19c7700c8e93a705662d87069901934.jpg

Yes, You need to know when you’re talking outside your area of expertise. But blind faith is also naïve:

The biggest IT disasters I’ve seen - millions of dollars lost and entire departments sacked - is where there’s a gulf between the users and the developers. The developers wander off, ignoring what their users really need, and do something else. By the time the users realize their blind faith in the developers was misplaced, it’s too late.

It’s True: If you’ve spent years writing Blender code day-in day-out you should be good at it. Far better than anyone else, including your users.

But if you’ve spent years creating Blender animations day-in day-out, you should be good at it too. Far better than your developers. After all, who knows more about flying aircraft? The people who make the aircraft, or the people that actually fly them? And God help any aircraft manufacturer who thinks they don’t need to talk to the pilots.

It’s possible to be appreciate and be thankful to Ton, Bastien and the blender devs - both BF and volunteers - without deifying them. I’ve spent much of my career in software engineering. Coders aren’t gods, though they can seem like it if you don’t understand what they’re doing. But learn C++, pick up a graphics programming book (Joey de Vries LearnOpenGL.com is excellent, there are similar Vulkan ones), start writing lots and lots of code, and you can do it too.

Coders are not gods: They are very human with very human traits. Some unfortunate traits in the software industry are not testing code properly (that’s a really big one), avoiding software maintenance (bug fixes, because it’s hard, and adding critical but mundane function, because it’s boring and unglamorous), compared to writing new code which is much more fun - and glamorous.

Good companies have good managers, test engineers and Quality Assurance to keep the coders on track. Otherwise, coders will ignore bugs and boring but essential function in favor of the bleeding edge: Happy coders. Unhappy users.

This may be where Blender is having problems: It’s unquestionably the buggiest program I have ever seen. I’d describe it as perpetually between alpha and beta quality. Not even beta quality, because at least with beta the problems you report get fixed. (Of the bugs I’ve submitted over the years, I’ve only ever had one decent response. The rest were a complete waste of time.)

As for “completely unable to do what they do”, you’re not only overlooking that we know how to use Blender better than the devs do, but there are many 3D artists out there who do understand what happens under the hood (There are some here on BA who really know their s***). We live in the age of game engines, but until very recently if you wanted a 3D application or game, you did the OpenGL/DirectX/Shader code yourselves (and not with node graphs!). There are 3D people out there - particularly with a gaming background - who know this stuff too, and function like overrides is more data structures and database management. This is a very well understood (mundane!) area.

You say “we criticize the players on the field because we are passionate”. Nope. I just want to get my work done. Passion is for sports teams and personal relationships. Passion is bad when it interferes with rational decision making, which is what engineering and business demands. And truth be told: passion is why I stuck with Blender too long.

Some Blender bugs I’ve hit over the last week:

• A day lost because a scene when rendered from batch suddenly wouldn’t display the same lights it did when rendered from the gui.
• A day lost because a different scene wouldn’t render because of this: ExceptionCode: EXCEPTION_ACCESS_VIOLATION Exception Address: 0x00007FF8525D7E78
• Blender crashing on an Intel Laptop(*) when editing a simple 40 polygon mesh. Comment from an onlooker: “Pathetic”
• 7 hours lost on a re-opened library override scenes because animated parts shifted without apparent cause.
• Hours spent battling library overrides bugs, glitches, and suboptimal UX.
• Time spent trying to decode Inane error messages, like:
  ImageInput::read_image() failed: Read error: hit end of file in targa reader. (Sure. Which one?)
  IMB_ibImageFromMemory: unknown file-format () (Sure. Which one?)
  GPUTexture: Blender Texture Not Loaded! (Sure. Which one?)
  Warning: Maximum instance recursion level reached. (Sure. Where? Also, 10 levels is a very silly limit.)
• Other crashes - several times a day.
• Other batch runs - exits without rendering with no error message.
• Blenders non-WYSIWG requiring re-renders.
• Time lost I could have been improving a shot to AAA quality because I was doing the above.

Things I am tired of hearing back:

• No one else is reporting this. It must be you.
• It must be something with your hardware.
• You’re an idiot newb.
• Report it via BugTracker / Suggest it on RightClickSelect, and it will be fixed.
• “Well why don’t you use Unreal/Maya instead and see if we care” (**)

(*) Blender has a long history of not working well on Intel Integrated graphics. Bug reports have been raised over it. Official response was it’s due to an obscure hardware glitch in the Intel Chipset which Blender can’t fix, yet every other DCC and game works fine. On an early bug report the dev responded they hadn’t tested the Intel chipset (because what self respecting 3D dev would use Intel Integrated graphics?) Unfortunately a lot of people, including newbs who can’t afford better yet, and pros when the GPU machines are all busy. I’d say if Maya can do it, then Blender should be able to do it too!

(**) Well, yeah. My next project will be a pilot on Blender/Unreal or Unreal/Maya. Question is: do you want to improve Blender so animators can stay on it? I’ve gained a lot of Blender expertise and it sucks to throw that away. But if Blender won’t improve and there are better options, of course people will gravitate to those. What do you expect?

@sozap

As for more ambitious projects and therefore more ambitious blender, we need to ask how far blender is from AAA grade movies ?

For asset creation - games and movies, it’s already there: When you export, Blender’s work is done.

For animation, when it works, Blender gives fantastic results. Look at Blender Artist gallery (at the ones done wholly on Blender). If you can that over multiple animated frames, you have something approaching Pixar level, done with EEVEE, at a fraction of the price of professional software.

Problem is, for animation to work, Blender’s work isn’t “done” with an export. Everything else has to work right up to the very end. One bug can delay a shot, and Blender has bugs aplenty. IME it’s very rare to complete a shot without fighting bugs, wrestling with subpar components (whether it’s time spent fighting them directly, or exporting to/from other programs which can do it), and a suboptimal UX. That’s what really slows you down.

@kilian89

I am not an expert here and have not really used links, overrides and so on. But from reading this thread it sounds like a complicated topic.

Overrides are a simple concept. I’ve worked on 3D software with overrides. BUT it was designed to support them from the very beginning, so it was easy. The problem for Blender is overrides were added late in Blender’s lifecycle. It’s very hard to add overarching function to mature software.

I am looking at fixing the code myself, but there’s a lot to take in, and code reviews are backlogged it might never make it into the main version. Found GN can get you only a limited distance as an alternative: You really do need working overrides.

@Kologe

I do agree to some extent, the Open Movies produced since the Blender Studio and the Blender Cloud were first introduced seem to dwell a bit more in some sort of comfort zone, compared to those from the days of DVD-sales campaigns (e.g. Sintel, Tears of Steel, Big Buck Bunny).

@Ace_Dragon

Therein lies a potential issue with having too much focus on developing with the Open Movies in mind. The Open Movie Projects today are really not that ambitious in any definition of the word compared to a AAA blockbuster (with up to perhaps 5 characters that does not require a lot of additional feature and tool development). The Institute needs to go bigger, they need to propose movies that would make Blender 4.1 alpha cry for mercy and thereby have more difficult development goals (since for one thing they have a lot more resources now).

Yes… The recent Blender Studio movies haven’t tried to boldly go where no Blenderer has gone before. “Fright Night” was nothing extraordinary. Note the ponytail! And while “Charge” was visually impressive, seemed it was showcasing recent additions, instead of looking for areas in need of improvement.

It would be worthwhile for BF to engage 3D artists outside of BF’s circle (‘Blender Bubble’) who can tell them far more, straight away, without the delay or the expense of doing another open movie. Do note @sozaps’s comments on Boon and Pimento. That’s very useful information for all animation artists to share, and should be formalized, but BF needs to be listening.

@Kologe

… is something none of the Open Movies ever was. And it is an absurd proposition, if you look at the kind of budgets AAA blockbuster movies consume (especially in animation).

Split that into two categories: AAA Studios - who already have working pipelines thank you very much (very understandable), and startups studios who want to do AAA-quality animations with a Blender pipeline.

I think you can do AAA-quality animations with Blender, but the biggest problems are bugs and subpar features (e.g. overrides, cloth). Importantly, AAA-quality isn’t just about the tools. It’s about being able to spend the time improving a shot. Time spent dealing with bugs or suboptimal/subpar components/UX is time you’re not improving your shot.

I just had a shot I had to wrap despite not being really happy with it, but behind schedule because I’d spent so much time bug fighting to complete it, instead of actually improving the shot.

@thinsoldier

I think the best we can hope for is they add/fix things (like links and library overrides) that would make a show like Boon and Pimento take half as much technical/trouble-shooting time and effort to achieve with the same sized team.

For Blender to be an attractive solution, it has be higher-productivity than its rivals. Not less! Buggy software is not high-productivity software. If one guy with Maya can do more in a day than one guy with Blender, then Blender has a problem.

@Ace_Dragon

The Institute could find these sooner if they only looked at bringing in artists from outside the Blender bubble. The artists they have now already know all of the workarounds and know how to deal with Blender’s quirks so as to not be too hard on the developers.

Blender-bubble is a big problem. Recently when I raised overrides on another forum my head was bit off by a Blenderhead who told me there’s an animation studio in the same building as BF so they’d already be well aware of any problems with overrides. aka. “We’re not interested in your feedback because we already know everything”

@thinsoldier

I don’t think they have nearly enough resources to do anything too ambitious.

If you look at their annual report, they do have resources and they are doing ambitious, but IMHO they’re doing the sort of bleeding edge function coders like to do, rather than what their users need.

Just one example: Render (the camera) vs Viewport (the eye). A real time waster is that Blender is not WYSIWYG, so you can waste hours rendering only to find you have something in / not-in the scene. And have to do it again. There have been at least 2 requests on RightClickSelect to fix this with +60 upvotes posted 2-4 years ago. Nothing’s happened.

Plenty of other low-hanging fruit on RightClickSelect ignored for years.

Obviously volunteers are free to work on whatever they want to, and it’s hard to tell what they work on and what BF is funding. e.g. Previously I was told one module is run entirely by volunteers. But I found it’s part volunteer, part BF-funded.

I’d argue BF funding should go where it best benefits the users:

• Bug fixes
• Bug fixes
• Bug fixes
• Code Reviews (See the ‘Guano’ thread - Code Reviews are backlogged. Implication: Bug fixes aren’t getting through.)
• Library Overrides (Append and Make Local isn’t viable for large projects).
• Low-hanging fruit on RightClickSelect to boost user productivity.
• Sculpting (I think far more widely useful than Simulation Nodes).
• Cloth (because nothing says non-AAA like a lame cloth simulation).

I’d like to take the foot off the gas on Cycles, Grease Pencil, Simulation Nodes, Animation 20XX, and even Geometry Nodes. Yeah, I don’t dispute GN is a vast improvement over the old particle systems, and the knowledge gained on those subsystems has to be kept alive by continuing work, but there’s other stuff that needs doing as well. Same with Animation 20XX, which is a nice-to-have. I wouldn’t say no, but the animation tools at the moment are fine. Bugs and library overrides are far bigger problems, even for animations.

And large scene optimization: Blender still has no LOD, no texture compression, no dynamic scene memory optimization. At the moment you either have to do this manually (using Override’s suboptimal UX) or by writing your own Python code (I write a lot of Python code but it would be better if I didn’t need to). Optimization is a real time waster in Blender, and is stuff Unreal (supposedly) does for you.

I’m sure there are people who have different interests and would rather prioritize Grease Pencil, Simulation Nodes, etc. I don’t debate that’s cool stuff and I’d love to see it too, and I respect volunteers freedom to work on whatever interests them.

I’d love to see everything in, but that’s not realistic at this point of time: When it comes to funding, BF needs to make (better) choices. Take a break from cool new toys and the bleeding edge. Fix the fundamentals.

I think Blender will live on for asset creation, but if it keeps neglecting the other issues, then animators will ultimately migrate, probably to Unreal, maybe even to Maya. Unreal is trying very hard to attract animators at the moment. BF is neglecting them. Look at how BF has ignored the library override issue. I’d say to BF: “Stop telling me I’m Breathtaking, and start listening to me.”

Options:

1. Raising this on BA and RightClickSelect has achieved nothing. Perhaps raise in with Ton directly. Yes, I don’t like to disturb the man and I’m sure his inbox is already overflowing. But I recall with the VFX Reference Platform discussions he said the studios never reached out to him. So maybe he isn’t aware. Rule of thumb in any organization: The guy at the top is the last to know.

2. Do it ourselves. I’ve raised this in the past here and on devtalk and no one else is interested. I do confess its weird not more people are bothered by this.

3. Raise money to do it ourselves.

4. Form a professional user organization to gather requirements, raise funds, and liase with BF, and lobby for or otherwise do our own improvements. Because RightClickSelect just isn’t working, and it could ultimately lead to bigger studios being able to put their trust in Blender. Because at the moment, I think they’d be mad.

5. Or jump to another platform.

Bottom line: Blender can be an AAA platform, but they need to fix the fundamentals before grabbing the bleeding edge.