How to properly work with linked files?

Support Basics & Interface
1

Sorry if this is posted in the wrong forum - feel free to move it if that’s the case!

Hi,

I’m revisiting Blender after many years apart, done a few smaller projects to get to know the basics again etc. But now I want to tackle a larger project, and coming from the vfx industry (mainly using Maya) I know the importance of working in a structured manner, as non-destructively as possible, and, whereas possble, set it up so that actions ripple through different aspects.

I’ve found that one method is by using linked files (similar to Mayas reference system), i.e modify an assets source file and all other files that have that asset in them gets the change. But I can’t really find any information about how to properly work with this system besides the basic workflow, meaning create an asset -> link it (or many of it) into an environment. Seems reasonable, but what if you wan’t to split it up further?

Let me give you an example of my ideal workflow:

1 - Model the asset in the modelling.blend file
2 - Link the model into the lookdev/material.blend file and add materials/textures etc.
3 - Link the model into the rig.blend file for rigging.
4 - Link in the rig file into the environment scene for animation.
5 - Cache/bake out out the animation.
6 - Link in the cached animation into the lighting scene, and then link in lookdev/material.blend (in this case only the materials/textures) which would automatically hook up to the correct alembics (based on naming I guess)

This way each step would not get gradually heavier as the scene accumulates more and more objects. It would also allow me to have other objects in the, lets say modelling scene (like reference images or temp objects) that is useful to have, but only there, if the model would have to be changed later on.
So would this be possible? In that case how? Meaning what would one link per step (all the individual objects or is a collection enough?) When to use “make proxy”? Because the only information I’ve found so far are examples of an entire asset that is all done is being linked in, and I can’t imagine that an asset has to stay in that one scene all the way through the project, right? Even though it’s just me and my personal project, I just assume that bigger productions would have to split up the workload so that the rigger and lookdev artist doesn’t just have to wait around for the modeller to finish his/hers modelling notes :slight_smile:

I’m also aware that “my ideal” workflow might not be possible, but any tips on getting closer towards something more split up, and away from working destructively/with unnecessarily heavy scenes are appreciated!

Thanks in advance!

2

Sorry this is an incomplete answer but have a look at andys talk at the blenderconference about the workflow on the production pipeline in spring.
I think he explains nicely how they linked files around

3

Blender have big problem. This problem are that people are not aware of this problems, they even do not want to hear about problem.
Imagine that you when your scene become too cluttered simply open new Blender and there model some part. Optionally you can make very rough blockout, just cube with proper dimensions and import in new- empty scene. When you finish modeling of part you simply import this in your project ( which are all time open ). Even if you model in center of world, with just one click you can position your new model in appropriate place. Not to mention that you can jump between projects in one click. Advanced users can make Xref so any change will be updated in project.
This and many other “little” stuff are real hero of the day. But why to do stuff in simple, easy, logic, intuitive way, when there is Blender way :nerd_face:
Ah and there is also layer system… So your scene can have visible in outliner only collection in which you work. In layer tab you can very easy set up visibility in outliner/ viewport, are this stuff locked, will this be rendered, disable modifiers, etc. Blender can do something like that… almost… but in very messy, complicated, x#wewrtt Blender way.

4

Yeah I watched that presentation before, it’s a really good one that provides lots of information regarding pipeline work and an overall look behind the scenes. But unfortunately it, like many others, stay at the “link the asset into the environment” level, which is reasonable consider the amount of information he had to share in just 40 minutes.
But who knows, maybe they did an entire character from start to finish in a single .blend file, and for someone like me, that might just be the way to go too. But if blender is designed to work that way it feels like it would be a major bottle neck if studios would want to scale up their blender usage as it prevents artists from working simultaneously.

I dunno I just want to find out the proper way of working (yes I know there are almost always multiple ways of doing the same thing) but I’m just overall curious about how blender is intended to work at a larger scale and not just for single image renders or one shot animations, and there isn’t much information about it (atleast that I can find but maybe I’m using the wrong term when searching) out there.

5

Not sure what you’re complaining about. Doesn’t sound like your’e describing anything Blender can’t do. And a few of those things I only know the Blender way of doing them because I couldn’t find or figure out the Maya or Modo way of doing them.

6

I’m not complaining about anything that Blender can or can’t do, simply because I don’t know what Blender can or can’t do in regards to this subject, That’s what I’m trying to learn. I’m sure there are ways for people working simultaneously/splitting up work in one way or another. But I don’t know if it requires custom scripted tools or if there are ways straight out of the box, because I haven’t found much information about this topic. And if it is as you say, that it can do what I’ve described then that’s fantastic.

But how?

Is it possible for example to link in the modelling into the lookdev scene but still have control over the UV’s in the lookdev scene? And in that case is there some setting that needs to be changed when linking/making a proxy or something? Is it possible to link in an animated cache from one scene/cache file and materials from another .blend and have them automatically hook up or do you still need to apply them one by one again? If not then it might be better to approach it in a different way.

It is information like this that I haven’t been able to find, which is why I’m asking about it here, so if someone knows something about this then I would be very thankful if you would share, and I’d be more than happy to listen to what you have to say. Even a link to a youtube video or a blogg post somewhere would be very appreciated.

If you have experience in working with linked file how do you do it? Do you split up an asset or create it (model/materials/rig etc.) in the same file and then link that into the environment/animation scene? If yes do you notice any performance hit for having all the components in the scene? And do you perhaps have any tips on how to deal with that? In Maya for example there is the option to turn off references (or links) that you currently don’t need in order to speed up the scene, maybe there is something similar in Blender?

Lots of questions and I don’t expect anyone to answer to all of them, but I’d love to hear some tips or how other people are working with linked files :slight_smile:

7

Like I say, people are nor aware of issues :grin:
And to cut this now. I don’t “complain”… sigh.
Blender can’t go further if users react like this. I want to Blender be better. I’m sick of this “Blender can do this or this, Blender can do everything, blah, blah”. First Blender ( like every other software ), can’t do everything. Second, often workflow is slow, complicated, illogical. Fresh example, today I wanted to array some bolts along spline… after searching YT I find way to avoid deformation. Overcomplicated mess, and after all I don’t get desired result. Maybe because tutorial is for older Blender ( I guess 2.8 ).
In C4D this is literally 2 clicks… but I don’t want to waste time with export - import for every Blender mess, so I duplicate objects manually.
PS. for C4D you don’t need YT, all is logic, intuitive, you try first time, struggle one min. and start using tool.

8

This is all personal bias. There are many things in Blender I find illogical and inconsistent. There are several million things you can do in C4D in 4 clicks that are IMPOSSIBLE to do in any other program without hiring a professional programmer to make a special case plugin for you. There are a few things in Blender that I feel are perfectly logical within the mind of someone who has been using blender for a while and understand how that way of doing it has a relationship to a way of doing something else in another area of Blender. Blender cares more about consistency between its many different feature areas and less about consistency with other programs. Most other programs also are like this. Maya usually does not care how things are done in C4D. Max usually doesn’t care how things are done in Maya. Modo usually doesn’t care how things are done in Max.

Nobody can look at maya or max or modo or c4d or blender for the first time and find it intuitive. Anyone with enough familiarity with C4D can ask a question about C4D and get a 5 word answer and immediately visualize what they need to do based on their prior understanding of C4D. It is the same with Blender.

If you are not familar, it is good to spend time with training resources, it does not matter which program we are talking about.

No amount of Blender sculpting and Mudbox sculpting and SculptGL sculpting will prepare you for the bad parts of Zbrush’s user interface.

No amount of Zbrush experience will prepare you for the bad parts of Blender’s sculpting user interface.

9

@thinsoldier @alekba Guys, please keep a support topic about linked files… about linked files. Thanks!

11

I am also very interested in this topic and haven’t found any current resources on at least One intended workflow.
I came from Max. There, I used Xref’s. It had checkmarks for overrides so it was pretty easy. I could look at what was referenced, and the details of it pretty easily. I work with CAD data, so my files are pretty big. I like to keep the data in one file, and do the rest of the work in another working file where I save versions.

One thing I can’t seem to find, is How the overrides work. There was a little bit of a workflow I remember seeing for 2.79 with I think an apache helicopter. I can’t seem to find it again, but it was using the proxy workflow. And he showed some of the things that come through. Since I work with CAD mostly, I rarely use bones. They seem like just a lot of extra work for precise work, when parenting does the job.

I think I’ve figured out a decent workflow, but I’d really like to see what blender users/developers actually intended, and why. I’ve seen a few examples, but nothing is really start to finish. Even the helicopter one didn’t go into details on how they made choices on the helicopter rig. And he still had to change some of them in order to fit the workflow.

There are quite a few things that I can’t quite figure out how they work, and that’s probably because I’m just assuming they work different than they were meant to. A good example is right click on an object in the outliner. Under ID Data you’ve got some options. As a new blender user, I don’t know what that means. So I go to the manual. Ha. ID Data > Unlink = unlink a data block from it’s owner.
Oh thanks. That’s really helpful. sarcasm

You can unlink, make local, Make single user (I don’t think that does anything at All at the moment so not sure why it’s even in the menu), etc. But it’s really difficult to find out what those things actually do/are for new users and at least one way they are intended to be used in a workflow.

So I’d love to see someone talking about how they are doing things, technically, rather than an overview of a large production. In my case, I almost Never have characters. And the animations have to be completed, start to finish, in 2-4 weeks. So I have to know how to do things “the right way” and then choose which sacrifices to use for each particular project.
In most cases for me, if I have a file that is 500MB of data, I’m going to need to put that data in a file and link it. But I haven’t seen any examples of doing that with blender Anywhere. But also most every product animation I’ve seen done by amateurs or even small studios, they modelled the data themselves and it’s fairly low poly. So close up detail shots look more like a game, which is disappointing.

If there are any good resources for workflow using linked files I would love to read/watch them.

12

In my limited experience “make local” just exposes a new heirarchy of things underneath that object that each need to be “made local” one by one, then their materials after that… basically the shit never worked for me.

The only thing I’ve ever tried to override is materials and while it works initially, when I save and re-open the file, the overrides are gone and have to be made all over again… basically the shit never worked for me. I assume the only thing that actually works is linking and animating character rigs.

13

Well I suppose that’s good to know. I’m linking all of my materials to an external file though, and editing them there. Then those materials will be linked and applied to my main imported scenes.

If I have to go back and edit the materials, I might try linking my working animation/lighting file all the way back to the materials scene. No Objects would have circular dependency, but tthat would definitely be setting myself up for some potential issues, so I’ll save some copies first. I haven’t gotten that far yet.
If that causes problems I could always just append a scene from the animation/lighting file to my materials file for material tweaking, then just delete it when I’m complete.

I haven’t run into any issue Yet, but I’m not all that far along on this project. Hopefully after I deliver it, it will go live online, then I can go back and make a video on how it all went that I share. Use that internet law to my advantage: The quickest way to get an answer online is not to ask a question, but rather to tell others what is correct. People are much more apt to correct you than to help you (unfortunately).

And if you hadn’t heard that before, this post is where I saw it, and I Love how they did it.

14

It crossed my mind the other day to test circular dependency of linked files but haven’t gotten around to it. I have tested linking entire scenes to another file and using the scene as the background scene of the main scene in the other file. It worked pretty well. Everything from the background scene is shown (except the world/environment lighting) but you can’t interact with it.

15

I Have already had it break a little. Some changes updated, some didn’t. Manually update linked file? No change. Close all copies of blender. Open file again, and it’s updated.
So not really a big deal since it didn’t really break I guess, and rather just didn’t properly update, but could definitely be an issue depending on frequency.

16

select linked collection instance,
ctrl+a, make instances real.
I can move the objects individually now but cannot enter edit mode.
Object Data Properties tab, make local the mesh data, now I can enter edit mode and adjust the topology.
I can select a few faces and assign a new material to them but I cannot edit the original material.
Select the original material and either make local or duplicate it.
I can now adjust colors and other values but cannot make changes to node groups because they are still linked from the other file and must be made local.
I cannot change a normal map image from srgb to non-color data because first it must be made local.
Made it local, still cannot change it. Undo. Made it a library override, still cannot change it. Undo. Crash.