Collection Instances from other files'

[tommy1441]

Hi guys,

I’m quite confused and curious if there’s a way to instance/reference a collection from another file into the working project without adding polys to the statistics (meaning adding vertex counts, etc.).

I have tried all possible settings when appending the collection in to the working file, but it still ends up adding vertices/faces/polys to the scene. And the huge problem is that there’s around 70 collections in separate files that I wish to append to the project (F16’s cockpit) without breaking or setting fire to the PC most of the collections that I wish to append on average have +40k vertices, even optimized, so…make the math!

Please any help in pointing me in the right direction will be gladly accepted.

PS. I have tried in 2.93.9, 3.1.2 and 3.2.0

[sozap]

Hello !

Well, sadly there is no simple solution…
In order to be able to display a 3D object, blender needs it’s data ( polys) to be stored in the memory.
Even if you instance a collection, and then can’t access these polys anymore, they need to be stored in order to be displayed.

You can start to save some polycount if you duplicate these instances, then the data only needs to be stored once but can be displayed several times.

For that you can use instanced collection, or object instances , either by duplicate them with Alt-D, or select several objects that are the same , and do CTRL-L → Link object data.

This is a way to save some memory and get better performances.

Linking / referencing from an external file won’t bring a lot of benefits for performances here.
You may try to append everything in the same .blend if that’s easier to manage.
As said, even when linked from another file, they need to be in memory in order to be displayed.

If I do the math given your numbers 70 collection * 40 000 vertices gives ~ 3 Million vertices.
It’s big, but unless you have a low spec computer, blender should be able to handle that.
What may be a bottleneck in that situation is the number of objects.

You may want to balance objects numbers, say you have 1 000 000 objects with 10 triangles each,
Chances are that you’ll get poor viewport performances when it comes to selecting objects.
Because when you’ll ask blender to select something it will be needed to go over that 1 000 000 list of objects to find wich one is under the cursor (to sum up basically ) .
Even if you use instancing there, it may be better to reduce the objects number in order to have better performances.

If you make 100 objects with 100 000 tris each, you’ll get a much better balance and better performances. You should be able to edit those objects without issue , and managing the scene should be smooth.

Of course, you can probably go for 1000 objects without issues.
But at some point if things get too slow check the object number too.

Hopes that helps to get started, in short, try to use several instance of the same data, either by using collection instance or object instances in order to save some polycount.
And check the object numbers too which may become problematic if you have too many.

Good luck !

[tommy1441]

Hi,

it’s understandable the reasons behind it, thanks.

@sozap

Even if you use instancing there, it may be better to reduce the objects number in order to have better performances.

Now, as far as reducing the number of objects is gonna be difficult, because as I mentioned is the whole of the F16’s cockpit instrumentation in one go…it’s food for thoughts then on how to tackle it without compromising the quantity.

Thanks!

[sozap]

Hum, when every objects will be in the same blend, you can look at object counts.
To reduce objects number you can merge some parts (CTRL+J) .

Do that only if you feel that selecting objects, or interacting in the viewport is getting slow.

Do a few different tests, like, find if there is room for instancing some parts.
Try to merge some objects, and you should be able to reduce the polycount a bit and make the scene manageable.

[tommy1441]

yeah, that’s the idea, to have them all in the same .blend for some renders and probably some animation of some dials and/or switches

I would have to think thoroughly the scene to see which objects I can merge or join to reduce object count…

This is an old render of what I did before I became much better at modeling and texturing…comparing this with the new one, this has ~50k vertices

Here’s the same panel…it took me by surprise, as I hadn’t actually compared the old and new one and caught me off guard, the new one has ~40k vertices, ~10k less than the old one and it has much more details and realism…

[sozap]

Super cool ! Both are nice and of course the second is much more improved !
Well done !

For the instancing, it’s hard to tell, indeed some mesh are repeating, but it may not be worth it.

How many objects on the average do you get by separated parts :
you’ve got 70 collections , how many objects as a whole, or per collections ?

How it behaves when you put all these collections in the same scene ?

[tommy1441]

Thanks!

@"sozap

For the instancing, it’s hard to tell, indeed some mesh are repeating, but it may not be worth it.

IK, that’s what I’m thinking.

@"sozap

How many objects on the average do you get by separated parts :
you’ve got 70 collections , how many objects as a whole, or per collections ?

It varies from panel to panel…it’s hard to say as I haven’t actually thought about it, now if I was to guesstimate the average would be 10-12 objects per panel, so that would be ~700 objects without counting the extras (ejection seat, the actual pit structure or consoles, etc.) but could be well off.
All the panels are on their individual files and collections, making it ~70.

@"sozap

How it behaves when you put all these collections in the same scene ?

I haven’t even began adding them to a single scene yet, as I am worried about the vertex count going over the top and making it hard to deal with…that’s why I began researching if collection instancing or something of the likes could help at least bring the whole scene to a decent level before committing to putting it all together.

[sozap]

Ok ! it’s worth testing to put everything in the same scene.
Unless your computer is very old , the object and poly count should be workable !

Good luck !

[tommy1441]

Quick test!

I instanced the collections and it seems to have worked, the most it has done is increased the vertex count to ~196k when instancing the first panel, after that it has stayed there and hasn’t changed a single bit…weird!

Or it could be that Blender sets a cap on instanced collections and keeps it there…DK, but it’s all good so far!

Obviously, I cannot animate or move any of the buttons or switches, but I guess I can make that particular panel “collection real” if needed.

[SteffenD (Steffen Dünner)]

@tommy1441

Obviously, I cannot animate or move any of the buttons or switches, but I guess I can make that particular panel “collection real” if needed.

If you want to keep everything linked and still be able to animate the switches etc. your best option is to rig them with Armatures / bones inside their respective collections and use overrides after linking them to be able to move and keyframe the bones.

[tommy1441]

Quick comparison between first time doing the project, and now. Sooo pleased with the difference

and Now (3 years apart)

[tommy1441]

@SteffenD

If you want to keep everything linked and still be able to animate the switches etc. your best option is to rig them with Armatures / bones inside their respective collections and use overrides after linking them to be able to move and keyframe the bones.

Thanks for the suggestion, but this means I’d have to go through ALL 70 panels and redo all the rigs they already have, and honestly I’m not that good at armatures and such…probably in the future and if I have time and the know-how I will change them.

[SteffenD (Steffen Dünner)]

So they already are rigged? How? Parented to empties?

Thanks to overrides (compared to the previous proxy-system) you can now also override the transforms of any object, not only Armatures. I hope this helps

[sozap]

@tommy1441

I instanced the collections and it seems to have worked, the most it has done is increased the vertex count to ~196k when instancing the first panel, after that it has stayed there and hasn’t changed a single bit…weird!

Indeed that’s a bit weird , but as long as it works !

You have few options here. In all cases it’s more a matter of convenience than performances.
As said earlier , no matter what you do, chances are that the scene will always be as heavy being linked or local, collection instanced or not.

Having collection instanced can be useful , and may solve the issue of having too many objects.
Also it allow to manipulate a bunch of objects as if they were one. Which can be useful if for instance all the panels aren’t modeled at the right position ( you need to place them in the final scene) .

Having collection linked , I’m not sure it’s super useful. If you find convenience in that keep it that way, or if not , I suggest that you append all the elements in the same .blend and work from here, so you can then change something without going back and forth.
One thing where linking could be useful however is if you plan to do several animation ( several shots) of the same cockpit.
Then indeed , linking from external files is quite relevant. That way you avoid having duplicate data and you keep animation files light in file size. You also get the benefit that you can change on panel (material / texture) and that change will propagate to all animated shots.

But performance wise it doesn’t make a difference. It’s a bit tedious to go back and forth from anim shot, to library .blend ( individual pannels) to change stuff, it’s a good balance that you need to find.

About animating these panels, indeed there are a few way.
Once again, if you only have one .blend for animation, I’d put everything local and animate objects right away from here.

If you have several shots, then the “regular” way of doing is like @SteffenD said, you create a simple armature/rig. And use Overrides that will allow to keep things linked but still allow to animate objects.
I think , you may not even need to create an armature and you should be able to animate objects.

Making instance real is another solution. I’d say if you link from external files Overrides will makes more sense and you keep the benefits of linking.
These things you should test them quickly to make sure you have the good technique right from the start.

Another thing that came to my mind related to performance, is about textures.
Because you have a lot of objects/textures, you may have issues with all these textures filling the memory. That can be solved anyway, but it would be hard to tell unless you put everything together.

[tommy1441]

Ok, so far so good, but DK what’s going on…I’ve rendered scenes more complex and with more objects and vertex counts as the scene RN (actual scene ~815k Verts, ~796k Faces, 366 objects, VRAM 0.6/2.0, etc.),…whenever I send to render (F12) Blender crashes…tried in 2.93, 3.1 and 3.2…and I still haven’t placed all the panels on the cockpit

My PC is not to shabby (Ryzen 5 5600X, 32Gb Ram, SSDs HHD and an Nvidia GT1030)…I suspect that the GPU is the one having issues, but then again, I’m rendering using the CPU as IK that the next upgrade coming for my PC is a new GPU (saving for a better one)…running Linux Mint OS

[sozap]

I see two possible issues (could be others) :

• You have disabled modifier at render time.
• You have too many big textures (tex resolution like 4k textures) , eating all the memory.

Best is to try a viewport render, if it’s fine there, something happend at rendertime only.
If not , it’s probably related to the textures, you can try to reduce texture size in the simplify panel.

128 is way too low, but as a starting point it will help to see if that’s were the issue is coming.
Eventually you may find a settings that fit the memory without loosing visual quality. So you don’t need to edit textures manually.

Good luck !

[tommy1441]

@sozap

Best is to try a viewport render, if it’s fine there, something happend at rendertime only.
If not , it’s probably related to the textures, you can try to reduce texture size in the simplify panel.

That’s was my second guess, although as I knew that I’d be adding all those panels, I’m using the models with the 2k textures.
BTW, I didn’t know you could reduce the texture size in the simplify panel…thanks for the tip

[sozap]

Depending on how many 2k images you’re using that can add up quickly…
one 2k image = 16Mb in ram, you can do the math to see if that fits.
Sounds not a lot, but each material may have several textures, and one asset may have several images.

[tommy1441]

That did the trick, I reduced the textures in the simplify panel to 1k and all worked fine, now IK for when I end up adding the rest of the panels

This is the entire port/left side, just needs the throttle assembly and some minor details to have the port side ready

[sozap]

Super cool ! very nice detailing ! it should be quite fun to work on this, even if it’s a bit challenging to manage !