Will four graphics cards (RTX 2080Ti) connected by NVLINK bridges (900-14932-2500-000) work properly in Blender and show as two graphic cards with 22GB of memory?
Is there any difference in speed between NVLINKX16-3SLOT-PB (RTX6000/8000 bridge) and 900-14932-2500-000 (GeForce RTX bridge) when connected to RTX 2080Ti?
Is it possible to connect 6x RTX 2080Ti with NVLINK bridges - will it work properly?
I don’t think you’ll find too many people among Blender users that have actually tried these configurations and know the answers. Most of us are struggling to get our hands on any single gpu right now, I leave aside two or four of them…
I think nvlink is supperted in latest versions of Blender and you should see 22gb as the total amount of Vram. But, in order to be sure about this, you should ask about this in the Nvidia forums. This question is somewhat too technical for most Blender users.
I know, Blender is for small studios and poor freelancers. But… I am one of these poor guys and I invested in a couple RTXs because these are my working tools. I believe there are more such freaks here…
I didn’t say you’re a freak, don’t get me wrong. If you decided to invest in multiple GPUs in order to boost your workflow that’s a great potential. Especially if this is your job and you earn a living doing this. But even blender users that have multiple GPUs for rendering don’t usually connect them via nvlink. So it’s a rather rare situation, and I think you have more chances to find answers inside the Nvidia forums.
Hi yfile,
You might be forging new ground with 4 cards in dual NVLink setup for Blender. Nice! Which OS are you using as this might matter? Cycles as of 2.90 does support 2 NVLinked cards. Not sure about 4. You might want to check with the E-Cycles dev (Mathieu) (https://blendermarket.com/products/e-cycles) as he does support NVLink so maybe he might have an answer for you. Checking in NVidia forums as birdnamnam suggests is a good idea also.
You also may want to check this article as it talks about issues with dual NVLink setups for Windows 10. Although this is a year old so hopefully things have gotten better with the drivers. https://www.pugetsystems.com/labs/articles/Unsupported-How-to-Make-Dual-NVLink-Work-on-Windows-10-1688/
Cheers and stay safe!
I couldn’t find much information on the Nvidia forum:
So it theory it should work.
Thanks for the pugetsystems link.
At the moment I have Windoze installed, however my main ‘workstation’ is MacBook Air M1 so I can install Linux on a PC as well – this is Cycles-only machine.
Other links I found:
I ordered two NVLink bridges so I’ll know in a few days whether it works.
You may want to post your question in the Blender developers’ forum https://devtalk.blender.org/ , the people who actually develop the code should be able to answer you from the software support point of view. Second on Matthieu! for sure.
There is also the NVidia forum where you may want to look whether the hardware works this way (I think it does, but don’t take my word for it) - oops you’ve done that… And finally the technical challenge of setting 4 up on a motherboard with proper cooling etc etc, that is one more thing to look at. … Post a pic when you are done!
I can only imagine you would be working animation professionally? I cannot imagine still image rendering of any one artist’s work would ask for more than one of those cards!
Things are changing Blender is moving center stage, more and more.
Hi yfile,
Good luck to you. We’re all very interested in how this will turn out. Please keep us posted.
Cheers and stay safe!
I’d like to know the rest of the rig. Will these GPUs be installed on a classic motherboard or in a GPU rack as an extension? There aren’t too many 2080ti models appropriate for this use. A blower style cooler is needed or else, the only viable solution is a closed loop cooling system. And of course, another factor is the PSU. A 1600w unit might be enough for up to 4 GPUs.
4x 2080Ti - mixed brands and cooling
ASUS WS X299 Sage/10G
i9-7900X
64 GB RAM
M.2 SSD
2x 1250W Chieftec
Everything mounted in the open rack at the moment. Graphic cards are connected with risers. If all goes well, I will design and order a custom case to be able to mount up to 7 graphics cards in the future.
Hats off… The motherboard is perfect for the job.
Just a note here. If the case is customizable, leave plenty of space between the GPUs.
Yep, probably 3-slot space. 4-slot would make the case monstrous.
I can confirm that 2x NVLink / 4x RTX 2080 Ti works great in linux (Ubuntu 20.04). Easy and straightforward:
I’ve not been able to run such a setup using Windoze – lots of graphics drivers crashes, weird SLI behaviours, e.g. triple (!) graphic cards SLI link reported in control panel. Sometimes NV panel reported connection that did not physically exist. Absolutely mess.
Some results for Linux (4x RTX 2080Ti NVLinked) and Windoze (4x RTX 2080Ti, SLI disabled):
Victor (switched to GPU + Auto Tiles)
Linux: 1:12,55
Win: 1:23,07
Victor animation (10 frames fragment – 2467-2476, 200 samples, Optix rendering + Optix denoiser)
Linux (1 Blender instance): 8:40
Linux (2 Blender instances): 5:55
Win (1 Blender instance): 10:03
My interior animation (30 frames fragment , 200 samples, Optix, about 4,5GB, 6,2 GB peak):
Linux (1 Blender instance): 6:44
Linux (3 Blender instances): 3:20
Win (1 Blender instance): 7:46
Win (2 Blender instances): 4:15
BMW
Linux: 0:07,54
Win: 0:08,20
Fishy Cat
Linux: 0:17,26
Win: 0:22,47
Barcelona
Linux: 0:31,21
Win: 0:31,59
Overall I’m very impressed with Linux/ Ubuntu results, especially concerning the animation rendering. Theoretically an NVLink configuration should be a bit slower than a non-SLI configuration, but as you can see Linux destroys Windoze in terms of performance.
What’s more, in most cases you can significantly speed up rendering by running several instances of Blender simultaneously using larger memory. BTW, I have no idea why such optimisation is not part of Blender (CPU calculation/preperation of the next animation frame during GPU rendering, when the CPU is idle).
So I’m now at the stage of adapting the Ubuntu interface and network remote connection for the Mac user.
Hi yfile,
Nice. Very impressive! Also impressed with Linux not so with Windows.
Now I can confirm that 6x RTX 2080Ti with 3x NVLink bridges works great in Blender:
I just had to move one graphic card from PCI-E Slot 6 to Slot 7 to make memory pooling work.
–––––––
Rendering results (Ubuntu 20.04, 6x RTX 2080Ti NVLinked with 3x bridges):
Victor (switched to GPU + Auto Tiles) – 1:04,78
Victor animation (10 frames – 2467-2476, 200 samples, Optix rendering + Optix denoiser)
1 Blender instance – 7:46
2 Blender instances – 4:51
My interior animation (30 frames , 200 samples, 1920x1080, Optix, about 4,5GB, 6,2 GB peak):
1 Blender instance – 5:28
3 Blender instances – 2:47
BMW – 0:06,70
Fishy Cat – 0:13,87 (Tiles: 128x128)
Barcelona – 0:21,15
Koro – 0:31,62
Hi Yfile,
just seeking some clarification, is this pool 3 x 22gb or 1 x 66gb when 6 2080Tis are there?
Does it render as a single tile or mutliple tiles? (as they use the distributed memory)
just curious if its become one logical “numa node”
thanks in advance and for the info supplied so far.
kind regards
Sean
This works out as three pairs of graphics cards with 2x 11GB each. This gives about 22GB memory (I read somewhere that this is not the full 22GB) for the application optimised for NVLinked setup (e.g. Blender) – equivalent to three graphics cards with 22GB each.
As you can see on the screenshot in my previous message – for Blender this is setup of 6 graphic cards with memory distributed between each of 3 pairs connected with NVLink bridge. The “Distribute memory across devices” is available when graphic cards are connected via NVLink bridges.
Randering is similar to a single graphics card - multiple tiles. However Auto Tiles add-on does not work well in some cases. Sometimes it needs to set smaller/more tiles to run effectively.
Is there any progress here? I have set up a Windows system with 2x RTX 3090Ti and cannot find a way to explicitly activate the NVLink in Blender. The NVLink is proven to be active, but the VRAM doesn’t show up twice as large in Blender, so I assume I still need to enable that. Do you have information about this? I can’t find an option to enable NVLink in Blender. Does anyone know where to put a tick?
Thnx!
