Experience with GPU expansion systems?

Support Lighting and Rendering
1

Hello,

I am looking at options for multiple GPU installations, either with or without SLI. The goal is to have CUDA cores I can allocate as flexibly as possible.

I am familiar with network rendering on multiple PCs, and with using one or two GPU cards per PC… I know some motherboards can handle more than 2 GPUs, but none that I currently own will. So I need to buy something: more PCs, or a way to leverage more GPUs into use.

I would welcome input from anyone who has worked with a GPU expansion cabinet… Can I load one up with 4 GPUs and have Blender recognize them? Can I access them individually thru Blender?

I am looking for a way to have:

  1. max GPU/cores available per $ spent;
  2. the high-dollar GPUs in a clean-room like enclosure with A/C and filtered air control I can’t afford for the office in general… essentially a controlled cabinet to keep things cool and clean.
  3. flexibility for rendering both animation frame sequences and still image tiles
  4. as little additional software licenses and upkeep

I would welcome comments from folks who have run GPU expansion systems… Satisfied?

All input appreciated.

2

so we run a render grid… currently at about 19GPUs (mixture of gtx580s and 590s) but have a few more which need to be looked at (pretty sure they have too much dust in them). We had a 64 computer cluster which we have slowly been turning into a GPU cluster.

We only load one per computer, because of a few reasons…(we mainly deal with animations so these points may not apply to you)

  1. with GPUs they only accelerate the main sampling… where for us we have a bunch of preprocessing time and post processing time, which is non gpu related. As a result, if we had 2 gpus in the same computer, the main render would be twice as fast but the pre and post processing time would be the same… if we had them in separate computers, these pre&post processing times would also be twice as fast.

  2. If a computer crashes out on a render… only one GPU is affected, not two… Also simplier to diagnose find out whats wrong instead of having to identify which card is the problematic card or cpu.

3

Excellent information! Can I ask: Windows or Linux? I had not thought about a crash taking out multiple GPUs… I just worry about how much space I would need to handle more PCs… I am hoping to find a solution that might not exist.

thanks!
rg