Cycles render farm brainstorming

Support Lighting and Rendering
1

I’m looking into building my own custom Windows render farm for Cycles rendering in Blender. While I have a pseudo-render farm that I built/coded for Anime Studio, most of the computers I use for rendering with that program don’t have a CUDA-compatible graphics card (and cannot have one for various reasons, like, because it’s a laptop). So I’m looking at acquiring several computers that I can outfit with a Cycles-friendly GPU.

I’d like to keep things as cost-effective as possible. My first thought was to pick up several computers that are last year’s model, especially if I can find a computer like that has a decent CUDA-compatible graphics card. I have no problem assembling computers from scratch, but it doesn’t seem like a cost-effective move compared to getting an already-assembled desktop with acceptable specs.

From what I can tell with Cycles rendering, what matters is not the computer’s CPU speed, the onboard number of CPUs, or how much motherboard RAM the computer has; what matters the most is the video card’s speed and how much RAM is on the video card. Is this a correct assumption to make?

As an aside, I’m not interested in using an online render farm service. Each shot in the project I’m working on is going to be gigantic. A lot of shots will end up being over 1 GB, with all of the image sequences (and accompanying alpha transparency image sequences) that I’ll be incorporating. I’m going to be rendering 1920x1080 png files stereoscopically at 30 FPS. A rendered 18 second clip creates about 3.35 GB of PNG files. That’s just too much data to be shuffling around on my internet connection. So I want it all in-house.

Anyway, I’m looking for opinions/advice – specifically, on approaches I might take, hardware that I should consider, and bargains that I should look into. I haven’t decided on a budget or on how many computers I want; I’m still in the very early stages of figuring out the best path to take.

Thanks!

2

Hi, the problem with GPU rendering is the limited video RAM.
You have at least go for 3,4 or 6 GB GPU and the most cost effective way to this is one system with several GPU.
One case, one power supply and a board with 3 or 4 (very expensive) PCIe slots.
Than you can, depends of your budget, put 1-4 GTX 780 6GB or GTX 760 4GB cards in this system.
You can also start with one card and extend the system later very easy.
Here is a spreadsheet with all cards:

/uploads/default/original/4X/1/3/8/1388c64bc2707267be2c0434c1a56037057fc4ab.pngd=1400534233

Here is the benchmark for 3 Titan and changed tile setting (3 Tiles).

http://www.blenderartists.org/forum/showthread.php?331149-The-new-Cyles-GPU-2-70-Benchmark&p=2656914&viewfull=1#post2656914

The benchmark file is on the first page, try it on your system.
One important advantage of one system is your electric bill does not explode as with 4 or more single systems.
For example, one big power supply is much more effective than 4 small cheaper power supplies.

Cheers, mib.

3

If you want a GPU based rendering farm, than you’ll want something like this perhaps:

http://www.pccasegear.com/sc/gyT - Prices are in $AUD, which will be more than most other places :stuck_out_tongue:

The important part to note is, you really do want to go with 6GB GPU’s, and multiple GPU’s per PC. In this case $1400 is spend on 2 6GB GTX 780’s, and the rest is on an 8GB RAM, Dual Module AMD CPU and a good PSU. You don’t need a powerful CPU for GPU rendering.

4

Thanks, mib2berlin. Currently, I have a EVGA GTX 580 (3 GB) on Windows 7 Home. I ran the test and got 1:56 using Blender 2.69. The Blender 2.70 alpha as well as a Blender 2.70 “Fastest Build” versions both took a lot longer – it was around 2:10 IIRC.

I have one other video card slot free on my motherboard. I’ve got a beefy power supply and could handle another big video card. The Titan seems to be overpriced for what it is. (Two 3GB GTX 580s render faster – and are cheaper – than one Titan.) Will this particular ASUS 6GB GTX 780 play nice with my EVGA GTX 580? http://www.amazon.com/Thermal-Direct-GPU-Contact-Graphics-GTX780-DC2OC-3GD5/dp/B00DWV3NM6

5

Hi zeealpal – the link you posted doesn’t really go anywhere on that site. Could you describe what hardware that link was supposed to show?

6

Hmm, the link was meant to show an example build for a GPU render farm, screenshot below

System.JPG1231×903 138 KB

7

Ah, OK – it does indeed take me to that screen. I wasn’t sure exactly what I was looking at. Thanks!

8

Hi, the GTX 780 6 GB should render the scene in 1.20 ~ in 2.70a but iirc newer cards are better on heavy scenes.
VRAM is not added with more GPU, on very heavy scenes you can use only the 6 GB card but if the scene fits in to 3 GB you can render with both cards.
To get the most out of you 3 GB card, drive the display with the 6 GB card (System use 3-400 MB).
I see no reason why the two EVGA cards should not work very well together, and fast.

Next step is, change your GTX 580 with upcoming GTX 880 8 GB. :yes:

Cheers, mib

9

Thanks for the info. I’m actually having trouble finding a place that sells that GTX 780 6 GB model for a decent price (in US dollars). Amazon only has it for $849, there’s none for sale on Newegg or on eBay…even EVGA’s web site isn’t selling it (there’s a button to be notified when it’s on sale): http://www.evga.com/Products/Product.aspx?pn=06G-P4-3785-KR There’s no problem finding the 3 GB model, though.

I see a Zotac 6 GB GTX 780: http://www.amazon.com/zt-70210-10p-gtx780-384bit-pci-express-displayport/dp/B00K9Q6C48 But that’s not EVGA – I’m not sure if there would be problems if there were two video cards from different manufacturers in the same system.

Actually, I may pick up another GTX 580 with 3 GB. It seems to be a much more cost-effective option than the GTX 780 (3 GB or 6 GB model), and my rendering speed will double. I can always pick up a 6 GB card for another computer later on, once everyone’s using the 8 GB model…