Render Farm Project

Hello dear Blender users! I am about to build my first (small) render farm. I have a lot of questions so it would be nice to get some help from someone who has experience with these sort of things. Here is my initial plan, please tell me if you think I should change anything.

1. Components

GPUs
It will be built primarly for Cycles, so I have decided to go for Nvidia GPUs, more specifically from the 5XX series since they seem to have the best computational performance.

On my main computer today, I have 2 GTX 580 3GB and they work great. Unfortunately there are very few 500-cards left. I can’t find any 3GB cards. Actually, there is only one store in Sweden who has many 580s left, and that is of the Zotax GTX 580 1,5GB AMP! Edition. They are priced at 270 euros which I think is a decent price. The only thing that is scary is the 1,5GB memory. It sure doesn’t feel good for the future. I will have to do further testing on my own, monitoring my 3GB versions and see how much my projects avtually use. But I think I could squeeze it in on 1,5GB. At least I could put the lighter scenes on my 1,5GB cards and put the heavier ones on my 3GB.
The Zotacs are stock overclocked but I plan on under-clock a little bit under stock clock, just so I can ensure 24/7 operation without straining them too hard.

Motherboard

I will go for a cheap 75 euro motherboard, the cheapest one that supports SLI (1 x16 and 1 x8),

CPU

Since I focus on the GPU I am going budget on the rest. I was thinking a cheap AMD one, perhaps the (pretty old) Phenom X4 which I can get for 75 euros. And other suggestions?

PSU

This is a tough one. I want it to be able to run 24/7 without a hiccup so I don’t want to cut back too much here. I will go with Corsair for sure since they have worked flawlesly for me and have a rumour of having great quality and durability. At first I was thinking about 750W per node, with 2 GTX 580 in each. I though it would be enough, with room to spare since the CPU is basically in idle while the GPUs are working, and vice versa. Nothing in the usb-ports, no drives, etc (more on that later). But now I start to doubt. Should I go for something stronger than 750 or would it be plenty?

CONNECTION

I will get a 8-port 1Gb/s switch to connect my nodes. One question, can a computer be connected to two networks? Because I want my main computer to be connected wirelessly to my regular router for actual web browsing, and a cable to my switch to talk to my local computers.

2. Software

Okay, so the obvious choice for this is to run some form of Linux. It is free, lightweight and much faster for Blender. Which distrubution would you recommend? Ubuntu? Debian?

I mentioned that I was planning to go drive-less in my nodes. I am not the most computer savy person so I might have to scrap this idea, but the ideal solution would be to load the OS over LAN and keep it all in RAM. That way, I can keep all my nodes identical (and save quiet some money from harddrives). Is this really doable? Except for loading Linux, it would be great if the Nvidia drivers and Blender was installed and ready to render on start-up, connecting to the network as a slave.
One problem though, I will have to have slightly different configurations for my different machines. One Slave will also act as a master, so on start-up it would have to load two instances on Blender, and connect respectively.

Plan B is to use a cheap 4GB usb-memory to boot the computers. If I load the entire thing into RAM, it would matter if the USB-stick is slow? They will be rebooted very seldomly. Is this possible and what kind of solution would you use?

To distribute the rendering I would use the built in network render in Blender.

One questions though, what is the beast way to actually control your computers remotely? I don’t want to connect a screen to them whenever I need to do something. Is a remote desktop the best solution when it comes to modifing my nodes? Or are there better ways of doing it?

3. Conclusion

I am planning on buying 5 nodes, with a total cost of 4500 euro (which is the maximum I can afford). That gives me 10 GTX 580s (12 in total with the ones I already have) which will hopefullt give me very good render times. It will be more focused on doing large volumes of “simple work” (simple scenes, long runtimes) than the other way around, so perhaps I can get away with the small memory of the GTX 580s.

Sorry for the long text, hope you could read it without getting too bored :smiley: If you have any thoughts or ideas, please write them!

(I have to decide rather quickly becuase the GTX 580 are very rare nowadays, only one model at one retailer left here in Sweden as I said earlier. So I have to move quickly while there is anything left at all to buy :-P)

Thanks in advance!

If there is no 580 go for 590 ^^. Love project. acctualy!, Will be great to see the rendering times.

Moved from “General Forums > Blender and CG Discussions” to “Support > Technical Support”

Thanks! :slight_smile: There are almost no 590s left either, but I will certainly try to grab the last 580s :smiley:

Do people have any software tips when it comes to the actual render distribution? Because I tride FarmerJoe, Brender and the built in, and unless I do something wrong, it doesn’t support file that use assets. When I do projects, I have characters and props in separate files and then link them into the current scene. When using network render it only sends the actual blendfile so nothing of what is linked shows up.

The only solution I can come up with is to render from a public folder on my work computer, so the node has complete access to the whole file structure. FarmerJoe did this but unfortunately it doesn’t seem to work on 2.6x :confused: Any other ideas. I don’t need anything fance, just a way for my nodes to automatically start rendering project in my public folder, and move to the next one when it is comlete.

Small update on the project! I ordered my GPUs today. I have been waiting and waiting to get them at the best price, and when they lowered the price with 15% on basically the last good model available in Sweden, I didn’t dare to wait anymore.

So I got 8x MSI GTX 580 Twin Frozr III Power Edition. Hopefully these cards will be very cool (20 degrees lower than the reference cards) and built for heavier load, enabling me running them 24/7 at 100% load. At least that’s what I hope :stuck_out_tongue: We will see if I end up underclocking/undervolting them just to be sure.

I would have loved to buy more, of course (almost took 10) but… well, I am not made of money :slight_smile: unfortunately! I do however already have 2x 580 Classified in my work computer which I guess will be able to pull their weight at least during night when I am sleeping. So I guess I have 10 cards at my disposal after all.