Hello dear Blender users! I am about to build my first (small) render farm. I have a lot of questions so it would be nice to get some help from someone who has experience with these sort of things. Here is my initial plan, please tell me if you think I should change anything.
1. Components
GPUs
It will be built primarly for Cycles, so I have decided to go for Nvidia GPUs, more specifically from the 5XX series since they seem to have the best computational performance.
On my main computer today, I have 2 GTX 580 3GB and they work great. Unfortunately there are very few 500-cards left. I can’t find any 3GB cards. Actually, there is only one store in Sweden who has many 580s left, and that is of the Zotax GTX 580 1,5GB AMP! Edition. They are priced at 270 euros which I think is a decent price. The only thing that is scary is the 1,5GB memory. It sure doesn’t feel good for the future. I will have to do further testing on my own, monitoring my 3GB versions and see how much my projects avtually use. But I think I could squeeze it in on 1,5GB. At least I could put the lighter scenes on my 1,5GB cards and put the heavier ones on my 3GB.
The Zotacs are stock overclocked but I plan on under-clock a little bit under stock clock, just so I can ensure 24/7 operation without straining them too hard.
Motherboard
I will go for a cheap 75 euro motherboard, the cheapest one that supports SLI (1 x16 and 1 x8),
CPU
Since I focus on the GPU I am going budget on the rest. I was thinking a cheap AMD one, perhaps the (pretty old) Phenom X4 which I can get for 75 euros. And other suggestions?
PSU
This is a tough one. I want it to be able to run 24/7 without a hiccup so I don’t want to cut back too much here. I will go with Corsair for sure since they have worked flawlesly for me and have a rumour of having great quality and durability. At first I was thinking about 750W per node, with 2 GTX 580 in each. I though it would be enough, with room to spare since the CPU is basically in idle while the GPUs are working, and vice versa. Nothing in the usb-ports, no drives, etc (more on that later). But now I start to doubt. Should I go for something stronger than 750 or would it be plenty?
CONNECTION
I will get a 8-port 1Gb/s switch to connect my nodes. One question, can a computer be connected to two networks? Because I want my main computer to be connected wirelessly to my regular router for actual web browsing, and a cable to my switch to talk to my local computers.
2. Software
Okay, so the obvious choice for this is to run some form of Linux. It is free, lightweight and much faster for Blender. Which distrubution would you recommend? Ubuntu? Debian?
I mentioned that I was planning to go drive-less in my nodes. I am not the most computer savy person so I might have to scrap this idea, but the ideal solution would be to load the OS over LAN and keep it all in RAM. That way, I can keep all my nodes identical (and save quiet some money from harddrives). Is this really doable? Except for loading Linux, it would be great if the Nvidia drivers and Blender was installed and ready to render on start-up, connecting to the network as a slave.
One problem though, I will have to have slightly different configurations for my different machines. One Slave will also act as a master, so on start-up it would have to load two instances on Blender, and connect respectively.
Plan B is to use a cheap 4GB usb-memory to boot the computers. If I load the entire thing into RAM, it would matter if the USB-stick is slow? They will be rebooted very seldomly. Is this possible and what kind of solution would you use?
To distribute the rendering I would use the built in network render in Blender.
One questions though, what is the beast way to actually control your computers remotely? I don’t want to connect a screen to them whenever I need to do something. Is a remote desktop the best solution when it comes to modifing my nodes? Or are there better ways of doing it?
3. Conclusion
I am planning on buying 5 nodes, with a total cost of 4500 euro (which is the maximum I can afford). That gives me 10 GTX 580s (12 in total with the ones I already have) which will hopefullt give me very good render times. It will be more focused on doing large volumes of “simple work” (simple scenes, long runtimes) than the other way around, so perhaps I can get away with the small memory of the GTX 580s.
Sorry for the long text, hope you could read it without getting too bored If you have any thoughts or ideas, please write them!
(I have to decide rather quickly becuase the GTX 580 are very rare nowadays, only one model at one retailer left here in Sweden as I said earlier. So I have to move quickly while there is anything left at all to buy :-P)
Thanks in advance!