How do render farms work?

Support Technical Support
1

I heard you can setup your own render farm for blender fairly easily though I don’t know how to do it yet~
I have some questions on how it works before I even consider looking for cheap underpriced computers.

  1. What am I looking for in terms of specs from the computers? ram? cpu speed? gpu speed? other? I am assuming I don’t need a monitor for each one?

  2. Does it help at all to hook up multiple computers to render 1 frame? Or does this not work because the farm delegates 1 frame per computer as needed instead of “helping each other” to render the same frame?

  3. is the speed increase 1:1? for example i have 2 comps of same specs so I get 2x the render speed?

  4. OS choice, does it matter?

  5. (opinion based q) is a commercial render farm better out right in terms of cost? I tried looking up costs and it feels expensive but I don’t know how long it actually takes. or how complicated their scene was.

thanks.

2

in regards to 5) I have a bunch of old HP Proliants from 2015 for CPU rendering and … hmmmm it worked pretty good until I came across Render Street and started to realize what a price difference it is to run a Server compared to just subscribe to their service (Render Street). I used to recommend running own servers before because the electricity bill compared to what a rendering service cost back then was a difference of day and night, making an own render farm much more feasible… But with subscription models in place for rendering services these days, there is really no point in doing it any longer. You safe much more money this way. I have now 5 servers at home which are retired since almost 2 years, haha.

3

There’s also Sheepit by the way, take a look and see if that suits your needs. If you have a couple of computers, you can start queing them up for points. Cost… free, if you don’t include time. =>

5

1.Any system will give you boost in rendering.

  • 4+ cores
  • 8GB+ ram (depending on how big your scenes are
  • GPu wise depending on scene size - memory buffer is important. Nividia allow GPu to “somewhat” share their memory. AMD allows you to mapp extra memory to GPU (on VEGA cards)
  • storage - best to use a central system to store rendered files.
  • https://techgage.com/article/nvidia-geforce-rtx-3080-rendering-performance/

2 .if you have a single storage (nas) for all systems, and setup blender scene to create “dummy” frame and “no override” options.

  • then first system tries to create dummy file, its successful starts rendering first frame.
  • second system tries to create first frame dummy file, but it already exists, so it creates dummy file for frame two, its successful, starts rendering frame two.
  • and so on…

3.dual system speedsups

  • if systems are identical, then nearly 1:1 Slight overhead on file creation if above is used, so maybe 1.95%+ speedup with two computers.

4.OS Linux/Windows - https://www.phoronix.com/scan.php?page=article&item=blender-290-performance&num=4

  • GPU rendering - Windows in slight lead (CUDA and OPTIX) in most scenes
  • CPU rendering - Linux is faster by noticeable amount

5.commercial render

  • depends how many projects you are rendering.
  • commercial render its more flexible. if you have a high priority project, just pay a tad more and you get a lot more rendering horse power.
  • home setup requires significat upfront costs, commercial does not.
  • home setup needs to have proper home power, as in if you setup 10+ computers, a single fuse is about 16Amps in Europe - @ 230V - so about 3600W… that is about 3-4 systems with multi GPU setups. if you need more you need to change come setup, or move them into differen locations where its conencted to another fuse. (but eventually you run out as there is still home limits)
  • advantage of home setup, your scene never leaves your hands.

I personaly have a small personal farm, but I do projects for fun, not proffesionally so it works for me. (that and love seeing that much hardware next to me hehehe)

6

thanks for the replies.

If i understood this correctly it tries to delegate frames correct?
What I was really hoping for was a way to render where each comp helps render the same frame… at least when doing test renders.

7

With the info posted already (thanks for the details @Grzesiek), seams like a good thread to ask the following (for a home render farm) for those of us completely new to farm rendering (as obvious as they seam) …

  1. What software is used to “communicate” with the Blender installs on the different computers and delegate the work out ?

  2. Can you mix OS across the farm ?

  3. If only using two PC’s in the farm (one being the main which was used for editing/making the scene/project) can you work on a second project while the first is being rendered ?

  4. If your scene was created using OpitX denoise, then the second pc would also need to have an Nvida card (more so, if project was created on a pc with certon hardware needed, the farm will also need it) ?
    If second pc has a non-OpitX compatible card what will happen ?

  5. if pc’s in the farm dont have GPU will they still render in CPU ?

  6. Single frame renders; based on answer 2 and dummy files, single images cant be rendered on a render farm ?

  7. For animations, the render output will be (sequenced, based on answer 2) stills or can it be a video file ?

8

1 ) actually you can use the remote desktop for windows from your main work computer. This is possible in Windows Server, just as it is with the home versions.

2 ) When using Blender as your render system, yes you can.

3 ) Yes. That is possible, and I tell you how just down below…

4 ) I know nothing about

5 ) when using Blender, yes they automatically do.

6 ) Single images are not possible to render using multiple machines at once (for your home render farm without add-on)… I mean at least not to my knowledge. (actually there are ways of doing it…)

7 ) It can not be a video file (unless you render multiple raw AVIs)… I will explain further, down below.

So here is how I have my set up:
All my Servers are for CPU rendering, even though I have low-power 1050ti’s installed on them (anything higher will require a power upgrade for your PCI cages)… as you will realize these Servers are NOT GPU servers. You will need you put a video card into your server, as Blender simply requires it (look up the OpenGL dependency on Blender and you will see why). They have much different redundancy layouts in terms of hardware and the way the power is managed. So… a GPU Server is MUCH MUCH MUCH more power consuming and WAAAAAAAAAAAAAAAAAAAAAAYYYYYYYYYY more expensive… I don’t know of a single Blender user who has a multiple Tesla equipped GPU Server at home (maybe there are people out there that have one… but you gotta be either crazy or just loaded with money). So CPU servers is usually what people go for when starting a render farm… now having talked about Servers… you can do the same stuff of course with multiple home Computers or Laptops. But using Servers is better in my opinion as they are made to handle power 24/7 and you can raid configure them, so if something goes wrong with a bad sector or you are at the end of your first SSD, there will be no errors or data loss… again redundancy here is the key.

So there are multiple ways of dealing with setting up Blender and one of them which I don’t use is Blender Network Render (2.79… I am not sure whether it is still around in 2.8 and up), even though it’s pretty good especially when you make use of your server’s raid configuration.
However I want all my servers to be connected to one main drive and render it on one disk… which doesn’t require Blender’s Network Render… all you do is to enable placeholder option and disable overwrite. I use Remote Desktop which is configured in a way where it doesn’t run into OpenGL issues. Usually when trying to run Blender over Remote Desktop on your server it will not work as it will prompt an error. The way I get around this is to have Blender running at start up on all servers. This is for Windows 2008 (Server). Of course before running this you need to do a lot of configuring, so if you don’t like all the crazy setting up hardware and software, using a home computer is easier.

Anyway, I could go on forever, but there are more than just the two ways which I have briefly described and there are lots and lots of videos in the internet on how to create a successful render farm. And some of them might be even better.

edit: a good Blender Network Render set up example (again I am not sure whether it is still around in 2.8 and up)

and the simple way , which my system is set up , sort of:

9

Thanks for the info and links…

Done a little research since last post and was gonna create a thread, guess i could start it off here…
Heres where im at and train of thought…

Gonna upgrade vid card in current pc soon.
Have a power supply, i7-8700K Coffee Lake 6-Core 3.7 GHz and a few hdds (maybe an SSD also) laying around and after vid upgrade will also have a GPU… so with a new m/b some ram and a case i can have another pc (pretty decent one at that). Also have an OLD linux sever fully running.

Current PC
  • Windows 10
  • i9-9900K
  • ZOTAC GAMING GeForce RTX 2070
  • 64GB of ram

Have an old server pc which is used for php, mysql and file server and was thinking of making a small render farm and adding it to the farm, but need a GPU.
Been doing alot of animations and if i could speed up the rendering a little, I’d be willing to spend 200 bucks on a GPU for it.

Server Specs

Was looking at a PNY NVS Quadro NVS 300 VCNVS300X1-PB 512MB DDR3 PCI Express x1

Good plan ?
Better why to go about this with resources at hand ?
Better/Different video card for server ?
Any feed back welcome.

10

…in my example I have HP ProLiant Servers (which only feature two riser cards, all the stuff a normal PC doesn’t limit you to), and they can only have a low voltage GPU … so I have had to use a 1050ti which doesn’t require external power, and is the only one in NVidia’s line up that caps it, everything else requires a server hardware upgrade which is near impossible todo on an HP dl380. It simply feeds of the on board power.

I am not sure about the supermicro x7sbl.
I think a bare minimum PCI 1GB GPU already is enough to run Blender (edit: basically what you have suggested)… I mean all you need is the CPU cores, right? It really depends on what your board’s PCI slot supplies on voltage… you might even have a power connector on your board to route additional power to a card which requires external power (my AMD Opteron has such, but it’s not enough juice to run a radeon RX, haha)… I really don’t know. Like I wrote, my HP dl380’s all run 1050ti’s … but 2 of them have also 1GB GPU’s of which I am not sure whether those are 210s or what

edit: this one is pretty decent too:

11

I want it ALL :smiley:
Im only a hobbyist and will not be rendering 24-7. If i can get a card which will contribute alittle also, why not ?

Thanks for the link.

12

You are very welcome

A 1GB low profile card wont do you too good in that case, I am afraid :slight_smile:
That 1 GB is not enough to handle even the slightest texture related work, even with both (CPU and GPU) rendering enabled.

13

Bucket Rendering is not in default Blender.

there are probably some scripts to run each blender on a seciotn of a scene, the issue is that at the end you’ll have to use another software to recombine the sections into a single image. Might be possible with Blender, but woudl again require some setup, liek a new scene with just node setup to input each “section” then merge final image at the end.

Will admit I have not played with the above.

14

With regards to point 4.

Mixing reder types (opencl, cuda, optix) and mixing denoisers can produce different results.

expectation is that opencl/cuda/optix will generate identical frames, but in the past that wasn’t the issue. Unfortunatley i only have OpenCL devices as such can’t confirm if they fixed that.

Note that some denoisers have isseus with temporal qualtiy. As in, static images will results in good quality, but when plaied in sequence, strange artifacts can appear.

Optix issues were present 1 year ago, unsure if this is fixed. you’ll need to test (and hopefully report back)