Currently collecting Render Farms who would like to participate in the next set of tests for 2021, if you own a farm and would like to be included please post below or DM me if your shy.
Hi all, I’m putting together a google sheet comparing render farms for Blender and I wanted to share my results with you.
So far I have only tested GridMarkets however I think the data is already worth sharing since it went against my intuition (sometimes it’s cheaper to use a faster machine weirdly).
I intend to add more tests and farms to the sheet soon and my hope is some other blender users will add their favourite render farm to the list as well.
Here is a graph of my results so far:
What this graph shows is that in the case of GridMarkets you are better off using a more powerful machine since it will complete the job much faster, negating the higher cost per hour. In my tests it worked out to be 1.95 times cheaper and 4.2 times faster to use the 1XP100 over the 16vCPU because your paying per hour for both but the GPU is that much faster.
Since all render farms report their prices using different units it makes it hard to compare their relative prices and expected performances. I suspect this is somewhat intentional but it means that users must carry out their own tests to conduct any meaningful price comparison between farms. In these benchmarks my aim is to find out the effective performance of render farms in GHz and their price in $ USD / GHzh.
I’m using the term effective frequency to mean the number of CPU cycles per second a single core CPU would have to exhibit to match the performance observed by the render farm. Basically it is a way of treating a render farm as if it were a single CPU (albeit a very high performance one).
I am measuring this by first calculating the render time of each benchmark scene I use on a local machine. Since the frequency and core count of the local machine is known we can estimate the number of CPU cycles required to render each .blend file. Some work could be done to improve this estimation (test on a wider variety of machines / operating systems, etc) but since it is the relative performance we care about, it should be sufficient for now.
The farms effective frequency can then be calculated using
eF = c / d
where:
eF = Effective Frequency (GHz)
c = Total CPU cycles required to render the benchmark (G CPU cycles)
d = Job Duration (s)
This is an especially useful way of comparing farms as allows you to take into account differences in the benchmark scene.
$ USD / GHzh is the most common way of comparing render farm prices but the units can be hard to understand, I find it helps to think of them in SI units:
so the GHzh is actually equal to
In this case the cycles we’re measuring are CPU clock cycles so you can think of ($ USD / GHzh) as the the number of US dollars you pay for 3600 x 109 CPU cycles.
I am again using the fact that we know how many CPU cycles it takes to render each test scene to estimate the price of each benchmark.
p = pi / (c / 3600)
where:
p = Render Farm price ($ USD / GHzh)
pi = Price ($ USD) to render job i
c = Total CPU cycles required to render the benchmark (G CPU cycles)
I have included all the .blend files used in the tests as well screenshots of the raw data as displayed by the render farms in this google drive folder https://drive.google.com/open?id=1PKcE9ZEPgIA6a54DYwWg-83LyySelElt
The test scene I used was a modified version of the Blender bwm demo scene (modified so that it is a 25 frame animation of the car on a turntable). I used different versions for CPU and GPU tests so that the scene file is optimised for both.
For each benchmark I carried out I took a screenshot of the render results as shown by the render farms user interface. here is an example of one from GridMarkets:
If anyone feels like running their own benchmarks I’d ask that they upload a similar screenshot to the provided google drive so that people can see where the data is coming from.
For full disclosure I created the GridMarkets blender add-on which is why I started with GridMarkets. It would be great to have feedback on the add-on as well.
Added benchmarks for Rebus Farm and Ranch Computing.
Notes:
Added benchmarks for RenderStreet. Their GPU rendering option is the cheapest I have tested so far 
just not as fast as some of the others.
Also ran some more tests for Ranch Computing since they added 30 free credits to my account given it’s new 
.
Added benchmarks for GarageFarm.net .
Notes:
Added benchmarks for SuperRenders.
Note:
Added an initial benchmark for Tresorio’s beta rendering service.
Notes:
Are you interested in adding cloud PC options as well? If so, I can run the benchmark on Paperspace; on a P5000 instance it will take about two hours, but only cost about $1.50 (+$5 monthly for storage). So if you are not in a hurry it can be a very affordable option.
Hey @blenderrocket, cheers for bringing up Paperspace! It look really cool.
If you get a chance to run a benchmark for them I’d be very interested in seeing the results. If you could say a bit on their pricing model and machine options as well that would be good to hear.
See the Benchmark Scenes tab in the google sheet for the links to the test scenes I’m using. The google sheet has got pretty complicated 
so feel free to post your results here if you’d rather not edit it and I’d be happy to add them for you.
I rendered the GPU modified v1 scene on a Paperspace P5000 machine running Blender 2.81a, which took 1h45m53s (manually timed, from 21:23:00 to 23:08:53). At $0.78 per hour this comes down to $1.35 total cost.
You basically get a remote desktop connection to a virtual PC running Windows 10 in their data center. For GPU rendering you would generally want one of their GPU instances, which are priced as follows:
P4000 $0.51/hr
P5000 $0.78/hr
P6000 $1.10/hr - this one should be about 30-40% faster than the benchmarked P5000 for GPU rendering
In addition to that you always pay a minimum of $5/month (50Gb) for storage per machine.
Compared to a renderfarm, you get a lot more control over your render, you can run any blender version/plugins you want, you can also use it for modelling if you have a weak computer and rendering starts instantly the moment you start it.
They are planning to add RTX GPU’s, but there is no ETA yet. If you render a lot, Shadow is also worth considering, where starting from April you can get a machine with an RTX GPU for a fixed monthly price. For blazing fast performance Xesktop also looks interesting with 10x GTX 1080Ti per machine at $6/hr.
Thanks for the benchmark @blenderrocket, I’ve added it to the sheet.
At that price I could definitely see people using a service like this if they needed a more powerful workstation but were not ready to upgrade their own machine yet.
I’ve actually tried Shadow in the past but it never occurred to me to try it with blender. I think it would be good to get some of these other desktop streaming services tested as well. I’m not sure how to take into account the subscription prices most of them have though, render farms are a bit easier to benchmark in that respect.
That 10x GTX 1080Ti Xesktop setup sounds very interesting though 
Added an initial benchmark for Xesktop using their 10x Nvidia 1080Ti GPU machines. I am actually really impressed by how it performs, it renders faster than most render farm options and is much cheaper as well.
Notes:
Any reason you prefer to use paperstreet over Xesktop @blenderrocket? Seems to me Xesktop is both cheeper and faster, but maybe having to use RDP is a deal breaker?
Added an initial benchmark for Xesktop using their 10x Nvidia 1080Ti GPU machines. I am actually really impressed by how it performs, it renders faster than most render farm options and is much cheaper as well.
Wow those results are really impressive!
- You connect to the PC over remote desktop connection, this is not the best protocol for pc streaming but I found the image quality was very reasonable and there was very little lag as well.
For a better experience, you could try using Parsec to connect to the machine, I have heard other people have good experiences with this for for example connecting to a pc at home for using Blender. It is designed for playing games and has really low lag.
Any reason you prefer to use paperstreet over Xesktop @blenderrocket? Seems to me Xesktop is both cheeper and faster, but maybe having to use RDP is a deal breaker?
I started using Paperspace when I only had a laptop without a GPU available which would lag horribly when modelling anything slightly complex in Blender, so I actually ran Blender on Paperspace all the time that I used it. Ever since I have moved to a pc powerful enough to run Blender locally, I have kept using it mostly for rendering animations and longer cycles renders. Compared to Xesktop it has a very good streaming protocol, Blender works without any issues and it starts up very fast. Also the low per hour cost means I have to worry less about keeping it running for too long, and my renders are almost never so complex that they need the power that Xesktop offers. However if you look at it solely as an alternative to renderfarms, Xesktop seems to beat Paperspace hands-down looking at your benchmark results.
Lastly I actually just found out about Xesktop yesterday as I was looking if there were any alternatives to Paperspace and Shadow available 
I’ve run two benchmarks for Xestkops 8x V100 GPU machine and updated the graph.
I realised I made a mistake for their last benchmark (The 10x1080Ti machine), I entered the finish time as 01:19:10 when it should have been 01:29:10 
. It is still a very competitive offering but not as stratosphericly powerful as before.
Hi haik-u. Andrew here from GarageFarm.NET
Great initiative. It had to take you a lot of work to put all this information together.
Here is some more information about our pricing and service:
More about priorities on our cloud CPU farm:
High: 100% price - 150 nodes per job
Medium: 50% price - 75 nodes per job
Low: 25% price - 50 nodes for all jobs on Low priority together
Soon we are going to launch a cloud GPU service, which works in the same way as CPU render farm (sending the scene directly from the scene with a Blender add-on, managing it over nodes in Web Manager) but it will use GPU cards to render the job.
We support Cycles, Internal and V-Ray for Blender. Here is more information about our support: https://garagefarm.net/blender-render-farm/
- Machine startup was very long, ~15 min to create the image and ~15 min to boot. But once it is up it is very fast.
Yes, the creation and the first boot of the image can be long, but then the next boots are much faster. When you finish working on a server your setup is saved so you don’t have to install your software from scratch when you use the server again.
but I found the image quality was very reasonable and there was very little lag as well.
We are now working on our own software for remote connection, it will be ready soon.
Here are links to our guide about renderfarms - You can learn what types of services you can find in the internet, useful testing techniques and many useful tips how to use online farms efficiently:
If you have any questions you can ask them here I will be happy to answer or you can use our 24\7 Live Webchat to get an instant response. 24\7 Live Webchat
Happy rendering!
Andrew
Hi haik-u,
Marius from RenderStreet here. Thank you for taking the time to create and update this topic.
I just wanted to make a few comments related to the way our farm works. One of the most important aspects is that our server allocation scales automatically with the job size. This means that bigger jobs automatically get additional machines allocated to them, so they can finish faster. We do this so we can offer a repeatable and dependable solution, which studios and freelancers can trust to include in their timeline. Also, for our Studio and on-demand plans, we always have availability. So no matter how big is a project, we can deliver it fast, without prior notice, reservation or advance booking. For big projects, allocating in excess of 200 machines per account is something we can easily do.
And there’s also our one-of-a-kind monthly plan RenderStreet One, which offers rendering for a budget. We have rendered over 37 million frames with it so far, including a lot of short movies. As it’s a flat-fee plan, it’s very easy to budget and it offers a very good value per dollar.
And finally, we are also the only farm supporting Eevee rendering. We’ve been supporting Eevee for over one year now, and we are continuously improving the technology behind it.
Just thought I’d mention these particularities, as our custom rendering platform is one of the main things that set us apart in the rendering space.
Ok ok fine ill join in.
Hi haik-u,
Derek creator of SARFIS here. SARFIS isn’t on this chart but I am really interesting to see where it would place compared to everyone else, We currently offer 32 thread machines for $1.50 / Machine Hour. If ya would like to test SARFIS PM me your SARFIS account name and ill give you some credits 
We are currently in an early access stage where we have 100% functionality but the GUI could use some polish. So as long as you can overlook some slightly rough layout SARFIS offers a pretty cheap, simple, and powerful rendering service with support for output nodes as well as multi layer EXR to whatever size your heart desires. 100 5GB multilayer exr files? no problem. 
Also id just like to add we support eevee as well but only in case someone really really wants to render with eevee, Using mesa on CPU to render eevee isnt really the best option so we don’t advertise it as a service… But hey 50GB of vram is interesting
@garagefarm Hi Andrew, cheers for the extra information! I’d like to try splitting the benchmarks up into the four render farm models you cover in your “How to use online render farms” article once I get some more examples of the different models. Also make sure to post an update here once those GPU machines are available, it’d be good to try them out.
@RenderStreet Hey Marius. Thanks for bringing up the machine count factor. Right now the test scene I’m using only has 25 frames so we don’t see the maximum theoretical performance for render farms which offer more than 25 machines at a time. I’d love to be able to test how all these farms compare when it comes to allocating massive numbers of machines but that can expensive fast. Also, I’m still thinking of how to best incorporate subscription models into these tests. Obviously they make things a bit more complicated since their price to performance score depends on how much the user uses them but I think they would still be valuable to include.
@LordOdin Hey Derek. Check your PMs for my username and I’ll get SARFIS added to the list 
. No need to worry about how “polished” your GUI is, this comparison is just concerned with price and performance right now. Even if you required your users to mail you their scenes on hard drives I’d be up for doing the comparison. I would start the timer as soon as I had posted the drive though 
.
Id be really curious to see how well a larger productions scene would work, I imagine CPU will slowly start catching up to GPU. That is if the scene can fit in the vram of the GPU. If you are rendering very large scenes GPU isn’t even an option.
Added Benchmarks for SARFIS.
@LordOdin From the tests I’ve done GPU (or to a lesser extent higher powered cpu machines) often work out to be the most cost effective options. Obviously most farms I’ve seen which do offer GPU machines can’t match the 200+ machines per account Marius mentions above. But then again for blender GPUs are so much faster that from the end users perspective it’s probably better to have 20 - 30 high powered GPUS than hundreds of CPU machines. Just look at XDestops 8x Nvidia Tesla V100 GPU virtual machine. That’s 1 machine with 8 very powerful GPUs attached but it outperforms many CPU render farms on both price and performance.
Then again, as you mentioned, GPUs add the issue of vram into the picture which can complicate things.
