New Open-source Render Farm Add-on (with GridMarkets)

Hi all. I’ve been thinking for a while about how to improve the submission process for Blender render farm add-ons.

Most of the solutions I’ve tried recently have either used a web-based frontend (requiring the user to manually pack their files before submitting) or a very basic add-on that can submit your scene but not much more. That’s fine if all you need to use a render farm for is to submit the occasional animation but compared to the solutions that exist for some of the other 3D software packages out there, I’ve always been left with the impression that there’s room for improvement.

I’ve been working with GridMarkets (a parallel computing platform) for the past few months to develop a new open-source (GPLv3) render farm add-on for Blender and I’m really excited to finally be able to share it with the community.

Aims

My aims were to create an add-on which solves some of the issues I’ve ran into in the past when using Blender render farms and which could be used as a template for render farms add-ons in the future. Most importantly I wanted the add-on to be easy for artists of all skill levels to use while retaining the advanced features that might be required by production artists. I’ve made a list of all the features that I’d want to see from a render farm add-on (see list below). I would love it if I could get some feedback from other blender users as well (feel free to be harsh :smiling_imp:).

Downloads

The current versions of the add-on can be downloaded here:

Implemented Features

  1. Add-on opens in a separate floating window
  2. Allows the submission of unsaved or modified projects.
  3. Uses BAT internally to automatically pack your scene (Including external assets and linked libraries) before submitting.
  4. Allows you to optionally maintain separate render settings for local and cloud rendering so that you can don’t need to worry about updating your scenes render settings before submitting.
  5. Supports multiple frame ranges
  6. Supports blender 2.80 and 2.79

Getting Started

Download the zip from one of the links above and install the add-on like any other. Optionally you can follow the instructions of the GitHub page for how to build the add-on for sources if you prefer. Once the add-on is installed you should be able to open the add-on by clicking the button added to the topbar:

Using with GridMarkets

Given the add-on is designed to work with GridMarkets you will need a GridMarkets account to actually use the add-on for submitting renders (signup page here: www.gridmarkets.com/signup). They start with 20 free credits so you can try out the platform.

GridMarkets currently supports Cycles and Blender Internal as render engines for Blender. It also fully supports python scripts and drivers which I know from experience can be an issue with some render farms.

Note - Right now the add-on uses the GridMarkets’ Envoy application for the actual uploading and auto-downloading of render results. There’s no reason the add-on couldn’t be made to work standalone but GridMarkets requires Envoy to be used for security reasons.

Submitting Jobs

After installing you can then simply submit your scene to render as shown in the below .gif.

Custom Jobs

By default the add-on uses your scene settings as the render settings so it should render exactly like you get from a local render. Alternatively you can create custom jobs which you can use to override one or more of blender’s render settings and use that job when you submit.

Projects Tab

Any project that you submit is added to the projects tab as a new project option. This means you can submit a project and then submit new jobs to the same project later without needing to re-pack and upload your files. You can also upload projects from this tab without submitting any jobs to them. This is useful when you want to check your project uploads correctly without starting a render job or if you want re-upload to an existing project.

There’s also the option to upload an existing .blend file using the “upload file as new Project” button. This is useful because it allows you to quickly upload multiple projects without actually having to open each one.

Future Ideas

A few ideas for future features that I would still like to implement include:

  • Preflight window
    so you can see all the files your uploading before submitting

  • Pack project to local zip file
    for if you just want to pack your project locally using BAT)Project status updates

  • Show upload percentage

  • Differential file upload
    only pack and upload the files which have changed

  • Add more support for more Engines
    GridMarkets already supports Redshift, Vray, Renderman and Arnold for their other services, it should be possible to use them with Blender as well.

  • Render Farm as a Compute Device
    I think that would be really powerful If it was possible to create an add-on which effectively adds a render farm to blender as a separate compute device. I believe this would require a custom build of Blender though :pensive:.

For now any feedback would be appreciated!

6 Likes

This is probably the most interesting idea I’ve seen in ages. It would sure bring 10 fold simplicity for basic users and artists.

1 Like

Error in Submission Settings: “Render engine VRAY_RENDER_RT is not currently supported.”

What about simulations? Can we do fractures with the fracture branch? Clothes with the other branch? What about all different branches out there? Is this only for vanilla 2.8?

@JuhaW It looks like your trying to submit a scene using the RT V-Ray render engine. GridMarkets doesn’t currently support blender to V-Ray (although it shouldn’t be too difficult to add) so I’m guessing that’s why it’s complaining. By default it will try to submit with whatever engine your scene has selected

image

But you can override this using a custom job setting.
image

@DavidRivera Right now it’s just vanilla 2.80 / 2.79. GridMarkets does support simulations for Houdini so it should be possible though.