Simple cloud rendering

Hello Guys,

Recently I’ve found that there is no simple way to render using cloud (like GCP/Amazon) - everyone actually buying some hardware and building ~$3000-5000 render farms or using some proprietary net-rendering solutions, which are unstable or have none security to be somehow used in a cloud environment.

So I decided to build the “BlendNet” addon for blender 2.8 with GCP provider support - “preemptible” instances could be really useful for the rendering purposes and could offer ~$0.5 per quite complicated cycles frame. GCP is offering $300 for anyone to play with and it’s easy to setup the project and actually run your render without any man-in-the-middle. If the addon will become useful - there is a number of things to add, like cost estimation, another providers to support, etc. So hopefully it will help you guys.

Here you can find the addon sources and some documentation:

Known issues:

  • Tested only on Linux - hopefully should work on MacOSX and maybe will work on Windows (but expect errors).
  • Initial configuration is quite hard for someone who never used GCP gcloud utility
  • Tested only on GCP with default project - so if you have some interesting configs, it could fail
  • Agents will have external ip to download external resources (updates, blender binary)
  • Some settings are not working - for example you can’t specify the existing manager IP, so only dynamic creation on GCP
  • Auto-shutdown of the Manager - so you have to stop it manually
  • Save of the Manager/Agent tasks information between runs - do not shutdown Manager until you get all the required information from it
  • Simple auto download the results - right now they are downloaded only if you previewing them
  • Not tested render with dependencies (images,caches) - so could contain some minor issues
  • Interface is responsive - but relies on user interaction, so if it’s not updating - just fly mouse around it
  • Providers interface is not looks good - will be rewritten
  • Animation (multiple frames) tasks creation glitches (sometimes stuck for some reason on some frames) and slow, but could be used somehow)
  • No Manager tasks control Addon interface

Please read the documentation and HOWTO’s on github repo Wiki to try to use the addon properly.

Thank you.

1 Like

As usual 25% of the plan are taking ~50% of time… But finally I prepared alpha version of the addon - I updated the header with the required information.

Hello Guys,

First demo test results: Cosmos Laundromat Demo from official blender demo page

  • Modifications: I just unpacked the resources from the blend file to test how they will be found and uploaded.
  • Reference render (tiles):
    • Configuration: 48 threads, progressive (2x Xeon E5-2680, 128GB, xUbuntu 18.04)
    • Render time: 10:50.10
  • Reference render (progressive):
    • Configuration: 48 threads, progressive (2x Xeon E5-2680, 128GB, xUbuntu 18.04)
    • Render time: 13:04.41
  • BlendNet render:
    • Configuration: Manager n1-standard-1, Agents 3x n1-standard-16 (highcpu have not enough ram)
    • Render time: 13:06.30 (without running manager/agents)

So I think it’s quite optimal right now - the same results with the reference progressive render is a huge value of the system. Of course we need to work on support for tile rendering, because 2min difference is pretty big if we will multiply it by thousands of frames. And remaining calculation still requires some improvement…

Some BlendNet render screenshots:

Results to compare:

Manager task response (some verbosity):

{"success": true, "message": "Got task status info", "data": {"state": "COMPLETED", "create_time": 1571786754, "start_time": 1571786884, "end_time": 1571787856, "project": "benchmark.blend", "samples": 600, "seed": 1400925772, "frame": 2472, "samples_done": 600, "remaining": 441, "result": {"preview": "a1daab67a6791b3b61f1b459b810bac055e63a5b", "render": "145808ba01785718ab73a894aeda492c1dca8eaa"}, "start_time_actual": 1571786975, "samples_per_workload": 100, "samples_acquired": 600, "workloads_taken": 6, "results_processing": false}}