Render Farm Guidance

Support Compositing and Post Processing
1

Greetings,
I’ve been doing some research online about various approaches to rendering with blender render farming. So far they are really good, but they involve the computers, the monitors, keyboards, mice and so fourth.
Here’s my current situation:
I have 6 computers ready to go and I may have 2 more to add to that soon. I have enough extra mice and keyboards, however, I don’t have enough LCD monitors for each computer and I really don’t want to purchase up to 5 or 6 additional LCD monitors.

Is there a way that I can link all of the computers together via a network switch or some other method rather than having to have 8 computers along with 8 monitors hooked up to each over a network? I do not know of another way to combine all the power of my machines and was hoping someone could give me their expertise on the best methods for render farms.
Thanks a bunch in advance!

2

I could write something up for you.
I’m no expert, but I’m getting good at the I.T. and networking thing since my Dad and I opened our business, and I’ve been wanting to set one up for a while. Dad won’t let me do it yet for some reason or another, but I only need two machines to set one up just to know exactly how :eyebrowlift2:. In fact now that I think about it, I’m going to start writing up a full tutorial because we do need a good renderfarm tutorial and it’d be cool to have it confirmed working by two people at once if you want me to keep you updated on the steps as I do it. :ba:

3

Are you comfortable with the Unix command line? If so, I’d recommend putting your favorite flavor of Unix/Linux on the machines and running them headless (no keyboard/video/mouse) after the initial install. You can connect to each machine using SSH (secure shell/terminal) from your primary machine to control them. If I’m speaking a foreign language, then get thyself to Google and start learning!

4

Your best bet is some sort of KVM – essentially a box to which you connect a single monitor, keyboard, and mouse, and from which you connect to a number of computer mice, keyboard, and video inputs on multiple computers. You can then switch the console to any of the computers individually. There are cheap, 4-computer versions, as well as larger versions depending on your budget. There are even IP KVM’s (a bit pricey) that will allow you to connect via the network to the console of any of the computers. I use the latter at work to connect to the consoles of computers I support remotely when their network cards are non-functional, or when I need to do admin stuff with networking turned off, so that remote desktop, VNC, or SSH will not work.

Alter, the Unix idea is good, but sometimes it may be necessary to have a console. If WCS doesn’t want to spend the money for some sort of KVM solution, the “headless” solution, with a single monitor he can attach at need will work, since most Linux systems will accept and recognize hot-plug keyboards and mice, although the graphical interface (X-Windows) will be problematical in some releases.

5

Good points whraven; since he’s only dealing with six or eight boxes, manually plugging and unplugging the kvm will probably be more cost-effective than eight-port KVM. Also: no reason to run xwindows if all you’re doing is running Blender headless in a render-farm. I know it’s a bit counterintuitive to run a graphics package without a GUI, but since all it’ll be doing is accepting input from the network and rendering to files, it shouldn’t be too hard. Then again, take me with a grain of salt because I have set up clusters of headless boxes, but always for apache/mysql, never for use with Blender.

6

I use vnc software to control all 3 of the computers that make up my renderfarm. The software runs automatically on the 3 upon start up. Then on my main computer, I use the vnc software to connect to the others. Then on my main computer in the vnc window, I have the complete desktop of the slave, allowing me to do anything I want with my mouse & keyboard. The only downside is each computer must have a name and a password to access it. I use TightVNC, which is free, so the downside of entering computer names and passwords is no big deal compared to the cost of KVM switches or having to plug & unplug KVM connectors.

Randy

PS I’m running all win xp & win xp pro on all the computers…

7

WOW! I’m gone for a couple of days, I come back to check and I see a nice little discussion going on here! AWESOME! Okay, pirohmaniac YES! I would be very interested in this tutorial that you speak of. My primary email that I check is [email protected] please stay in touch with me either on here or email so that I can follow you on this tutorial. I am now very intrigued!

Alterscape, Yes I am quite comfortable with it actually. I’m a computer science new media major and several of my classes were ONLY Linux oriented using Vim and a large number of terminal navigation and commands including accessing servers via SSH logins. However, even though I have a VERY high respect for Unix, a number of my machines are Windows based and I REALLY don’t want to tamper with the operating system whether it be installing an additional Unix based OS on them. Some of these machines are already currently Linux machines and I suppose I could just use a program like Putty to run SSH commands into or from the windows machines. I honestly need an alternate option from what I’m currently looking at. At the moment, I only have a Windows based networking render farm concept going, and I REALLY could use a Cross Platform system since I own 5 windows PC’s, two Linux PC’s and two Macintosh machines. I am slightly hesitant to try this All Linux route Alterscape. Have you successfully been able to get this method to work for you?

Whraven, agreed! I have looked at KVM’s. I am quite tight on money at the moment which only makes my situation that much more complicated, but Christmas is coming up… I would definitely be interested, this is beginning to become a little overwhelming now! LOL! I suppose this happens with Render Farming though.

Alterscape, I am really just looking for the least costly approach with the ability for this to not be too hard on my Noob experience. For my animation Class, one of the other group members, (Not me obviously) had set up a render cluster, so I know nothing about that either. I feel like I’ve gotten way over my head now, except that I have all of these machines at my disposal and have some EXTREMELY complicated scenes coming up that I need to render, and my Quad Core machine will not be able to do it alone.

Randy, you make it sound very simple, is there a type of tutorial that you followed when using TightVNC or was it just that simple for you?

All of you have given AWESOME feedback and I thank each of you for taking your time thus far, PLEASE, if any of you can keep talking me through any more of these ideas. I feel like I have been given the general over view of approaches and software, but I’m still feeling fairly lost in terms of what the very best approach would be for a guy who is 1) Low on funds
2) Has 8 computers all with varying OS’s running. I need something cross platform
3) Is fairly confident in my technical skills, but I need a little more direction since I am still new to the ins and outs of render farms.
Finally, I do have time. This does not have to get done by next month. I’m just trying to get a head on it. If any of you can reply again and give more helpful info, I seriously appreciate it! Any of you can also email me at [email protected]

8

Since Blender runs on all those platforms, and some form of remote control (be it VNC or SSH) also runs on all those platforms, it seems like any cross-platform renderfarm solution ought to work, though you’ll have to configure/start it in different ways on different machines.

The most basic solution would be to manually start rendering some subset of frames to each machine (eg: box #1 gets frames 1-1000, box #2 gets frames 1001-2000, etc…) and then save the frames to a network share (SMB is probably easiest, since Mac, Windows and Linux can all access it) and combine them using ffmpeg or a similar tool… or you could look into 2.5x’s built-in renderfarm support, etc. I’ve used the simple approach; someone else may be able to tell you more.

9

Yes, TightVNC was that easy to set up. I knew nothing about networking or vnc software when I started setting my system up and I think it took me longer to figure out how to set up the router to assign IP addresses to each computer than it did to set up the vnc software. I looked, TightVNC is windows and unix only, so I don’t know if it would work on linux (I would assume it would, but if it did, why isn’t it mentioned?) and mac os isn’t mentioned. So I think you would have to find cross platform vnc software.

I used FarmerJoe software to control everything because it’s written in python and perl. I used to know perl (but it’s been years since I used it) and I’m learning python. But since blender’s python api has changed so much from 2.49 to 2.5x I would assume it wouldn’t work in 2.5x. FarmerJoe was nice as after I had it running on all 3 pcs, all I had to do was hit the ‘submit’ button from the scripts window once I selected the script and it handled everything. Then my farm renders and my main pc is free to work on other things. If I’m not using my main pc, I can start FarmerJoe on it, and it becomes a grunt as well.

Another good choice is Loki render, I’ve heard good things about it and it’s written in java, so it can run on any computer with java and it will work with any blender version.

Randy

10

Just a short note on VNC and SSH: they work fine once you set it up, and that is how I do over 98% of my remote support. The KVM (or just having a spare monitor/keyboard/mouse to plug in at need) is required when network problems block the VNC or SSH connections (or, in my case, when I have to disable the network connections for security reasons).

Oh, and GUI applications work just fine over SSH.

11

Alright, I’ve actually been thinking it over more and in the next couple months I’m going to be shelling out money anyway, so does it really make sense for me to purchase a whole bunch of extra monitors, mice and keyboards anyway? At this point, I’m actually thinking not. Besides, if I run it over the network, all I’m really doing is allowing the individual computers to render whatever available scenes have not been rendered, to be rendered. Thus, I’m not combining the power of my machines, I would only be letting them take individual jobs at one time and this will not work. Most of these machines are extremely out of date. A lot of them are just old Pentium 4 machines with 1 gb of ram in each. The second I hit animate, these computers will just crash and be useless. What I need to do is combine the power of all the machines to take care of one frame at a time so that I can make use of the individual power of the machines right? Is this where the KVM cables come in?

12

If you’re really into your Linux skills, you can set up a single LTSP server which will act as the hard drive to your slave machines. The slaves all do a network boot from the server. This way, if you ever have to change anything (e.g. upgrades and updates), you only have to update in one place. Also, if you acquire any additional machines, you just plug them in and go - no setup required! It can get pretty gnarly to set up though…

It also sounds as though you should be looking into render farm management software. There’s a few options around, e.g. DrQueue, FarmerJoe, Loki Renderer etc. I believe Blender 2.5 has/will have network rendering support too, although I might be wrong on that one.

Steve.

13

I consider my Linux skills to be intermediate. I’ve had a few classes in which all programming was based on the Vim command line so navigating through the terminal, executing commands and such was grasped and utilized frequently, but I’ve never set up an LTSP server, nor do I know what it is. The problem there is, unless you have a great link for a tutorial, I would need to start searching online wherever I could find the best instructions. When you say, connect them all together, are you talking about using KVM cables, or CAT5 cables into a switch or what? Sorry but I’m new to setting up a render farm/server so I’m still trying to understand everything.

14

Since I have used FarmerJoe, I’ll explain how it works to help you understand it a bit better. FarmerJoe requires one computer’s harddrive to be a shared network drive, forget how I set that up now. When I hit the submit render button, it copies the .blend file to the network drive and sends the information to the master. The master assigns each slave 1 frame of animation. Each slave starts blender via console commands and blender (without user interface) renders one frame, saving it to the network drive. When the slave finishes, the master assigns it the next frame that is not assigned to any other slave. The process repeats until each frame is rendered. Not sure how LokiRender works, but basic principal is the same, Master assigns frames to slave machines, slaves render and saves the output.

You mention that you Pentium 4 1gb machines would crash if told to render your scenes, so that would be a problem, because each machine has to be able to load blender and be able to render a frame. If your scene is too much for those machines to handle, you’ll have to split the scenes up, as in maybe render the backgrounds, then render the foregrounds, then composite the two together.

Hope this helps,
Randy

15

I’m completely open to using Farmerjoe as well. I have had an animation that I’ve been trying to render on my Core 2 Quad machine and this scene has been crashing numerous times even on a computer as strong as that one with 8 GB’s of ram too. This is why I am concerned about trying to attempt to render on these older Pentium 4 machines taking on individual frames, without actually combining the power through a different method.

Plus, what about varying hardware? Is it okay that three of my machines are AMD, four others are Intel and then one more is a power pc G4. Is this okay that not all machines will match in terms of hardware and even OS? Rendering the foregrounds, backgrounds and other various objects separately will take far too long. There is no way that I can do it that way. Any other thoughts anyone?