I’ve been looking for a good network render script/system for a while, but everything I’ve seen so far (instinctive blender, piovra, drQueue, etc.) has had some drawback. Although they are all excellent software, either they are not supported on all three major OSes (Win/OSX/Linux), or require setups, installations, etc. So I wrote my own system, which will soon be ready for release. The entire thing is designed around the idea of complete simplicity and ease of use.
FEATURES & BENEFITS
Setting up is very easy. Server setup involves simply copying a few files, and then using a web-based point & click interface to configure the system. Client setup is even easier - just run an executable and you’re done.
No knowledge of the render network is required. New clients can be added or removed at any time. There is no need to know IP addresses (it works even with dynamic IPs for the clients), and there is no need to mess with the server at all. Just double-click an executable on the client, and you’re ready to go.
No installation required on the clients. You don’t need to install any scripts, libraries, python, etc. Actually there is not even any need to install Blender itself! The client executable will take care of everything.
No permissions needed. The original design was to run the whole thing off some cheap USB thumb drives that I got for free. So you don’t need any local-disk access at all on the client machines - everything is done off the USB stick. This makes it possible to run even on locked-down systems like kiosks or university networks where you only have a few megs of quota - you can, of course, run it off a local disk too.
Automatic directory structure duplication means that as long as you use relative paths, you don’t have to worry about textures, scripts and other resources being unavailable. You can set up a directory structure to match the one on your main machine, and all the clients will automatically clone it.
Web-based administration. Upload new jobs & resources, manage your directory structure, set job priorities and view statistics about the various render clients, all through your web browser. Monitor your render farm on the go from a PDA.
I will be posting screenshots in the next few days; the first release will be coming up in a week or two.
wonderfull… sounds absolutely lovely. And I do hope it’ll end up in the cvs.
1 thing… what I’ve found missing in basicly all network renderers till now… They all seem to have 1 host computer that directs what is going on, and then lots of slaves that do the actual rendering. I wonder if it’s possible to make all the computers (including the host/master) render… that way it would be worthwhile even for someone like me who owns only 2 computers
Or better yet, make that a setting that you can choose.
Yes, this is possible…to explain further how it works: the server is basically a set of PHP scripts on some webserver. Having a webserver with PHP is the one ‘system requirement.’ But many people (OSX & many Linux users) already have Apache preinstalled, and it’s easy to set up on Windows too. Most webhosts also support PHP. MySQL is supported, but optional. So getting a webserver is by far the hardest part of the whole setup, and in my opinion it’s not too hard, considering how easy everything is afterwards.
Once the webserver is set up, the clients can query it whenever they want, to obtain new jobs, or download updates, transmit status info, etc. Because the clients initiate the transfer, the server doesn’t need to know any IPs or other confusing stuff. This also means that you can run multiple clients on one machine, or run a client on the same machine as is hosting the server.
I was going to have a screenshot to post today, but unfortunately I was working on the Mac client when OSX decided to erase my USB drive for no reason So I lost a couple days’ worth of USBlender work, plus a Blender scene that I’d been working on for 2 weeks. I will have some screens as soon as I redo that stuff.
Oh hang on… having a php server is a requirement here???
Damn… well that kind of makes the "The entire thing is designed around the idea of complete simplicity and ease of use. " idea you put in your first point rather silly. Apollux was right apparently… too good to be true.
I Have apache & php installed but it’s not easy really… and I don’t think a lot of people are going to install it for this purpose only… I know I wouldn’t.
Heh, in fact, for sane people, I would consider setting up a php-capable webserver a lot more troublesome (especially on doze) than making a network share. heh. But well! My stupid opinion!
I agree: Installing a net server of any kind isn’t a kid’s job. Specially considering the sceneario where you go to your office LAN with the USB device on your pocket … somehow you got authorization to use the LAN’s machines on the iddle time, but as soon as you mention “Web Server” the SysAdm will answer “No No!”. Honestly if I where the SysAdm, even been myself a Blenderhead I would give you the same “No No!” answer… there is just too much at stake there.
Second Issue: If the clients are going to initiate the transfers, and you are designing this as an works-out-of-the-box solution, you really need to find a way around the various firewalls that most likely are going to be running on the network.
Third Issue: You are working on a USB device, from what I’ve seem here on the university labs, 128 MB is the most popular size… now, on those 128 MB you gota fit Blender (for multiple OSs), the Web and PhP servers (for mutliple OSs), your .blends, the textures needed by those .blends… and most important, enought space for the rendered frames… giving just a ball park figure, you are about 100 MB short.
I considered the difficulty of webserver install, but I think it’s overestimated, consider the different options:
Mac OSX. Go to System Preferences, Sharing tab, click ‘Start’ for the web sharing option and there you go.
Windows. Download the Apache MSI installer; it’s just like a regular Windows installer, easy to use, the webserver is ready.
Linux. I guess it’s more difficult here, but there are still RPMs available, and if you use Linux you probably know how to install Apache. Many Linuxes come with a webserver installed already.
Web hosting. Remember that this doesn’t need a server ‘machine’ just a webserver account. Many people have online webhosting already, with bandwidth quotas of several gigs (good enough if you’re transferring only a few scenes and a few hundred JPG frames). Almost every web host I’ve seen has PHP set up already.
That having been said, I’m trying to find an easier way to do this. Maybe for the next version I will make an executable for the server, instead of PHP scripts, and include a very minimal server within it.
Once the server is taken care of, you can just copy the usblender folder into your web location and you’re done.
Response to various concerns:
Permission on an office network, I’m not sure why this is a problem. Remember that only the server machine needs a webserver. The server could be in your home, or on a laptop you bring in, or whatever. The clients don’t need to do any webservering.
Also, all servers must communicate between the client and server somehow; so there has to be some kind of host listening for new instructions at some point. Personally I think it is less secure when all the clients are in ‘listen’ mode than if only one machine is doing the listening.
Firewalls: port 80 is actually the only port that is open on almost all firewalls. Other scripts using custom ports would have problems; USBlender would not.
Size: I tested on a 16MB USB stick; my target is 128. Blender is approx. 5 MB per OS. The webserver and stuff is only on the server, not the clients. The clients have a single .exe that is about 50-200K in size. It only works on one .blend at a time so figure 1-10 MB even for quite a complex file. Similarly, it only needs space for 1 rendered frame. Even if you render at BMP hi-res, that will only be 4 or so MB. The extra space (~100MB) is used as a texture cache; plenty of space.
Yep, there will be a variety of options. You can use a preinstalled blender, manually add a blender to the clients, or put it up on the server, where it is auto-downloaded to the clients.
I have reconsidered the idea of a webserver - I agree that it doesn’t fit with the ‘easy to use’ claim. I want to concentrate on the client for now, so version 0.1 will still use the PHP system (apologies to those who don’t have access to a webserver; you’ll have to wait a little longer). Version 0.2 will include a VERY limited webserver built into an executable. It will basically be a hard-coded version of the PHP. I did something similar in the past…maybe I can find the code…
I haven’t been able to work on USBlender the past few days, but here’s a screen while you wait. This is an early draft of the client status viewer, but it shows generally what the web admin will feel like.
- you can, of course, run it off a local disk too.
