ok, I have impletmented 2 1/2 versions of something like this in java, so I can probably help you with some of the problems that you will run into. (I am working on the third at the moment, It will be about a 50k download, and will do all the things that have been listed already,)
Really, I think your Idea is a better way, sql databases are definately the way to go - A lot of the program is going through the data base to see what frames (I stored everthing as frames, each with a blend file name, the name of what ftp server that file is on, and i will implement checksums, so that clients can keep a local cache of files, so that when someone redoes a render with a slightly better texture, no new ones have to be downloaded) suit the querying machines ram etc.
The only real bottleneck in this system is the network speed, so I advise
Save things as png’s - these can have up to 64bits including alpha layer, and re compressed very nicely.
Have support for multiple file transfer protocols - you will know the best way to do this with c++
Ummm, don’t use packed textures - If they are seperate, they can be cached as above.
The recieving server (the one that is getting the frames back) and the sending server (that is handing out blend files) should be able to be seperate.
I m going to finish my java version of this, as I will probably end up adding support for a sort of blend cvs (using linked objects) nd various other functionality. If you run in to any problems, email me (bobthevirus[at]orcon.net.nz) and I can tell you how I did it, but more importantly than that, how I did it for the fiirst two versions (which didn’t work.)
Also, contact me if you have some set php/mysql tasks, or for bug checking, or whatever. I can’t promise I will be of use, but I will get back to you immediately if I can’t do something, so you can give it to someone more skilled.
Nice! - Would very much like to know how you did the “render in parts” thing with Blender. Can’t get that to work from the commandline.
The BOINC client is about 2.5MB and the BURP enabled Yafray render is around 8MB, so 50k compared to 10MB is quite an achievement! (or is this without the renderer?)
Exactly, and it also enables the servers to be located anywhere in the world and simply connect to the MySQL dbase to get or send info.
Yafray only supports very few formats at the moment, but this is going to be possible once I add the Blender renderer. Idea noted. - Thanks!
(almost) Everything is using the public http protocol. In that way the clients can even be run behind very aggressive firewalls and proxies. I could have created another protocol, but why do that when http works brilliantly and even supports caching on proxy-caches close to the users.
The media-files are distributed individually. I’m using md5 hashing and size-checks to make certain that identical files (even across animations) will not have to be resent without reason.
Also, I plan to use a network of high-bandwidth servers to distribute media and further compress files using loss-less compression algorithms.
Clients will cache files until they run out of (a user configurable amount of) diskspace.
It already is.
Actually it is seperated into these parts:
Input and splitter (handles upload from maker and creates workunits, hashes media and connects to next level: )
Distributor(s) (provides a place where clients can download data, media and workunits)
Receiver and encoder(s) (provides a place where clients can upload rendered frames and encodes final results in several formats)
Publisher(s) (provides a place where the encoded/finished result can be downloaded by the maker and interested users - also handles the finished animations uploaded by the maker)
When you have finished making your animation (post-processing, sound etc.) you will be able to upload it to the publishing-server(s) for free hosting.
I’m probably going to need a helping hand to do bug-checks of the PHP code at some point. I’ll remember your kind offer and get back to you then.
News:
I just rendered the first textured frame using the system. Everything from upload to distribution, rendering and returning the rendered image worked!
(the image rendered was a raytraced box on a plane with a texture on it…I told you that I’m not an artist )
Now I need to start working on the encoders. non-compressed Tga-files are already supported, XviD AVI streams will be up next, then the old MPEG and then other formats such as streaming video etc.
TGA will be considered “full-quality”. It is a pretty heavy download for most people but is very good for stills (or if you are a quality-freak with a lot of time on your hands). XviD produces >DVD quality, so this will probably be the preffered download format. MPEG will be there for compatibility purposes (if at all).
There has to be some server somewhere that is coordinating the work, and periodically polling the other computers to check their health and progress. Presumably they talk to a process on the worker-computer which launches the Blender as a child-process. (There needs to be a way for Blender to inform the parent of its progress, e.g. through a shared-memory segment and semaphore/mutex.)
The master, coordinating server must make decisions like when to order a worker to stop. The master server must also be restartable, so that if it crashes and restarts the work is not lost: it can re-connect with the children, who can bring the master up-to-date even if the Blend has already completed, and so on.
It would be tricky work but things like this have been done before and there should be a careful search, e.g. on SourceForge, for that “prior art.”
I call this the “Master”-server. Physically it is probably going to be located the same place as the input- and splitter-servers (probably even on the same machine).
About work and communication:
It works the other way around. It is not the master server polling the clients but the clients (renderers) polling the master-server for work. It goes like this:
Client: Hi! - I’m ready to do work! [estimated number of seconds]
Master: Ok, give me some specs so I can see if I have any that suits you
Client: Ok, I’m doing 1.2MFLOPS, have 512MB ram and 3.2GB free HD space
Master: Ok, go to one of the following download sites and download workunit XXXX and this list of media files if you need them.
[list of download sites and required media-files follows]
Client: Nice! I’ll start right away - bye!
Master: Bye! - Have a nice day.
The client goes to one or more of the download locations and fetches files (if needed). When finished rendering it uploads to another server. There is support for multiprocessing - WUs (workunits) can be running, uploading and downloading on the same time. If a machine has several CPUs 2 WUs can even run concurrently. (tested and it works!)
Some of this is also mentioned on the “Getting started”-page on the project website.
As mentioned before I plan (at a later stage) to add support for trickle-messages (messages between client and server while the client is working). This way the server can stop a client if, for instance, the animation render is aborted.
Thanks!
I really like that so many people support this idea!
News:
I have cleaned up one of my brainstorms and made a page with some info about what I intend to do and how far I am. There are probably a lot of technical terms etc. but it should give you a quick overview of the project. You can find a link to it on the project front page in the news.
And to those of you who have already tried to register: There’s no work yet and there is no windows client. I will make a closed alpha-test with 10-15 people as soon as I get the windows version of the client up and running (about 2 months or so). Just before that the database will be wiped clean, so there is no need to register now - your users will be deleted.
Ok, I rendered the first homemade full animation using Yafray. It worked pretty well. However, there are a couple of things about yafray that has made me drop the idea of using it as primary renderer.
Instead I used a couple of days to mod Blender v. 2.33a to run under BOINC. It turned out to be much easier than expected.
I rendered a couple of frames using this setup today - and it worked right out of the box.
Blender has some features that make it easy to use:
Everything can be packed into a single small .blend-file
Just as Yafray it runs on everything (most platforms)
It’s opensource
I won’t need to write an exporter for it
So now I’m moving forward to design some kind of website where users can upload animations to be rendered.
The idea is that you simply pack all the textures in your animation’s .blend-file, set your render-settings, save it and upload it.
The test that I announced earlier will only cover the Linux client and will be held when I finish my work on the feeder (the part that generates workunits/frames from the .blend file). It is going to be a “proof of concept”-test and the main server will be located on my desktop computer (ie. nothing fancy).
I guess I don’t need more than 5 people to sign up for it - more would be too much for the tiny bandwidth I have.
If any of you have an interesting Blender animation that I could borrow to use as a test that would be nice. Please remember to include textures etc.
To participate in the test you will have to have a linux system.
I have already tried this system. Blender part is still new and may have problems. You only get to do a free render for two hours. It would be great if I could find a render farm that was totally free! I am looking forward to this system more.
BTW . . . I run Mandrake 9.2 (it is on a duel boot system). I would love to participate on this test run. I have a somewhat fast computer too! As soon as I upgrade to Mandrake 10, I can access the internet through my nvidia networking card.
Count my comp in. Also i think that maybe the priority of the render coming from a computer should depend on how many renders the computer itself has done for the project, more renders = more points = higher priority of renders being sent in from the computer; kinda hard to explain for me right now.
Comp Specs:
P4 w/ HT @ 3.0gHz
1 gb RAM
and the best for last: an internet connect @ <= 28.8kbs
I did 30 wu for BOINC seti@home in about a week with the comp working about 6hrs on average a day so my comp should be about handle quite a bit. If possible, how about being able to download multiple frames so ppl with a slow connection dont have to be on the internet downloading all the time
The idea is somewhat different. I guess renderplanet uses their own computers? (Didn’t take that good a look).
BURP (or whatever its name is going to be) is going to use your computer - that way you’ll see a far larger number of CPUs than those 46 that renderplanet has. (SETI, for instance, has more than 500.000 active CPUs connected to a system quite like the one I plan to make).
The advantage to renderplanet (if they DO use their own computers) is that they can probably begin the animation before because they have much faster network connectivity than BURP will have using the internet. So in some cases these two projects will use the same amount of time for the same animation.
BURP, however, will be far superior when it comes to lengthy animations and animations with large rendertime/frame.
And BURP is free =)
Also i think that maybe the priority of the render coming from a computer should depend on how many renders the computer itself has done for the project, more renders = more points = higher priority of renders being sent in from the computer; kinda hard to explain for me right now.
Yes - I understand. The idea is that it is going to “cost” you a few of the credits that you earn when rendering frames to start your own animation with high priority (ie. start it now). It will be completely free to use the “Start when network is ready” option.
Good question.
If you are talking about the final release date where the software is fully operational it very much depends on how much help I get on the way.
For instance I’m a programmer, not a designer, so if I have to design and layout a website myself it will atleast take a couple of months extra.
The thing is that most of the difficult stuff (client/server backend stuff) is finished so now it’s time to do all the easy stuff that simply takes time:
designing a website
finding people who are willing to sponsor webspace on fast servers
writing documentation, support and FAQs
porting the client to other platforms than linux
making the project known to blender-users
finding people who are willing to sponsor harddrives for the masterserver
etc.
Each of the points above can potentially take months. I hope that at some point someone with experience in webdesign will join and help out - that would cut down the workload on me pretty much, and would allow me to focus on the rest.
This is a great project, which would significantly add to the entire Blender FREEdom experience.
Since I’m not much of a code jockey, I offer you my web design services.
I realy need this. I have spent nearly 5 hours rendering a yafray animation and then It crashes on me halfway through a render and leaving no files behind :<
You have to render again, but it usually goes realtime or faster. It wouldn’t go any faster in a dedicated post-production software package because they both would be doing the same thing- adding frames to an animation one by one. But seriously, render into individual frames, and then pull them into the sequence editor and render out as an avi or mov. It will take you an extra minute to do, but in your case it would save you 4 hours and 59 minutes
As a side note, renderfarms ALWAYS render into individual frames and must be compiled into an animation later. Multiple computers can’t be accessing a single avi file and add frames to it- especially since the frames usually don’t get rendered out in order.
