Blenderfarm

[ideasman42]

Single image renderer- can render 1 image accross multiple Pc's

1) Make sure all blender installs have the blenderfarm script on them.
2) Point all the PC's to the same network directory. (Export>Blenderfarm>Configure this Node)
3) Run a few PC's as Nodes (Export>Blenderfarm>Send & Be a Node (slave))
4) Open a file to render and send to the farm.
......
5) To view, goto (Export>Blenderfarm>View Rendered Tiles (Poll until rendered))

the file removing is very safe and dosent recursivly remove whole dirs.
Youll need blender 2.4+Python.

EDIT - UPDATED FOR BLENDER 2.43
http://members.optusnet.com.au/cjbarton/blenderfarm.py

[Marty_D]

woohoo! Going to have to give this one a try. Thanks, will see how it goes.

[Marty_D]

All right. Well, the script actually worked. Which is way better than any other solution I've tried so, hats off.

In case anyone doesn't know, got the netork to play by mapping the shared folder to a driver letter in explorer (windows of course) because blender can't actually see the network.

Trouble is, using a two machine farm the rendering process is actually slower than if I just render on a single machine?

testing specs: 2.4ghz Pentium 4 w/1.5G ram and 100mb ethernet, 2ghz Pentium4-M w/512M ram and 10mb wireless ethernet.

Seriously though, thanks. At the very least I can send a file around and still work locally.

[Jedrzej_s]

Wow !!! Sound great !!! Can I render Blender scene with YafRay on accross multiple Pc's ?

P.S. Cambo <- Why don't all yours scripts are on your website of scripts, example: Blenderfarm, new sculpt mesh ?

[ideasman42]

Hey, Marty_D thanks for testing, glad you had some sucess.
About rendering comparisons/speed tradeoff.

This script would be usefull if you had 3+ PC's and your were rendering a print sized, raytraced image.
For smaller files, there is not much point in splitting them up, bacause the overhead of file transfur, copying the images, and each blender converting to renderdata is too high.

As for net releasing the scripts on my website? - will ray_pick.blend is an experement and somthing Id like to re-write or heavely modify. Dosent work in all the ways Id like it too, Blender dosent support propper remove doubles yet either (in python). and the renderfarm- same deal, its not 100% done the way Id like.

Hey, on the topic of renderfarms, I had an idea how to realy easerly render an animation across a network- with multiple PC's.
Add 3 options into Blender.
* Touch before rendering.
* Never overwrite frames.
* Render random frames.

Turning all these options on and rendering to the same path across a network, would distribute the animation between all the PC's.
- Any thaughts?

[MacBlender]

another area where you'd find a speed increase would be on massive levels of faces shown and with everything turned on, if your just rendering a cube using multi-cpu rener is stupid thing to do.

cambo: so are you goign to extense this into animation or leave it at single renders?

MacBlender

[Jedrzej_s]

Cambo <- THX for answer, but: Can I render Blender scene with this script and with YafRay on accross multiple Pc's ?

[kattkieru]

Originally Posted by cambo Hey, on the topic of renderfarms, I had an idea how to realy easerly render an animation across a network- with multiple PC's. Add 3 options into Blender. * Touch before rendering. * Never overwrite frames. * Render random frames. Turning all these options on and rendering to the same path across a network, would distribute the animation between all the PC's. - Any thaughts?

Well, depending on how fast your network is, network latency plus rendering times should make that a fairly feasible system for a small renderfarm (ie Pixar's thousands of computers would kill the server, but for 5 to 10 machines it'll be alright).

It'd be really nice for the server to talk directly to the clients, though. Having those clients constantly making lists of images rendered by stepping through the image list seems wasteful...

However, Python does have socket support. You could configure clients to talk to the server by typing the server IP address into clients and passing back and forth information on A) what's been rendered and b) what needs rendering.

This is the best solution, I think, because if the render client crashes it's left a fake file as a standin until it's finished rendering. However, a client who's finished rendering, written the file, and sent the "File Rendered" message to the server can die all it wants. This also keeps the list of files needing rendering in one place only, whereas before it was constantly being generated by the clients.

Thoughts?

Incidentally, your script is pretty cool as is. Reminds me of 3DS R4, back when the world was young. It used to do the split-into-tiles thing across a network for single frames. However, these days it really is better to render single frames (except, as you said above, when rendering huge print-format resolution images).

I think that, if you got the single frame renderfarm thing going with decent client / server UIs, the community would praise you.

[pildanovak]

Originally Posted by cambo As for net releasing the scripts on my website? - will ray_pick.blend is an experement and somthing Id like to re-write or heavely modify. Dosent work in all the ways Id like it too, Blender dosent support propper remove doubles yet either (in python). and the renderfarm- same deal, its not 100% done the way Id like.

this actually answers my question from the thread about ray_pick. I wanted to add some functionality, but I see it would be a waste of my time.

[Unimatrix]

I know for single frames at work, we use a Quad-Core Mac (or whatever is the latest and greatest) to do print images in lightwave. It is far more effiecent we've found to have one super fast machine working on it and then move the employee to another computer to work on something else while it's rendering. Our renderiing grid seams best when tackling large numbers of frames.

Again, on a small scale <10 node render farm, this could be useful for users, but on a massive scale or using a distributed rendering method, I'm not sure how well it work in praxis.

[karpov]

Hi Cambo,

I've just bought a new box (amd64) with ubuntu running on it and trying to use BlenderFarm with my windows laptop.

But I have some troubles to work with it.
launching send and be a node
Somes errors (like ' no module named blenderfarm')

In the mean time, %| , I have some restriction on my network. (using smb)
I only have acces from Windows to my 'share node' directory on Linux and not from Linux to Windows ( problems of password )......is your script can run in such conditions (!)....and if the answer is 'yes', what am I doing wrong ?

(maybe my some wrong read/write permissions )

[ideasman42]

Probably try and get the new CVS blender render server running. There are some Bugs in Blenders python API that prevented me from finishing this script to work well as well as the face I only do realtime modeling so I havnt got much reson to test the script.
Sorry to give up on this project.. its just not somthing I need at the moment.

[kaso]

I'm trying your script. It seems works very fine. Tnx for your work.

[greboide]

karpov, this can be a matter of taste but i wouldnt use ubuntu if i was you try gentoo instead, these permissions trouble i guarantee you will have no more and your machine will be faster too.

Cambo you did a great job, i think that you should finish it when you have time to it, the worst things that we do in life is quit our projects, it gets that bad taste in the back of the tongue, how it is working you can stop revising it i cant tell you what to do and not to do far from me. I will test this when i have the time and the machines, about the server /client thing wouldnt it be best to dont have any servers at all, only clients? i dont understand the minimum about net protocols and these stuff to know if what i am asking is ridiculous or not so take it easy.

[proghettolab]

Hi cambo,

I'm trying to use your script with 2 PCs: one linux slave (debian) and the other WinXP master. In WinXP when I set the Network Patch all goes well, but the same thing in linux returns this message:

Error - python script error: check console, but I haven't window console and I don't know anything about.

Why this error?

pMB

[proghettolab]

Originally Posted by proghettolab Hi cambo, I'm trying to use your script with 2 PCs: one linux slave (debian) and the other WinXP master. In WinXP when I set the Network Patch all goes well, but the same thing in linux returns this message: Error - python script error: check console, but I haven't window console and I don't know anything about. Why this error? pMB

Maybe is the python version. I controlled blender dependences and are both 2.3 and 2.4. I have to wait for the new blender version with only 2.4 dependence. You don't?

pMB

[kattkieru]

Originally Posted by proghettolab I'm trying to use your script with 2 PCs: one linux slave (debian) and the other WinXP master. In WinXP when I set the Network Patch all goes well, but the same thing in linux returns this message: Error - python script error: check console, but I haven't window console and I don't know anything about.

Prolly because you launched Blender from Gnome or KDE... open a terminal window and type "blender", then try again. When the error pops up you can go back to the terminal window and see what the issue is.

[proghettolab]

Originally Posted by kattkieru Prolly because you launched Blender from Gnome or KDE... open a terminal window and type "blender", then try again. When the error pops up you can go back to the terminal window and see what the issue is.

The message is:

Using Python version 2.3
Unable to load: libtiff.
Try setting the BF_TIFF_LIB environment variable if you want this support.
Example: setenv BF_TIFF_LIB /usr/lib/libtiff.so
Traceback (most recent call last):
File "<string>", line 765, in ?
File "<string>", line 760, in main
File "<string>", line 115, in configMe
File "<string>", line 72, in getConf
File "<string>", line 59, in newConf
IOError: [Errno 13] Permission denied: '/home/arvi/.blender/scripts/bpydata/bfarm.conf'
Vertex3f: 1
Vertex3fv: 2
Normal3f: 1
Normal3fv: 1

The script wants python 2.4+ How can I set 2.4 (that I've installed) instead of 2.3?

Any suggestion?

pMB

[ideasman42]

permission deny means you dont have permissions to write the file. try login as admin?? or install your blend file in a different dir.

[JA-forreal]

This is a great network rendering script Cambo, thanks! I look forward to any updates that you might make to it. It would be nice to have this script included as a regular Blender script.