Multiple threads==two scanlines?

The other day whilest sitting on the ThinkTank™ (you can guess what that is ;)) and pondering Blender and some of the new features in 2.37 I remembered threads support. I hadn’t given much thought to as I don’t own any dual core or multi-proc systems but an old idea I had about network rendering came into my mind.

I had an old idea of network rendering for just stills. Right now from the way I understand it rendering across the network for blender renders an entire frame in a single scanline. If this is distributed among two or more computers rather than lightening the workload it just says: “you render this frame and I’ll render this one, report back once you’ve done this and I’ll give you another.”

That’s all well and good but doesn’t do a bloody bit of good for stills.

My original idea was to use xparts/yparts and use the same concept as above, so rather than doing whole frames it does smaller sections of each frame so it’s still effectively sharing the single frame.

Back to the ThinkTank™. I read somewhere that enabling the “threads” button actually starts two different scanlines to render your image. So I thought to myself: if this is possible on a single dual core chip (or multiple CPUs too) can this be done over a network? If it’s possible to render a single frame in two scanlines locally can it be done by sending n number of scanlines over a network to waiting render nodes?

Is this feasible? Or more accurately: is this even worth it? Which idea is better? Which is easier to implement…if at all.

Let me know what you think.

The two scanline threads still render from the same data in memory, working accross a network, you’d need to maintain data integrity if nodes update data in the shared memory.

Martin

Very interesting idea, Dittohead. Quick thought: could field rendering perhaps make itself useful over a network or does it operate on the same principle as what Theeth said?

The problem is that before actual rendering begins, which makes use of multiple threads, all of the precalcing stuff has to be done, like geometry calculation and shadow map generation. Each thread needs to have access to this info as it renders, and it just uses the hardware on the mainboard of the computer to do it: it’s just calls to and writes to the same memory space.

What you’d have to do is get Blender to allow as many threads as you have processors available (two is the most on any consumer machine you or I can buy), then create a network of Linux boxes (with gigabit Ethernet) running as a cluster. Once that’s going, it could possibly work.

The problem is that before actual rendering begins, which makes use of multiple threads, all of the precalcing stuff has to be done, like geometry calculation and shadow map generation. Each thread needs to have access to this info as it renders, and it just uses the hardware on the mainboard of the computer to do it: it’s just calls to and writes to the same memory space.

it depends how “long” the render will be as to whether this is worth it.

for a render that will take over an hour. the few seconds- minutes of precalc would be worth it to save 30 mins on the hour render.

its only once you get past that threshold of value that things become a problem.

the only stills you would need to thread across networks are the damn long ones anyway.

Alltaken

this would be even cooler, if you could just use an entire networked computer system as one multithreaded machine. I would enjoy this, since i have a 50 port switch and a bunch of old hardware sitting around collecting dust. An asymetric networked multiprossesing system, then you would just use the kernel to pass out all the blender threads around… only thing is, someone has to write that kernel… hmm… i sense a new linux branch…

Um… Beowulf?

i was speaking in more of terms of having one universal OS as opposed to a cluster of many, but yeah… Now if we can only get one of those superclusters for rendering…

Thanks for the answers guys, I understand a little more now.

It is called border rendering in Blender jargon - go to your camera view and press shift+b to try it out “the manual way”.

While working on some code for BURP the past days I tested the idea about using multiple computers to render a single frame. Splitting a frame into 4 pieces for frames that take about 2 hours to render seems like a good idea. The precalculations usually don’t take more than a few mins, so you get quite a speed increase. For frames that take more than a day just up the number of pieces.
For some reason it is sometimes faster to cut it using horizontal cuts than vertical ones… must be something to do with optimization…

Hint for those who try it out themselves:
For the linux people you can easily combine the parts afterwards by using imagemagick:
convert ./part1.png ./part2.png ./part3.png ./part4.png -coalesce output.png
Remember to select RGBA output in the render pane.

(I was using a modified version of Blender, made for BURP, to get it to render only part of the frame by using commandline arguments)