I am not a blender user. I work for a computer lab at a university. We have a render farm (Deadline) available for our users. Per request I set it up today for one of them to run with Blender 2.55beta. With applications like 3ds max and Maya I have seen very good utilization of the computers. With Blender I only see 3% of CPU usage on the farm machine during rendering. This looks like as if only one core is being used. Is there way/settings to make more use of the machine?
After looking at that file some more I found the threads. It was set to one. After switching to auto-detect the rendering really took off. For 10 frames the render time went from 1:02hrs to 7 minutes.
We have ten dedicated machines for rendering. Those are dual six core with 24 GB of RAM each. The render farm software lists because of hyper threading 240 cores and 240 GB total. When I ran my tests with the blender file from the lab user I set the task size to 10 frames at a time per computer.
In some ways, you discovered that “you were being constrained by an unrecognized and unwanted limit,” more than you found “the proper setting to which this parameter should always be set.”
Clearly, with the thread-limit (somehow) set to “1,” Blender was prevented from using more than “1” core or CPU within your awesome system. “Obviously wrong, but it is Doing What You Said.”
“Auto,” AFAIK, detects the number of cores and assigns one thread to each one of them.
This is a perfectly appropriate and reasonable strategy, but with one very important caveat: the system must possess enough RAM to actually be able to keep all of the assigned cores simultaneously busy. (And of course… the underlying infrastructure of the machine has to actually be beefy enough to service the simultaneous demands of all of them. Which, presumably, your servers can do.)
My cautionary tale is simply this: it is possible to over-commit your physical resources, and should this (inadvertently…) happen, the pragmatic outcome is considerably worse than “bad.” Obviously in your case, “1” was very wrong. But in the general case, “Auto” may or may not be the best alternative.
Know what your system is realistically, actually capable of … without “overclocking” or other such “salesman’s tricks.”