The speed you are experiencing can vary from distro to distro.
Older versions of Mandrake and Suse are known to be sluggish.
Secondly, the voodoo chipset is dead. Under WinXP I always had to use unofficial drivers to get some decent performance out of it. Worked ok under linux. But the linux performance for THAT chipset was worse than the unofficial drivers for windows. As far as I know, the windows drivers where modified hacks of voodoos original drivers. Point being that even a simple geforce4mx which is dead cheap nowadays will get you a huge boost in opengl performance, both for windows and linux.
Another thing that is important to consider, you have to make sure that under linux your OpenGL accelaration is activated. You can test that by issuing the following command in a terminal window:
glxinfo | grep rendering
if this produces “direct rendering: Yes” then you’ve got accelaration working.
Another thing to consider is which version of Blender itself you download.
For linux there are always two releases, one with shared libraries and one with static libraries.
The one with static libraries does not use 3D acellaration AT ALL! If you use that version, screen performance is gonna suck big time. Especially when you consider that Blenders entire interface is done through opengl.
You say you have only 256mb ram. That can be quite little when using mandrake or suse. These are bloated desktop distros and have way too many background services activated by default. In other words, they will certainly suck up more of your RAM then other distros which are more conservative with the services they run in the background.
If your up to it, you could try debian, or if your not ready for that yet, you could always go for ubuntu(gnome) or kubuntu(kde).
Personally, I use gentoo, but that might be too extreme for you. (You compile everything from source and define exactly what you want down to the finest detail.)
Now about the Windows/Linux swap file thingy.
Windows uses a variable swap size. This is good when you dont have a lot of ram and your apps need more, then windows will dynamically adjust for that. BUT!, and thats a big but, when rendering, you absolutely do not want to use swap. You want to make sure that your scene can load into ram entirely, otherwise it can drastically slow down your rendering performance.
Linux on the other hand uses a separate partition for swap. This alone has a speed advantage, since the harddrive doesnt have to look for a specific file inside a partition, but simply jumps to the partition directly.
If you put your swap partition at the beginning of the harddrive, e.g. as the first partition, you can gain a bit of a speed boost.
The disadvantage is of course, that your swap is set at a certain size. If your needs should exceed what you have reserved for swap, then blender or yafray, or any other app for that matter, will most likely crash.
However, like I said before, in any case, you wanna make sure you hit the swap as little as possible.
Some people notice that no matter how much ram they have installed, linux will try to take up most of it. This is sort of true, what linux does is use a lot if ram for cache. This is not really considered true ram usage, since as soon as an app requires more ram, the relevant portion of the cache is emptied immediately. The cache is just there so that programs can be reloaded more quickly.
From my experience, linux has always run faster, no matter on what I installed it. Unless the grafx card sucked and wasnt properly supported under linux, then desktop and 3d performance suffered.
As with all things, your mileage may vary.
However, I have certainly noticed that blender is way more stable under linux than under windows. But once again, your experience might be completely different.
In the end, you need to choose whatever works best for you. Be it windows or linux.
Good luck.
p.s.
Pleasepleaseplease get rid of that old voodoo card. Its a disgrace to your spanking new sempron! 
. I also hate KDE, it seems to run slower than Gnome (in my opinion). On my AMD64 rig, I always used TWM (please dont point fingers and laugh! :D), it didn’t seem to work with Gnome or KDE. Also, I think that the real-time performance of Blender seems to work a little better on Windows (or maybe I’m just crazy). 
If you can’t seem to get it to run faster, you might want to try compiling Blender yourself (the new ones seem easy enough to compile, even on AMD64). Your real-time performance is probably caused by graphics driver problems, if you say it runs better on Windows. I have a 16MB Nividia Geforce 2 GO in my laptop, with worse specs than yours (1.5ghz celeron, bleh…), but it could still run Blender like a charm. And it worked much better on Linux I might add! Or maybe you just need to upgrade your RAM like Dittohead said