raytracing / rendering question

I’ve got a question about raytracing (at least I think it is raytracing related). Currently I am rendering my project (the cathedral/dome from the WIP) for cubic QTVR (6 images: front, right, left, top, bottom). The question is: why do the frames that show only the bottom floor or a wall take longer (up to 25 %), while the frames that show the whole rest - columns, pillars, ribbed vaults etc. - need much shorter render times. I thought that it should be less rendering time when all that is in the camera focus is a plane (well, with texture of course). But obviously I’m wrong.
What is happening there?

If the parts with pillars have the background showing behind it, then it’s gonna take shorter - it’s much easier to render a simple background compared to an intricately textured plane.

Another guess is the procedural textures…there’s more detail to render in a textured plane than a pillar.

Otherwise I’m stumped. :-X