I noticed my renders were really fast, until I put in a plane with a picture for a background . Suddenly, the renders start taking hours. The picture is a high resolution picture, but I cant figure out, why a picture is so much more render intensive than an even higher density angular map.
Its an interior, with a good deal of glass.
Both of them are used by the raytracer, and it has something to do with my windows, but without the background picture, the windows add little time.
The same happens here when I add a plane even with no textures.
When I scale the plane making it bigger, I notice that the renders become longer.
I think it’s due to the bouncing of light on the plane, since I usually use ambient occlusion.
So, it makes a sense: larger plane, more bouncing light to calculate, increased render time.
Anyway, I’m not an expert it’s just my thought.
I dont know if blender can cull AO calculations. I am using it, but I turned off almost everything shadow related. I turned off AO, and it did seem to cut the time by 66%.
This is caused by raytracing. I don’t see inside the problem very well, but it’s some kind of searching issues in the Octree.
However, you have to disable the Traceable option for all big objects. Or move them to another render layer. Heavy detailed small objects can’t be raytraced at a time with simple huge billboards.
If you think this sucks, support my idea on Blenderstorm 
Or another one by Robert Tiess.
I think moving the background closer to the rest of the model helped. I kind of bunched everything together for the final render, and it didn’t take nearly as long as I expected, considering the bump up for OSA and the resolution size.