Rigid body simulation instable

Hello all,
I am working on an animation, which consists of a number of sticks arranged as in the screenshot:

All objects are active, the floor is passive. All objects have Location, Rotation and Scale applied. They have been positioned manually to fit perfectly.
The “fun part” of the problem: A scene only containing the objects and the floor is about stable. Although there are no other forces and the objects are perfectly aligned, they move a little but that does not have any negative impact on the animation.
Having set up that physics scene I am trying to put the animation to the next level: I have added a domain object around these objects and a smoke simulation.
The not so funny part: Now the selected object instantly starts to jiggle and falls down after 10 frames.
There is no interaction between the smoke simulation and the objects. And even funnier, the wrong behaviour remains wrong after moving the smoke objects to another layer.
I don’t know what else parameters I could change:
There is no object bounciness defined. In the Scene settings, speed is defined to 1, steps per second 200, solver iterations 500. There is no change if I change the number of iterations to 1000. Changing the steps per second to a low value around 20 lets the pyramid crash completely. Negative impact is the larger the bigger the value is: If I change the value to 500, more objects become instable. If the value is around 60, the instability starts at a later frame.
All objects have the same weight of 2kg, collision shape of the vertical objects is a box as the objects are more or less “boxy” with a bevel modifier. (It does not make a difference if I apply the modifier or not.) Collision objects of the horizontal objects is a convex hull. Side fact: If I change the collision shape to mesh, all objects seem to explode right at the second frame… Another finding: defining a mesh and a Collision margin of 4cm lets all objects come to rest “flying” 4cm above the floor.

Any ideas beyond my desperate trials?

Thanks a lot in advance.

maybe sharing the .blend would help

I have scaled all objects by factor 5. The simulation becomes more stable with that. Nevertheless, it is not really convincing. I have rendered a “teaser” animation, which you can find here: http://www.youtube.com/watch?v=9rtb67o92G4 . Some of the sticks fall through the table…