Falling in succession!?

I want to create a animation where the pieces of the object falls in succession (bottom-to-top).
The idea is to use the physics simulation to simulate the falling and hitting the floor of the pieces.
That I can figure out how to do, but I need help with the other part: not having all the pieces fall at the same time.
Another way to explain it is that I want to have the object “crumbling apart”.

To illustrate the falling effect I’m after I did a simple animation: http://chris.hindefjord.se/Blandat/falling.ogg Of course extremely simplified…

Any ideas are welcome! :?D