OK, I may have found a source for your problem but not necessarily a solution. It seems to have something to do with the nature of the Orco mapping used for the initial image texture application – except for one of the 6 faces of the cube in your .blend, it doesn’t seem to be stable, but instead changes based on the object rotation, at least in terms of the maps baked from it, which I didn’t expect.
What I did was use the green mat (Orco projection of the plaster) to bake 3 new maps using Full Render, with a 90 degree rotation on Z axis between each bake. Then I looked at the resulting baked images in the UV Texture Editor.
As expected, the lighting for each of the unwrapped cube faces changes with the rotation, since it’s baking the lightmap as well as the diffuse. But what I didn’t expect is that the portion of the image texture (plaster Tex) applied to each face except the top one also changes. The top face stays the same except for minor lighting shifts. I can’t quantify how these shifts are made in terms of the portion of the image texture being used on one face or the other because it’s too hard to “read” the texture accurately, so it’s hard to figure how this may be causing the probs you’re having, but it is unexpected – I would think that the same map would bake out regardless of object orientation, at least in terms of what part of the applied image texture bakes out onto which face.
Further testing shows that the portion of the image baked on each face is not really shifting depending on object orientation, but because the green mat is just a Nor channel bump on an RGB hue, the diffuse texture being baked out looks very different depending on lighting conditions – it took a couple of passes looking at the baked maps up close (pixel level) to see that the underlying pattern wasn’t really changing. Back to square one.
EDIT YET AGAIN_________________
Eureka (I think) – after baking out the map from the Orco material, when you re-applied it to your unwrapped cube using UV, you left the “Cube” mapping option enabled – this should be on “Flat” for UV mapping. You’ll get the diffuse only from the bake, though, so no specular will show and the shadowing from the bump will be slighty different.