It might be because of the Orthogonal mapping.
As far as I know thats a mapping relative to the three orthogonal-axis.
if you place something outside the centre and rotate around it the dimensions get distorted.
I don’t thing it’s a good mapping method for this kind of thing.
They usually use it for 3d birds eye view stuf.
Just like in a lot of adventure games.
Clear location of your empty and your cube. When the camera is in ortho, the size of objects comes from the distance from camera to global origin. When your empty was not at the global origin, the camera would get closer and then farther form global origin.