I have rendered a model using Cycles and Blender Internal
In Cycles the shadows are dithered like real shadows are. Whereas in Blender Internal, the shadows are very sharp and unrealistic.
I tried all combinations including but not limited to Ambient Occlusion.
Why is that the shadows produced by Blender Internals are so unrealistically sharp?
Or do I know less about how deep the rabbit hole goes?
Several options - in BI you can choose to add lights that illuminate, but do not contribute to specular or shadow, and this can soften the harsh shadows you are finding. There are also options to increase the samples for the ray traced shadow buffer I think, but I’m sure you can figure it out if you use a google search to look up threads on blenderartists for that.
But if you choose not to have specular or shadow, then anyway there would be no shadow at all.
Whereas I want soft shadows
No, I don’t think I made myself clear. You would set up your lighting with your shadows, then add new lamps that do not cast shadow to illuminate the area covered in shadow, low power, and that will break up the shadow and soften it.
Enable the Soft Size value in the lamp / shadow settings for supported lamp types
Just turn on soft shadows for your lamps. Shadows are based on the size parameter for most lamp types in BI. You will also have to increase your samples.
Guys thanks a ton. The explanation by all of you was really helpful. I was able to achieve image more or less what cycles created by way of BI soft shadows 
Very minor shadow difference. The cycles shadow still looks more realistic comparatively
Below are the images.
The first one is by Cycles and the second one by BI
