AO simulates totally ambient lighting where there is no light coming from any specific direction, so it makes sense that it would “fight” shadows. AO is a dirt shader that darkens creases to kinda simulate shadows (actually more like a lack of radiosity and spec from angled light into corners) - but not shadows. You should either bake AO, or make two render passes - one with shadow and the other with AO, and then composite them however you want to get the effect you want.Another option would be to use AO "Sub"tract.