normal map inversion across an image?

I’m having a problem with my normal maps inverting across an image.
Here is an accidental example in an otherwise nice tutorial.
Hi-rez example:
The mortar is correct in the lower right, but as one looks up, the mortar goes flat, then becomes extruded ridges. I’m having the exact same problem, although in my case I’m using a UV-mapped tangent-space normal map on a cylindrical (inside) wall. Don’t know what else to do or if this is just an intrinsic problem to normal maps.
(even then I’m not sure what I can graduate up to)

This happens with normal maps, they don’t create any additional geometry, just the appearance of it. The direction of the light can change the level of apparent depth.


Are there any workarounds? I don’t seem to see this problem in general, especially with real-time engines that use hi-poly normal maps mapped to low-poly objects.