Sometimes, but not always, (which is confusing), even something like a simple mesh plane that is subdivided quite a bit and then given a texture material, causes this. So I use an image texture, and have also generated a non-color version to use for displacement. I add the second material and texture (non-color image) and then the displace modifier. After adjusting the strength down, I can see the effect of the modifier. But the problem is that the high and low points of the displacement don’t match up to the colored texture on the mesh. The mesh has been UV unwrapped, and the color texture applied the way I want. I make sure that I set “UV” as the mapping for the displace modifier. Is there a way to be sure that both images map the same way, so that the peaks and valleys line up correctly?
Can you post an example file where that happens?