How to turn off symmetry in displacement modifier's object setting?

In trying to figure out how to make the displacement modifier respond to the location of an object, I tried setting the coordinates to a basic cube and found it did exactly what I wanted…almost.

For some reason, every time I use this setting, the object setting creates this weird symmetry that makes the mesh’s displacement reflect across the global Y axis and I don’t know why, I don’t see a setting to turn off that symmetry.

I do have a temporary workaround which is to create a vertex group with only half the mesh weighted and assign it in the displacement modifier, but obviously I can’t move the entire vertex group weights in the exact same way with the object, the vertex modifiers aren’t that flexible.

Am I really the only one who’s encountered this?