UV Project Modifier - Blurry results

I have a piece of high resolution LIDAR terrain imported, as well as an area of orthoimages. The ortho is on a separate plane which matches the terrain. I’m trying to use the UV Project modifier to project the UVs of the plane onto the terrain below like this:

For some reason the result on the terrain isnt right. After applying the modifier I can see that the UV map has ended up multiple times larger than the orthoimages.

The modifier works on smaller objects in the same scene like this plane and cube:

How would I make it project properly on the huge terrain objects? Does size really matter?
Thanks in advance for any help.