Choose What Rigid -Body is Affected by a Force Field

I have a conundrum,

I have a bunch of rigid bodies that I would like to travel to different locations.
On their voyage to their location I’d like them to be able to interact and collide with any other rigid-body it meets along the way. (This could be hundreds of rigid-bodies)

One way I thought of setting the target destination would be to use a force field. However, is there a way to make it so the force-field will only affect one rigid body of my choosing? At the moment when I try it it affects all of the rigid bodies.

Something like this would be great…

If not, is there any other method for getting a rigid body to travel to a specific target location but be able to collided with other rigid-bodies along the way?

Any ideas welcome.

Thanks in advance!