HELP: How to randomize cycles displacement materials across particle system objects?

I am trying to use a particle system in Blender to create a field of objects (in this case bumpy rocks) with a procedural texture material on each object where each object has a randomized bumpy pattern (using cycles “true” displacement).
Based on feedback from another thread (and a discussion on Stack Exchange), I was able to get the Random output from the Object Info node to modify the way a procedural texture is mapped to a emitted object (which is extremely helpful). See below to see layout (where I display the procedural texture as a diffuse color rather than displacement):

https://i.stack.imgur.com/i1ZIK.png

As you can see in the image above, the procedural texture pattern appears different for each ball, which is awesome (or at least could be quite awesome if I made a better-looking procedural texture lol). However, if the exact same setup is used for displacement, the patterns are identical on each ball (in other words it doesn’t work).

https://i.stack.imgur.com/fFrLw.png

Some feedback I’ve gotten would indicate that what I’m trying to do is currently impossible in Blender (shout out to @Secrop for being very helpful in this discussion) , but I am definitely all ears for workarounds (or better looking forward to hearing what I did wrong here that would make this work if rectified).

One workaround is to covert the particle system into a bunch of separate objects (though that does seem to be extremely slow and of course not as convenient).

Here is a link to download the file:
https://drive.google.com/file/d/1GkfZMhIYJId7pX0k0A8BxKn2HUa8WY_m/view?usp=sharing

Thanks in advance!

You know what the problem is… but it’s possible to minimize in pratical terms.

For example, if your particle system is from a group of 100 or 200 different objects, you can use something like 10000 particles and still have a great diversity that fools the eye. I mean, 200 objects for cycles is not that much, and unless your public is somewhat autistic, most people won’t even notice that there are repetitions from a group of more than 100.

That’s a good work-around because I can have the colors and bump maps vary across all objects, but actually have the displacement vary across 10-25 objects. That breaks up the monotony. I didn’t know you could have particle system emit a random selection from a group of objects. That’s quite useful! Thanks Secrop!