Per object/material indirect passes?

AOVs and Cryptomattes are great but when you use either one of them to change the color of an object or material, the indirect light remains the same, leading to inconsistent indirect lighting, as shown in this glaring example :

Original render:

Color change using cryptomatte (the floor is blue where it should be yellow):

How can this be addressed?

Only way I can think of is to render out in isolation. Image based on light A + image based on light B = final result.

I used 2 spot lights to make the indirect light more prominent for this example but in most situations the light source(s) is/are shared between multiple if not all objects, especially when using HDR lighting. So this wouldn’t work.
Of course I can just turn on and off Glossy and Diffuse in the object properties and make multiple renders but I was mostly wondering if there was a single render solution like cryptomattes and AOVs provide for direct lighting.