I’ve also had this issue before, it’s actually a bit of a flaw in ray tracing renderers. Some renderers tackle this issue by using the REYES pipeline, however that sort of architecture isn’t really smoothly compatible with Cycles. Adaptive subdivision is meant to mimic an aspect of REYES, but unfortunately
it still over tessellates things and is still in an experimental state. It also doesn’t occlude objects outside the camera view, since this could lead to various consequences with Cycles. In general REYES is not really suited to path tracing because path tracing requires pretty much all the geometry in the scene to work.
This is bit sneaky to advertise my work, but I’ve been developing an addon that uses ‘shader based displacement’ so to say instead of tessellation based displacement. It’s effectively like relief/parallax mapping that you see in game engines. The main point is that it produces accurate silhouettes, which replicates true displaced geometry quite nicely. There’s almost no memory overhead for it to work other than the displacement texture itself. It has its pitfalls, but I’d really like to get it to a productionish ready standard soon.
It’s in early days right now, but I have some promising features in R&D and I’m trying my best to develop it on top of university work. I made a thread here for more info:
https://blenderartists.org/t/super-parallax-addon/