I’m just playing around with the Octane version of blender, testing render speeds and VRAM limitations on both Cycles GPU compute and Octane Render.
I put a 4k exr map as displacement map (adaptive subdivision) on a simple plane and cranked the default 1920 x 1080 render resolution to 400%. I hit render and Cycles instantly used up all 24 GB of my GPU.
With Octane, a similar scene used around 4-7 GB I believe.
So I was kind of surprised that I was able to run into the GPU VRAM limitation so easily with Cycles. All it took was the combination of a single 4k displacement map and 7680 x 4320 px render resolution. Can anyone explain the math behind why this is happening in Cycles?
Not only performance, but there’s a variety of features that do not work with it yet and other things subtly broken: https://developer.blender.org/T53901
For now, you can try the following and see if it helps since there’s a bug where it will double-subdivide the mesh leading to extremely high memory usage https://developer.blender.org/T85030 :
Disable adaptive subdivision on the modifier
Set both Viewport and Render to 0
Re-enable adaptive subdivision
See if that helps a bit (if it doesn’t, you can try also limiting the level of subdivision in Cycle’s Render → Subdivision → Max Subdivision setting)