Wikipedia says that Optix can use “CPU memory”. What does it mean? Doesn’t Optix rendering fail when VRAM has run out?
Yes, that is what it means. It will lose a bit of performance but it will render using the system memory, in theory.
“With CUDA and OptiX devices, if the GPU memory is full Blender will automatically try to use system memory. This has a performance impact, but will usually still result in a faster render than using CPU rendering. This feature does not work for OpenCL rendering.”