It’s a bit weird material, consisting of a mix of diffuse, glossy, refraction and another glossy from the glass.
Could enable different passes from render layer properties and look where the noise comes from, and check the values if there are any fireflies. There’s both.
The material has glossy and refraction components with quite a few bounces, so noise is expected. Those also suggest their caustics might be the cause. Disabling caustics gets rid of some of the noise but not all of it. Indirect pass still has some fireflies, and setting clamp indirect gets rid of those and help with the noise.
Another way is to cheat. Could make only the camera see the material like it was, but make the rest of the material where it bounces see it the other way. Simplifies the rays a lot and gives a cleaner result without adjusting render settings. It doesn’t look the same though which is expected.
I didn’t enable denoising in render layer properties, which is an option to reduce the noise further.