Baked ambient occlusion is speckly

I’m baking ambient occlusion on a fractured cube, with large pieces, with the material just being white diffuse. The parts of the surface that are flat and facing outwards, with nothing to be obstructed by, have several black dots on them in the baked texture. How do I get rid of the dots, or at least reduce the number of them?

In the World / Gather panel, increasing the number of samples or changing to Approximate will give a smoother look to the baked texture

If that doesn’t help show screenshots of your result and attach or post a link to your blend file that shows the problem when the Bake button is pressed