The default cube has a material. If one deletes it (the cube), then adds a new one, the new cube doesn’t have a material by default. It doesn’t seem to need one, so what is the purpose of the default cube’s default material?
The same reason there is a default cube to begin with. The default cube gives a new user something to select and play with. The same is true for the default material. It gives a new user a chance to see the different material options, without needing to add a new material. There is also no other reason to have the world color defaulted to Blue. Blender could easily start without a world material, without a default lamp, and without a default camera. But, having all these things allows a newb to start blender and press F12 and see the wonders of the default grey cube in a blue world.
If you do not want to see the cube each time you start Blender, delete it and press Ctrl-U to save your preferences.
I guess that makes sense. I suppose I’m a bit bemused by the fact that a cube renders fine without a material at all, but I guess having one means that the panels show values at least. It’s probably a good sign that I ask such a question - three months ago the answer would have been obvious!