You also have to model moving all those points if you want to animate anything.
As for time taken to render: as pointed out, most real objects bounce or absorb light in the first millimeter or so. So you could, for most things, render a very thin, dense point cloud just under the surface (like using the solidify modifier with a negative offset) instead of trying to fill the entire volume, and still be physically accurate.
But for very bright lights or thin materials, things need to be rendered more deeply. And things like water and air need to be completely filled, though not so densely as solids.
Which brings up another problem: even though air is very sparse compared to a solid, there’s a lot of air volume. Air is about 1.2 kg/m³, where human muscle is about 1100 kg/m³, and steel is like 8000 kg/m³.
A human body takes up about 0.07 m³, while the volume of air surrounding a human in a scene is at least 2 m³ (a 1m x 1m square the human is standing on, 2m high), and that’s just where they’re standing. So you’ve got 1.2 kg of particles surrounding a human who is filled with 77 kg of particles.
Seems like not much, but even a small bedroom is 30 m³ (about 3m x 4m floor, and about 2.5m high), or 36 kg. Move up to a 2-car garage and we’re around 170 m³ (7.5m x 7.5m, 3m high), or 204 kg. A typical Wal-Mart store has a volume of almost 90000 m³, or 106000 kg of air.
Start rendering parking lots, forests, little houses on a prairie, and so forth, and the number of particles in your solid objects become irrelevant. Put a lake in there and you’re done.