I know exactly what you mean, and there’s an absolutely fabulous addon to do it (which also gives you a bunch of other really handy camera-control features) called Photographer.
It sets a standard for brightness values regardless of where they come from, images, mesh lights, sun, etc. This way you can use real world values and get really nice results without having to play with it too much. When you change the ISO, aperture and shutter speed of the ‘camera’ (settings Photographer gives you) it will change the DOF, motion blur length (if enabled) and brightness automatically.
As for how forgiving Filmic is, that is part of its design. If you want to more closely match a particular camera, you’ll need to export the EXR and run it through a profile for that camera, though there is a lot to learn if you want to do that stuff right. An alternative is to just crush the whites a little bit after the render rather than trying to push lights so bright that they blow out.
This is because Filmic (if I recall correctly) has 22 stops of dynamic range, while 15 is pretty good for professional photography cameras these days. That means ‘pure white’ in Filmic is more than 100 times brighter than white on a professional camera.