To counteract that, I set up a grayscale normal map layer in texture paint, and I’m trying to bake some AO into it to accentuate the geometry via normal map.
Problem: when I hit the Bake button, AO is being baked into every texture but the grayscale one. On a scale from very to world-endingly, how wrong am I doing it?
First, be aware that you can select specific mesh parts and make them sharp. To do so, enable autosmooth in the mesh properties, let it at 180°. Then in edit mode select some vertices, edges or faces and in the toolbar > shading tab, you’ll find some buttons to make them hard or smooth. The hard edges become blue.
Second, if you bake on the low poly instead of a temporary highpoly, it will look awful.
For a face model I would at least put a subsurf modifier (with several divisions), add some crease edges here and there and bake the occlusion and the normal map (in colors). Then I would delete the subsurf. The results are way better this way.
As for your baking problem, ensure you have an image file for it and you have it selected in the node editor. Oh wait, You’re on Blender render. I don’t know a lot about it but I think you would have to select the right image in the texture tab or put it on top. At least you can temporarily go in cycles for the baking.