As Ikari said, Non-Color maps are not really colored maps even if they have color. Normal maps, bump maps, SSS overall maps, glossy roughness maps, displacement maps, etc. are generally Non-Color because they aren’t affected by the Linear color space inside Blender. They are being applied as information (how high, how deep, as values between 0-1, etc.) where color maps are anything you visually see with your eyes that describes how light would reflect off the object and give it it’s color. This is, diffuse color, glossy color, refractive color, etc.
I’m not going to go much into linear vs non linear besides the fact that monitors are designed to see things in a strict way. Black has a value of 0 and white has a value of one. Another way of putting it blue, red, and green can go no higher than 255 with white being 255, 255, 255 and black being 0, 0, 0. This is non linear. Linear tries to mimic the real world where white isn’t just white sometimes. It’s REALLY white.
What happens when you select Color vs. Non-Color is that Blender’s Color space is Linear. The image you created as a texture was most likely created in non linear color space. To rectify this, 3d artists used to have to attach a gamma node after the texture and before the color swatch and set it to .45. You would have to do that for EVERY color node which, holy crap if you miss one and it bugs you, “Why does my texture look all washed out.” Well, you forgot to add the gamma node.
With Blender you don’t have to worry about that anymore. The “Color” or “Non-Color” dropdown is essentially just asking if you need to use a gamma node set to .45. If it’s a color map, yes, set that textures gamma correction to .45. Is it a bump map? Set it to non-color and do not apply the .45 gamma correction. What you should be worried about, however, is how to crush blacks and clamp whites in Compositor because when you output it for everyone else to see, it will need to be delinearized. In other words, you’ll need to try your best to fit the color ranges back into that 0-1.0 value or 255,255,255 color space.
So why the fuss? There are a lot of advantages. Color correction gets a big boost because you don’t have to lose as much color information which helps for several reasons. Most importantly is that the world doesn’t work within confinements of numbers and when trying to reach photo realism in 3D, it also helps not to have a color space confinement. It allows for lights to match actual real world numbers and Global Illumination calculations receive light above 1.0 values so it can help light the scene when the light bounces around.