UV Color Texture from High or Low Poly mesh?

Hi, only been piddling in Blender (well, any 3-d) for about 2 months and I have a question. I am asking this question based on my limited knowledge of normal maps, which is that they are usually baked from a high-poly version of a mesh and applied to the low-poly version to make the model look nice without requiring as much rendering power. If that is wrong, please correct me.

Anyway, my question is, when you UV unwrap to get an image to do your color texture on, do you want to get that from the high-poly version or the low poly? I’m not sure exactly how applying a normal map works, so I want to know if it ‘stretches’ or warps the color texture at all or if it just fakes the appearance of 3-d quality like a bump map does.

All maps are applied to low-poly mesh UV. High-ply low-poly thing is for generating Normal map and AO only.