Question about assigning materials

I’m a beginner at Blender but I’m currently trying to make a building. I’m trying to select faces (glass windows) on the building and trying to change their colour but when I create a new material it always gets assigned to the whole building, can someone help me out?

The first material you create for an object is assigned to all of the faces. When you create other materials you can assign them to selected faces