Model doubles when using the Mirror modifier

Whenever I use the mirror modifier, I end up with double faces. I find out when I move/delete a face and there is another face in the same position.

How can I prevent this?