I think this is because you turn view around a location somewhere outside your model.
The easiest fix is to select the model and press <num block ,>. This centers the view aound the current selection. It even works on vertices ( in edit mode indeed).