deselecting

I am struggling to deselect objects after selecting with the b key. I have selected objects in a box with the b key but there are a few objects caught in the box which no matter what angle i come from will always be there, i do not want to select these few. i have tried shift and right mouse button on the unwanted objects when they are selected but rather than deselecting they change from orange to yellow and back, apparently it has something to do with a stupid ACTIVE OBJECT thing. is there something i am doing wrong or is there a part in user preferences where i can set the program so that by right clicking on an unwanted object and clicking M or H or any other key that is not usually used for this function i can deselect it rather than going through a huge amount of hassle where the object i want to deselect infuriates me by changing from orange to yellow but never deselecting? please help i am used to sketchup where clicking and shift selects or deselects but as there are only two states rather than 3 this is not a problem. blender has to few keys for too many things.

shift-rightclicking toggles selections, when using circle © or box (B) select, you need to use the middle mouse button to unselect

Another approach would be to hide (h, Alt-h to unhide) the objects you don’t want to select or move (m) them to another layer.

thanks the hide option works brilliantly.