Frustrating object deselect experience.. like why is this so rubbish compared to maya, 3ds etc?

[Ztreem]

I found this addon, it should do what you want.

https://blenderartists.org/t/ray-cast-selection/1141012