How to prevent world background to be visible through a glass material? (blender 2.79)

I used a world background as a purplish color, but I dont want it to be seen,
I used the glass material on the windows at the right of the picture, and I select film --> transparent, under the render tab.
I remember there is a way to still use the glass material without absorbing the world background. How do I do that? !