I stumbled upon what seems to be the file for the material forest scene so that I can use it in the background for rendered mode (the one that actually matters). But, it’s in an exr fie that seems to cause a glitch. Whenever I rotate my view by any amount, the background blinks random colors. Don’t know why.
First check that you have perspective projection in your viewport enabled. Using orthographic projection causes similar behaviour to what you’re describing.
Most (I think all even?) of the default HDRIs come from hdrihaven.com, which is a great site with tons of HDRIs under CC0 license. I think the HDRI you are referring to is this one: Ninamari Teien.
If your problem is caused by the file type, using the one from the website should fix it. The website also offers much higher resolution versions 
It is strange that problem you are talking about. You could share screenshots and .blend file.
About what you ask in the title, being in Material View mode, you open the drop-down menu on the top right “Shading”, you click on the little gear icon. In the new window that opens, you scroll down and there you will find an item HDRIs and on the right an “Install” button. It is not convenient for you to use very high resolution HDRIs there.
Edit:
Just in case I understood the opposite of what you are asking:
You can select your HDRi also from World tab:
Also you investigate how to manage nodes to rotate the hdri, or adjust colors.
Blinking “random” colors does sound like orthographic view indeed.
That said, I have had some glitches lately with the later versions.
Garbling screen, objects going semi-invisible in rendered preview etc.
For me, it helps just going to solid then back to rendered view.