Radeon Pro Duo slow in Cycles?

I recently purchased a Radeon Pro Duo (polaris) after watching this video: https://www.youtube.com/watch?v=To_N2xT6Xa8

The thing is, my rendertimes with it are far higher than CG Geek’s, and rendertimes are often unstable.

Example with the BMW benchmark:
CG Geek (at 5:05 in the video): 101 seconds
My first render: 227 seconds
Second render (just after, without changing any setttings): 180 seconds
My GTX 1070 in the same build: 187 seconds

Also when I used the AMD card with the newest driver (Radeon Pro Software Adrenaline Edition 17.12.2), I kept getting this error message when I tried to render bigger scenes:

AMD GPU "OpenCL error (-4) CL_MEM_OBJECT_ALLOCATION_FAILURE

I then downloaded one of the older drivers (17.Q2.1), and got rid of that problem.
But the card is still way slower than expected.

Another thing I noticed is, that the AMD card is really slow when rendering a completely unlit scene.
Whereas the GTX 1070 just plow through, rendering it in a few seconds, the AMD card uses 163 seconds rendering the black tiles, which is just ridiculous.

I am, of course, using the latest version of blender, where support for the AMD cards are said to be ‘on par’ with the CUDA-based ones.

My specs are:
Intel Core i7-4790K 4.00 GHz
AMD Radeon Pro Duo (polaris)
MSI Geforce GTX 1070 OC edition, 8GB GDDR 5, 256-bit
16GB RAM