Hm. Seems to work here…
Can you upload a simplified scene file that shows the problem and post a download link here for us to have a look? Make sure to include the specular map!