Clone brush is cloning from material diffuse colour, not texture.

I’ve pressed ctrl+lmb to pick the part of my texture I want to clone from, but it isn’t cloning the texture, it’s cloning my material diffuse colour. (I’ve purposely turned my diffuse colour to red to be sure).
Anybody know how to get it to clone from my texture? I’m in multi texture at mo but it doesn’t work in glsl either.