Did you animate the alpha of both colours in the colourband?
As a last resort you can always move the object to an invisble layer (key on frame 2 on a visible layer, key on frame 3 on an invisible layer) or move it behind the camera, but those are of course ugly solutions which shouldn’t be necessary.