For bump mapping Eevee uses derivatives provided by the hardware. GPUs shade in blocks of 2x2 pixels which means derivatives can be computed “for free” by reading neighboring pixels within that block, and I can imagine it causing this type of artifact.
The solution may be to manually compute the derivatives at the cost of performance. Not sure it would even be considered a bug at this stage, there’s dozens of render quality issue like this that could be improved.
For image textures you also ideally need cubic interpolation for smooth bump results, which Eevee does not support currently. But there is linear interpolation, which is better than closest which you effectively get with a procedural brick texture.