It’s because of this vertex, and others like it.
The normals for the vertices are determined by the angles of the edges connected to them, which is a straight edge going up and down and another at a sharp angle, so the angle of the normal used for smooth shading is a mix between them both and this results in shading that corner of the face as if it was pointing a different direction from the other corners.
To make the sides look flat you have two options:
Add an edge loop around this bevel with faces that are flat on both sides, I’d do this by selecting the edge and adding a 2-segment bevel with the profile set to 1.0 so it’s flat, make it wide enough so that the new edge inside the inset area is like halfway between the two existing edges, confirming it, and dissolving the new edge the bevel made inside the inset area. Really you can make the bevel used to add another edge loop as wide as you want, just be aware that the shading between the inset area and the flat sides will be blended together within this area.
Use something like Y.A.V.N.E or Weighted Normals Calculator to make custom normals for those vertices based on the angle of the larger face on the side. YAVNE gives you finer controls and will let you just modify the normals for a specific area and set how important different faces are when it comes time to decide which face to use for custom normals, Weighted Normals Calculator will do the whole object automatically. Custom normals don’t work with modifiers so you’ll have to apply your mirror modifier and then use it.
It’s high time we had a toggle for face weighted normals.