Yes, the HDRI is in the world node setup, but that doesn’t mean you cannot add anything else.
You can set the background to not be transparent and render out different colours/shaders depending on whether it is seen by the camera or reflected around in the scene. This can be either done with a mix RGB or mix shader. I use the mix RGB below, but both work fine.
What you can do is to add a mix RGB in the world node setup and mix it with the desired visual background. The mix factor determines which one is used, so you can plug in a camera ray light path in so that only the background visible to the camera will be set to the background colour.
Mix RGB has one limit compared to mixing shaders - you cannot have individual background strengths.
The alternative is to do this in the compositor. You can render out the scene with transparent background on like you already have done, and use the alpha over node to replace transparent parts with a colour (or image) of your own: