Go to Object Tab of properties of duplicated object.
Under Relations Extra panel change its tracking axes. Set axis to +Z instead of Y and Up axis to Y instead of Z.
Default settings are working well for constraints but not for particles.
Or
You can edit this object and rotate its mesh to have its height aligned to its Y axis.