Is there any solutions to make my character moves in the right way? Million thanks
You will have to go and modify the bone weights by hand. This is why people usually model in T-Pose, it makes automatic weights much easier to rig.
if you really really really dont want to adjust weights, you could delete the armature and all associated vertex groups, and then go back and adjust the arms into a T-pose, and then make another armature with the arms in the right spot and try automatic weights again.
Thanks a lot for your advice!