So I removed True Directional Movement and all the other mods that were required for it but I'm still having this same issue. They would also just not be connected to the shoulders while performing actions in general. The physics were fine but they would kind of wiggle around the shoulders when I walk or run. When I originally installed TDM, I had an issue where my capes would move out of sync with my character.
I'm not too worried about having the old Skyrim combat and more importantly, I'm really sick of trying to get mods sorted out and I just want to play the game. I wanted to try out True Directional Movement on special edition and ultimately decided that I didn't want to keep it.