I just played it some more. All was well until I was looking at a merchants menu and the mouse went wonky. I had to turn the speed way down but when I got out of the menu it was fine again. Then at sea I got into a battle and there were no problems at all in the battle, the boardings, or the rest of the voyage. I'm going to play POTC for a bit and see if this happens there.