It's supposed to come from the use of the sword hand: if you pass on the left side of the road (in your carriage or cart), then you are still free to draw with your sword-hand (the right, in most cases, unless you were a witch) to strike down any attacker who may be standing in the road or approaching head on....

Perversely, the English always joust with the short lance from the right hand side, so that the sword arm (the strong one) can direct the lance accurately across the body to the left-attacking opponent. Why? To hit the shield-protected left hand of his opponent, thereby giving him a sporting chance of survival. Of course.

Ils sont fou, les Anglais.