As far as I know a gps derives direction from difference between the current position measurement and the previous measurement. So it shouldn't matter at all whether the antenna is upside down or turned sideways.

The only effect of holding it upside down is that the metal of the mounting bracket might shield some of the satellite signals. The ceramic piece on one side is the actual antenna, so I guess it would work best when that is facing upwards.