Unfortunately, that's expected behavior. I doubt that you have a default gateway assigned to two different interfaces or IP addresses. I'm sure that you've just got two default gateways for the whole machine, and it's choosing which one to use via some internal algorithm that doesn't take source into account.