It figures out which is the best based on certain criteria that I don't know the ins and outs of, but I imagine it includes round trip time as a significant factor. I'm guessing that those machines are all topologically close to each other and similarly configured, so I suppose it makes sense for it to flip-flop. It doesn't really make any difference, as it's always getting time info from all the servers you have configured, and they should all be close enough that it's not going to make any difference on the time.

So, short answer, it's probably perfectly fine.