If timestamp and/or size are different on each side, rsync has to check checksums of each side. This might be very slow if the empeg is involved, like you have seen in the case of editing id3 tags.

Pim