My understanding was that they were punished for worshipping a god other than yaweh (granted, part of the worshop was via a whole bunch of sexual rites, including homosexuality among others).

The Bible states that the men of Sodom were "exceedingly wicked and sinful against the LORD" (Gen. 13:13), and that there were not even ten who were considered righteous in the entire city (Gen. 18:16-33). You are correct in saying that the city of Sodom was not destroyed specifically because of homosexuality. However, homosexuality did abound, so much so that it is mentioned in the Bible (Gen. 19:5). I am not aware of any translation that states that Sodom was specifically destroyed because of the abundance of homosexuality.