The amount of data per square inch on a CD is/was fixed. There is no compression. When deciding on the final standard size of the disk, there was a known, fixed ratio trade-off between physical handling size and the amount of music. They simply set the standard size in a place where most of the known "long" recordings would fit, while still offering an easy-to-handle, compact size for the media.

I'm sure that fitting the 9th onto the disc was one of their concerns when making the decision. But it certainly wouldn't have been the only piece they considered.