As far as SACD and DVD-Audio go, there are many hurdles that are more significant than customers wanting higher-quality sound.

First, most people still have, at most, a CD player in their cars. Most people probably consider this when buying music, as much of their music listening time is probably in-car. So that puts DVDA out of the running right away. Some SACDs also have CD data written on them so that they can be read by regular CD players. But then you have to consider the fact that such a disc is significantly pricier than the same title on regular CD. And these people have no interest in buying a new player for home, either, especially one that's going to cose several hundred dollars.

Second, where are these high-quality masters coming from? I could be wrong, but I think that most music is mastered at qualities that don't read the level of what SACD is capable of. Of course, you could go back to pre-digital mastering, but then you've got a big noise floor in almost every case. That can be compensated for, but it takes a lot of effort and is going to only be applied to back catalog titles that are likely to sell well, and, honestly, I've already owned at least three copies of Dark Side of the Moon. That leaves new recordings, that the studio knows is going to sell well, and I've got zero copies of the latest Britney album and I want to keep it that way.

Third, the real advantage of these better formats isn't higher-fidelity recording, whose advantage is questionable, but super-binaural recording, which means 5.1 encoding on both of those formats, IIRC. Okay, that's not a hurdle, really, but it does mean that there's no advantage to mp3s, which are mono or stereo only. I suppose many of the arguments about mastering can be applied here, too, to a lesser extent.

In summary, CDs caught on not because of better fidelity, but because of convenience. You can't play records in a car, cassettes and 8-tracks are annoying to deal with because of the lack of and/or slowness of track searching. Notably, DATs were close to CDs in fidelity, but had the inconveniences of cassettes. Then again, MiniDiscs were also close to CDs in quality and had potentially better features and still failed. Of course, CDs had a good foothold by then. I suppose that this really ends up confirming what you were saying. Sorry; I got distracted.