Actually, that is exactly the correct way to test encoders. Some of the best double-blind tests have been done this way. I actually saw one published in an online article. Unfortunately, I don't have the link to it any more. Still, it was a perfect test protocol.

The protocol for the test was complex, and it involved multiple CDs with the samples in a random order, like this:

- Original WAV
- Encoder A
- Encoder B
- Encoder C
- Original WAV

The first one (the "reference" track) was always the first in a given group. The last four were always randomized on the burned CD. So the person listening to the CD did not know which of the remaining four was simply a repeat of the reference track.

Several CDs with different randomizations were made and assigned serial numbers. The order of the tracks were kept in a spreadsheet, but the contents of the spreadsheet were not made public until the end of the test. The listeners only were told to compare each of the four successive tracks to the original reference track and then rate them.

The repeat of the reference track among the comparison tracks was the "control". It should have always gotten a perfect score. Sometimes, it didn't. One's ears can play tricks.

__________
Tony Fabris