You could try finding some different lower-impedance headphones. I don't know what the specs are on the earbuds that ship with iPods these days, but it's worth looking into.

Edit: The Apple web site says they're 32Ω. You could try the Denon AH-C351K, Etymotic ER-6i, or Sennheiser CX300-B headphones, which are all far lower impedance. Or you could try a portable amp, but that's probably not an ideal solution.