The problem with the mp3 walkman idea, is that you get about 30 mins of decent audio, unless you want to spend hundreds on flash cards or micro drives..
You could probably put an amp in an empeg sized device (I have an old Panasonic 30 watt sitting at home which is the size of a 2.5 inch HD) if designed it with only one HD slot.. However, you'll run into problems with heat and also computer device interference...
My JVC head unit has an option to turn off the internal amp to reduce these problems, which I do, as I use a big external amp...
Jazz