How about mounting a pair of phono sockets or a 3.5mm stereo jack socket on the rear of the PC and wiring them to the internal speaker wiring? The levels would be too high for the HiFi but you could easily fix that with a pair of series resistors and a preset potentiometers.

This obviously won't suit you if you're a HiFi buff but it's cheap and easy. You could even leave the socket dangling at the back of the PC rather than start drilling holes and generating swarf.