In your situation it's rare (but certainly not impossible) for PSUs to exhibit this fault behaviour.

IMO you need to provide more feedback regarding your troubleshooting vs. the suggestions already given to you in this thread to get anything better than a mere guess at either: CPU, motherboard, VGA etc. etc.