People can send me email attachments of any size and because of how POP3 works, I have no way to know I do or don't want the file until after it's downloaded.

I agree with everything you said, Tony, but I wanted to point out that POP3 isn't the culprit in that case, it's your mail client software.

I use a program called PopTray to check my mail, and it will show me the message headers (and also indicate whether or not the message has a file attachment) without downloading the whole message. Most of the time, I can tell if the messages contain desirable content just from a quick scan of the from/to/subject fields. I can then delete any of the messages from the server before launching my e-mail client and actually starting a download.