Hi everyone,

I've been using POP3 since the dawn of time and never saw a reason to change it because I did all my email reading from one machine, or through a web client if I was traveling. Once in a while I'd leave town and leave the client running and not be able to get my emails, but besides that rare and minor annoyance it worked, so I didn't fix it.

Well, the iPhone changes all that now. I got one of the spiffy new ones and now I want to have my email with me all the time. I also want my emails to go to my Eudora mailboxes I've used for around 15 years. I think that means the end of POP3.

My other little quirk is that I don't like to delete emails. I have a permanent corpus of emails and I like to be able to go back and search them from time to time. That seems incompatible with the IMAP philosophy of keeping the email on the server. I don't need that on the iPhone, of course -- I'd like to be able to delete emails off the iPhone and still have them hit the Eudora mailboxes.

What's my best alternative to configure my two email clients (iPhone and Eudora) to make it all work the way I want it? Since I don't delete emails, how can I make IMAP work for me?

I'm running a couple of home servers (Debian Sarge and OpenBSD) so those could enter into the mix if it results in an elegant solution.

Thanks in advance. These are probably stupid/obvious questions... I just never paid much attention to email because it's "just worked" for me.

Jim

Set up IMAP, migrate all your folders and old messages over, then set Eudora and the iPhone to IMAP.

The phone defaults to only downloading the newest 200 messages in a folder, and will only grab more if you manually scroll to the bottom of the mailbox. It also only automatically checks the inbox. With IMAP, if you nuke a message it will move to the Trash folder on both the phone and Eudora. IMAP gives you the same state in all places.

Eudora, like pretty much any IMAP desktop client, will download all your mail and cache it locally to make searches pretty quick.

If you are setting up your own mail server, I'd highly recommend an IMAP server that uses maildir formatted mailbox, and not mbox, especially with how much mail you have. The difference is that maildir creates a folder then makes a file for every message. Moving, deleting, and other common operations are much less IO intensive on the server since it doesn't have to pull a message out of the middle of a solid mbox archive. I've run Courier IMAP with MurderFS, err, Riser for quire a few years with no issues. Even migrated the 10 or so users between 4 servers now.

I think one of the issues you are thinking of will be a non-issue. You are concerned about leaving emails on the server filling up your iPhone right? Not how it works on mobile devices. You just tell the iPhone to only keep the last 1 day, 5 days, 1 week, 2 weeks or all of your mail. Options won't be exactly that but near enough.

Your desktop machine can be told to download headers or the whole of the email and because it has much larger resources won't bother with only syncing recent emails.

You should also be able to "subscribe" to folders on the iPhone so that once you file an email to a folder you can still get important ones if necessary.

IMAP cr4ps all over POP3 (IMO but I doubt too many will argue). If you have a decent server, you can run filters on the server so that mails are filtered on any client automatically. A simple one is moving marked spam to a junk folder saving you from deleting it from your inbox all the time. Means you don't have set up clients individually either and worry about which client has downloaded the mail etc etc.

EDIT: Probably the one issue is that you need a server to run IMAP. Not many ISPs will offer that service and if they do space will be limited. GMail can do it - depends on whether you want to keep your current email address or just forward it to GMail and use a GMail address.

As everyone else has alluded to, IMAP is actually a better match for what you do that POP3. I have several gig of email on my IMAP server (about a gig of mine and the rest other users). I access it from my phone, Thunderbird, Outlook and webmail.

If you end up running you own server, whatever you do make sure you don't end up running the UW IMAP server. Courier IMAP is good, but maybe a little unnecessarily complex. I use Dovecot which is easy to set up and performs very well even on very old hardware.

Also, using Dovecot you can get away with sticking with mbox style mailboxes due to Dovecot's excellent indexing system. My multi-gig IMAP server is a lowly PII 400MHz with only 386MB of RAM and moving/deleting within several hundred meg mbox files is pretty immediate.

While maildir helps with performance, it does complicate things like incremental backups, due to the way it moves files around and changes their names when their associated message changes state.

I am tempted at the moment to move to GMail hosting for my mail.

There is a third option with GMail, let them host the email side of your own domain so that you get to use GMail with your own rather than a GMail address.

I agree with Andy about staying away from the UW IMAP server. I use BincIMAP (with qmail) for my server.

Aside: Does the iPhone support IMAPS and SMTP+TLS?

That made me sit up, because I'd always thought my distrust of non-mbox folders meant I was stuck with UW. So I downloaded and installed Dovecot (it's not in the docs that I could find, but it turns out you can do preauthenticated IMAP-over-SSH using "ssh server exec /usr/libexec/dovecot/imap"), but disappointingly it's not noticeably faster. I think what this probably means is that UW itself has got faster and more efficient over the years -- I'm using version "imap-2007".

Peter

That is very surprising, I didn't realise UW was still under active development. I don't remember what the last version of UW I used was, but moving to dovecot was a huge improvement. It wasn't just a little bit faster, it was massively faster (while also placing much less load on the server).

Yes. It even defaults to trying to use them, and then will ask if you want to drop to unencrypted if it can't connect when setting up the account.


Out of curiosity, why do you not trust maildir? It's been supported by many programs well for over a decade now, is far easier to find messages when even digging around on the command line, and also trivial to change things even without a mailserver running. I can mark a message read with mv, move a message to another folder with mv, or copy one with cp. And I can obliterate a message with rm. mbox, good luck moving a message manually in a quick way.


I would think an incremental backup of an mbox server would take longer and use up more disk space though. Move one message out of the 10 year old inbox into a 5 year old archive mbox file, and now you have both files being fully backed up again. Or, if you have something trying to generate diffs, you have this heavy process trying to make the diff files for the two folders. Maildir, one file disappears in one folder, one appears somewhere else, and the disk space used is the size of the message.

As far as server recommendations, Courier may indeed bit a bit complex for you use, and I had forgotten about Dovecot (it wasn't around when I first set up mine). I did try it out about a year ago and a friend set one up for his work, and does say it works well. It does also support Maildir just fine, so either format is an option.

No, incremental backup of maildir is a pain, if you want your backup to be usable. Unless your incremental backup code is clever or you choose to remove files on every backup.

For example, I use rsync scripts to backup all of my data twice a day. Just about anything else is easy to backup (excepting database servers, where I do a dump and sync that), maildir is a pain.

Backup up mbox files with rsync is easy, only the changes get sent, most of the time the files are just being added to so just the new messages get transferred.

We you do it using maildir however, because the files get moved and renamed when you read them, things are less smooth. It means that you end up with multiple copies of messages on the backup server.

I could fix this I guess by making my rsync scripts remove deleted files on each sync, but I have it set not to so I can get things back when I delete them by accident.

So what you really need to incrementally backup maildir properly is either:

- a script that creates a snapshot of each backup using symlinks on the backup server (there is such a script out there)
- a tool that uses hashes to spot files that have moved/renamed and move them on the backup server

I really should be using a script that does the first of these options and then maildir would be less painful, but I've never yet go round to setting it up.

I'm not doing it today, but if I was setting anything up now, it would be google apps for domains. As much as google is taking over my life, the last thing I want to do is run my own mail server.

Now, to thread hijack with more iphone questions.

1) Notes - 3rd party or first, is there a solution today for syncing notes up and down on a PC (online is fine, but no Mac involved)?

2) To-Do's - Same as above. Can they be integrated with the calendar? AFAIK there's no todo list facility at all?

3) Insurance - Now that It's Only A Phone and not a New Revenue Sharing Model, can the damn thing be insured like a phone? AppleCare is one of Apple's biggest competitive disadvantages at the moment.

Matthew

No real need for a script to do it either, beyond knowing where the last backup was. The --link-dest option in rsync takes care of all the symlinks that people used to use cp -R for. I started using that option at work to hang onto more builds without chewing up so much disk space, and it's worked well so far.

As far as the backup method and mailbox type, it depends on the backup program. For some reason my mindset of backup is still stuck in Windows land, where it's uncommon to see effective use of things like rsync and such. Instead, a ton of backup utilities there would do what I had talked about above, simply looking for files that changed, then throwing the entire contents out to tape or wherever. Time Machine on OS X works in a similar way, moving the entire file over to the backup if a bit inside of it changes.

Oh, didn't know about that new feature I'll have to investigate (though it will mean adding more intelligence to my current scripts).