"No one ever got fired for buying IBM."

First off, I think that Exchange is full of fail, from needing a lot of handholding, to costing a huge amount, to failing to implement standards, etc. To me, that's risk. It means an excessive amount of IT expenditure and an assurance that you're always going to be a 100% Windows shop.

Let me give you an example of a non-obvious Exchange-related IT expenditure. One would assume that backing up your users' mail is important. And, one would assume, if that's the case, it's important to be able to restore their mail. Now, you can back up Exchange's mailstore blob and restore it without significant effort. However, if a user realizes that they need an email from a few weeks ago, the only thing you can do is restore the entire mailstore blob. Which means that all of the emails received in the last few weeks are gone. For all your users. Now, you can restore that blob, let the user copy the pertinent email out, and then restore the current blob, but that means probably hours of downtime for your entire mail system. If you want to be able to restore individual messages without downtime, you'll have to invest in a commercial backup solution and their Exchange client. (As far as I know, there is no free/open backup server that can deal with Exchange datastores; I believe that Microsoft licenses that specification to the backup vendors.) But that's the sort of thing you need to consider, and that Microsoft doesn't tell you. In fact, it's entirely possible that you could implement an Exchange solution without realizing this and then be up the creek when your CEO requests that email.

Second, even if you don't buy that argument — that those factors are not risk — you have to ask yourself: is that reduction in risk really worth that increase in cost? Now, you can argue that as long as your users are happy with mediocre (sadly, not a terrible assumption) that Exchange/Outlook is good enough. Personally, I feel it's my responsibility as an IT guy to improve the workflow of my users where possible. (That said, Exchange and Outlook do work together well. They're like two Legos that fit together perfectly, as long as you're willing to overlook that all of the other faces have been chewed up by the dog.)

Third, is it risk to your users, to your company, or to your job? I'm not going to make any arguments about the virtue of those assessments, but make sure you know what risk you're reducing. Don't lie to yourself.