I've got two undergrads doing a summer project in C# (building me a software voting machine). This is with students who know Java well but didn't previously know C#. Why? Because C# is theoretically portable (eventually) to an Xbox 360 (thus making for a clever, low cost voting machine). Turns out, the Mono people are way behind the Microsoft people, in terms of supporting the latest and greatest language features (notably templates, which were added in .Net 2.0 as well as apparently significant library support). I haven't dug into this much yet, myself, but my students are convinced that the new features in .Net 2.0 are essential.

As such, if you write with C#, you accept portability as a problem. What you get back is much better integration with the Microsoft universe of COM objects. Also, unlike Java, the language knows about arrays of structures and will allocate them inline, unlike Java which allocates an array of pointers to class instances. As such, if you've got "high performance" needs, and by that you mean to process large arrays of data, C# gives you much more control over what your data looks like in memory. That, all by itself, may be a significant reason for going with C#.