If you change cards, you have to do a hard reset of the phone and let it format the flash space. The internal and SD storage are then joined together into one contiguous space. Microsoft uses the "secure" part of SD to also lock the card to the phone, preventing accidental writes if it's put into another device. If you want to reuse the SD card later, you have to completely wipe it out too.
From doing research on it, it's not raw sequential read/write speeds that are critical. It's more important to have better random access and seek times. Some users reported really bad instability with the "fastest" class of SD cards due to them being optimized only for sequential.