The fast drive is about 25/30GB full and the slow one is only about 15/40GB full.

The slow drive is the system drive if that could make any difference.
The swap partition is on the system drive. Paging fsck or swapping its data will cause lots of time-consuming seeks if you're fscking a partition on the same drive. But I'm still surprised that there's a factor of six difference. Maybe the fsck data structures for 30GB's worth of disk just fit in memory, and for a 40GB disk they don't and it thrashes swap?

The actual disk performance -- whether data rate, spin speed, or seek time -- makes a lot less difference in these circumstances, unless the slow disk is actually dying (and doing a lot of retries), which seems unlikely.

Peter