The only other times I've seen severe slowdowns like this are when there are duplicate MAC addresses on the LAN and when routing relies on ICMP redirects, but I can't imagine either one of those is your issue.