I figured I'd throw this odd one out there to see if anyone has seen similar and has any ideas.
Software: CentOS 5 running kernel 2.6.18
Hardware: Multiple Dell PowerEdge 1950s and a Precision 490
Network: Broadcom 5708 in the servers, 5752 in the Precision, also an Intel card in the Precision.
The problem is that whenever these boxes talk on the LAN, they show extremely slow performance. 30k/s to a remote office that other boxes can easily pull down files at 1000k/s. At first we thought it was networking related with the switches the servers were on, so they tried a different switch. Then a different patch panel. Then just running direct network wires to different switches in the server room. And then the Precision was set up on a different switch and also had the same problem.
The idea came up that it could be the broadcom NICs somehow, possibly due to the tg3 module. So toss in the Intel NIC into the Precision, with the onboard NIC turned off, and same result, even with the NIC now using the e100 module. Also talked with Dell about the servers, they suggested turning off some odd feature called "TOE" that does some network offloading from the CPU to the NIC. We pulled the RJ11 enabler key off the motherboard, same issue.
A Fedora Core 3 box running 2.4 on the same network port sees the expected megabit speeds to the remote office, and near 100mbit speeds on the local LAN.
Currently, the server is being loaded with CentOS 4.4, and I'm trying to put Red Hat Enterprise 4 back on the Precision to see what happens next.