Strange networking problem

I posted about this issue a while back, but I really made a mess of the thread and made it very confusing. Reading it now I'm not even sure what I was saying myself Anyway, here it goes:

I do cheap/free IT work for my family's church, and they're having a heck of a problem. They have two buildings, with underground ethernet connecting the two of them. It was thought that they were close enough together to not need fiber (which I wanted them to use), but after everything was set up there didn't seem to be a problem. Now, however, they've been experiencing very slow speeds in the old building (which receives the connection from the new building).

In attempting to troubleshoot it, I took the end of the underground cable and attached my laptop to it. I got great speeds, just what their cable contract was rated for. I then plugged it back into the switch it was connected to, and the speeds plummeted. I tried attaching a different switch and got the same problem. After a little more testing, though, I started noticing that my upload speeds were terrible or the tests weren't even working.

One thing I noticed was that when I connected the end of the long run to the second switch I had, I was getting strange readings from the LED for that connection. It would turn on for one second then off for one second. When it was off it looked like it was blinking nearly too rapidly to see, so I don't know if that means anything.

Is the problem merely that the run is too long? It doesn't seem like it's more than 300', which I know is near the limit. I'm pretty certain that the plans for construction put it at well enough under 300' that they weren't too worried about it. They ran two ethernet cables but both have the exact same problem.

After troubleshooting I set everything back the way it was, but now the connection doesn't work at all. Today I'm going back and I'll try getting them set up temporarily with a WDS connection between the buildings with some routers I picked up. They won't be connecting to the second router wirelessly, so the throughput drop shouldn't be a problem, it just probably won't be ideal.

Is there any way to salvage this wired connection?

What type of cable is the long run between the two buildings? CAT5, CAT5E, CAT6? I would guess EMF interference of some sort, or signal not being consistently strog enough to reach destinantion at all times. Just a guess.

It's always possible that the cable is damaged. How is it run underground? Is it in some sort of conduit? Is there any possibility of pulling a new cable? There are devices that will test a cable to see if it is capable of supporting the data you want to pass, like the Fluke CableIQ, but they're pretty expensive and I don't know if there's anywhere you could rent one.

Oh, you might also try forcing speed and duplex settings.

I believe the cable is Cat 5e.

It's possible the cable is damaged. I was getting some connectivity earlier, though, and I would have thought it was an all or nothing kind of deal.

I'm currently trying to set up two WRT54GL's for WDS but that's not going well either. I can't seem to get the two to pair. I've been following some instructions I've found online (the Tomato site, I have Tomato installed), but none of the instructions I've found deal with the type of setup I'm using. They all assume the first router is the main router, when I want hang the first router off the edge of the network, essentially as a WAP, and connect the second router to it over WDS. I'm not sure how to configure the first one, though. I've turned off DHCP, but I don't know of I should do that or not. It looks like I have everything configured correctly, with perfectly matching wireless configurations (SSID, channel, encryption, password, etc - everything except IP addresses), but it won't work...

*edit*
D'oh! I was using the wired MAC addresses for each router. I switched them to the wireless MACs and everything's working. Well, at least it is in the same closet, we'll see what happens when they're 200' apart.

After this I can look into the wired connection more. I just needed to get them up and working with any kind of connection!

Cat 5e has 8 wires in it. 10 and 100mbit speeds only use 4 of those wires, with two dedicated to sending and two to receiving. Gigabit will use all 8, and uses the wires differently. Damage to the cable may impact 10 and 100 speeds differently then gigabit.

What link speed is being negotiated? If it's higher then needed (Gigabit when 100 will do) try forcing it down a notch like Bitt suggested. If the switches don't allow this, try to find a 100mbit only switch for one end.

Thanks, guys. I'll try that the next time I'm there. The lines start at a gigabit switch but end up at a 10/100 (gigabit is for the rest of the new building).

It's also curious, though, that both lines would be affected the same way if they were damaged. I'd expect different behavior..

Both lines? I was under the impression that there was a single ethernet cable. Is that not the case?

See? There I go confusing things again.

Yes, there are two ethernet cables running through the conduit from the new building to the old one. There may be a third, unterminated line, but I haven't tried it yet. That will be the next step, I think. I'm not really sure why they ran multiple lines. Perhaps in case something like this happened. But if none of them work, then it doesn't matter how many are in there.

If I didn't mention it: I noticed today that at the start of the conduit, in the mechanical room of the new building, someone did have the foresight to include a pull chord for new/replacement cable. But you know what some other a-hole did? They filled the end of the conduit with friggin' cement! WTF?

Perhaps the 10/100 gear and the Gigabit equipment is not 100% compatible at extreme cable lengths.

Try inserting a simple 10mb or 100mb hub between the gigabit switch and the long cable.

Or get another gigabit device in place of the 10/100 unit to test with

Another test would be to manually force half duplex mode on the link. Try that at each speed.

Get a 300 foot cat 5e cable and carry the entire long cable and 10/100 end gear to the gigabit switch location. See if you can get it to work all in one place.

Thanks for the suggestions. I'll try these the next time I'm over there. At least the WDS is working pretty well, but the speeds aren't great. On short notice the only thing I could buy that I trusted with Tomato/Tomato USB was a pair of WRT54GLs. If we can't get the wired connection working again I'll have to work on improving the wireless one, maybe 802.11n and some directional antennas would help. The distance for the wireless is definitely less than 300 feet or the length of the ethernet run, but it has some cement walls to go through.

Triple check the ordering of the 8-wires at each of the RJ45 connectors -- if any of the colours are out of sequence then it could cause issues like what you have reported here. Ditto if somebody just did straight-through wiring rather than splitting one pair (usually Green) onto pins 3 and 6.

And once again, DX is your Friend: Cable Connectivity Tester (highly recommended!).
Or this one (slightly cheaper): the exact one I have here.


-ml

Did you prod at it to verify that it really is rock hard cement? Usually the custom is to fill the end of the conduit with "duct seal", which is a grey firm putty substance, easily removed if needed. This keeps critters and drafts out of the tubing.

Cheers

I do have a cable tester and I'm pretty sure I used it in this case and everything checked out. Would that make things change though? I would think it would be bad from the start...


How firm? I prodded it pretty darn hard with my finger and felt no give. It felt cold and hard like cement, but I'll try poking it with something sharp the next time I'm there.

It should be very firm, but not "hard". A sharp/pointy object should easily dent/penetrate it.

That device will check to see if the cable's terminators have their pins connected correctly, or if there's a break in the cable, but it doesn't do anything to test the cable's ability to carry a signal.

That said, it's more important than usual to make sure that the correct pairs are being used when you're approaching length limits.

For less than $4, *delivered*, I don't really expect it to replace a very expensive cable analyser.

But then, the Cat-5(e) labelling on the cable itself generally assures the ability to carry a signal, especially in a mostly straight/protected run inside conduit.

In real life, nearly all faults are of the connection variety, and the little DX gadget is quite adept at pointing them out.

Given that there are two such cables in the conduit here, I find it rather unlikely that the cables themselves are BOTH bad; it's much more likely they've not been terminated correctly, or that the GigE switch at one end might be the culprit.

Cheers

My point is that when approaching the length limit on data cabling, you get more likely to come across transmission errors. There are testers that can tell you if a particular cable can reliably pass certain kinds of traffic. These are very expensive, and what I was referring to in a prior post. All the $4 tester does is see if the connections are terminated properly and that there are no breaks in the line. I can see the former for free, and the latter almost never happens after installation.

If Matt had one of those sub $4 gizmos, then we probably wouldn't even have this thread to discuss them in.

I don't have the $4 one specifically, but I have the tool. I'll be testing it later today.

Okay, I went by my church late this morning, and looked at a few things. First, I tried testing the lines. My cable tester showed no problems there. Next I tried attaching a new switch to both ends of the lines, and had the same problems. Lastly, I tried poking what I thought was cement, and while I don't have experience with the substance Mark mentioned, I'll eat my hat if this stuff isn't cement.

I've created an album of photos and videos to demonstrate. I have a photo of my cable tester, a video of a test, and photos and a video of what I assume is cement.

Just thinking out loud:

What router/switch (brand / model) do you have on the new building, where the network is being originated? What brand/model do you have in the old building?

Have you then tried to force 100Mbps on all the ports involved (switches of the old and of the new building)?

What about redoing terminations for all cable ends?

Nice pics.

Yup, looks like cement for sure. And your cable continuity thing seems happy, though one thing it won't get right is the wire order in the plugs -- if both ends have it exactly wrong in the same way, then the tool will still be happy.

So look closely at the wire colours in each end and ensure they are done correctly. Or just cut off the RJ45 plugs and redo them correctly.

Again, the order must be P1W, P1C, P2W, P3C, P3W, P2C, P4W, P4C,
where P1,P2,P3,P4 refer to each of the four pairs of wires (orange, green, blue, and brown), and "W" means the "white" half of a pair, and "C" means the coloured half of a pair.

The exact colour order should be identical at each end of the cable (conduit). P1 can be orange or green, P2 is the other of orange or green, and P3 is blue, P4 is brown.

Cheers

That doesn't look like ethernet cable coming out of the cement. Is it maybe some sort of sub-conduit?

Also, when you end up running new cable, run fiber. Yeah, it's a little expensive, and, yeah, so are the converters you'll need, but you won't be running at the long end of the spec.

Image removed pending cure. Thanks k447

Something funky with the _labels_ for the wire colors in that diagram.

Yes, yes. Colors are correct, lables on rt side are fubar.

Doug's story of getting his internet fixed (possibly) reminded me to update this thread.

I had a cabling company come out to the site yesterday to test the ethernet lines running under the ground through the conduit. This is something I can't do as they have the fancy $8000 device for this purpose. Well, the cables passed the first test when it found the pins all lined up, but then while it was checking for consistency in the line, one of the techs pointed out the distance reading on the device:

441 feet

Clearly the problem had been found. This raises a couple questions for me, the primary one being how it ever worked (for two years) to begin with! I'm also wondering why nobody who was putting in the cable had questioned how much cable was being laid out.

The cabling guys were also dismayed that the end of the conduit had been filled with concrete, by the way. They had no explanation for that.

So I've arranged an appointment for the fiber termination and ordered the converters. Hopefully this will all be fixed by Monday!