Boring background info

In one of the recent threads here, I mentioned my disappointment that Verizon FIOS service isn't very friendly to people who use their own home routers. If you just use FIOS for Internet you're generally fine with your own router, but if you also have TV services, and want to use Video on Demand, remote DVR scheduling, etc., you run into problems unless you use the Verizon-provided router.

The crux of the issue is that Verizon uses TR-069 to perform some automated setup/configuration of the router, and to figure out what set-top boxes are on the network, so that port forwards can be maintained to provide connectivity to the set-top boxes. You can set these port forwards up yourself, but if the Verizon-provided modem doesn't have a valid external WAN address, it won't report things back to the Verizon mothership, and you're out of luck.

The Rube Goldberg solution that people have come up with requires using three (!) routers: the primary router on the network perimiter, the Verizon-provided Actiontec router to handle some of the TV stuff for the set-top boxes (guide data, VOD, etc.) and a third router whose only job is to serve the external internet address to the Actiontec's WAN port via DHCP, tricking it into thinking it's the primary router. This is what it ends up looking like on my network:



The goal is to remove "lenny", who's doing nothing except fooling "larry' into thinking he's "barney".

Digging into the weeds, I learned more about the technical details of this TR-069 communication between the Verizon router and the mothership, and I started to think it might be possible to use some software to send the correct data back to Verizon instead of using hardware to trick the router. Though the outbound connection is SSL-encrypted by default, I found out that one can set the URL for that connection to an HTTP URL on the local network, which opens up the possibility of a man-in-the-middle proxy where we can do any substitutions we need to do, then send it out over SSL to the "real" configuration server.

I've made some good progress on this, but one piece I still haven't solved is how to decode some of the configuration values that the router obscures using what seems to be a primitive cipher. I thought I had solved it, but there's something else going on with it that I can't quite figure out, so I thought I'd pose it here as a question, because we've got a bunch of smart people who love solving hard problems.

The puzzle

So, with all that out of the way, here's where I'm stuck.

There are a couple of passwords stored in the router's config that are used to authenticate to Verizon's servers, but they're stored in some sort of obfuscated format, and they need to be presented to the server in plain text. The good news is that it's trivial to store a value in the config that uses the encoding/encryption scheme, so I can generate an arbitrarily long list of raw/cooked text pairs, in an attempt to reverse-engineer the encoding scheme.

The first insight I had from just eyeballing the encoding scheme is that it uses an HTML-entity-like encoding for some characters, e.g.:



So, ampersand, followed by a hex number, followed by a semicolon indicates a non-printable ASCII character, with other regular ASCII characters mixed in. The next observation was that, once you decode these HTML entities, the lengths of the raw and cooked values are identical. Seeing this, I started to think it was a simple 1-to-1 mapping of ASCII values, but shifted in some way, a-la ROT13 / Caesar cipher.

I tried to measure the length of each of these shifts for each input character with some simple input, e.g.



So, we start with the first character of the input string (the number zero, ascii 0x30) and add 0x56 to get 0x86 in the output. Then the second character is zero again, but this time we add f4 (wrapping around using mod 256) to get 0x24, which is a dollar sign in ASCII.

Doing a few more of these short sequences, I noticed a pattern:



The next step was to see if the shifts were the same for every pair of input/output strings, and in my initial testing, it looked like they were. I came up with a sequence of shifts that seemed to work for most of the inputs I tried, but for longer / more complex strings, there would be slight differences in certain character positions, e.g.:



This suggests that whatever scheme they're using is doing some minor perturbation of these shifts based on... I don't know what, really. To illustrate further, here are some more samples from some sequential tests I ran (ellipses indicate that the pattern repeats):





Things get even murkier when you start using longer strings. Here are some results from some random test runs (encoded output omitted for brevity):


In these examples, the shifts of character positions 6 and 13 are off by one in some cases. So close, yet so far!

Once you start throwing in punctiation, things get even weirder:



So, there's clearly a pattern, but I can't for the life of me figure out what the algorithm is doing to induce these slight changes in how each character is shifted.

Sadly, I never took any crypto classes, and I was never all that great at math, so I'm sort of stumped on what's going on. I was hoping someone here might have some insight on other things to try, or what the algorithm could be doing behind the scenes to introduce these slight changes.

I'm sure there are probably some better forums on the Internets to ask this kind of question, but I know folks here like a challenge, so I thought I'd post it here first. And, if not, I thought it was a fun story to tell about hacking hardware and trying to make it work better, which is something dear to the hearts of most folks here.