Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#227686 - 17/07/2004 17:23 Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle thing.
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31600
Loc: Seattle, WA
Interesting problem with my Rio Central. Anyone know what to do exactly?

- Because I use hierarchical playlsits on the Rio Central, I now exclusively use Jemplode to update it. (Emplode won't see the hierarchy on the Central and no longer works with it.)

- I'm on Jemplode 56.

- I've got a group of MP3 files on my central, a large album of 38 tracks.

- This album was uploaded to the Central, it was not ripped on the Central.

- Okay, I admit it, it was a bootleg recording of a Rush concert. Hey, I buy *all* their albums including their live albums and DVDs, I spent nearly 400 dollars on concert tickets, plus another 100 for merchandise on this tour. I'm entitled.

- I seem to faintly recall something had gone wrong during the upload, but it was a while ago so I'm not sure. I thought I checked it and made sure everything was OK, and it seemed to be. So that might not have anything to do with it.

- On the Central, track 34 is missing from this album. It is not missing from my car player, and not missing from the PC's hard disk. The PC's hard disk was the source location for the upload when I originally sent the files to the car player and the central.

- I don't seem to recall the track being missing on the Central before, but when I look at it in Jemplode, it's just not there, no matter which view I use (playlists, all tracks, artist, etc.).

- Today when I was working with updating other tracks on the Central, I was using Mike Schrag's riorid utility to refresh the RIDs so I could synch playlists. It gave me an error:

Unknown fid type for fid 83920 titled ""
Synchronize succeeded.


Note that 83920 is decimal for 147D0 hex.
Also note that there are two space between "fid type" and "for fid" in that error message. Doesn't come across in the BBS posting very well.

- "Interesting", I sez to meeself. Let's find out what this mystery FID is.

- http://192.168.0.4:12078/content/147D0?_extended=1
(The above is the HTTP query syntax to stream or download an MP3 file directly off the central.)

- It sends me an MP3 file. Which happens to be a complete and properly tagged track 34 from that album. It's my missing file!

- When viewing the Central in Jemplode, I see no FID numbered 83920 (147D0 hex). The "All Tracks" view skips over track 34 of that album, and skips over that FID number.

- However I can easily download both 147D0 and 147D1 onto my hard disk using the above HTTP command, and I get a proper MP3 file for the *0 file, and a proper ASCII FID with all the correct tag/title/etc information for the *1 file.

- I can download the track right before it (147C0 and the 147C1 ascii tag data) and it's all fine, too. It's the prior track on the album. Other than the fact that they have different track names, track numbers, and different RIDs, they seem fine to me.

Except...

Here's something interesting I notice. It may not have any bearing, or it may be the whole reason this is happening. Not sure.

Okay, you know how the track number on an ID3v1.1 tag is supposed to be last character of the comment field? In other words, ID3V1 doesn't spec a track number, it specs 30 characters for comment, but ID3V1.1 specs 29 for comment and the last byte of comment for a track number. And the track number in ID3V1 is an 8-bit binary value, it's not ASCII.

Well, I notice that these tracks all have that byte in the ASCII FID data as the start of the comment field. For instance, here is track 33's tag (FID 147C1):

source=30th Anniversary Tour Bootleg, May 26 2004, Starwood Ampitheatre, Antioch TN
length=13339746
duration=416000
offset=1704
artist=Rush
samplerate=44100
comment=!;RID=cb44ac3b97fd01e95de211aa598839d8
fid_generation=1088061181
genre=Progressive Rock
ctime=1088061181
tracknr=33
type=tune
year=2004
codec=mp3
bitrate=fs256
title=Xanadu (Live 2004 Bootleg)
rid=cb44ac3b97fd01e95de211aa598839d8


Now notice what's happened there. Jemplode, when it imported the track originally, Did this: It looked at the comment field of the ID3V2 tag and saw it was blank. Then it looked at the ID3V1 comment field and saw it had one byte in it, a 33, an ASCII exclamation mark. It didn't know that byte was actually just the track number at the end of the comment field, it didn't know to ignore it. So it took the exclamation mark, and made that the comment field.

Then, when RioRid came by later (I had refreshed the RIDs on the Central after uploading that album), it correctly appended the RID to the comment field.

Okay, so no big deal you say. And it's not a big deal because that track, track 33, worked fine.

But what is 34? An ASCII quote character.

I'll bet the fact that it's a quote had something to do with it. Here's what FID 147D1, the "missing" file, looks like:

source=30th Anniversary Tour Bootleg, May 26 2004, Starwood Ampitheatre, Antioch TN
length=12920120
duration=400000
offset=1709
artist=Rush
samplerate=44100
comment=";RID=d710471cda01232aab00a70bed0e31a7
fid_generation=1088061181
genre=Progressive Rock
ctime=1088061181
tracknr=34
type=tune
year=2004
codec=mp3
bitrate=fs256
title=Working Man (Live 2004 Bootleg)
rid=d710471cda01232aab00a70bed0e31a7


Okay, here's what I'd like to know if possible...

1. Find out if the "ID3V1.1 track number becomes comment field" thing is a Jemplode bug?

2. Is the fact that an ASCII quote was part of the comment field right before the RID comment the reason I get an error when running RioRid?

3. Does any of that have anything to do with why I'm not seeing that track in Jemplode when I browse the Central?

4. How do I make that track reappear on the Central?

Any other ideas?
_________________________
Tony Fabris

Top
#227687 - 18/07/2004 00:30 Re: Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle thing. [Re: tfabris]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31600
Loc: Seattle, WA
Okay, online with Mike...

Tested: Putting a quote (ascii 34) in an empty comment field and then synching makes the file disappear from the central. But the FID still exists on the hard disk and looks just fine. It's merely missing from the database.

Mike's current working theory is that something about the protocol or the database build wants there to be a closing quote if there's an opening quote in the field. Not sure yet.

At this point, I don't care about the parsing thing where Jemplode thought that the track number was part of the comment field. Mike is pretty sure that was an old bug in Jemplode that's been fixed. And in fact I couldn't reproduce it even though I tried.

But simply putting a quote in the comment field causing the file to disappear, well, that's pretty scary. Hopefully we can find out exactly where the error is happening and have Jemplode implement some kind of a work-around.
_________________________
Tony Fabris

Top
#227688 - 18/07/2004 00:36 Re: Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle th [Re: tfabris]
genixia
Carpal Tunnel

Registered: 08/02/2002
Posts: 3411
I wonder if you could edit the comment field to include an escape character before the quote. I presume that JEmplode has one, you could try a backslash for starters.

Alternatively you could probably just edit the quote out of the *1 file, and I bet JEmplode would find it again.
_________________________
Mk2a 60GB Blue. Serial 030102962 sig.mp3: File Format not Valid.

Top
#227689 - 18/07/2004 01:20 Re: Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle th [Re: genixia]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31600
Loc: Seattle, WA
Oh yeah, fixing it so the quote isn't there any more solves the problem, that's for sure. Of course, on the Central, editing stuff on the hard disk is tricky if you don't have a USB keyboard to get a shell. Mike helped me solve the problem by writing a little utility that could delete a FID via the standard protocol. I erased the 0 and 1 fids and added the file again and the problem was solved.

I'm just concerned that it might happen again to me or to someone else, so I'd like to get to the bottom of why a quote messes up the Central so we can find out where the work-arounds need to be implemented.
_________________________
Tony Fabris

Top
#227690 - 18/07/2004 07:50 Re: Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle th [Re: tfabris]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
where the work-arounds need to be implemented.


In the comment-loading code from the ID3v1 tags. Technically, Tony, in v1.1, there's 28 characters for the comment. The next one must be a null, and then you're allowed the binary track number.

Unfortunately, it looks like your bootlegged tracks don't correctly null-terminate the comment, so Mike's code picks the whole thing up as the comment.

It's a broken tag, so the fix (in this case) ought to be in the tag reader.

However, it would be conceivable that a comment might naturally have a single quote in it, in which case, you're still broken. The behaviour of the *1 parsing is ambiguous WRT to quote symbols, so IMO, it's JEmplode's job to simply throw mismatched quotes away (after complaining, obviously).
_________________________
-- roger

Top
#227691 - 18/07/2004 13:42 Re: Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle th [Re: Roger]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31600
Loc: Seattle, WA
Quote:
Unfortunately, it looks like your bootlegged tracks don't correctly null-terminate the comment, so Mike's code picks the whole thing up as the comment.

Actually, that wasn't the case. The Comment fields were full of nulls, 29 nulls followed by the track number. They were correct. Note that I tagged the files myself. The tracks may have been bootlegs, but I always make sure my tags are right.

Quote:
However, it would be conceivable that a comment might naturally have a single quote in it, in which case, you're still broken.

Exactly. We're trying to find if the problem is on the Central itself, or if it's only in Jemplode. If the latter, it's a bug fix. If the former, since no bugfixes are going to happen there, then we need to implement a work around in both emplode and jemplode.
_________________________
Tony Fabris

Top
#227692 - 19/07/2004 06:20 Re: Abandoned FID? Possible Jemplode and/or RioRid bug? Very subtle th [Re: tfabris]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Quote:
The Comment fields were full of nulls, 29 nulls followed by the track number.


Bug in JEmplode, then.

As for the dodgy quote causing a problem with the Central (which could happen, even if the comment/tracknr field is correctly parsed), that's a bug in the Central. As you say, it ain't gonna get fixed, so you will need a workaround in JEmplode for it.

See what happens if you have a comment field (or other field) with a matched pair of quotes.
_________________________
-- roger

Top