Hijack Web interface

Hello,
I have been unable to view the playlists on my empeg through the Hijack web interface for quite some time now. I installed charcoalgray's interface and still no luck. When I click the link to view/browse playlists, it just shows an Unattached Items link and nothing else, if I click that link then it will display an empty playlist.
I know I have playlists, because I can go through them on the actual empeg.
Also I have Hijack v309.
Any help would be great,
Thanks,
Justin

What does this show for you:

http://your.empeg.ip.addr/?FID=101&EXT=.htm

-ml

I have attached the page it shows me.....

Hmmm.. if you know how to, could you please send me a .tar or .zip file containing ALL of the files ending in "1" from /drive0/fids/ (and /drive1/fids/ if your unit has two disk drives), plus all of the files from the same place(s) that end in "0" which are under 1000 bytes in size if possible. Basically everything from there EXCEPT for the multi-megabyte tune files themselves..

Could take you a bit to assemble it, but with that data I can fix whatever's wrong in Hijack.

Thanks

Here you go...

If the file you tried to attach was larger than 200k (I'm guessing it was), then Mark didn't get it from that post. You're going to have to email it to him.

Mmmmm.. I received the fids.zip that you emailed me, thanks.

The problem appears to be that your player is using the new fids directory structure design, which will eventually become the default for post-v2.00 player software.

However, the current player software, while it supports this new design (for forward compatibility), does not normally use it by default. And Hijack does not currently support it.

How the heck did you end up with this on your player??

What version of player software, and more importantly, Emplode (or JEmplode or emptool) are you using with your player?

Thanks

I am using v2.00 beta 13 on the player, and the most up to date versions of Jemplode and Hijack, per the Jemplode auto-update function. As to how the new structure design came to be on my player, I am stumped.

By any chance have you ever used mp3tofid? Because that can generate the new style FID structure. Read this

- Trevor

Have you EVER used "mp3tofid" ?

Apparently version 3.00 (and higher) of mp3tofid uses the new format.

It is important to determine how this happened, because whatever did it was buggy -- it created duplicates of the root playlist and several other files -- your music directory structure is basically corrupted at this point, with some things in the new format, and others duplicated in the old format. No guarantees as to which copies of the playlists will be used when sync'ing the player with Emplode ...

-ml

Yep, I had tried out mp3tofid in the past. Is there anyway to undo the corruption to the filesystem without losing music?

Is there anyway to undo the corruption to the filesystem without losing music?

Once a new-scheme file exists, the player will use that from then on, until the FID is deleted and re-created -- even though all FIDs created by the player (e.g. when told to do so by Emplode/Jemplode/Emptool) will be old-scheme.

This means that, for FIDs that exist both as new-scheme and old-scheme, the new-scheme one is the one the player is using, and thus probably the one you want (but check a few first with "ls -l" to make sure you're keeping the newer one).

Peter

Mmmm... any chance of having Emplode remove duplicates at some point in the future?

The biggest problem here is that this player has two copies of fid 100 -- the root playlist. The one in fids/_00000/100 is correct, and the one stored as fids/100 is basically empty except for "Unattached Items".

In Hijack, I need an easy way to detect up front whether the new format exists on a player, so I can avoid the time-costly lookups for new-style fids when none exist.. before falling back to the old-style regardless.

??

So if the files under /drive[01]/fids/ are thought of as long hex numbers, which when zero padded on the left would be in the form YYYYYXXX. To "fix" your player, you must delete all files from /drive[01]/fids/ which are duplicated as /drive[01]/fids/_YYYYY/XXX

For example, your player has both /drive0/fids/100 and /drive0/fids/_00000/100 -- a no-no! You can safely delete /drive0/fids/100 in this case. There are others like that as well.

And back to the original problem of the web interface, I may update Hijack soon to cope with the new structure, if only I can figure out a cheap way to tell when a new structure exists..

Cheers

The biggest problem here is that this player has two copies of fid 100 -- the root playlist. The one in fids/_00000/100 is correct, and the one stored as fids/100 is basically empty except for "Unattached Items".

If mp3tofid writes a "complete set" of files including FID 100, then presumably it's not meant to be run on a non-empty player?

In Hijack, I need an easy way to detect up front whether the new format exists on a player, so I can avoid the time-costly lookups for new-style fids when none exist.. before falling back to the old-style regardless.

Does Hijack stat files that often? The player just looks for the new name first and, if it's not found, the old name. It already has to look in two places for each fid (i.e. each drive); looking in four places doesn't seem to have caused any user-visible slowdown.

Peter

Mmmm... any chance of having Emplode remove duplicates at some point in the future?

No chance whatsoever -- duplicates are simply not visible over the protocol, so Emplode couldn't fix them if it tried. Nor is it likely that the player itself would fix the situation: if you muck with the filesystem at the command-line, you should expect to need the command-line to fix it -- it's simply impossible to "fix" every single way a user could get stuff wrong!

On the other hand, deleting a fid in Emplode/whatever will delete all duplicates of that fid if they exist.

Peter

In Hijack, I need an easy way to detect up front whether the new format exists on a player, so I can avoid the time-costly lookups for new-style fids when none exist.. before falling back to the old-style regardless.

Does Hijack stat files that often? The player just looks for the new name first and, if it's not found, the old name. It already has to look in two places for each fid (i.e. each drive); looking in four places doesn't seem to have caused any user-visible slowdown.

Hijack already tries to be smart about which drive to try first, based on where it most recently found a related item. And I'd like it to be smart about this too, if possible.

This is just for the web interface, which when one clicks on a LARGE playlist can take a very long time to generate the page -- completely stat/IO bound while doing so.

I think for now I'll just use a config.ini flag to turn on/off support for the new format, so that plain vanilla systems won't be impacted at all.

This will be out shortly, untested, in Hijack v310.

Cheers

Okay, try again now with Hijack v310.. it should find all of your playlists now.

If mp3tofid writes a "complete set" of files including FID 100, then presumably it's not meant to be run on a non-empty player?

mp3tofid writes a complete set of fids. But it relies on rsync to transfer the fids
to the player. The rsync command as suggested in the docs wipes away all
fids not created by mp3tofid.

Going back to emplode does not do this, of course. So before
going back to using emplode, you should probably resync all fids
to the old scheme, having recreated the fids with mp3tofid's -o switch.

Pim

Mmmm.. I just tried this out on my player, and noticed that "emptool" doesn't seem to use the new structure when uploading playlists -- it just drops them into /drive0/fids/ instead of into /drive0/fids/_00000/

???

Yet another case of "the man" trying to keep us Linux users down.

Mmm.. or is that just the "way it works" even for Emplode?

Doesn't hurt anything, but just means that new playlists and tunes don't take full advantage of the new directory structure -- a third party tool could be used to shuffle them into the right pigeonholes, I suppose. Like a shell script.

-ml

Doesn't hurt anything, but just means that new playlists and tunes don't take full advantage of the new directory structure -- a third party tool could be used to shuffle them into the right pigeoholes, I suppose. Like a shell script.


Like this?

Mmm.. or is that just the "way it works" even for Emplode?

That's just the way it works. Emplode/Jemplode/Emptool don't get a vote as to where the file goes: they all just say to the player "create FID 0x2340 for me" and never even get to find out which drive it's on. In order for "protocol" (Emplode and friends) to create new-scheme FIDs, you need to be running a post-2.0 build of the player -- which no-one outside Empeg Towers currently is.

Peter

Mmmm.. I just tried this out on my player, and noticed that "emptool" doesn't seem to use the new structure when uploading playlists -- it just drops them into /drive0/fids/ instead of into /drive0/fids/_00000/

Yes, the player will recognise and read new-scheme FIDs, but any new FIDs created by the player (i.e. by Emplode etc.) will be old-scheme.

Whether just editing a playlist will make it migrate from new-scheme to old-scheme depends on how Emplode was written -- I can't remember now whether it deletes and recreates the FID (which would count as a new FID and thus be old-scheme) or whether it overwrites in-place (which would preserve the new-scheme FID).

Peter

Something similar to that, yes, but not that exact script.

Cheers

Like a shell script

I have come up with this:

#!/bin/sh



# unless you remove this line, this script runs dry

echo=echo



for newfid in /drive?/fids/_*/*

do

        # try not to use external commands for speed

        oldfid=${newfid#drive?/fids/_}

        oldfid=${oldfid#0}

        oldfid=${oldfid#0}

        oldfid=${oldfid#0}

        oldfid=${oldfid#0}

        oldfid=${oldfid#0}

        oldfidhi=${oldfid%/*}

        oldfidlo=${oldfid#${oldfidhi}/}

        oldfid=${newfid%%/*}/fids/${oldfidhi}${oldfidlo}



        if test $newfid -nt $oldfid

        then

                # move the new-scheme fid if it is newer

                $echo mv -v $newfid $oldfid

        else

                # delete it otherwise

                $echo rm -vf $newfid

        fi

done

However, it does not take care of the effect that mp3tofid uses a different
algorithm to distribute fids across drives than (j)emplode/player. It should
work if you only have one drive, though.

Onother gotcha is that file date comparisons may not work if your player's
idea of the date is off. mp3tofid/rsync stores the file dates of the source
files, while the player uses the current date of the player.

Pim

I have come up with this:

Oops, I misunderstood. Mark was talking about a script that
actively uses the new scheme. My example script was
meant for tearing it down and taking duplicates along with it.

Pim

Apparently version 3.00 (and higher) of mp3tofid uses the new format.

It is important to determine how this happened, because whatever did it was buggy

The only "buggy" thing I can think of is not using "--delete" with rsync, or using something
other than rsync (FTP?). Either way, old fids are not deleted and duplicate fids arise.
Duplicates can also arise when using the old directory scheme; you could end
up with duplicate fids on both drives.

Fixing this is easy in either situation. Just run rsync again, this time with the --delete switch.

Abandoning mp3tofid and its new directory scheme does not seem to pose problems
for (j)emplode, but it is probably safest to revert to the old scheme.

Pim

Sounds safe enough, either way now. The leftover bits don't hurt anything, but getting rid of them is a Good Thing (tm).