Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Page 2 of 4 < 1 2 3 4 >
Topic Options
#248684 - 13/02/2005 05:52 Re: empeg file structures [Re: mlord]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
OK, point taken.

Many years ago I was struck by something I read in Apple's "Human Interface Guidelines" (their guide to GUI design - everyone should read it). This was:-

"Consistency should be valued above idiosyncratic cleverness"

In this instance I guess I was just getting carried away being idiosyncratically clever.

Tags files are in:-)

Top
#248685 - 13/02/2005 06:01 Re: empeg file structures [Re: ukengb]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Soemthing else I need to consider - tags.

Specifically:-

rid - do I need to bother with this. Since the fid num. will be persistent and unique? Just use the fid?

fid_generation - is this the time it was created? Why would that be important?

ctime - of what?

bitrate - always preceded by 'vs' (Rio Receivers use 'fs'). Can do, but why the prefix?

I'm getting there. Thanks.

Top
#248686 - 13/02/2005 08:22 Re: empeg file structures [Re: ukengb]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
Inspired by all the questions you're asking, I've started actually documenting the file structures.

Go to http://www.differentpla.net/node/view/387 and take a look.

If there's something I've not covered, or something that could be explained in more detail, post another comment here and I'll go back and update the document. Give me a short while to include all of the questions that have already been asked in this thread.
_________________________
-- roger

Top
#248687 - 13/02/2005 10:43 Re: empeg file structures [Re: peter]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
Quote:
There's no way the player stats 13,000 files on every startup


Fair point. ISTR that there's some kind of simple sanity check in there, though.
_________________________
-- roger

Top
#248688 - 13/02/2005 10:54 Re: empeg file structures [Re: Roger]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14472
Loc: Canada
It doesn't stat() them, but it *does* read the fids directories and subdirectories, which gives it a list of what is present (at a fraction of the overhead of stat()).

-ml

Top
#248689 - 13/02/2005 11:03 Re: empeg file structures [Re: Roger]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14472
Loc: Canada
Good descriptions, Roger! Nice to see that our reverse-engineered view was mostly correct! I think the description there of the byte-ordering will confuse people, though.

cheers

Top
#248690 - 13/02/2005 11:42 Re: empeg file structures [Re: mlord]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
Quote:
I think the description there of the byte-ordering will confuse people, though.


Yeah, I wasn't particularly happy with how it came out, either. I've dropped that paragraph.
_________________________
-- roger

Top
#248691 - 13/02/2005 14:43 Re: empeg file structures [Re: Roger]
bonzi
pooh-bah

Registered: 13/09/1999
Posts: 2401
Loc: Croatia
Ah, excellent reference!

Mr FAQmeister, would you please put a link to it from some convenient place?
_________________________
Dragi "Bonzi" Raos Q#5196 MkII #080000376, 18GB green MkIIa #040103247, 60GB blue

Top
#248692 - 13/02/2005 15:34 Re: empeg file structures [Re: bonzi]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31563
Loc: Seattle, WA
Done.
_________________________
Tony Fabris

Top
#248693 - 13/02/2005 16:29 Re: empeg file structures [Re: Roger]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Damn, that's sweet.

Thanks for taking the time to put this together.
_________________________
- Tony C
my empeg stuff

Top
#248694 - 13/02/2005 20:05 Re: empeg file structures [Re: tfabris]
bonzi
pooh-bah

Registered: 13/09/1999
Posts: 2401
Loc: Croatia
Heh, there was a discusion a few weeks ago about using Wiki for technical doc or similar site. Granted, Wiki can be handy, but having Tony around is much better!
_________________________
Dragi "Bonzi" Raos Q#5196 MkII #080000376, 18GB green MkIIa #040103247, 60GB blue

Top
#248695 - 13/02/2005 20:28 Re: empeg file structures [Re: tonyc]
oliver
addict

Registered: 02/04/2002
Posts: 691
I liked the notes on the rid. specifically the...

"Oh, and the R stands for Roger. Cheers."
_________________________
Oliver mk1 30gb: 129 | mk2a 30gb: 040104126

Top
#248696 - 13/02/2005 21:41 Re: empeg file structures [Re: oliver]
tman
carpal tunnel

Registered: 24/12/2001
Posts: 5528
I thought it was Rio ID or something but now I know

Top
#248697 - 14/02/2005 07:28 Re: empeg file structures [Re: Roger]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Well done Roger. Wish I'd been able to read that before figuring it all out for myself.

Couple of questions that relate to my project.

How important are the offset and bitrate tags?

Reason I ask is that the data I have readily to hand doesn't include the former and only the actual rate for tha latter, no indication of whether it's variable or fixed rate. I guess both can be obtained by looking at the actual file, but for speed I want to avoid that.

I know that the Rio Receiver doesn't use offset, even though it's a defined tag, it is not sent and currently ALL tracks are tagged by the server as 'fs', even though they're mostly 'vs' (I didn't write that bit), but it plays perfectly.

So can I get away with some innacuracy of the bitrate prefix and no offset specified? Is the RioCar as unconcerned about these as the Receiver seems to be?

Top
#248698 - 14/02/2005 08:18 Re: empeg file structures [Re: ukengb]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
Quote:
How important are the offset and bitrate tags?


Not particularly. They're hints to the caching code, but the player ought to be able to cope without them. Just leave them out for your files and see what happens.
_________________________
-- roger

Top
#248699 - 14/02/2005 10:44 Re: empeg file structures [Re: Roger]
pim
addict

Registered: 14/11/2000
Posts: 474
Loc: The Hague, the Netherlands
Implementing offset tags should prevent the player from loading id3 tags.
This is benificial if they contain album art or lyrics.

Pim

Top
#248700 - 14/02/2005 12:01 Re: empeg file structures [Re: Roger]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4172
Loc: Cambridge, England
Quote:
Not particularly. They're hints to the caching code, but the player ought to be able to cope without them. Just leave them out for your files and see what happens.

Some releases do fail to cope if the bitrate is absent. None should fail to cope if it's inaccurate, though. If you're going to guess, it's more conservative to (a) overestimate and (b) go for vs rather than fs.

Peter

Top
#248701 - 15/02/2005 05:58 Re: empeg file structures [Re: peter]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Guess I'll have to do some trial and error testing.

But, since I do know the bitrate, just not whether it's vs or fs, would it help to send just the numeric value and leave off the 'vs'/'fs' prefix?

Top
#248702 - 15/02/2005 07:30 Re: empeg file structures [Re: ukengb]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
No. It most definitely needs two characters before the number. The parsing's not particularly flexible. Just put 'vs'.
_________________________
-- roger

Top
#248703 - 11/03/2005 14:29 Re: empeg file structures [Re: ukengb]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
I'm getting to the end of this, but have a slight problem. I am trying to arrange that nothing is required on the actual empeg in order to sync - all the control is from the PC/server side. So I want to run rsync as a daemon when on AC power, my sync script will then set the empeg read/write to do the sync and read only afterwards.

The trouble is that rsync will NOT start unless root is mounted read/write, but at the time that config.ini is run (and rsync would be started) of course, root is read only.

I can only think that I'll have to start rsync with a 'site exec rsync --daemon' via ftp from within the sync script, but is there another way to get the daemon running on startup?

Top
#248704 - 12/03/2005 19:04 Re: empeg file structures [Re: ukengb]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Still struggling with rsync since after the sync it stops the 'ro' command from working, which leaves some filesystems 'rw' and when I rebooted in this state it totally trashed the drives and had to rebuild from scratch. OK, that's why I'm using one designated 'test' empeg for this, but I am surprised.

What is the problem with leaving them rw when rebooting? What's the difference with a normal linux PC which normally runs rw?

Also, should I quit the player app while syncing?

When jEmplode syncs there's a neat graphic that displays and it 'locks the UI'. Is there any way I can do these with command line scripting?

Top
#248705 - 12/03/2005 20:15 Re: empeg file structures [Re: ukengb]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14472
Loc: Canada
A "normal PC" also mounts drives r/o before rebooting. This is "normally" done for you as part of the system shutdown/reboot scripts in /etc/init.d

Cheers

Top
#248706 - 13/03/2005 17:59 Re: empeg file structures [Re: mlord]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Ah, that explains it. So it's entirely possible then that rebooting my system with it rw caused such catastrophic failure on my empeg?

I need to fiddle with rsync some more as it's very sensitive to things being rw or ro. It seems to be that specifying a log and/or pid file in rsyncd.conf prevents it starting when ro, but even without that, rsync can prevent the ro operation succeeding - keeping a file open maybe?

So is it OK to sync with the player app still running?

What about a nice graphic while syncing? Is it even possible to do this from the command line?

Top
#248707 - 13/03/2005 18:09 Re: empeg file structures [Re: ukengb]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Quote:
What about a nice graphic while syncing? Is it even possible to do this from the command line?

pim's rsync binary supports a graphic progress indicator with the --empeg-progess switch. Do a forum search for mp3tofid and rsync and you'll find a script for handling the sync situation properly with respect to mounting drives, etc.
_________________________
- Tony C
my empeg stuff

Top
#248708 - 13/03/2005 18:24 Re: empeg file structures [Re: ukengb]
mcomb
pooh-bah

Registered: 31/08/1999
Posts: 1649
Loc: San Carlos, CA
Quote:
I need to fiddle with rsync some more as it's very sensitive to things being rw or ro.


In the meantime you might want to consider using ext3 (see my sig for details) to decrease the likelihood of trashing your drives if you accidentally reboot when they are mounted rw again.

-Mike
_________________________
EmpMenuX - ext3 filesystem - Empeg iTunes integration

Top
#248709 - 14/03/2005 07:44 Re: empeg file structures [Re: mcomb]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
I will probably upgrade to ext3 once I know that my rsync routine is working correctly - I don't want to have to rely on ext3 to catch my errors.

However, I am a bit confused. Does Hijack now automatically contain the ext3 patches, or do you have to use the 'special' hijack, an older version which has had the relevant patches applied?

Top
#248710 - 14/03/2005 08:20 Re: empeg file structures [Re: tonyc]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Quote:
Quote:
What about a nice graphic while syncing? Is it even possible to do this from the command line?


pim's rsync binary[/url] supports a graphic progress indicator with the --empeg-progess switch. Do a forum search for mp3tofid and rsync and you'll find a script for handling the sync situation properly with respect to mounting drives, etc.


That's the rsync I'm using, but there are a couple of problems.

mp3tofid runs the empeg as the client and the linux server (with the music on) as the rsync server and although that's an option I'm exploring, I'm not getting any progress indication. I am capturing output of the rsync process so I can report on the outcome.

Would that prevent the indicator from working?

I did notice that there's a libresolv alongside the modified rsync, but as it already existed on the empeg I didn't install it.

Is that a different version required for the indicator to work? Pim?

The other issue is that I really want the empeg to be the rsync server (i.e. rsync running as a daemon) and control it all from the (linux) music server and if rsync is the server...

Would the progress indicator patch still work?

I suspect not.

I have been exploring the use of hijack's popup feature, but can't make that work either. I've tried a site exec via ftp or echo directly from the command line (via telnet) - the former returns an error and the latter does nothing.

What configuration is required to make hijack do this?

I've got [output]notify=1 in config.ini and I can read /proc/empeg_notify, but writing to it still does nothing. In desperation I added [input]notify=1 (seemed a logical idea:-), but that made no difference either.

I want to get the popup messaging working anyway, but ideally a graphic would be better during the transfer.

Can I somehow force a graphic to display?

As for setting the rw/ro etc, it's not a problem basically figuring when to do what, the problem is that rsync can cause the ro operation to fail (hanging onto a file probably) and that's what can be a disaster. Pim doesn't use a log or pid file on the empeg (in mp3tofid it's the client not the server). I wanted to make use of those files and indeed Roger uses them in his example of an empeg as an rsync server, but I have discovered this 'ro fail' problem. As I said, I need to play with rsync some more to ensure it does what it's supposed to do.

Top
#248711 - 14/03/2005 13:57 Re: empeg file structures [Re: ukengb]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
I guess I can't see the forest through your trees. What's so bad about having rsync run on the empeg as the client? Sounds like someone already solved 90% of the problem, and you're tossing that aside and inventing a new problem, but in the opposite direction. What needs are not met by having rsync do a "pull" from your music server instead of having the music server "push" files to rsyncd on the empeg?
_________________________
- Tony C
my empeg stuff

Top
#248712 - 14/03/2005 15:18 Re: empeg file structures [Re: tonyc]
ukengb
member

Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
Quote:
What's so bad about having rsync run on the empeg as the client? Sounds like someone already solved 90% of the problem, and you're tossing that aside and inventing a new problem, but in the opposite direction. What needs are not met by having rsync do a "pull" from your music server instead of having the music server "push" files to rsyncd on the empeg?


What I am doing here with rsync is merely part of a larger overall project that ties up an iTunes controlled music collection and makes it available to networked Rio Receivers and also can be easily sync'd to a RioCar/empeg. The basic concept is that all this is controlled on the main server (or with iTunes of course) and shouldn't involve messing around on the empeg itself - just connect it and tell the music server to sync, exactly as (j)Emplode do not expect/require any such interaction with the empeg itself.

Not only that, but Roger has implemented an rsync server on an empeg so I'm not the first to to attempt this.

You might imagine from this that what I am doing here is not the same as mp3tofid, e.g. I do NOT read the mp3 tags or the file folder structure that contains the mp3s. I have it all in the relevant databases and is much faster as a result. Hence mp3tofid provides somewhat less than the 90% you propose.

In any case, the problems I currently have are not really related to whether I'm pushing or pulling the data.

I'd like to get to the bottom of why the hacked rsync still doesn't display progress info.

Should I quit the player first? mp3tofid doesn't, but from what I've read it would appear to be a good idea. How can I do that without it re-starting?

I can't get hijack to popup messages. I've copied and pasted Mark's commands but it doesn't work. What else might need doing to enable this?

Is there any way to display a graphic on the empeg's display, i.e. just from the command line?

These are all issues that will help me to tidy up and tweak the UI of my stuff and help in sorting these issues would be much appreciated.

Top
#248713 - 14/03/2005 16:27 Re: empeg file structures [Re: ukengb]
cushman
veteran

Registered: 21/01/2002
Posts: 1380
Loc: Erie, CO
Quote:
I can't get hijack to popup messages. I've copied and pasted Mark's commands but it doesn't work. What else might need doing to enable this?

echo 'popup 5 Test Message' > /proc/empeg_notify

or in a c program, open() /proc/empeg_notify and use write() to write "popup x message" to it.

Quote:
Is there any way to display a graphic on the empeg's display, i.e. just from the command line?

Not unless you write your own program to do this.
_________________________
Mark Cushman

Top
Page 2 of 4 < 1 2 3 4 >