Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#15754 - 28/08/2000 13:17 Making programmer's lives easier...
hamzy
journeyman

Registered: 11/08/2000
Posts: 51
Loc: TX, USA
Hello empeg-team,

I was wondering if you could make some changes to make programming on the empeg easier?
Could there be an option to make the player set the disks Read/Write when it exits (instead of read only)?
Executing the rwm command takes a while to complete.
Modifing /etc/fstab to change the defaults to rw doesn't seem to make a difference.

Also, it would be helpful to set the execute bit for programs that we download to the empeg.

While I am on the topic, I am seeing some strange things:

1) On boot up:

Mounting proc
Mounting first music partition
Mounting second music partition
Tried to mount /dev/hdc4 but got error 6
Error mounting partitions (possibly already mounted)
Remounting first music partition read-only
Remounting second music partition read-only
No secondary hard disk

2) When I make df work with ln -s /proc/mounts /etc/mtab
then I cannot sync anymore. Emplode fails during
stage 2 with error 0xfffffff8

I notice that it says "Deleting items" first, but I
didnt delete anything within emplode.

3) every sync does an fsck (which takes a while).


Top
#15755 - 28/08/2000 13:42 Re: Making programmer's lives easier... [Re: hamzy]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31563
Loc: Seattle, WA
Could there be an option to make the player set the disks Read/Write when it exits (instead of read only)? Executing the rwm command takes a while to complete.

I know nothing about Linux or programming for Linux, but it seems to me there's two problems with that:

1) Your complaint is that rwm takes too long, so you don't want to have to do it yourself when the player exits. But if the player did an rwm upon exit, then it'd just make it take that long to exit the player. So you'd be forcing the end-user to sit through an rwm every time they exit to the shell, even if they didn't need readwrite access during that session.

2) I'm told that you have to make sure to ROM the drives before starting up the player again. If you weren't the one typing RWM in the first place, then you might not remember (or even know) to do that.

Also, it would be helpful to set the execute bit for programs that we download to the empeg.

Is that something that's normally done in Linux? If not, isn't there a good reason for that?

___________
Tony Fabris
_________________________
Tony Fabris

Top
#15756 - 28/08/2000 13:53 Re: Making programmer's lives easier... [Re: tfabris]
hamzy
journeyman

Registered: 11/08/2000
Posts: 51
Loc: TX, USA
Actually I was hoping that it could always execute in read/write mode when I have my "development hat" on.
Then, I could switch into read only mode to listen to music.
That way, I would avoid waiting for any switch between modes (whether I do it or the player does it).

Yes. In unix, programs have to have an execute bit set before they can be run. Otherwise they are considered data.

That was the main reason why I was switching to r/w mode because you cannot change the bit in r/o mode. However, I can see other reasons why one might like to be in r/w mode at all times.


Top
#15757 - 28/08/2000 14:05 Re: Making programmer's lives easier... [Re: hamzy]
trevorp
member

Registered: 08/06/2000
Posts: 144
Loc: Ft Lauderdale, FL
If the player was left in rw mode at all times, then pulling the player out of the dash/pulling the power plug out will cause an fsck, as the partition will be marked dirty. This is annoying, and bad for the data.

The reason it is ro is so that you can pull the power at any time, and not have to worry about corrupting your filesystem.

It would be no fun to have to do a START | SHUTDOWN equivalent every time you turned the player off..

-Trevor

-----
Mk 2, Green 12GB 080000349
_________________________
-Trevor

-----
Mk 2, Green 12GB, Tuner, 2.0b11, 080000349

Top
#15758 - 28/08/2000 14:15 Re: Making programmer's lives easier... [Re: hamzy]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31563
Loc: Seattle, WA
Yes. In unix, programs have to have an execute bit set before they can be run. Otherwise they are considered data.

That's not what I meant. I know that the execute bit has to be set for a program to execute. What I meant was: Is it normal for a Linux machine to automatically tag every uploaded files as executable? And if not, isn't it that way for a reason? I'd think that the Empeg would behave like any other Linux box in that respect, and would behave the same way for the same reasons.

___________
Tony Fabris
_________________________
Tony Fabris

Top
#15759 - 28/08/2000 14:27 Re: Making programmer's lives easier... [Re: trevorp]
hamzy
journeyman

Registered: 11/08/2000
Posts: 51
Loc: TX, USA
Yes, I know being in r/w mode puts you in more danger. That is why I am asking it to be an option. Not the default. I understand why it is in r/o mode all of the time. Perhaps it can be called "expert mode" with the appropriate "Danger Will Robinson" dialog box.


Top
#15760 - 28/08/2000 14:34 Re: Making programmer's lives easier... [Re: tfabris]
hamzy
journeyman

Registered: 11/08/2000
Posts: 51
Loc: TX, USA
It is not normal, but it would not hurt anything (other than look strange in the developer image. The consumer image couldnt tell).

What I meant was to add an option to set the execute bit in emplode. That way, the default would be as usual. One way to accomplish that would be to add a "new executable" and "new other" to emplode. The "new other" would be to allow non-music files to be transfered and the "new tune" could filter non music files out (that are in huge drag&drops).

I think we can accomodate both worlds here ^_^


Top
#15761 - 29/08/2000 02:09 Re: Making programmer's lives easier... [Re: hamzy]
kim
member

Registered: 21/07/1999
Posts: 140
Loc: Helsinki, Finland
Executing the rwm command takes a while to complete.

You can also force mount not to check the drive when remounting:

/bin/mount -n -o remount,rw,nocheck /drive0

That way you'll have read-write access immediately, though, the disk may still be in need of fsck.

Kim


Top
#15762 - 30/08/2000 14:24 Re: Making programmer's lives easier... [Re: hamzy]
dglidden
journeyman

Registered: 21/08/2000
Posts: 62
A little more info on why 'rwm' takes so long. This is not 100% perfectly technically accurate but close enough:

empeg is using the ext2 filesystem for its partitions. The equivalent of the FAT under DOS/Windows is stored on ext2 (and other UNIX filesystems) in multiple locations on the drive. The larger the filesystem, the more locations that superblock backup is kept. That way if one of them gets corrupted, there are multiple copies to refer to instead of just one backup as is the case with FAT. Since even a small empeg still has nearly a 6GB filesystem as its data partition, thats a LOT of copies of the superblock scattered across the disk. One of the pieces of information contained in each superblock backup is the "Dirty bit" mentioned a little earlier. That is a bit that is set any time the partition is mounted read-write and cleared anytime the partition is unmounted. (when you "rom" you are unmounting the partition first, then remounting it read-only which does not affect the "Dirty bit.") A reason "rwm" takes so long is because it's having to update a lot of copies of that "dirty bit" in all the superblocks stored across the filesystem. Another thing that mounting the ext2 filesystem "rwm" does is just do a quick "sanity check" that all those backups blocks are in reasonable shape and the filesystem doesn't need checking.

One way of avoiding this is to do as others have said by using the "check=none" option to the mount command.

Another option that would have to be done by the empeg people is to originally make the filesystem with the "sparse superblocks" option. This just tells the ext2 "formatter" to not make so many superblock backups, which makes mounting large filesystems much quicker. (It also tends to make them less reliable overall since the fewer copies you have available, the more likely it is that a single corrupted copy will not be recoverable.) You can't do this yourself without reformatting your data partition...


Top
#15763 - 30/08/2000 14:27 Re: Making programmer's lives easier... [Re: hamzy]
dglidden
journeyman

Registered: 21/08/2000
Posts: 62
What I meant was to add an option to set the execute bit in emplode. That way, the default would be as usual. One way to accomplish that would be to add a "new executable" and "new other" to emplode.

I'm not sure I agree. I'd much rather see the programmers doing other more generally-useful stuff. Making a file executable under linux is as simple as the command line:

chmod +x [filename]

If you're doing development on/for the empeg, I can guarantee that you're spending at least 100x more time overall on writing code and compiling than you are at marking your files executable. :) I'm not convinced that taking the time to add an additional option to emploade so it would "chmod +x" for you is worth the effort.


Top
#15764 - 31/08/2000 05:41 Re: Making programmer's lives easier... [Re: dglidden]
altman
carpal tunnel

Registered: 19/05/1999
Posts: 3457
Loc: Palo Alto, CA
We're moving to sparse superblocks on the production line now, which will speed up remounts.

The long term aim is to move to a journalling fs - I've been looking at reiserfs, which compiles cleanly but doesn't work yet. We have to get this working for another OEM product, so I suspect it'll leak over at some point. It would mean reformatting, though...

Hugo



Top
#15765 - 31/08/2000 09:46 Re: Making programmer's lives easier... [Re: altman]
Henno
addict

Registered: 15/07/1999
Posts: 568
Loc: Meije, Netherlands
It would mean reformatting, though...

Oh Noh . . .



Henno
mk2 6 nr 6
_________________________
Henno mk2 [orange]6 [/orange]nr 6

Top
#15766 - 31/08/2000 13:09 Re: Making programmer's lives easier... [Re: altman]
dglidden
journeyman

Registered: 21/08/2000
Posts: 62
The long term aim is to move to a journalling fs - I've been looking at reiserfs, which compiles cleanly but doesn't work yet.

Doesn't work on the ARM platform or at all? I've been using it at home for several months on my Intel Linux workstations and it's been pretty much bulletproof for me. I've run some pretty intensive torture tests on it against my file server and it's held up against any evil thing I can think of to do to it so far. There's also the advantage that reiserfs excels at handling directories with very large numbers of files in them. Kinda like /drive0/fids... Although it seems to me that reiserfs consumes more memory than ext2 does, so in a tight-memory situation like on the empeg, that disadvantage might outweigh the advantages.

We have to get this working for another OEM product, so I suspect it'll leak over at some point. It would mean reformatting, though...

There is always the kluge of doing ext2-resize as small as it will go, create reiserfs partition in empty space, move data from ext2->reiserfs, resize ext2 smaller, resize reiserfs bigger, move data, repeat as necessary... :) Although users may just prefer to erase and reinstall their mp3s than wait for this nonsese...

Or you could consider ext3 if it ever comes along in a stable form. Converting from ext2 to ext3 is supposed to be as simple as "umount /drive0; mount -t ext2 /dev/hda4 /drive0" since the physical structure is the same.


Top
#15767 - 31/08/2000 14:10 Re: Making programmer's lives easier... [Re: Henno]
eternalsun
Pooh-Bah

Registered: 09/09/1999
Posts: 1721
Loc: San Jose, CA
Heh, a journaling file system can give you a whole lot more in return :) Like for instance, being able to mount the disks read/write while the vehicle is in motion, for instance.

Calvin


Top
#15768 - 31/08/2000 15:53 Re: Making programmer's lives easier... [Re: dglidden]
altman
carpal tunnel

Registered: 19/05/1999
Posts: 3457
Loc: Palo Alto, CA
No, it doesn't work at all. I suspect an alignment issue, as these are rather common where ARMs are concerned (can't load word from non-word-aligned position, can't load halfword from non-halfword-aligned position).

The product which is having reiserfs has more memory, so this isn't so much of a problem. ext3 is all well & good, except it journals data as well as structure which would slow down syncs rather a bit...

If we ever do switch, I suspect we'll be supporting both fs types for people who don't fancy reloading all their music :)

Hugo



Top