250 GB Drives

Posted by: Ross Wellington

250 GB Drives - 21/01/2008 23:33

Hi,

Has anyone tried out the 250 GB drives from Western Digital (like the WD2500BEVE)? I have generally used the 120GB Toshiba or Hitachi in sets of 2.

While reading comments about drive reliability (Newegg), the Western Digital comments were generally not complimetary.

Were there any difficulty in using the Bigdisk builder during the install or was it as silky smooth as I experienced with my multiple Empegs and 120GB drives?

Any recommendations?

Thanks,

Ross
Posted by: altman

Re: 250 GB Drives - 23/01/2008 04:35

ISTR you'll definitely need the hijack with LBA-48 support - the 120GB drives are under this limit and so don't need it.

Apart from that... good luck...!

Hugo
Posted by: mlord

Re: 250 GB Drives - 23/01/2008 15:05

Yeah. My BigDisk builder images include Hijack for the LBA48 support, and should work (with a tad of luck).

The real question becomes, what new player software limitations will be encountered with these..

smile
Posted by: Ross Wellington

Re: 250 GB Drives - 11/03/2008 14:21

Hi,

Guess who's trying to install 250 GB drives?

Right, having a little trouble with it.


I have followed (I think) the big disk build in the FAQ.

I used your big disk builder (created 09/28/2006) that I had great success with ror multiple 120 GB drives.

This formatted (I assume it also partitions it too), the drive and stressed the disk successfully.

I then used the new hijack car2_v2.01_hijack (created 06/23/07) as directed in the FAQ.

After the reboot, it found the drive but not the file system (see attached). This occurred after several attempts.

==============
Start of list
==============

e000 v1.04for help):
Copying kernel...
Calling linux kernel...
ide_dat

Uncompressing Linux..................................... done, booting the
kernertition (1-4)080
p
Partition number (1-4): 2st
l.ro
Linux version 2.2.17-rmk5-np17-empeg52-hijack-v464 [email protected]) (gcc
versionder (6-16709, default 6)

Command action
2.95.3 20010315 (release)) #2 Fri Oct 27 01:00:29 EDT 20060 36): Poll
result=0 on
Processor: Intel StrongARM-1100 revision 11 (1-4)

p
Par
Checking for extra DRAM:e 0xaaaa read 0x0080sp
c1000000: wrote ffffffff, read e28cc001
First cylinder (11-16709, defa
NetWinder Floating Point Emulator V0.94.1 (c) 1998 Corel Computer Corp.555
read 0x0080
Last cylinder or
empeg-car player (hardware revision 9, serial number 4010385

Page cache hash table entries: 4096 (order 2, 16k)ault
16709):0x000-0x007,0x038 on irq 6
POSIX conformance testing by UNIFIX

Linux NET4.0 for Linux 2.2
Using default value 167
Based upon Swansea University Computer Society NET3.039

Command (m for help): n
NET4: Linux TCP/IP 1.0 for NET4.0, default 1):
IP Protocols: ICMP, UDP, TCP212H9AT00,
TCP: Hash tables configured (ehash 16384 bhas


Hex code (type L to l
ttyS02 at 0xf8030000 (irq = 16) is a SA1100 UART
hda: hda1 < hda5 hda6 > hda2
Signature is 636f6972 'rioc'pe of partition 6 to 82 (Lin
Found custom animation at offset 0x9c388

Command (m for hel
Tuner: loopback=0, ID=-1db2
Scheduling custom logo.

Disk /dev/hda: 255 h
empeg display initialised.indersSK: ext2 filesystem
empeg dsp audio initialised
empeg dsp mixer initialised
empeg dsp initialised= cylinders of 16065
empeg single channel IDE

Calling ioctl() to r
Probing primary interface... music partition read-onlypo
ide_data_test: wrote 0x0000 read 0x8080

ide_data_test: wrote 0xffff read 0x8080th error 16: Device or resource busy.

ide_data_test: wrote 0xaaaa read 0x8080
Pr

ide_data_test: wrote 0x5555 read 0x8080page for additionalmezone:
Atlantic/Sou
ide_data_test: wrote 0x0000 read 0x8080

ide_data_test: wrote 0xffff read 0x8080t determine runlevel - doing soft
reboo
ide_data_test: wrote 0xaaaa read 0x8080

ide_data_test: wrote 0x5555 read 0x8080 instead of reboot from the command
lin
ide_data_test: wrote 0x0000 read 0x8080

ide_data_test: wrote 0xffff read 0x8080: cannot open /var/run/shutdown.pid

ide_data_test: wrote 0xaaaa read 0x8080
kftpd: listening on por
ide_data_test: wrote 0x5555 read 0x808ay.g dsp
ide_data_test: wrote 0x5555 read 0xdbfb -a: use
/etc/shutdown.allow...
hda: WDC WD2500BEVE-00WZT0, ATA DISK drive4231

ide0 at 0x000-0x007,0x038 on irq 6

hda: WDC WD2500BEVE-00WZT0, 238475MB w/8192kB Cache, CHS=30401/255/63, LBA48

-r: reboot afte
empeg-flash driver initialized
smc chip id/revision 0x3349 -n: do n
smc9194.c:v0.12 03/06/96 by Erik Stahlman ([email protected])


SMC9194: SMC91C94(r:9) at 0x4008000 IRQ:7 INTF:TP MEM:6144b MAC
00:02:d7:28:0f:0 -c: cancel a running shutdo

c
RAMDISK: Loading 320 blocks [1 disk] into ram disk... done. **
the "time" argument is mandatory! (try "now
VFS: Mounted root (ext2 filesystem).
empeg-pump v0.03 (19980601)
Press Ctrl-A to enter pump...VFS: Mounted root (ext2 filesystem) readonly.
change_root: old root has d_count=1
Trying to unmount old root ... okay
Freeing unused kernel memory: 4k initempeg init 0.8
I see this is a developer image!
Mounting proc
Mounting first music partition
Tried to mount /dev/hda4 as reiserfs but got error 19
VFS: Can't find an ext2 filesystem on dev ide0(3,4).
Tried to mount /dev/hda4 as ext2 but got error 22
Error mounting partitions (possibly already mounted)
Remounting first music partition read-only
No primary hard disk
Remounting second music partition read-only
No secondary hard disk
Press 'q' now to go into development mode. You Have Zero Seconds To
ComplůStarti
ng player
Timezone: US/Pacific
player.cpp : 385:empeg-car 2.01 2004/07/06.
! tags.cpp : 61:Failed to open tags (0xc0041002).
Prolux 4 empeg car - 2.1434 Jul 5 2004
Vcb: 0x4086d000

==============
End of list
==============


I also tried to partition it manually from another FAQ, which seemed to work, but I had trouble with formatting the drive. I eventually scribbled on the boot partition and started over with your programs.


Here's where I get embarrassed now, what am I missing? I know, I know you warned us all that there might be problems with this size drive...

Have others been successful with 250 GB drives?

Got any ideas?

Thanks,

Ross

Posted by: tfabris

Re: 250 GB Drives - 11/03/2008 14:46

You said you used:

- Builder (bigdisk)
- Hijack (latest)

Somewhere after using the builder, you're supposed to put on the actual *player software*, which is different from the builder. The serial output seems to indicate you did do that, but you didn't say which one you used. Anyway, Mark made a bigdisk version of the player software installation, make sure you used that.
Posted by: mlord

Re: 250 GB Drives - 11/03/2008 16:03

It really looks as if the kernel currently installed isn't a (newish) Hijack one -- except that it *is*.

I'll look around and see if I can find a 250-300GB IDE drive here to connect up, and see if the LBA48 support works for me here with that.

I don't think it's been tested on anything other than a 160GB drive when I first implemented/tested it here.

Does anyone out there know of any empeg with a 160GB or larger drive installed ????

Cheers

Posted by: mlord

Re: 250 GB Drives - 11/03/2008 16:06

Ross,

Connect up a serial cable, and hit Control^C to get a command prompt on the empeg. Then do this:

cat /proc/partitions

and this:

fdisk -l

And post the outputs of those.

Thanks
Posted by: Ross Wellington

Re: 250 GB Drives - 11/03/2008 17:03

Hi,

Yes, I followed Mark's new instructions in the FAQ.

I installed hijack 488 prior to any install as the instructions indicate. Then the Big Disk Builder, then the player hijack that was recommended.

I thought the player software was the car2.v2.01_hijack.

I have tried this on 2 different known good Rio's and 2 different disk drive cables. I have completed the above install and tried v2b13 Hijack as well.

I started with a new 250 GB drive.

Here is the result of the above requests:

<CTRL C>

Restored terminal settings
Remounting first music partition read-only
No primary hard disk
Remounting second music partition read-only
No secondary hard disk
Abnormal player termination
Player received SIGINT, user interruption
Switching to shell-player loop
Starting bash.
empeg:/empeg/bin# cat /proc/partitions
major minor #blocks name

3 0 134217727 hda
3 1 1 hda1
3 2 40162 hda2
3 3 24097 hda3
3 4 134110620 hda4
3 5 24034 hda5
3 6 16033 hda6
empeg:/empeg/bin#
empeg:/empeg/bin# fdisk -l

Usage: fdisk [-l] [-b SSZ] [-u] device
E.g.: fdisk /dev/hda (for the first IDE disk)
or: fdisk /dev/sdc (for the third SCSI disk)
or: fdisk /dev/eda (for the first PS/2 ESDI drive)
or: fdisk /dev/rd/c0d0 or: fdisk /dev/ida/c0d0 (for RAID devices)
...

I assume this is what you really wanted.


empeg:/empeg/bin# fdisk /dev/hda -l

Disk /dev/hda: 255 heads, 63 sectors, 16709 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 5 40131 5 Extended
/dev/hda2 6 10 40162+ 83 Linux
/dev/hda3 11 13 24097+ 10 OPUS
/dev/hda4 14 16709 134110620 83 Linux
/dev/hda5 1 3 24034+ 83 Linux
/dev/hda6 4 5 16033+ 82 Linux swap
empeg:/empeg/bin#


These were the same partition assignments that I got when I manually partitioned the drive too.

By the way, if anyone else is trying to upgrade to these drives, they seem to be in short supply.

Thanks for the help,

Ross
Posted by: Ross Wellington

Re: 250 GB Drives - 11/03/2008 17:30

hi,

One other thing that might be worth mentioning is that when the player powers up, it looks like a blank music partition as in "End of playlist" and obviously, emplode doesn't see the player although the PC host recognizes the USB port cable being installed.

Thanks,

Ross
Posted by: mlord

Re: 250 GB Drives - 12/03/2008 02:06

Originally Posted By: Ross Wellington
hi,

One other thing that might be worth mentioning is that when the player powers up, it looks like a blank music partition as in "End of playlist" and obviously, emplode doesn't see the player although the PC host recognizes the USB port cable being installed.

Thanks,

Ross


Yeah. The log messages indicate that the kernel doesn't see a formatted music partition on /dev/hda4. But the builder ought to have created/formatted that, so something is amiss.

No worries, though. We do have the expertise in house for this sort of thing. smile

At some point soon, I'll dig out a suitably large drive and run through the motions with it on one of the empegs here. And then fix whatever might be wrong.

It may take a week or three though, so do be patient.

But not too patient, as my memory isn't the greatest. wink

Cheers
Posted by: Ross Wellington

Re: 250 GB Drives - 12/03/2008 03:17

Hi,

I'm not in a real hurry, and appreciate your "in-house" expertise.

Thanks,

Ross
Posted by: Ross Wellington

Re: 250 GB Drives - 12/03/2008 23:19

Hi,

Some additional notes about this thread.

I was able to see the 250 GB drive (244 GB after partition & format) on the EMPEG Vital Signs, Boot log, and partition data with fdisk and cat.

I was able to manually partition & format the drive and using v2b13 in the Empeg and emplode would see the drive as 128 GB (no VBA-48). It showed up on the EMPEG Vital Signs, Partition table, and in emplode on USB.

When I manually partitioned and formatted the drive for 250 GB, it would show 244 GB on the EMPEG Vital Signs, the correct Partition Table (same as the big builder does), and emplode would see the player on USB. Emplode completed the file system check, and when rebuilding the database, the EMPEG rebooted and emplode crashed (asking the familiar "Do you want to send this crash dump to Microsoft for analysis" message). Emplode did however play music "ta-da" like it completed the process at the same time emplode crashed.

Does emplode need to be fixed too?

Ross
Posted by: Shonky

Re: 250 GB Drives - 13/03/2008 00:25

Originally Posted By: Ross Wellington
Hi,
Does emplode need to be fixed too?

From what you are saying, maybe it does. Tried jemplode?
Posted by: mlord

Re: 250 GB Drives - 13/03/2008 00:44

Originally Posted By: Ross Wellington
I was able to manually partition & format the drive and using v2b13 in the Empeg and emplode would see the drive as 128 GB (no VBA-48). It showed up on the EMPEG Vital Signs, Partition table, and in emplode on USB.


Do keep in mind, please, that *anything* you do with a drive larger than 128GB installed in an empeg/Rio car unit is meaningless unless Hijack is installed from start to finish.

Which I'm sure you've made a point of doing, but it does bear repeating here. Anything that writes to the disk (fdisk, emplode, database rebuild..) without a Hijack kernel will corrupt the drive.

Cheers
Posted by: Ross Wellington

Re: 250 GB Drives - 13/03/2008 05:35

Hi,

Yes, I understood that. I just wanted to make sure the drive could be addresses within the limits of the available software.

It can be.

It was just a warm and fuzzy that the drive electronics were not the issue.

Thanks,

Ross
Posted by: Ross Wellington

Re: 250 GB Drives - 13/03/2008 05:39

Hi,

Haven't tried jemplode yet. Wanted to get everything else correct with the partitioning (I think it looks good - not a software guy though), formatting, and the file system first.

Thanks,

Ross
Posted by: Roger

Re: 250 GB Drives - 13/03/2008 17:05

Originally Posted By: Ross Wellington
Does emplode need to be fixed too?


In a general sense, yes it does. The odds of that happening are about as slim as a slim person who's lost a lot of weight recently.

In this specific case, I can't think of a reason why emplode would crash just because you've got that much space available. If it's reproducible, use JEmplode. If not, it's just one of those things.
Posted by: Ross Wellington

Re: 250 GB Drives - 13/03/2008 17:22

Hi,

It's very reproducible. I was able to have it happen 3 times after each new partition, format, & system load.

I guess I need to load jemplode and see what it does.

Ross
Posted by: mlord

Re: 250 GB Drives - 23/03/2008 13:43

I'm taking a short break from tweaking my Gray-Hoverman to begin diagnosis of the super large drive issue that Ross has reported here.

There's a 320GB desktop drive hooked up to beater here now, running the bigdisk builder. I'll follow it all the way through loading music and see if any problems can be identified for fixing.

-ml
Posted by: mlord

Re: 250 GB Drives - 23/03/2008 14:11

Originally Posted By: mlord
There's a 320GB desktop drive hooked up to beater here now, running the bigdisk builder. I'll follow it all the way through loading music and see if any problems can be identified for fixing.


Yep. The /dev/hda4 partition never got formatted. Something's wrong with the bigdisk_builder here. Still digging..
Posted by: mlord

Re: 250 GB Drives - 23/03/2008 14:33

Mmm.. the /empeg/bin/scrawl command craps out for me here, causing the build script to abort:

empeg:/# /bin/make-drives
Setting up swapspace version 0, size = 24670208 bytes
Adding Swap: 24092k swap-space (priority -2)
Making first drive...
scrawl(91): user memory violation at pc=0x020046e0, lr=0x02004608 (bad address=0x4014a000, code 2)
pc : [<020046e0>] lr : [<02004608>]
sp : bffffd04 ip : bffffd04 fp : bffffd1c
r10: 40145128 r9 : 02002bc4 r8 : 02020538
r7 : 0200242c r6 : 4000c2a8 r5 : 0202aae0 r4 : 0202b5c8
r3 : 4014a000 r2 : f00f1bb1 r1 : 00000003 r0 : 00000001
Flags: nZCv IRQs on FIQs on Mode USER_32 Segment user
Control: C073D17D Table: C073D17D DAC: 00000015
Function entered at [<020045ec>] from [<02004c64>]
r4 = 0202B5C8
Function entered at [<02004c44>] from [<02004b88>]
r4 = BFFFFD70
Function entered at [<02004968>] from [<020048d8>]
Function entered at [<02004798>] from [<02004200>]
r4 = 020041EC
Function entered at [<020041ec>] from [<02020500>]
r4 = 020041EC
Function entered at [<020204dc>] from [<02002440>]
r5 = 00000002 r4 = BFFFFE04
Function entered at [<0200242c>] from [<4007cfec>]
Function entered at [<4007cee4>] from [<02002b38>]
r10 = 4001D858 r9 = 00000000 r8 = 00000000 r7 = 00000000
r6 = 02002B14 r5 = 00000000 r4 = 4001E5EC
/bin/make-drives: line 1: 91 Segmentation fault /empeg/bin/scrawl "Making $1 filesystem..."


I believe scrawl is just a program to dump progress messages to the front panel display, so we can nuke it, or write an Open Source replacement for it that actually works.

-ml
Posted by: mlord

Re: 250 GB Drives - 23/03/2008 14:42

Originally Posted By: mlord
I believe scrawl is just a program to dump progress messages to the front panel display, so we can nuke it, or write an Open Source replacement for it that actually works.


Here's my replacement for /empeg/bin/scrawl:

#!/bin/sh
echo "POPUP 9999 $*" > /proc/empeg_notify

Requires a Hijack kernel, of course, but we already have that in the new builders.
Posted by: mlord

Re: 250 GB Drives - 23/03/2008 15:18

Originally Posted By: mlord
Originally Posted By: mlord
I believe scrawl is just a program to dump progress messages to the front panel display, so we can nuke it, or write an Open Source replacement for it that actually works.


Here's my replacement for /empeg/bin/scrawl: ...

I'm still retesting things here. I'll post a new builder_bigdisk_v3.upgrade file once it's all testing and working.

-ml
Posted by: Roger

Re: 250 GB Drives - 23/03/2008 16:46

Originally Posted By: mlord
I believe scrawl is just a program to dump progress messages to the front panel display


Yeah, it is. As you point out, with Hijack, you can just use /proc/empeg_notify.
Posted by: mlord

Re: 250 GB Drives - 23/03/2008 19:41

Originally Posted By: mlord
I'll post a new builder_bigdisk_v3.upgrade file once it's all testing and working.


Gag.. this is taking forever, but that's okay -- I been going between the workshop (antenna stuff) and this for much of the afternoon now.

The 320GB drive I'm using here also requires more swap space for the mkfs to work, so the script now uses *both* hda3 and hda6 for swap. And I've updated it to zero hda3 afterwards, so that the dynamic data partition starts off clean instead of semi-random.

And other updates. Now running the (hopefully) final test from scratch.

-ml
Posted by: mlord

builder_bigdisk_v3 now available - 23/03/2008 20:11

Okay, I've updated the bigdisk builder on my web server. Could somebody please copy the new version over to riocar.org (I think there's a mirror of this stuff being kept there) ?

The new version has been tested on a 320GB drive here, and worked fine for me. I have not yet installed any tunes onto the drive, but I'll try that as well, just to confirm things.

Cheers
Posted by: drakino

Re: builder_bigdisk_v3 now available - 24/03/2008 01:20

Mirrored here: http://riocar.org/upload/bigdisk/
Posted by: mlord

Re: builder_bigdisk_v3 now available - 24/03/2008 01:29

Thanks, Tom!

EDIT: I just noticed a double-negative typo in my index.html file, so if you could pull that one again...

Thanks!
Posted by: drakino

Re: builder_bigdisk_v3 now available - 24/03/2008 03:38

Fixed
Posted by: mlord

Re: builder_bigdisk_v3 now available - 24/03/2008 13:10

Here's the new init script that I used for builder_bigdisk_v3. This also eliminates the scrawl and make-drives utilities, as they have been rolled into this one instead to save some execution time RAM.

Attached and inline (We really need a <pre> tag!!):
Code:
#!/bin/sh

log() {
        echo "$1"
        echo "POPUP 9999 $1" > /proc/empeg_notify
}

failed() {
        log "$1 failed"
        /bin/swapoff /dev/hda3
        /bin/swapoff /dev/hda6
        sync
        exec /bin/bash --login
}

makedrive() {
        dev="/dev/$1"
        log "Making $1 filesystem"
        /bin/mkfs.ext2 -v -s 1 -i 131072 -m 0 $dev || failed "mkfs $1"
        log "Tuning $1"
        /bin/tune2fs -c -1 -i0 $dev || failed "tune2fs $1"
        /bin/sync
        /bin/sync
        log "Mounting $1"
        /bin/mount -n -o rw,nocheck $dev $2 || failed "mount $1"
        /bin/sync
        /bin/sync
        log "Making directories"
        /bin/mkdir $2/fids || failed "mkdir /fids"
        /bin/mkdir $2/var || failed "mkdir /var"
        echo "[hijack]" > $2/var/config.ini || failed "config.ini"
        log "Remounting ro"
        /bin/mount -n -o remount,ro $dev $2 || failed "remount"
        /bin/sync
        /bin/sync
        log "$1 completed"
}

# echo everything to the serial console:
set -x

/bin/mount -n /proc
log "Builder image"

DRIVE1=""
if [ -e /proc/ide/hdb -a -e /proc/ide/hdc ]; then
        log 'hdb,hdc both exist?'
        exec /bin/bash --login
fi
[ -e /proc/ide/hdb ] && DRIVE1=hdb
[ -e /proc/ide/hdc ] && DRIVE1=hdc

/bin/mount -n -t ext2 -o ro,nocheck /dev/hda4 /drive0
if [ -d /drive0/fids ]; then
        log "Drives already built"
else
        /bin/umount /drive0

        /bin/mkswap  /dev/hda6
        /bin/swapon  /dev/hda6
        /bin/mkswap  /dev/hda3
        /bin/swapon  /dev/hda3

        log "Tuning hda5"
        /bin/tune2fs -c -1 -i0 /dev/hda5 || failed "tune2fs hda5"

        makedrive hda4 /drive0

        if [ "$DRIVE1" != "" ]; then
                log "partitioning second drive"
                init_drive /dev/$DRIVE1 || failed "init $DRIVE1"
                DRIVE1="${DRIVE1}4"
                makedrive $DRIVE1 /drive1 || failed "init $DRIVE1"
        fi

        /bin/swapoff /dev/hda3
        log "Zeroing hda3"
        /bin/cat /dev/zero > /dev/hda3

        /bin/swapoff /dev/hda6
        log 'Done!  Testing disk..'

        /sbin/hdstress
fi

exec /bin/bash --login
Posted by: drakino

Re: builder_bigdisk_v3 now available - 24/03/2008 14:26

Originally Posted By: mlord
(We really need a <pre> tag!!):


I'm assuming here that you want the code to not be put into a 150px tall scrolling box. If thats correct, I've just modified it to no longer try to calculate the height and cap at 150. This isn't retroactive though, but future uses of the code block should work like it is in your post now.

Random note for any web developers here, the built in developer tools with Safari 3.1 made this really easy to track down. I haven't done much web debugging in any browser lately, but this feature really impressed me. All I did was right click the area I was interested in, and selected Inspect Element. I could then tweak the height tag to verify it was changing what I wanted. It was also displaying the CSS hierarchy to explain where all the attributes were coming from. Really wish tools like this existed years ago when I actually did web stuff for a living.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 24/03/2008 16:39

*Much* better, thanks!
Posted by: oliver

Re: builder_bigdisk_v3 now available - 24/03/2008 18:58

Originally Posted By: drakino
Random note for any web developers here, the built in developer tools with Safari 3.1 made this really easy to track down. I haven't done much web debugging in any browser lately, but this feature really impressed me. All I did was right click the area I was interested in, and selected Inspect Element. I could then tweak the height tag to verify it was changing what I wanted. It was also displaying the CSS hierarchy to explain where all the attributes were coming from. Really wish tools like this existed years ago when I actually did web stuff for a living.


Firebug for Firefox does the same thing, granted it isn't built into the browser like safari. It's also awesome for debugging async callback requests (ajax).
Lately I've been using it to optimize my websites, as I can see every single request and how long it took to download the requested file.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 25/03/2008 03:25

Hi,

I tried this and aside from a few minor difficulties, I was able to run the new Big Disk Builder v3 on a 250 GB Western Digital Parallel ATA drive.

A couple of problems I ran into the first time were, it completed the build normally (I think). I executed the Big Disk Builder v3 and then the car2_v2.01_hijack. Emplode (Windows XP Home) recognized the drive, showed available space of 232 GB. No hijack 488 loaded.

I loaded Hijack 488 onto it and then it wouldn't see the drive in emplode, though hijack worked.

Retried same procedure except this time it told me that the drives were already built. Same result.

I scribbled on the boot of the drive and retried the same procedure. Same result, good build, good player software, good hijack (button lights worked).

I uplugged the player, replugged it back in, good hijack v488. Retried emplode. This time it worked. Emplode saw the player, hijack buttons worked, verified vitals (drive shows up as a 244 GB though (not sure why - large format or reserves or what).

I am now loading a small set of 12 GB of music to it (mix of mostly .wav files and .mp3 files.

I'll let you know how it turns out.

But so far, it looks REAL GOOD!!

Thanks alot Mark, you are good!

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 25/03/2008 03:28

Hi,

Forgot one thing, the disk builder FAQ link to the Big_Disk_Builder (Mark Lord's Special Disk Builder) is broken.

New file name....

Thanks again,

Ross
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 25/03/2008 10:08

Fixed. Thanks.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 25/03/2008 11:02

Originally Posted By: Ross Wellington

I am now loading a small set of 12 GB of music to it (mix of mostly .wav files and .mp3 files.

I'll let you know how it turns out.
But so far, it looks REAL GOOD!!


I think we may still have a problem with it, so don't get too carried away just yet.

My theory is, that since the mkfs needed a *lot* of virtual memory (swap) before it would work, then the fsck will probably have the same issue.

Maybe not immediately, no, that would be too obvious. smile
But perhaps after you've painstakingly added 120GB of tracks to the beast -- *then* it might croak, with no easy means of recovery.

So another builder update will be needed before you can trust it on big drives. We need to change the partitioning scheme, as Roger suggested a while ago, to allocate a much larger swap partition on /dev/hda6.

The code that currently partitions the drive, is in a closed-source binary called init_drive, which is why I didn't fix it right off on the current pass through.

So I'll have to write a shell script to replace that binary, and then we can update the builder one more time. And I might even remember to co-locate the new swap (hda6) alongside the root (hda5) partition, in an attempt to minimize head movement when swapping.

And perhaps integrate the builder functionality into a new universal upgrade file, that includes the player software etc.. for a one-step install. But this would likely also require replacing the closed-source init binary with something more flexible.

And we should probably arrange for the swap to *always* be enabled whenever the player boots up, which will require another init change to force a mkswap on it each time around.

Cheers
Posted by: Roger

Re: builder_bigdisk_v3 now available - 25/03/2008 12:16

Originally Posted By: mlord
But this would likely also require replacing the closed-source init binary with something more flexible.


Code:
while true; do /empeg/bin/player; /bin/bash; done


...is pretty all that init does. OK, OK that isn't the full story: it watches for particular exit codes from player to decide whether to start bash or not.

I wanted to extend it in v3 to map certain exit codes to different actions, and to put the desired exit code in the "Quit Player" protocol request packet (e.g. to run rsync in non-daemon mode and then restart the player when it was finished), but we never got around to it.

init is a binary because we didn't want the shell loaded when we didn't need it, and it's closed source because Hugo wrote it, and his code's always embarassing ;-)

No, seriously, it's closed source probably because it didn't seem worth the effort to open it.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 25/03/2008 13:13

Originally Posted By: Roger
init is a binary because we didn't want the shell loaded when we didn't need it, and it's closed source because Hugo wrote it, and his code's always embarassing ;-)

No, seriously, it's closed source probably because it didn't seem worth the effort to open it.


Quite understandable, and not difficult to replace it either. smile

Thanks.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 25/03/2008 13:16

Originally Posted By: mlord
Code:
..
if [ -d /drive0/fids ]; then
        log "Drives already built"
else
..

Ahh.. I should also remember to have it still build DRIVE1 in that case, if necessary.
Posted by: peter

Re: builder_bigdisk_v3 now available - 25/03/2008 13:18

It's also responsible for mounting the filesystems (the job done in full-size Linux by "mount -a"). The stuff about starting bash or not has to be right for synchronisation to work (synchronisation makes the player exit with a return code that causes init to respawn the player binary only, and not enter bash or reboot the unit). You can probably determine the value in question by running the player from the shell, then making it exit using "emptool --restart".

You also need to not be starting bash if it's not there (consumer image).

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 25/03/2008 13:27

Originally Posted By: peter
It's also responsible for mounting the filesystems (the job done in full-size Linux by "mount -a"). The stuff about starting bash or not has to be right for synchronisation to work (synchronisation makes the player exit with a return code that causes init to respawn the player binary only, and not enter bash or reboot the unit). You can probably determine the value in question by running the player from the shell, then making it exit using "emptool --restart".

You also need to not be starting bash if it's not there (consumer image).

Peter


Thanks. For now, I'll just ignore it, and prepend an "init" script to run before it if I need anything special. The script would then exec the init binary after everything is set up.

But the partitioner is somewhat trickier to write anew, and higher on my list of Things That Ought To Get Done.

Cheers
Posted by: peter

Re: builder_bigdisk_v3 now available - 25/03/2008 13:29

Originally Posted By: mlord
And we should probably arrange for the swap to *always* be enabled whenever the player boots up, which will require another init change to force a mkswap on it each time around.

I'm not so sure that's a good idea, at least for in-car use; I bet Linux starts using its swap, if available, at much lower memory pressures than are generated during normal car-player use. So the disk would keep getting spun up. Plus, I have a feeling the player issues a swapoff() during startup anyway, in case a previous synchronise died and left it enabled.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 25/03/2008 14:15

Originally Posted By: peter
I bet Linux starts using its swap, if available, at much lower memory pressures than are generated during normal car-player use. So the disk would keep getting spun up.

That's generally tunable to taste using /proc/sys/* interfaces, and I suppose we also happen to have the full source code if we care to get fancier.

Quote:
Plus, I have a feeling the player issues a swapoff() during startup anyway, in case a previous synchronise died and left it enabled.

That's easy enough to intercept. smile

Does the player normally try to enable swap during a sync?
Note that swapon -a uses /etc/fstab for the "-a" part, and if swap partitions are not listed there, it has no effect.

swapoff -a should work regardless, though.
Posted by: peter

Re: builder_bigdisk_v3 now available - 25/03/2008 16:21

Originally Posted By: mlord
That's generally tunable to taste using /proc/sys/* interfaces, and I suppose we also happen to have the full source code if we care to get fancier.

Sure, I was just suggesting that most people's taste would be "not at all" during normal playback, which is more easily achieved with swapoff() than /proc/sys/*.

Quote:
Does the player normally try to enable swap during a sync?
Note that swapon -a uses /etc/fstab for the "-a" part, and if swap partitions are not listed there, it has no effect.

IIRC it's not enabled for the whole sync, but it's certainly enabled if fsck needs to be invoked. The player uses swapon(2) not swapon(8), so /etc/fstab isn't consulted, but of course it wouldn't be hard in Hijack to make each call to swapon(2)/swapoff(2) actually do more than one partition if that's what your new disk setup needed. I'm also pretty sure it uses "/swapfile", or some such name, as the argument to swapon(2), rather than the actual partition device, so you could point that symlink somewhere else if it helped.

Peter
Posted by: altman

Re: builder_bigdisk_v3 now available - 25/03/2008 23:25

Whilst I won't disagree that *some examples* of my code are nasty, I don't think init was my code smile

ISTR this was Mike's work. Lots of unixy-shelly things needed to be done there to make stuff work as expected. I remember ctrl-C not working in shells launched by it for a while!

Hugo
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 25/03/2008 23:37

Hi,


Mark said....
Maybe not immediately, no, that would be too obvious.
But perhaps after you've painstakingly added 120GB of tracks
to the beast -- *then* it might croak, with no easy means of
recovery.


Right again!

The player loaded 7 GB and then stopped with a load of warning meaasges. The warning messages had the track number, song title, artist, album as required and then an 8 character hexadecimal suffix. There was a warning file for each song not loaded. I re-sync'd for grins, and it sync'd fine with empty directories for the songs not loaded onto the drive.

Doesn't anything like The Beatles?

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 05/04/2008 12:12

Originally Posted By: Ross Wellington
Have you had time to work on the new builder for large drives?


Originally Posted By: mlord
the partitioner is somewhat trickier to write anew, and higher on my list of Things That Ought To Get Done.


Not yet.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 05/04/2008 13:49

Okay thanks>

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 05/04/2008 14:09

Originally Posted By: mlord
the partitioner is somewhat trickier to write anew, and higher on my list of Things That Ought To Get Done.


Okay, so I did some preliminary work on it this morning.
It's been about a decade since I last looked at low level partition info on IDE drives, so it took a while for it all to come back.

It's not rocket science, but it's not trivial either.


Basic partitioner strategy:

Use GETGEO or IDENTIFY to get drive C/H/S + LBA parameters
Do standard CHS geometry scaling to taste, and to comply with expectations for Large LBA geometries.


Heck, just get them from my /proc/ide/hd?/geometry entries.. duh!.

Construct partition tables as follows:

Extended partition /dev/hda1:
/dev/hda5: 16MB root (or maybe 32MB to give room for more apps)
/dev/hda6: 64MB swap

Other primary partitions:
/dev/hda2: 32MB spare (normally, nothing uses this space)
/dev/hda3: 16MB dynamic data (or maybe double it?)
/dev/hda4: many GB everything else

The current partitioner likes to round up partitions to even cylinders, but we really do not need to do this if we don't want to, apart from padding the MBR out to a full 63 sectors for universal compatibility.

Anyone have any comments or important things to point out or suggest?

In particular, would the player binary actually use a larger dynamic data area?? I know we could put longer bookmarks (and a longer current running order) in their if we left enough space, and we already know how to patch the player binary for those..

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 05/04/2008 14:14

As for implementation, there're a few methods I'm currently considering:
  • write a C-program to just do it all, or
  • use a shell script, with pipes into /bin/fdisk for the low level details.
  • stick /bin/gawk onto the image and use an awk program piped into /bin/fdisk for the low level details.

The first choice means I could take and incorporate it all inside the Hijack kernel, if that turned out to be useful.

The scripting choices may be more work, but are far less opaque in the end (easier to understand, test, and modify).

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 05/04/2008 17:13

Hi,

I had a question in reference to the partitioning and caching of the current song. When a block of the current song is loaded, does any of it get loaded into a disk partition or does it go directly to the RAM for caching? Is this a PIO transfer or DMA?

The reason why I ask is I play mostly .wav files and have noticed that after it has played the first ~30 seconds to 1 minute of the song, there will be short break in the song during the next block fetch. After that, it doesn't have a break in the music (for up to hours). If a song is skipped forward or backward, it will do the same thing (as expected - It's doing a cache flush, right), and play the song for a short time have a short break, then resume the song(s).

A couple of other questions...

1) Will increasing the player RAM help expand the cache or is the cache a fixed size and the RAM would only expand the program space for 3rd party programs, display data, etc?

2) If the RAM was expanded for cache would it's performance be compromised due to a larger cache-flush requiring more time to do it? Will the break be more frequenct or a longer pause?

3) Is the cache 4 way set associative? If not, would that help the performance for the problem I mentioned above?

4) Does the player use the on-disk drive cache? Can it be utilized for cache queuing (sort of like L1, L2 cache so to speak)? Would that improve the performance if it was used?


The problem I mention above is only a minor annoyance. Mostly, I am really asking for an understanding of how things work, for general interest.

Is there a reference that defines the function of the partitions?

By expanding the partition sizes as you mentioned above, how would that improve performance?

Thanks again, and please excuse my ignorance on the operation of the player.

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/04/2008 00:59

Originally Posted By: Ross Wellington
When a block of the current song is loaded, does any of it get loaded into a disk partition or does it go directly to the RAM for caching?

The player software loads it into a malloc'd buffer in RAM. Which means that the Linux kernel reads it into the page-cache (RAM), and them copies it to another location in RAM specified by the player software. Note to software developers: use mmap() instead of read() to avoid the memcpy and 2X memory requirement!!

Quote:
Is this a PIO transfer or DMA?

PIO. The empeg hardware completely lacks any provision for DMA from any device.

Quote:
1) Will increasing the player RAM help expand the cache or is the cache a fixed size and the RAM would only expand the program space for 3rd party programs, display data, etc?

2) If the RAM was expanded for cache would its performance be compromised due to a larger cache-flush requiring more time to do it? Will the break be more frequenct or a longer pause?

Since PIO is used (100% CPU during disk reads), and since the player software actively fights the Linux kernel's automatic caching, the result will be no different. Things will be jittery, possibly with a playback gap, while software is filling RAM with pre-read tracks. Since the RAM is larger, it will take longer to fill it. Where expanded RAM is useful, is in holding the larger data structures that come with large drives -- the filesystem metadata, and the empeg player's database.

Quote:
3) Is the cache 4 way set associative? If not, would that help the performance for the problem I mentioned above?

Dunno to the first question, and not likely to the second. X-way set associative memory caches only help with data that's already in RAM. The problem in the empeg is that there's not much RAM, and the data is on disk to begin with.

Quote:
4) Does the player use the on-disk drive cache?

The player is unaware of it, and the drive automatically uses it.

Quote:
Can it be utilized for cache queuing (sort of like L1, L2 cache so to speak)? Would that improve the performance if it was used?

Already happening. You already see the results.

Quote:
Is there a reference that defines the function of the partitions?

I think it's probably somewhere in the empeg FAQ.

Quote:
By expanding the partition sizes as you mentioned above, how would that improve performance?

Not. But a large drive is more likely (than a small drive) to have large numbers of tracks, possibly exceeding the design limits of the player software. One such group of limits has to do with the maximum number of FIDs ("tracks") in the current running order (or in a bookmark list), and how well this playlist is remembered across power cycles.

The memory for the current running order (and bookmarks) is actually the dynamic data partition on disk. Making this partition larger, in conjunction with some software changes (eg. a modified version of the set_empeg_max_fid hack), could alleviate those particular limits.

* * *

EDIT: A somewhat related discussion came up on the Linux Kernel Mailing List (LKML) last week. I had raised some issues there regarding the 2TB drives that are soon to hit the market, and the ensuing discussion eventually touched upon the same immediate issue that you have experienced: memory requirements for filesystem checks (fsck).

Ted (Tso) suggested that, for a 2TB filesystem, the machine might need as much as 2GB of memory for fsck to be guaranteed to run without crashing..


Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/04/2008 12:57

Here is my first crack at a script to reinitialize an empeg partition table, using larger sizes for the root, swap, and dynamic data partitions.

After comments here, I'll integrate it into a new set of bigdisk_builder images.

Please have a look, and let me know if the sizes look good, and if you can find/fix any potential bugs.

In particular, you can install this script onto an empeg, and run it -- by default it will not actually commit changes to disk, so it is safe to play with.

EDIT: updated again. Current version sticks with an old-fashioned 16MB root filesystem instead of something larger.

Thanks
Code:
#!/bin/bash

function del_partitions(){
        echo "d"
        echo "1"
        echo "d"
        echo "2"
        echo "d"
        echo "3"
        echo "d"
        [ "$SPECIAL4" = "4" ] || echo "4"
}

function enable_errcheck(){
        echo "v"
}

function add_partition(){
        echo "n"
        echo "$1"
        [ "$1" != "l" -a "$2" != "$SPECIAL4" ] && echo "$2"
        echo
        echo "$3"
        if [ "$4" != "" ]; then
                echo "t"
                echo "$2"
                echo "$4"
        fi
}

function make_partitions(){
        del_partitions
        enable_errcheck
        [ "$UNITS" = "K" ] && echo "u"
        add_partition e 1 "+$SZ_HDA1$UNITS"
        add_partition l 5 "+$SZ_HDA5$UNITS" 83
        add_partition l 6 ""                82
        add_partition p 2 "+$SZ_HDA2$UNITS" 83
        add_partition p 3 "+$SZ_HDA3$UNITS" 10
        add_partition p 4 ""                83
        echo "a"
        echo "5"
        echo "p"
        echo $COMMIT
}

function errcheck(){
        DOECHO=0
        ERRCHECK=0
        while read x ; do
                [[ "$x" == Disk\ ${DEV}:*             ]] && DOECHO=1
                [[ "$x" == Command\ \(m\ for\ help\): ]] && DOECHO=0
                [ $DOECHO -eq 1 ] && [[ "$x" != *does\ not\ end\ on* ]] && echo "$x" >&2
                [[ "$x" == *\ unallocated\ sectors    ]] && ERRCHECK=1
                if [ $ERRCHECK -eq 1 ]; then
                        if [[ "$x" == *out\ of\ range.          \
                           || "$x" == *got\ EOF\ thrice*        \
                           || "$x" == *No\ free\ sectors*       \
                           || "$x" == *unknown\ command*        \
                           ]]
                        then
                                echo "ERROR: fdisk: $x" 1>&2
                                exit 1
                        fi
                fi
        done
        exit 0
}

## First parameter is the device path, eg. /dev/hda:
[ "$1" = "" ] && echo "missing device path" && exit 1
DEV=$1

## The default is to "just pretend", and not actually do anything for real.
## Supply "COMMIT" on the command line to actually write the new partition info.
COMMIT=q
[ "$2" = "COMMIT" ] && COMMIT=w

## Figure out how big $DEV is, in kilobytes:
DEV_KB=`sed -n -e"s/  *[0-9][0-9]*  *[0-9][0-9]* *\([0-9]*\) ${DEV##*/}\$/\1/p" < /proc/partitions`
if [ "$DEV_KB" = "" ]; then
        echo "$DEV: not found"
        exit 1
fi

## Newer fdisk versions don't always prompt for partition number 4, but the empeg does:
SPECIAL4="4"
[ "`fdisk -v`" = "fdisk v2.10d" ] && SPECIAL4=NONE      ## empeg

## minimum partition sizes, in $UNITS (defined below):
SZ_HDA5=16      ## root
SZ_HDA6=64      ## swap
SZ_HDA1=$(( SZ_HDA5 + SZ_HDA6 ))
SZ_HDA2=32      ## spare
SZ_HDA3=32      ## dynamic data
SZ_HDA4=        ## FIDs, use all remaining space

## allocation units, normally "M" (megabytes), or possibly "K" (kilobytes):
UNITS=M

## "K" kilobytes: only for testing here on a tiny drive I have:
SZ_TOTAL=$(( SZ_HDA1 + SZ_HDA2 + SZ_HDA3 + 128 ))
[ $(( SZ_TOTAL * 1024 )) -ge $DEV_KB ] && UNITS=K

echo
[ "$SPECIAL4" = "4" ] || echo "$DEV: ${DEV_KB} KB"
make_partitions $COMMIT | fdisk $DEV 2>&1 | errcheck || exit 1
echo "Success"
Posted by: peter

Re: builder_bigdisk_v3 now available - 06/04/2008 14:12

Originally Posted By: mlord

/dev/hda5: 16MB root (or maybe 32MB to give room for more apps)

How well does that work with stock upgrade files? Currently, owners of (already built) big drives can apply any stock upgrade, and then immediately a new Hijack (only), and be fine. IIRC the stock car-player upgrade files contain complete root filesystem images, not tarballs (we only invented embedded tarballs for Jupiter). So they'll come out as 16M, not 32M, filesystems.

Mind you, I suppose anyone installing a stock upgrade is going to eliminate any "more apps" from their root filesystem anyway. I think the recommendation has to remain that "more apps" live on hda2, hda4, or hdb5.

Quote:
/dev/hda6: 64MB swap

That's definitely a good idea, though.

Quote:
In particular, would the player binary actually use a larger dynamic data area?? I know we could put longer bookmarks (and a longer current running order) in their if we left enough space, and we already know how to patch the player binary for those..

The table of offset/length pairs which set-empeg-max-fid already patches, is the only thing which tells the player how big a dynamic data area it has. So no, it won't use a bigger area automatically, but you already know how to tell it to use a bigger area. Nor (if building a new drive from empty) is it important to keep the offsets/lengths compatible with the stock player -- as long as users know they can then never use a stock player on that drive.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/04/2008 16:37

Originally Posted By: peter
Originally Posted By: mlord

/dev/hda5: 16MB root (or maybe 32MB to give room for more apps)

How well does that work with stock upgrade files?

It works fine -- the extra space is simply ignored, and one ends up with a 16MB root filesystem, with 16MB of unused spaced at the end of it, just as already happens on some big drives today.

Quote:
Currently, owners of (already built) big drives can apply any stock upgrade, and then immediately a new Hijack (only), and be fine.

Mmm.. I was going to say, no that's actually unsafe to do, because if the player ever runs (even once), it will write to the disk (dynamic data), and possibly screw up (write to the wrong part of the disk) due to lack of working LBA48 support in the kernel.

But in practice, the entire dynamic data partition is still well below the LBA28 boundary, so it actually will work and is safe to do -- just never try to remount the music partitions r/w.

In practice, though, there is no reason to ever use a "stock" .upgrade file on a bigdisk player, as there are hijacked versions available to use instead.

But it could indeed get a bit messy, because if we go ahead with 32MB root on bigdisks, then we'll have a mix of 16MB and 32MB players out there, and the 32MB .upgrade files won't work on the 16MB players (or worse, the firmware might happily write out all 32MB..).

So, I guess the solution for space on root has to be to use the spare partition, and perhaps update /bin/init etc.. to automatically mount it at boot, and have rw/rwm mount it r/w along with the others.

Ugh. I guess I'll first check and see what the firmware does with a 32MB root file on a 16MB partition..

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/04/2008 16:53

Here is what happens now on a 20GB disk I have here.

First, the "before" view, from a stock .upgrade file:
Code:
Disk /dev/hda: 16 heads, 63 sectors, 38760 cylinders
Units = cylinders of 1008 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1             1        66     33232+   5  Extended
/dev/hda2            67       132     33264   83  Linux
/dev/hda3           133       165     16632   10  OPUS
/dev/hda4           166     38760  19451880   83  Linux
/dev/hda5             1        33     16569   83  Linux
/dev/hda6            34        66     16600+  82  Linux swap


Next, the "after" view, from running the new partitioning script:
Code:
Disk /dev/hda: 16 heads, 63 sectors, 38760 cylinders
Units = cylinders of 1008 * 512 bytes

Device Boot    Start       End    Blocks   Id  System
/dev/hda1             1       163     82120+   5  Extended
/dev/hda2           164       229     33264   83  Linux
/dev/hda3           230       295     33264   10  OPUS
/dev/hda4           296     38760  19386360   83  Linux
/dev/hda5   *         1        33     16569   83  Linux
/dev/hda6            34       163     65488+  82  Linux swap


Of course, the idea is not to run this on an empeg that has already been set-up, but I just wanted to show what would be done differently.

Mmm.. since the root filesystem (/dev/hda5) is now stuck at 16MB, perhaps we should increase the size of the spare (/dev/hda2) partition?
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/04/2008 17:00

Ross:

Could you perhaps install that script onto one of your already-built 250GB drives, and run it and post the results here?

Eg. Save the script as /reinit_partition_table.sh, and upload to the player via FTP:

ftp your.players.ip.addr
(login as root, no passwd)

site rw
put reinit_partition_table.sh
chmod 755 reinit_partition_table.sh
site ro
quit


Then connect over serial, hit Control^C to get a shell prompt, and run the script:

/reinit_partition_table.sh /dev/hda


It will pretend to repartition the drive, and it will print out the results of what would have been. But it won't actually hurt anything for now.

I just want to see if the copy of fdisk on it correctly handles the 250GB drive.

Thanks
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/04/2008 17:13

Originally Posted By: mlord
Ross:

Could you perhaps install that script onto one of your already-built 250GB drives, and run it and post the results here?


And if you want to help things along even more, you could then have it make the changes permanent, by adding the word COMMIT to the end of the command line when running it, as in:
Code:
/reinit_partition_table.sh /dev/hda COMMIT

This will destroy any music on the machine, and you must then re-install the player .upgrade image again after running it.

Give that a try, add some tunes, add some more tunes, and generally beat on things. If it all seems to be working, then I'll roll it into a bigdisk_builder, and respin the .upgrade files etc..

Actually, I might eliminate the need for the builder entirely.. perhaps having Hijack automatically create a partition table when it sees a blank drive. Except for the music partition (/dev/hda5). Then the player software .upgrade files could format the swap on boot, and create/format /dev/hda5 to fit.

That might be nice. I wonder what I'm forgetting here?
Posted by: Roger

Re: builder_bigdisk_v3 now available - 06/04/2008 17:50

Originally Posted By: mlord
That might be nice. I wonder what I'm forgetting here?


When the drive's not blank, but you want to rebuild it anyway? ...

Yeah, I know that you can just cat </dev/zero >/dev/hda for a bit, but...
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 07/04/2008 04:20

Hi,

I'll give it a try tonight.

Thanks!

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 07/04/2008 11:49

Originally Posted By: Ross Wellington
Hi, I'll give it a try tonight.

Mmm.. I've thought about it some more, and it will not work for you as a standalone script -- it still needs to have the filesystems formatted and the like.

So I'll spin a builder image that incorporates it. Later today.

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 07/04/2008 12:47

Originally Posted By: mlord
Originally Posted By: Ross Wellington
Hi, I'll give it a try tonight.

Mmm.. I've thought about it some more, and it will not work for you as a standalone script -- it still needs to have the filesystems formatted and the like.

So I'll spin a builder image that incorporates it. Later today.


Okay, you can get builder_bigdisk_v4.upgrade from my web server at
h t t p : / / r t r . c a / bigdisk /

This is totally untested, but might work.. smile
Posted by: mlord

Re: builder_bigdisk_v3 now available - 07/04/2008 12:49

Originally Posted By: Roger
Originally Posted By: mlord
That might be nice. I wonder what I'm forgetting here?


When the drive's not blank, but you want to rebuild it anyway? ...

Yeah, I know that you can just cat </dev/zero >/dev/hda for a bit, but...


I've now added a FORCE argument to the builder script.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 08/04/2008 04:33

Hi,

The new Builder.v4 is installed.

It came up with the Drives Already Built message and wouldn't build the filesystem until I deleted the fids and var directories.

It looked like it worked correctly, including the new swap modifications. I then installed the car2_v2.01_hijack on it.

Emplode saw it and it had the correct capacity (233GB) which decremented with playlist entries.

It is currently loading 20GB of songs over USB.

I'll let you know how it works out.

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 13:54

Originally Posted By: Ross Wellington
Hi,

The new Builder.v4 is installed.

It came up with the Drives Already Built message and wouldn't build the filesystem until I deleted the fids and var directories.


Yeah, I've got to add a front-panel prompter for that situation.

Meanwhile, though, one can connect over serial, and force it to run with this:

exec /sbin/init FORCE

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 13:57

Quote:
It is currently loading 20GB of songs over USB.


If you can get the ethernet working, it is noticeably faster than the USB for everything, including uploads.

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 14:01

Originally Posted By: mlord
Originally Posted By: Ross Wellington
Hi, I'll give it a try tonight.

Mmm.. I've thought about it some more, and it will not work for you as a standalone script -- it still needs to have the filesystems formatted and the like.

So I'll spin a builder image that incorporates it.


One issue I had while preparing this, was that the initial partitioning is still done by the firmware (?) prior to the /sbin/init script being run -- where we then want to redo the partitioning.

This is not a big problem, because we leave the root filesystem in the exact same place/size in the new partitioning scheme, so programs don't suddenly crash when we do that.

But the kernel doesn't normally allow use of the modified /dev/hda partition table until after a reboot (some of my own ancient code). So the v4 builder image includes a new prerelease Hijack kernel that gets rid of that restriction / safety-valve.

Problem solved. Ain't source code wonderful? smile

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 14:22

Originally Posted By: Ross Wellington
Hi,

The new Builder.v4 is installed.

It came up with the Drives Already Built message and wouldn't build the filesystem until I deleted the fids and var directories.

It looked like it worked correctly, including the new swap modifications. I then installed the car2_v2.01_hijack on it.


Ross, could you check whether or not it still has 64MB swap space now? (and 32MB dynamic data, as well..).

Eg. When you can, just do cat /proc/partitions from the serial port, and post the results here.

I'm trying to figure out how the firmware decides to write a partition table or not. Both the builder and v2.01 .upgrade files include "partition drive" chunks (directives to the firmware), but I haven't yet puzzled out what the firmware does in reaction to those.

Thanks.
Posted by: peter

Re: builder_bigdisk_v3 now available - 08/04/2008 14:50

Originally Posted By: mlord
the initial partitioning is still done by the firmware (?) prior to the /sbin/init script being run

This is /linuxrc in the initrd.
Code:
RAMDISK: ext2 filesystem found at block 0
RAMDISK: Loading 320 blocks [1 disk] into ram disk... done.
VFS: Mounted root (ext2 filesystem).
empeg-pump v0.03 (19980601)
Press Ctrl-A to enter pump...VFS: Mounted root (ext2 filesystem) readonly.
change_root: old root has d_count=1
Trying to unmount old root ... okay

By "pressing Ctrl-A" (i.e. sending \x01 down the serial port) the upgrader stops /linuxrc from mounting the disk-based root partition on / and exiting to {/dev/hda5}/sbin/init -- and, instead, causes it to enter "upgrader mode", where the commands available include partitioning a drive, and filling a partition from a gzipped filesystem image down the serial port.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 16:16

Originally Posted By: peter
Originally Posted By: mlord
the initial partitioning is still done by the firmware (?) prior to the /sbin/init script being run

This is /linuxrc in the initrd.
Code:
RAMDISK: ext2 filesystem found at block 0
RAMDISK: Loading 320 blocks [1 disk] into ram disk... done.
VFS: Mounted root (ext2 filesystem).
empeg-pump v0.03 (19980601)
Press Ctrl-A to enter pump...VFS: Mounted root (ext2 filesystem) readonly.
change_root: old root has d_count=1
Trying to unmount old root ... okay

By "pressing Ctrl-A" (i.e. sending \x01 down the serial port) the upgrader stops /linuxrc from mounting the disk-based root partition on / and exiting to {/dev/hda5}/sbin/init -- and, instead, causes it to enter "upgrader mode", where the commands available include partitioning a drive, and filling a partition from a gzipped filesystem image down the serial port.


Yeah, I see the ext2 initrd (aka. "ramdisk") image in flash (/proc/flash_b0000), but it's mostly just one big closed-source executable.

What I don't know (for sure), is exactly what it does when the .upgrader encounters a "DRIVE" chunk, which seems to mean "create a partition table", sometimes.

Where is the decision made as to whether or not to overwrite the on-disk partition table with a new one? Does the Windows-based upgrader decide, or does linuxrc on the initrd decide?

I'm just wondering if an altered swap/spare/dyndata partition is reverted when a v2.01 or v3a11 software .upgrade file is applied over top of the built disk.

Because all of the .upgrade files seem to include the "DRIVE" chunk for hda, not just the builder .upgrade.

EDIT: What I call a "DRIVE hda" chunk, is a type 0x20 chunk inside the .upgrade file.

Cheers
Posted by: peter

Re: builder_bigdisk_v3 now available - 08/04/2008 16:27

Originally Posted By: mlord
What I don't know (for sure), is exactly what it does when the .upgrader encounters a "DISK" chunk, which seems to mean "create a partition table", sometimes.

Are you dead sure it's "sometimes" and not "always"?

Quote:
Where is the decision made as to whether or not to overwrite the on-disk partition table with a new one? Does the Windows-based upgrader decide, or does linuxrc on the initrd decide?

I'm just wondering if an altered swap/spare/dyndata partition is reverted when a v2.01 or v3a11 software .upgrade file is applied over top of the built disk.

Because all of the .upgrade files seem to include the "DISK" chunk for hda, not just the builder .upgrade.

This was all done before I even started at Empeg, but I expect that all stock upgrade files unconditionally repartition the first drive. That would mean that as long as we didn't move the music partition, we could in theory have fiddled with the other sizes from version to version. We never did change anything, though.

I still think that if the goal is to get more swap on Hijacked players, the easiest thing to do would be mkswap /dev/hda2 (or hdb6), and suborn sys_swapon and sys_swapoff to additionally do their thing on the second swap-partition whenever invoked for /dev/hda6. IMO you really don't want to create a population of players incompatible with stock upgrade files.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 16:42

Originally Posted By: peter
IMO you really don't want to create a population of players incompatible with stock upgrade files.


That's no big deal. We've already got respun .upgrades for 2.01 and v3a11, and with those nobody really needs the originals.

But I do want to be able to increase the swap partition and have it stay that way. This may require replacing the ramdisk (initrd) image, I suppose, with something more open source that we can hack.

But first we need to figure out what it does, and what it does not do.

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 16:51

Originally Posted By: mlord
But I do want to be able to increase the swap partition and have it stay that way. This may require replacing the ramdisk (initrd) image, I suppose, with something more open source that we can hack.

But first we need to figure out what it does, and what it does not do.


Mmmm.. since it is running Linux at that point, I suppose I can simply have the Hijack kernel disallow overwriting a valid empeg partition table from the linuxrc script.

Then we could leave the initrd alone, and just continue to customize things in the init script instead.

Mmm..
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 16:57

Originally Posted By: mlord
Originally Posted By: mlord
But I do want to be able to increase the swap partition and have it stay that way. This may require replacing the ramdisk (initrd) image, I suppose, with something more open source that we can hack.

But first we need to figure out what it does, and what it does not do.


Mmmm.. since it is running Linux at that point, I suppose I can simply have the Hijack kernel disallow overwriting a valid empeg partition table from the linuxrc script.

Then we could leave the initrd alone, and just continue to customize things in the init script instead.


..Or just remove the repartition chunk (type 0x20) from the respun .upgrade files completely. Yeah, that's probably better to do.

Then those respun files will be compatible with empegs built with any builder, rather than being locked to the default factory build scheme (which we've now outgrown).

And should someone accidently install a factory software load by mistake, it will destroy the partition table, and then not be able to find the music partition. We could provide a special builder image to restore things, or just make the stock bigdisk_builder clever enough to handle that.

Mmm... Gotta poke at it to see how it really works now..

Cheers
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 08/04/2008 17:12

Originally Posted By: mlord
Originally Posted By: peter
IMO you really don't want to create a population of players incompatible with stock upgrade files.


That's no big deal. We've already got respun .upgrades for 2.01 and v3a11, and with those nobody really needs the originals.


I respectfully disagree. I'm on Peter's side here. Mark, if your respun upgrades will create players whose music partitions will get hosed by rolling back to a stock upgrade, then there should be gigantic blinking red warning letters all over them.

I understand the need to move forward instead of backward, and I know that altering the partition table can offer important advantages. But so far, you've managed to make a lot of wonderful new things possible for our players without breaking backward compatibility with the stock files. It would be nice if that trend could continue.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 17:31

Originally Posted By: mlord
Originally Posted By: Ross Wellington
Hi,

The new Builder.v4 is installed.

It came up with the Drives Already Built message and wouldn't build the filesystem until I deleted the fids and var directories.

It looked like it worked correctly, including the new swap modifications. I then installed the car2_v2.01_hijack on it.


Ross, could you check whether or not it still has 64MB swap space now? (and 32MB dynamic data, as well..).


Never mind. I just tested it here, and discovered that it fails -- it never creates the 64MB swap at all.

The fdisk program was missing from the builder image, and the copy of /bin/bash on the builder is different from the copy used on the developer software images -- missing features that I used in the new script.

I'll fix both of those, and put up a V5 shortly at the same place, but it may yet need more work to force the later software .upgrade files to not destroy what it does.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 08/04/2008 22:03

Hi,

I guess I was wrong about the swap size. This is the results of the cat after loading the 20 GB.


empeg:/empeg/bin# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 24097 hda3
3 4 244091610 hda4
3 5 24034 hda5
3 6 16033 hda6
empeg:/empeg/bin#


On a positive note, it did complete the sync and successfully loaded the 20GB of songs.

I'll try the new build when its available.


As far as the ethernet, that's the next project...

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 22:46

Originally Posted By: Ross Wellington
I'll try the new build when its available.


Okay, there's a V5 builder there now, and a V5 of v3alpha11 to apply afterwards. This combo works for me.

I don't have a v2.01 there yet.

Cheers
Posted by: mlord

Re: builder_bigdisk_v3 now available - 08/04/2008 22:51

Originally Posted By: mlord
Originally Posted By: Ross Wellington
I'll try the new build when its available.


Okay, there's a V5 builder there now, and a V5 of v3alpha11 to apply afterwards. This combo works for me.

I don't have a v2.01 there yet.

Cheers


Okay, 2.01_v3 is there now too.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 08/04/2008 22:56

Hi,

I'll check them out!

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 08/04/2008 23:24

Hi,

This is what I got with the new v5 build....

Done!
+ echo 'POPUP 999show_message("Done!")
9 Done!'
+ exec /bin/bash --login
bash-2.03# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 64228 hda6
bash-2.03#

Is this correct?

I will install the new v5 player hijack now.

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 08/04/2008 23:42

Hi,

This is what I get with the v5 builder and the v5 player software installed:

empeg:/empeg/bin# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 16033 hda6
empeg:/empeg/bin#


and


empeg:/# cd drive0
empeg:/drive0# ls -la
total 16
drwxr-xr-x 5 0 0 1024 Feb 29 00:36 .
drwxr-xr-x 15 502 220 1024 Apr 9 2008 ..
drwxr-xr-x 2 0 0 1024 Feb 29 00:36 fids
drwxr-xr-x 2 0 0 12288 Feb 29 00:32 lost+found
drwxr-xr-x 2 0 0 1024 Feb 29 00:36 var
empeg:/drive0#


If this looks good, I will load 20 GB of songs on it tonight.

Thanks for all of the hard work!

Ross

Posted by: mlord

Re: builder_bigdisk_v3 now available - 09/04/2008 01:38

Originally Posted By: Ross Wellington
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 64228 hda6

That looks correct, but..

Quote:
This is what I get with the v5 builder and the v5 player software installed:

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 16033 hda6

... but not that. And it should not have been possible to get that, either. I wonder what I did wrong there?

EDIT: You didn't happen to reinstall Hijack in between the builder and the software installer, did you.. a no-no with the new layout.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 09/04/2008 02:11

Hi,

No, I used the procedure that you gave to a recent poster to cause a rebuild to occur:


rwm

rm -r /drive0/fids

rom


Then I executed the v5 builder, after it completed, I sent the previous message.

I then executed the v5 hijack. After it completed, I sent the second message.

I did notice one thing during the build. While writing inodes, with the previous builders, the inode total was 29797. The last v5 builder had a total of 29798.

It looks like the player hijack reset the swap space again.

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 09/04/2008 02:13

Originally Posted By: mlord
You didn't happen to reinstall Hijack in between the builder and the software installer, did you.. a no-no with the new layout.

Yeah, that had to be what went on.

With the new V5 scheme, I've updated the Hijack kernel in all of the V5 files, so you should not overwrite it by installing a previous version in between.

Doing so resulted in the swap space being reduced back down to 16MB from 64MB. Not harmful, but it means that things will crash later (insufficient swap) once the drive is filled up a bit.

You may have been mislead by the original labelling on the files -- I've now updated the index file on my site to indicate that all of the V5 stuff now has hijack v488+ incorporated.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 09/04/2008 02:29

Hi,

I just re-checked the partition tables again and they are the same as I reported earlier (16M swap).

I also verifed that I used the correct files from your website. BTW, I like the way you re-arranged them.

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 09/04/2008 12:33

Originally Posted By: Ross Wellington
Hi,

I just re-checked the partition tables again and they are the same as I reported earlier (16M swap).

I also verifed that I used the correct files from your website. BTW, I like the way you re-arranged them.


Mmm.. okay, I must have been too gentle in my kernel hack to prevent this issue on your specific drives.

V6 is now available, and uses a larger hammer in that code. Untested here (not that it matters, it seems), but I have a good feeling about it.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 10/04/2008 02:29

Hi,

I'll check it out.

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 10/04/2008 03:25

Hi,

I used the same process as I described in the previous email.

This is what I got with the v6 builder....


bash-2.03# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 64228 hda6
bash-2.03#


...... This looks correct ......


This is what I got with the v6 Player Software afterwards....


empeg:/empeg/bin# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 16033 hda6
empeg:/empeg/bin#


...... Looks like the same thing happened again. What am I doing wrong? ......

Do I need to install the FIDS patch that you mention on your website before I install the player software? This shouldn't affect the swap partition should it?


BTW, Its nice to have the player display messages to see what is going on during the installs. What is the "prevented 48258" message?


Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/04/2008 11:17

Originally Posted By: Ross Wellington
This is what I got with the v6 Player Software afterwards....

empeg:/empeg/bin# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 16033 hda6
empeg:/empeg/bin#

...... Looks like the same thing happened again. What am I doing wrong?

It's not you, but rather the frickin closed-source firmware and Windows installer program. They insist on destroying a perfectly good partition layout, even when *not* running a builder image. I'm just using a bit of guesswork to try and prevent that.

Quote:
BTW, Its nice to have the player display messages to see what is going on during the installs. What is the "prevented 48258" message?

Yeah, I'd really love to have some serial port messages, except the stupid software won't work over anything other than pathetically slow serial -- even on Linux we at least have an ethernet-based upgrader (open-source, too!) that runs at perhaps 100X that speed.

I really don't know why I persist in wasting my time to help Windows users with software that I don't need/use here. Stupid me, I suppose.

As for the "prevented 48258" message, there was also a "prevented 0" message too, which goes by so quickly you don't have time to read it. And hopefully also a "prevented 63" message.

Sector 0 is the master partition table, sector 63 is the linked-list entry for hda5, and sector 48258 on your disk is (0 + 63 + 63 + 63 + (2 * 24034) + 1), which was supposed to be the partition table linked-list entry for the swap partition, hda6. But I may still not have that one correct. EDIT: Yeah, looks off by that final +1. Fixed.

If it manages to prevent all three of those, then the partition table layout should not change.

I guess it didn't manage it. I'll try and beef up the messages some more, though that's very difficult to achieve here.

I'll respin it again (V7), with a couple more messages. This time, run the builder (I'll remove the safeguard, so you won't need to manually un-build the drive first), then run the software .upgrader, and watch carefully once the empeg logo appears on the screen (after the kernel is reflashed).

Write down the numbers you see, and post them here, along with the output of "fdisk -l /dev/hda".

That should be enough info for me to nail it properly next time around (or we might get it this time..).

Cheers
Posted by: peter

Re: builder_bigdisk_v3 now available - 10/04/2008 11:49

Originally Posted By: mlord
It's not you, but rather the frickin closed-source firmware and Windows installer program. They insist on destroying a perfectly good partition layout, even when *not* running a builder image. I'm just using a bit of guesswork to try and prevent that.

Did your earlier correct guess that it's the CHUNK_PUMPHDA (0x20) in the upgrade file that tells the firmware to do the re-partition operation, not work out? Omitting that would seem a more robust solution than preventing partition table rewrites on a sector-by-sector basis, particular if the actual sector numbers can vary with disk geometry.

And though most of the Windows installer is closed-source, the code it uses to do the upgrading is actually visible in the Emptool open-source distribution, lib/protocol/upgrader.cpp.

I still also think that you should arrange that, whatever the disk geometry, the music partition ends up at the same sector offset it would with the stock image. And that the easiest way to achieve that would be to repurpose, not resize, the existing partitions.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/04/2008 12:37

Here is what your disk probably looks like with bigdisk:

Code:
Disk /dev/hda: 255 heads, 63 sectors, 16709 cylinders
Units = cylinders of 16065 * 512 bytes

Device    Start     End    Blocks    Id  System
/dev/hda1     1      11     88326     5  Extended
/dev/hda2    12      16     40162+   83  Linux
/dev/hda3    17      21     40162+   10  OPUS
/dev/hda4    22   16709 134110620    83  Linux
/dev/hda5     1       3     24034+   83  Linux
/dev/hda6     4       5     64228+   82  Linux swap

Or, viewed differently:

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl    Start     Size ID
 1 00   1   1    0 254  63   10       63   176652 05
 2 00   0   1   11 254  63   15   176715    80325 83
 3 00   0   1   16 254  63   20   257040    80325 10
 4 00   0   1   21 254  63 1023   337365268092720 83
 5 00   2   1    0 254  63    2       63    48069 83
 6 00   1   1    3 254  63   10       63   128457 82


The master partition table is always at sector 0.
The first extended partition list entry is always at sector 63.

But where is the second one, for the swap partition?
It will be at (63 + 48069 + 63) = 48195.

You can verify this, after running the bigdisk_builder (but before installing the software), using this command:
/bin/read_sector.sh 48195

The last line of output should end in 55 aa.

And sure enough, my off by one error in the last attempt would have missed preventing that from being overwritten..

So the current v8 should actually work, I think.

Cheers

Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/04/2008 12:47

Originally Posted By: peter
Did your earlier correct guess that it's the CHUNK_PUMPHDA (0x20) in the upgrade file that tells the firmware to do the re-partition operation, not work out? Omitting that would seem a more robust solution ..

No, that would only work with the new images. But I'm also trying to prevent a major mishap should somebody get confused and try to install an older image. Much more robust this way. It just failed on the last attempts because I didn't have the math entirely correct. What it's doing now is rather simple: disallow upgrader writes outside of the existing /dev/hda5 partition. Period. It's just taking a long time because I don't run MS Windows here, so I cannot actually test it very easily.

Quote:
I still also think that you should arrange that, whatever the disk geometry, the music partition ends up at the same sector offset it would with the stock image. And that the easiest way to achieve that would be to repurpose, not resize, the existing partitions.


Nope. That doesn't help us. The current spare partition is still too small for 64MB of swap, and some people already use it as an adjunct to the root partition. So it stays as-is, and will not be" repurposed". And the dynamic data partition also could use some growing space to handle the larger running orders that may appear on these new huge drives.

The alternative is to use a swap file, rather than a swap partition, and to place that file onto /dev/hda4. Except I remember that 2.2.xx kernels happen to perform much better when swapping to a partition rather than a file (fixed in newer kernels).

There's nothing sacred about the factory layout that has long since been abandoned -- there'll never be another factory load to worry about, so we can simply use whatever new format we like. With safeguards to prevent accidents when somebody uses an ancient factory build.

Cheers
Posted by: peter

Re: builder_bigdisk_v3 now available - 10/04/2008 13:06

Quote:
The current spare partition is still too small for 64MB of swap

True, but largely irrelevant as it would give a total of 48MB (in fact, due to round-up-to-cylinder, actually nearer 56MB), which is a big improvement on 16.
Quote:
and some people already use it as an adjunct to the root partition

Not on brand-new disks they haven't built before, they don't. I don't know of any third-party add-ons that demand to live on the spare partition and not, as an alternative, the music partition.
Quote:
And the dynamic data partition also could use some growing space to handle the larger running orders that may appear on these new huge drives.

Again true, but largely irrelevant as you've already solved that problem with set-empeg-max-fid, which on typical large disks increases the size to nearly 24MB.
Originally Posted By: mlord
The alternative is to use a swap file, rather than a swap partition, and to place that file onto /dev/hda4. Except I remember that 2.2.xx kernels happen to perform much better when swapping to a partition rather than a file (fixed in newer kernels).

I respectfully suggest that putting a swapfile on /dev/hda4 in order to have enough swap to run fsck on /dev/hda4 is in itself not a great idea wink

Peter
Posted by: peter

Re: builder_bigdisk_v3 now available - 10/04/2008 13:14

Originally Posted By: mlord
With safeguards to prevent accidents when somebody uses an ancient factory build.

Our "ancient" factory builds reflash the kernel before they pump the drive. (Who would have thought the old man had so much blood in him?) There's no safeguard you can put in Hijack to prevent that, because Hijack isn't running at that point.

Peter
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 10/04/2008 13:53

Quote:
I really don't know why I persist in wasting my time to help Windows users with software that I don't need/use here.


Because you are AWESOME, and because you're essentially the only person on the planet with the skills, tools, and knowledge to do it right now.

Do you realize that if it weren't for you, most of the empegs out there would be paperweights by now? You're almost singlehandedly keeping this community going by doing things like this. I'm proud to know you.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/04/2008 14:19

Originally Posted By: peter
I respectfully suggest that putting a swapfile on /dev/hda4 in order to have enough swap to run fsck on /dev/hda4 is in itself not a great idea wink

Heh. Yeah, that too! smile

But I just remembered the real reason for wanting to hack the kernel here: I'm getting rid of the builder image altogether. There's no reason that a (new) oridinary player .upgrade file couldn't include builder functionality, except for this tricky little problem.

Cheers
Posted by: peter

Re: builder_bigdisk_v3 now available - 10/04/2008 14:35

Originally Posted By: mlord
I'm getting rid of the builder image altogether. There's no reason that a (new) oridinary player .upgrade file couldn't include builder functionality, except for this tricky little problem.

FWIW, it was originally done that way out of sheer paranoia. Ensuring that "normal, everyday" upgrade files weren't capable of wiping the user's music by formatting the partition, helped reassure us that we wouldn't accidentally trigger that outcome.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/04/2008 16:54

Okay, there's a fresh set of v9 installers on the site now. These worked for me here a moment ago, and I'm rerunning them again now just for background pleasure. smile

The rather simple workaround is to block all early writes to the raw disk device /dev/hda, unless it lacks a valid empeg partition layout, and allow everything else to pass through.

Nice cool feedback on the "pumping" now, too. One can even see where the upgrader suppresses sending long streams of zeros over the serial link. smile

That'll do it for now.

-ml
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/04/2008 18:29

Originally Posted By: mlord
Okay, there's a fresh set of v9 installers on the site now. These worked for me here a moment ago, and I'm rerunning them again now just for background pleasure. smile

Okay, they worked for me.

You'll have to nuke /drive0/fids/ again before trying them, though.

Cheers
Posted by: altman

Re: builder_bigdisk_v3 now available - 11/04/2008 00:25

Originally Posted By: mlord
The player software loads it into a malloc'd buffer in RAM. Which means that the Linux kernel reads it into the page-cache (RAM), and them copies it to another location in RAM specified by the player software. Note to software developers: use mmap() instead of read() to avoid the memcpy and 2X memory requirement!!


There were good reasons for this; ISTR they are to do with the player running mlock'ed to prevent any non-explicit disk access that the kernel might think is a good idea when the disks are spun down (which then causes blocking at kernel level, as opposed to us explicitly managing the spin up/down in a disk manager thread). JohnR or Mike can probably elaborate on this.

The actual data volume is pretty small in terms of copy overhead so we didn't delve further into the 2.2 kernel to try and fix this.

Hugo
Posted by: altman

Re: builder_bigdisk_v3 now available - 11/04/2008 00:26

Random fact: the "pumping" term comes from how Acorn used to restore their R140 unix boxes in case of being trashed...
Posted by: mlord

Re: builder_bigdisk_v3 now available - 11/04/2008 01:04

Originally Posted By: altman
Originally Posted By: mlord
The player software loads it into a malloc'd buffer in RAM. Which means that the Linux kernel reads it into the page-cache (RAM), and them copies it to another location in RAM specified by the player software. Note to software developers: use mmap() instead of read() to avoid the memcpy and 2X memory requirement!!


There were good reasons for this; ISTR they are to do with the player running mlock'ed to prevent any non-explicit disk access that the kernel might think is a good idea when the disks are spun down (which then causes blocking at kernel level, as opposed to us explicitly managing the spin up/down in a disk manager thread). JohnR or Mike can probably elaborate on this.

The actual data volume is pretty small in terms of copy overhead so we didn't delve further into the 2.2 kernel to try and fix this.

Hugo

No complaints on the day -- your little mp3 player is a great machine, Hugo!

But for future designs.. mmap(). It can even play friendly with mlock/mlockall().

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 11/04/2008 03:37

Hi,

I had to work late tonight, couldn't get to this until now.

I installed using the same procedure tonight and this is what I got after the v9 builder...


bash-2.03# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 64228 hda6
bash-2.03#


This is what I got after the v9 player software....


empeg:/empeg/bin# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 64228 hda6
empeg:/empeg/bin#


!!!!!!!!!!! Looks Good !!!!!!!!!!!

Emplode sees it, it has the correct storage capacity displayed (232 GB).

I will load 20 GB onto it and see what it does.

Always wondered what the term "pumping" meant. I just paralleled it is the DOS world of formatting. Thanks for the clarification.

I certainly appreciate you supporting us poor saps that are stuck with Windows. I was playing with Windows when it was Windows 286 (really archaic) and it has definitely improved since then <grin>. I also tested & qualified UNIX V and SCO UNIX products Graphics drivers. Linux is really looking better every day.

The Empeg wouldn't have near the power without the well-thought-out software and hijacks you participate in the continual development of. You have worked hard on the big disk builders for me and the community. Thanks!

Definitely like having the display messages - they give a hint as to what's going on !!!! They do fly by very quickly and I first saw 02 then 62, blur, blur, then the pumping of hda5.

I'll let you know how it works out.

Thanks again,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 11/04/2008 11:40

Originally Posted By: Ross Wellington
Definitely like having the display messages - they give a hint as to what's going on !!!! They do fly by very quickly and I first saw 02 then 62, blur, blur, then the pumping of hda5.


Yeah, somewhat useful, I believe. And there's still just enough room on the line for a %complete indicator, too.

I may add that when I get around to fixing the errant pump message that happens on each boot, as the root filesystem is mounted (and written to.. wonder what's up with that?).
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 12/04/2008 03:40

Hi,

It completed loading 22 GB of songs without error and everything looks right.

I will load another 30 GB of songs on it and see if it looks good.

So far, So good....

I'll keep you informed.

Thanks alot,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 13/04/2008 03:13

Hi,

Looks like we have a winner here (at least as far as I have tested it)!!!!!

It loaded 54 GB of songs (3428 songs, 113 Albums, 283 Playlists) and worked correcly. Most of these are .wav the rest are .MP-3s.

Do I need to install the FIDS patch still or was that part of the drive build/player software load?

Throughout the next few days (maybe weeks), I will load it up, purchase another 250 GB drive, and load around 400 GB of my CD collection on it. Can the patch be installed at anytime in the process? I will probably erase what is loaded and start anew, so if I need to apply the patch before loading, it's no big deal.

I really need to get my Ethernet up before I load that much onto the drives. Just no time to work on it yet.

I'll let you know if I run into any snags along the way.

You did good Mark!!!

Thanks again,

Ross

Posted by: mlord

Re: builder_bigdisk_v3 now available - 13/04/2008 14:43

Originally Posted By: Ross Wellington
Do I need to install the FIDS patch still or was that part of the drive build/player software load?

The program to do the patch should already be in /bin on the software load, but you do still have to run it (once). I didn't get round to automating that part (and it is disk drive specific, so it cannot be done in advance).

Quote:
Can the patch be installed at anytime in the process?

Earliest is best, like before any music is installed. But I've used it on my (pre-loaded) players here without ill-effects.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 14/04/2008 04:14

Hi,

Okay, I will start from scratch and install the FIDS patch before music load. The patch needs top be run every time a build and player software load is performed since it is drive specific, right?

BTW, how large of drives will the new build/player software accomodate? How large for the FIDS patch?

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 14/04/2008 11:33

Originally Posted By: Ross Wellington
Hi,

Okay, I will start from scratch and install the FIDS patch before music load. The patch needs top be run every time a build and player software load is performed since it is drive specific, right?


I wouldn't bother reloading the stuff that's already there -- it should be fine. But if you build other new disks from scratch, then just run it earlier on those.

The patch just need be installed once after software installation. If you ever reinstall the software, then reinstall the patch.

If you later want to duplicate your drive for a second player (with identical drive type), then just clone the drive, and no patching is required if this is done as a whole drive copy.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 14/04/2008 22:48

Hi,

That's what I will do then.

I will be duplicating the drives for multiple RIOs in my cars and around the house.

I thought the ones around the house can be a larger 3.5 inch drive (with additional cooling, larger power converters, possible separate power for the drives - common grounded of course, chassis tray modifications, etc)

That's why I asked about how large of drives were possible with the new build.

Thanks,

Ross
Posted by: peter

Re: builder_bigdisk_v3 now available - 15/04/2008 13:53

Originally Posted By: Ross Wellington
I thought the ones around the house can be a larger 3.5 inch drive (with additional cooling, larger power converters, possible separate power for the drives - common grounded of course, chassis tray modifications, etc)

That's why I asked about how large of drives were possible with the new build.

Receiver Edition? (but beware this)

Peter
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 17/04/2008 21:45

Hi,

Been busy, I will check this out too.

Does it handle .wav's?

Thanks,

Ross
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 18/04/2008 05:36

you can't stream .wav with the kind of network connection that the receiver and the empeg have. Not fast enough.
Posted by: peter

Re: builder_bigdisk_v3 now available - 18/04/2008 07:07

Originally Posted By: tfabris
you can't stream .wav with the kind of network connection that the receiver and the empeg have. Not fast enough.

On real Receivers, you're right; while it's impressive that that HomePNA stuff worked at all, it never worked as well as it was supposed to. (Receivers have Ethernet too, which is certainly capable of streaming WAV, but we thought HomePNA users would be in the majority, and didn't want to make them second-class citizens.) For Receiver Edition, it works fine, although you need a server that's prepared to give you the WAV files in the first place. Most don't, as they're intended for use with real Receivers which can't "decode" the WAV format even if they could stream it fast enough. With a custom server, I've decoded Freeview DVB-T radio on the server and sent it as WAV to a Receiver Edition car-player.

If you're still worried about WAV bandwidth, the Receiver Edition is a v3 build, so you could send it FLAC instead.

Peter
Posted by: Attack

Re: builder_bigdisk_v3 now available - 18/04/2008 19:42

Originally Posted By: mlord
Okay, there's a fresh set of v9 installers on the site now. These worked for me here a moment ago, and I'm rerunning them again now just for background pleasure. smile

The rather simple workaround is to block all early writes to the raw disk device /dev/hda, unless it lacks a valid empeg partition layout, and allow everything else to pass through.

Nice cool feedback on the "pumping" now, too. One can even see where the upgrader suppresses sending long streams of zeros over the serial link. smile

That'll do it for now.

-ml


I have 2 160 GB disks I want to install in my Empeg. Will the new disk builder setup both disks at once or will I need to do the disks one at a time?
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 18/04/2008 20:15

Hi,

I only did mine a disk at a time to make them both bootable if necessary. Maybe the new version already does that, I didn't try it.

I haven't tried it with both drives, but I think Mark was trying to do that if you look at one of his previous responses in this thread.

I could try it with my 250 GB drives if you need me to...

Ross
Posted by: Attack

Re: builder_bigdisk_v3 now available - 18/04/2008 21:52

Originally Posted By: Ross Wellington

I could try it with my 250 GB drives if you need me to...



No, I will try it myself this weekend.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 19/04/2008 11:07

Originally Posted By: Attack
I have 2 160 GB disks I want to install in my Empeg. Will the new disk builder setup both disks at once or will I need to do the disks one at a time?


It should do both of them.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 24/04/2008 15:06

Hi,

I checked this out with 2 each 250GB drives using v9 builder.

It looked like everything was going well. It completed the build of hda correctly. It created the proper inodes for hdb until the end of the process it asked the following question which I had a 50/50 shot at getting it wrong. The following list shows what happened....

+ DRIVE12=hdb5
+ logshow_message("Making ")
'Making '
+ echo 'Making '
Making
+ echo 'POPUP 9999 Making '
+ /bin/mkfs.ext2 /dev/
mke2fs 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
/dev/ is not a block special device.
Proceed anyway? (y,n) y
ext2fs_check_if_mount: No such file or directory while determining whether /dev/
is mounted.
/dev/: Invalid argument passed to ext2 library while setting up superblock
+ /bin/swapoff /dev/
swapoff: /dev/: Invalid argument
+ /bin/swapoff /dev/hda3
+ /bin/swapoff /dev/hda6
+ log 'Done!'
+ echo 'Done!'
Done!
+ echoshow_message("Done!")
'POPUP 9999 Done!'
+ exec /bin/bash --login
bash-2.03#

It looks like it got an unexpected response.

The partitions appear to be built identical (correctly?)....

bash-2.03# cat /proc/partitions
major minor #blocks name

3 0 244198584 hda
3 1 1 hda1
3 2 40162 hda2
3 3 40162 hda3
3 4 244027350 hda4
3 5 24034 hda5
3 6 64228 hda6
3 64 244198584 hdb
3 65 1 hdb1
3 66 40162 hdb2
3 67 40162 hdb3
3 68 244027350 hdb4
3 69 24034 hdb5
3 70 64228 hdb6
bash-2.03#


Did I just answer wrong or is there a problem?

Mark, what do you think?

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 24/04/2008 17:39

I wonder what I was smoking when I did that to the script? blush

Fixed in builder_bigdisk_v10.upgrade at the usual place.

Thanks for testing things!
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 24/04/2008 21:28

I'll check it out.

Thanks again,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 24/04/2008 22:36

Hi,

Ahhh much better!

Once again the IDE_GUY proves that he is the "Master" and not the "Slave" of code.

It might be good when we are all done (very close now), that you sync the versions (like v10 for the builder and player software).

BTW, who's RobV? Anyway, Happy Birthday RobV.

Good job,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 25/04/2008 00:37

Originally Posted By: Ross Wellington
BTW, who's RobV? Anyway, Happy Birthday RobV.

That would be Rob Voisey, who first emailed me on August 14, 2000:

Originally Posted By: Rob Voisey, empeg Limited

Hi!

Your Registration Number: 10862

We are pleased to announce that we have commenced volume shipping of the new car player. You have received this email because, according to our records, you confirmed your interest in purchasing a player some months ago and we have now reached your position in the queue.

If you are no longer interested in purchasing an empeg car, please disregard this email.

To place your order, please visit our online store at www.empeg.com. Be sure to enter your registration number where prompted. You may order a maximum of three players with your registration number.

Note that the tuner module is not ready yet - you will be able to order one as soon as it becomes available, but this could be a couple of months from now. You will not be charged for shipping, and the module simply plugs on to a connector on the car mount.

Shipping of players is underway now, and new orders should ship within 2-5 working days.

Best Regards

The empeg Sales Team.
[email protected]
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 25/04/2008 21:19

Hi,

In case anyone is interested, here is some information about available music database size with the larger disks.


120 GB drive = 117 GB each for 234 GB total with Emplode

Drives like the Toshiba MK1234GAX, & Hitachi



250 GB drive = 232 GB each for 465 GB total with Emplode

Currently the only 250GB ATA-6 (IDE) drive I know of is the Western Digital WD2500BEVE (I was able to purchase a few for $125.00 US this week). I have not seen the reported boot problem that I have read on the web with either of the first 2 drives. These drives are available but not widely available and stil are inexpensive.


I really appreciate Mark for helping us have this additional capacity.

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 26/04/2008 17:11

Hi,

Okay, got Ethernet working thanks for the Ethernet FAQ default addresses. The FAQs for the Empeg are really great!

One problem encountered though, I have to disable my Kerio Firewall for access to the Empeg (of course it is turned on right now and any time I am not syncing a music database).

I went into the Kerio allowed network access fields in the advanced options and added the new Address/Masks and still no luck.

Is anyone using Kerio and can point me to the correct method to allow the Rio to be excluded from the filters list or allow the network address?

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 03/05/2008 05:19

Hi,

Any one using Kerio Firewall with an Ethernet connection?

I could use a little assistance with this if you are able to help.

Thanks,

Ross
Posted by: Attack

Re: builder_bigdisk_v3 now available - 03/05/2008 15:02

Originally Posted By: Ross Wellington
Hi,

Any one using Kerio Firewall with an Ethernet connection?

I could use a little assistance with this if you are able to help.

Thanks,

Ross


I don't use Kerio Firewall anymore it was about 4 years ago when I stopped. You need to allow the local network but I don't know how since I don't use it anymore.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 03/05/2008 15:06

Hi,

I think I will just have to work this with Kerio. probably something that I'm stupid about again.

Thanks for replying,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 04/05/2008 00:58

Hi,

I checked with Kerio. They no longer support the Kerio Firewall that I have and sold it off to sun-belt. Deleted Kerio.

Bought Norton Internet Security 2008, that solved the problem.

Thanks,

Ross
Posted by: Attack

Re: builder_bigdisk_v3 now available - 04/05/2008 01:21

Originally Posted By: Ross Wellington
Hi,

I checked with Kerio. They no longer support the Kerio Firewall that I have and sold it off to sun-belt. Deleted Kerio.

Bought Norton Internet Security 2008, that solved the problem.

Thanks,

Ross


Please remove all Norton software from your computer. It's got to be in the running for the worst software ever made. I've been using NOD32 for more than 2 years now and they just released SmartSecurity that has a Firewall built in. I haven't tried it since I use NAT.

Edited to add: I feel the same as Bob the Angry Flower does about Norton.
Posted by: tman

Re: builder_bigdisk_v3 now available - 04/05/2008 11:24

Yup. Got NOD32 here as well. Works well and it isn't a huge memory hog like Norton.
Posted by: BartDG

Re: builder_bigdisk_v3 now available - 04/05/2008 13:36

Agreed. NOD32 is really the best. Norton probably one of the worst. It entangles itself so deeply into your system, IMO it should be regarded as spyware.
Posted by: tman

Re: builder_bigdisk_v3 now available - 04/05/2008 13:48

The enterprise versions of Norton et al are usually better because they don't have the fancy UI and all the other crap bolted on.
Posted by: frog51

Re: builder_bigdisk_v3 now available - 04/05/2008 19:55

Still pretty poor though: badly written cludge-together
Posted by: caseyse

Re: builder_bigdisk_v3 now available - 04/05/2008 23:55

I started using NOD32 a year ago, migrating from Norton business, and I've been happy with the performance. I recently upgraded to their Smart Security package (w. firewall), as I had been using the old Sygate Office Network firewall, which hasn't been supported in years since Symantec purchased it, for the purpose of eradicating its competition - the best software firewall on the market. NOD32's firewall is very unobtrusive and has all the configuration options I would want.
Posted by: ecoen

Re: builder_bigdisk_v3 now available - 24/02/2010 01:19

I had a disk crash in my Rio Car & have installed a new 250GB WD2500BEVE drive. I have tried using both builder_bigdisk_v3 & builder_bigdisk_v4, but either way when I install one of your tweaked upgrade installs (car2_v2.01_hijack.upgrade or car2_v3a11_hijack.upgrade) I end up with the same result:

The Hijack Vital Signs show:
Mk2aL 32MB 244GB

However, Emplode 2.1 or Jemplode 70 both show (and only allow me to fill up to) 128GB of space.

Since I actually removed both a good and a bad 80GB drive that were close to full, I cannot load all of my music back onto the player.

Help! (please)
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 24/02/2010 04:39

I'm surprised by the emplode and jemplode issues you describe. That's the first I've heard anyone mention that happening.

Emplode 2.10 ... Well that's got known issues and shouldn't be used anyway. (only 2.00 is known to be stable.) But Jemplode should do just fine, so I'm surprised. First I'd look to make sure that the shell thinks you've got 250gb there. Vital signs might be reading the IDE hardware directly. I don't know the command required at the shell to show the disk size of drive0, but I'm sure someone here can help with that.

Next, if the shell really reports 250gb, then I'm curious about exactly what goes wrong with jemplode, i.e., exactly what you're loading and how, and exactly what error jemplode is giving.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 24/02/2010 05:37

Hi,

I've loaded up a lot of 250GB WD drives on multiple Mk2a's without problems.I have used the V2.01 Player Hijack Software with the big builder software that Mark Lords crafted. Haven't tried the latest Big-Builder for any size drives yet.

It ought to work. The 244GB on the Vital Signs is correct. When using Emplode (I haven't use J-Emplode), you should see the player capacity as 232GB for one drive and 465GB for 2 drives at the bottom of the GUI window. You could try Emplode.

If you have a lot of info (tags), in your playlists, you may have to adjust the cache reserve to prevent memory errors (mem err).

Anyone been successful with SATA drives yet. They are getting bigger...

Ross
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 24/02/2010 15:51

Yeah. So I think that somehow, despite having used the bigdisk builder, his drive0 is still formatted to only 128gb.

What's the shell command to show the disk size at the shell prompt?
Posted by: wfaulk

Re: builder_bigdisk_v3 now available - 24/02/2010 16:04

df
Posted by: peter

Re: builder_bigdisk_v3 now available - 24/02/2010 16:05

Code:
fdisk -l /dev/hda

Peter
Posted by: wfaulk

Re: builder_bigdisk_v3 now available - 24/02/2010 17:34

Yeah. There's a difference as to disk size (and partition configuration) and filesystem size. I assumed he was looking for filesystem size, but maybe not.
Posted by: peter

Re: builder_bigdisk_v3 now available - 24/02/2010 17:54

Well, really we need both, because as Tony says, one of our candidate diagnoses at this point is that the filesystem was somehow not created the full size of the disk.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 24/02/2010 19:55

It's probably the same old issue, of the builder refusing to build a drive that's already partially built.

So he's gotta wipe what's there, before the bigdisk builder will actually rebuild it.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 24/02/2010 20:07

Originally Posted By: mlord
It's probably the same old issue, of the builder refusing to build a drive that's already partially built.

So he's gotta wipe what's there, before the bigdisk builder will actually rebuild it.

Okay, in case that's the problem, I've now released bigdisk builder v6, which will unconditionally wipe out any existing installation.

Give that one a whirl.
Posted by: ecoen

Re: builder_bigdisk_v3 now available - 25/02/2010 00:14

Thanks for the feedback, but it sounds like I was doing everything by the book... so now I'm going to try the alternative route.

I just finished a manual partitioning and formatting of the 250GB drive on the Empeg. I am installing the car2_v2.01_hijack.upgrade now. Crossing my fingers...

-Ed
Posted by: ecoen

Re: builder_bigdisk_v3 now available - 25/02/2010 00:34

So far so good.. Jemplode 70 showing 232.54GB capacity!
Posted by: hodge555

Re: builder_bigdisk_v3 now available - 15/03/2010 18:44

I'm new onto this forum so please bear with me as I've not read this entire thread but I thought I might share my experience with upgrading my empeg with you anyway. My upgrade process was using the Windows software versions.

Empeg / RIO rebuild with multi-drives

This is my experience during a rebuild with 2x 250GB WD2500BEVE drives on Empeg MkII (sn 080000452).

Using builder_bigdisk_v6.upgrade

both drives initally connected as master/slave

failed to find drives (bad pump) when both drives connected
unplugged drive on end of cable (slave) - then success

unplugged first drive - plugged in 2nd drive on the end of the cable - failed
plugged 2nd drive to first cable point - failed

changed jumpers to make 2nd drive master and plugged it back in on the end of ribbon - success

hence drive needs to be master to work, ribbon point not relevant
hence with 2 drives you must do them one at a time

Whilst each drive is plugged in individually and set to master then run the software installer.
hence for 2 drives you should repeat the process.
used car2_v2.01_hjack.upgrade

Note - even as Master the drive now needs to be on the first ribbon connection (not the end) or it fails

Once complete for both drives, connect both as master/slave and it works fine.
Capacity found as 360GB
Posted by: Robotic

Re: builder_bigdisk_v3 now available - 15/03/2010 20:10

Originally Posted By: hodge555
Capacity found as 360GB
Wow!

Congratulations!
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 15/03/2010 20:30

Originally Posted By: hodge555
Note - even as Master the drive now needs to be on the first ribbon connection (not the end) or it fails


This is interesting. My first reaction is to say that's a symptom of cable trouble or IDE header trouble, and that it's got nothing to do with the builder software.

Mark, can you confirm or deny the concept that the drive's position on the cable could affect the builder? Is there anything, even at the lowest level, that could cause that behavior he observed?

I can understand possibly needing to have only one disk drive plugged in for the builder to work. But having it need to be on a certain header? That just sounds like hardware/header/cable trouble to me.
Posted by: mlord

Re: builder_bigdisk_v3 now available - 16/03/2010 02:54

The cable is not keyed for master/slave, so a single drive should always be at the end of the cable to reduce signal ringing.

If it doesn't work on the end, then there's a cable/connector fault.

Cheers
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 16/03/2010 04:49

Originally Posted By: mlord
If it doesn't work on the end, then there's a cable/connector fault.


...and, according to the behavior he cited, it was an intermittent fault, making it look like the position on the cable was some sort of a requirement.
Posted by: peter

Re: builder_bigdisk_v3 now available - 16/03/2010 16:37

Originally Posted By: hodge555
2x 250GB WD2500BEVE drives
[...]
Capacity found as 360GB

Well, hang on, 360GB is a lot but it's not as much as 2x250GB. Is that a typo for 460GB, or could there be some problem where one of the drives got formatted at 128GB by mistake? 232GB+128GB is suspiciously 360GB...

Peter
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 19/03/2010 05:01

Hi,

I haven't had to worry about drive placement on the cables. Formatted alot of Dual 250GB drive Empeg systems. I haven't used the latest builder though.

Yeah, he should see 244GB for each drive on the Empeg Vitals display (push and hold the round knob on the player for a few seconds, turn it till it highlights Vital Signs, push the knob again).

The first line should say Mk2a: 16MB, 244+244GB.

In Emplode (Windows) it should say at the bottom of the display window 465GB.

That would indicate that you have 2 drives with full formatted (pumped) capacity.


****************************************************************
Anyone tried to use a PATA-to-SATA adapter yet with the 250GB or larger 2.5 inch SATA drives? WD has some 750GB & 1TB drives in the 12.5mm high form factor. I would like to have one drive plus the PATA-to-SATA adapter (I have already modified one for proper connectoring).

Have to get the PATA to SATA to work first (if it will work at all), then I'll probably run out of playlist RAM, adjust cache, re-drill the mounting bracket for the 12.5mm height, and so on, and so on. Maybe it can't work and it will put me out of my misery. Seems like a logical upgrade path for the Empeg, now that PATA laptop drives seemed to have topped out at 320GB.

Am I out of my mind Mark? Probably my next Empeg project...

****************************************************************

Thanks,

Ross
Posted by: hodge555

Re: builder_bigdisk_v3 now available - 28/03/2010 10:52

Vital Signs does say 244+244GB so OK on the build.

The 360 is what Emplode (Windoze file version 2.0.19.0) displays as the capacity on the bottom right corner.

My appologies for any confusion.
Posted by: Shonky

Re: builder_bigdisk_v3 now available - 28/03/2010 21:52

Hijack shows the "raw" capacity as reported by the drive and doesn't care what partitions are set up as I understand it.

Emplode shows the partition capacity and this is probably wrong.

So I would say from what you're telling us, your build is not ok.

run these two commands:
fdisk -l /dev/hda
fdisk -l /dev/hdc
Posted by: hodge555

Re: builder_bigdisk_v3 now available - 28/03/2010 22:18

Yep, you are correct, it looks like the first drive has been setup incorrectly.
Couldn't be the second one could it, nah too easy...

empeg:/empeg/bin# fdisk -l /dev/hda

Disk /dev/hda: 255 heads, 63 sectors, 30401 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 5 40131 5 Extended
/dev/hda2 6 10 40162+ 83 Linux
/dev/hda3 11 13 24097+ 10 OPUS
/dev/hda4 14 16709 134110620 83 Linux
/dev/hda5 1 3 24034+ 83 Linux
/dev/hda6 4 5 16033+ 82 Linux swap

empeg:/empeg/bin# fdisk -l /dev/hdc

Disk /dev/hdc: 255 heads, 63 sectors, 30401 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hdc1 1 11 88326 5 Extended
/dev/hdc2 12 16 40162+ 83 Linux
/dev/hdc3 17 21 40162+ 10 OPUS
/dev/hdc4 22 30401 244027350 83 Linux
/dev/hdc5 * 1 3 24034+ 83 Linux
/dev/hdc6 4 11 64228+ 82 Linux swap
Posted by: Roger

Re: builder_bigdisk_v3 now available - 29/03/2010 05:16

Originally Posted By: hodge555
Yep, you are correct, it looks like the first drive has been setup incorrectly.
Couldn't be the second one could it, nah too easy...


Looks like it. You could swap the drives over, install the developer image, and then partition the (now) second disk manually.

Or you could just run the builder again...
Posted by: hodge555

Re: builder_bigdisk_v3 now available - 29/03/2010 18:01

Oddly the first drive appears to have been a troubled initial build in that it had 104.89GB unallocated and a rebuild from scratch (starting with bigdisk_v6 etc) on it's own, didn't clear that.

I removed it and manually cleared all partitions and then rebuilt it again and it came up as full size this time.

Re-assembled it as second disk and re-ran car2_c2.01_hijack.upgrade and it's now fine.

Total capacity now reported as 465GB :-)

I know I don't really need the build on both drives but it's proved handy this time as the drive with all the MP3's on were on the working drive. I'd only put 60GB on so far but that can take a while to do with a 10meg nic.

thanks for your help
Hodge

PS for confirmation:
empeg:/empeg/bin# fdisk -l /dev/hda

Disk /dev/hda: 255 heads, 63 sectors, 30401 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 11 88326 5 Extended
/dev/hda2 12 16 40162+ 83 Linux
/dev/hda3 17 21 40162+ 10 OPUS
/dev/hda4 22 30401 244027350 83 Linux
/dev/hda5 * 1 3 24034+ 83 Linux
/dev/hda6 4 11 64228+ 82 Linux swap
empeg:/empeg/bin#
empeg:/empeg/bin# fdisk -l /dev/hdc

Disk /dev/hdc: 255 heads, 63 sectors, 30401 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hdc1 1 5 40131 5 Extended
/dev/hdc2 6 10 40162+ 83 Linux
/dev/hdc3 11 13 24097+ 10 OPUS
/dev/hdc4 14 30401 244091610 83 Linux
/dev/hdc5 1 3 24034+ 83 Linux
/dev/hdc6 4 5 16033+ 82 Linux swap
Posted by: jarob10

Re: builder_bigdisk_v3 now available - 02/06/2010 06:11

Originally Posted By: Ross Wellington
Anyone tried to use a PATA-to-SATA adapter yet with the 250GB or larger 2.5 inch SATA drives?


My attempt has been unsuccessful so far, using a WD2500BEAS SATA drive + adaptor.

The attached is the output immediately after running builder_bigdisk_v6.upgrade - are there any obvious errors?
Posted by: mlord

Re: builder_bigdisk_v3 now available - 02/06/2010 11:12

That output all looks good (apart from being mangled by hyperterm).

Next step is to install the empeg software onto the drive.
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 02/06/2010 19:31

Originally Posted By: mlord
(apart from being mangled by hyperterm)


To clarify:

Copy/Paste from Hyperterminal is horribly broken on XP and later. Use "Capture to text file" instead of copy/paste. Explained in a callout note in the relevant FAQ entry.
Posted by: jarob10

Re: builder_bigdisk_v3 now available - 03/06/2010 22:32

Ok I've loaded the software on, then tried an ambitious 60GB upload, and the sync failed with the following:

- Synchronise failed while checking media. Read on socket failed (error 0x8004003d)
- Player stuck in reboot cycle with '0000.-1 sigkill err' in a box
- emplode then crashed

I then manually rebuild the database and synced a single album - success

I am now attempting a 2GB transfer. After about 5 tunes were uploaded the sync appears to have stalled with some error messages on screen, along with some chatter over the serial port - see attached (sorry I didn't have the Capture Text feature running at that time, so its a mangled copy and paste job).
Posted by: mlord

Re: builder_bigdisk_v3 now available - 04/06/2010 18:59

Probably running out of memory, due to the increased usage requirements of managing big drives.

I cannot remember if swap is normally enabled or not during a sync (?), which would help if it was.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 05/06/2010 06:22

Hi,

Is he having a problem with Cache size? I had to adjust mine for large playlists.

I wouldn't think that the SATA interface would use any additional memory. Is the SATA adapter reporting back unexpected information that has to be thrown away or conflicts with configuration files?

I guess I will play with this sometime next week to see how big a drive I can get to run too.

Ross
Posted by: Roger

Re: builder_bigdisk_v3 now available - 05/06/2010 06:39

Originally Posted By: mlord
I cannot remember if swap is normally enabled or not during a sync (?), which would help if it was.


It is, but only on the primary disk.
Posted by: jarob10

Re: builder_bigdisk_v3 now available - 05/06/2010 07:47

Is there any way of increasing swap for the period of the sync?
This is a single drive installation btw (only room on the drive tray for 1 sata drive + adaptor)
Posted by: mlord

Re: builder_bigdisk_v3 now available - 05/06/2010 15:35

Originally Posted By: Ross Wellington
I wouldn't think that the SATA interface would use any additional memory.

No, not the physical device driver, but rather the filesystem layer on top. Bigger disk == bigger filesystem == bigger data structures to keep track of everything.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 08/06/2010 03:11

Hi,



Mark Says...
No, not the physical device driver, but rather the filesystem layer on top. Bigger disk == bigger filesystem == bigger data structures to keep track of everything.




That was one of the reasons why I condensed by tags and file information (lots of information in tags) - to accomodate the existing structure.

I ran out of memory alot until I shrunk things down alot and adjusted cache.


Ross
Posted by: jarob10

Re: builder_bigdisk_v3 now available - 08/06/2010 20:24

Did you run out of memory during the initial sync (like me) or during normal player use?
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 09/06/2010 03:20

Hi,

No, during normal player use. Actually, after a good sync on the first or second item in the play list. I would get a NOMEM xxxx on the display.

Ross
Posted by: frog51

Re: builder_bigdisk_v3 now available - 30/07/2010 23:37

Hi Ross, quick query - what Reserve Cache did you use? I'm running a 250Gb drive, used the v10 big disk builder, and the latest 2.01 with hijack upgrade and jEmplode sees 232.59Gb so that is all good, but before I start to load up the new disk with songs I thought I'd try and get everything right.

Completed:
Build
Telnet - love the fact Mark's latest has telnet daemon built in, so no need for the additional hassle of installing that one

Things I want to do:
Set_max_fids - Can't run it (I get a text file in use error)
Fidsift - is this needed any more?
Set up the cache correctly
Posted by: mlord

Re: builder_bigdisk_v3 now available - 31/07/2010 10:14

Originally Posted By: frog51
..
Set_max_fids - Can't run it (I get a text file in use error

Stop the player application first.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 31/07/2010 14:55

Hi,

I used 16, but my file sizes are quite large.

Ross
Posted by: frog51

Re: builder_bigdisk_v3 now available - 31/07/2010 20:59

Mark - yep, I went through it in my head, and that should have been obvious :-) Got it now... (must be getting old)

Ross - will give that a shot, think I used 10 with previous incarnation on v3 alpha but it is obviously a tad different with this.

Slowly adding tracks on - am up to 'ACDC' this evening - currently doing it in chunks bigger than a couple of gig has had issues. Will see how it goes.

Cheers

Rory
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 01/08/2010 02:11

Hi,

I loaded chunks of 10GB sometimes and it was a little slower.

Mark, is V3 alpha a good base for the larger drives? I thought that we required 2.01 Software.


BTW, I think I recently found a problem with the file system getting too large. I recently loaded an additional 10GB to one drive set which loaded fine - completed normally. That made around 455 GB on the system drive set. When I tried to load on another 2 GB, it didn't complete. While debugging through the serial port using Hyperterminal, I noticed that it can't find the Tags.

(! tags.cpp : 61:Failed to open tags (0xc0041002).

Emplode just croaks, after it starts to rebuild the database, remounts the partitions, stalls and dies with a core dump for debug.

It takes forever to build the database upon power boot. Normal time when powered on from front button panel.


From the FAQ...

1) I performed fscks of both drives, took forever, per the FAQ. No Change.

2) I tried to restore the partition from all of the alternate locations (24577 was best), took forever, per the FAQ. No Change

3) Re-installed Emplode. No Change.

4) Tried 3 other Empegs that hadn't had the additional 10GB installed - they all work fine.

5) I tried to use USB (pronounced Ugh), instead of Ethernet, same thing.

6) Drive cable is good, other drives worked.

I think I broke the database. I don't think that Emplode is hosed, I think it is a player thing.

Funny thing is after it completes building the database, it runs fine. It plays fine, still takes forever to build the database if it is powered off.

I want to remove some and add some more with this system. With this system, I can't even get past initialization in Emplode to play with it. I could clone the drive set from another of my RIOs, but, I am curious as to what happened.

I will probably dump the drives contents and start a new database image anyway.

It's a Mk2a, 16MB RAM, 244+244GB, Player Software car2_v2.01, Builder_Bigdisk_v10.


Serial Port output string:


empeg-car bootstrap v1.02 20001106 ([email protected])
If there is anyone present who wants to upgrade the flash, let them speak now,
or forever hold their peace...it seems not. Let fly the Penguins of Linux!

e000 v1.04
Copying kernel...
Calling linux kernel...
Uncompressing Linux..................................... done, booting the kerne
l.
Linux version 2.2.17-rmk5-np17-empeg55-hijack-v505 ([email protected]) (gcc version
2.95.3 20010315 (release)) #1 Wed Dec 31 11:10:05 EST 2008
Processor: Intel StrongARM-1100 revision 11
Checking for extra DRAM:
c1000000: wrote ffffffff, read e28cc001
NetWinder Floating Point Emulator V0.94.1 (c) 1998 Corel Computer Corp.
empeg-car player (hardware revision 9, serial number 40103852) 16MB DRAM
Command line: mem=16m
Calibrating delay loop... 207.67 BogoMIPS
Memory: 15000k/16M available (996k code, 20k reserved, 364k data, 4k init)
Dentry hash table entries: 2048 (order 2, 16k)
Buffer cache hash table entries: 16384 (order 4, 64k)
Page cache hash table entries: 4096 (order 2, 16k)
POSIX conformance testing by UNIFIX
Linux NET4.0 for Linux 2.2
Based upon Swansea University Computer Society NET3.039
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
TCP: Hash tables configured (ehash 16384 bhash 16384)
IrDA (tm) Protocols for Linux-2.2 (Dag Brattli)
Starting kswapd v 1.5
SA1100 serial driver version 4.27 with no serial options enabled
ttyS00 at 0xf8010000 (irq = 15) is a SA1100 UART
ttyS01 at 0xf8050000 (irq = 17) is a SA1100 UART
ttyS02 at 0xf8030000 (irq = 16) is a SA1100 UART
Signature is 636f6972 'rioc'
Found custom animation at offset 0x9c388
Tuner: loopback=0, ID=-1
show_message("Hijack v505 by Mark Lord")
empeg display initialised.
empeg dsp audio initialised
empeg dsp mixer initialised
empeg dsp initialised
empeg audio-in initialised, CS4231A revision a0
empeg remote control/panel button initialised.
empeg usb initialised, PDIUSBD12 id 1012
empeg state support initialised 0089/88c1 (save to d0005d00).
empeg RDS driver initialised
empeg power-pic driver initialised (first boot)
RAM disk driver initialized: 16 RAM disks of 4096K size
empeg single channel IDE
Probing primary interface...
ide_data_test: wrote 0x0000 read 0x8080
ide_data_test: wrote 0xffff read 0x8080
ide_data_test: wrote 0xaaaa read 0x8080
ide_data_test: wrote 0x5555 read 0x8080
ide_data_test: wrote 0x0000 read 0x8080
ide_data_test: wrote 0xffff read 0x8080
ide_data_test: wrote 0xaaaa read 0x8080
ide_data_test: wrote 0x5555 read 0x8080
ide_data_test: wrote 0x0000 read 0x8080
ide_data_test: wrote 0xffff read 0x8080
ide_data_test: wrote 0xaaaa read 0x8080
ide_data_test: wrote 0x5555 read 0x8080
ide_data_test: wrote 0x0000 read 0x8080
ide_data_test: wrote 0xffff read 0x8080
ide_data_test: wrote 0xaaaa read 0x8080
ide_data_test: wrote 0x5555 read 0x8080
ide_data_test: wrote 0x0000 read 0xdbfb
ide_data_test: wrote 0xffff read 0xdbfb
ide_data_test: wrote 0xaaaa read 0xdbfb
ide_data_test: wrote 0x5555 read 0xdbfb
hda: WDC WD2500BEVE-00WZT0, ATA DISK drive
hdb: WDC WD2500BEVE-00WZT0, ATA DISK drive
ide0 at 0x000-0x007,0x038 on irq 6
hda: WDC WD2500BEVE-00WZT0, 238475MB w/8192kB Cache, CHS=30401/255/63, LBA48
hdb: WDC WD2500BEVE-00WZT0, 238475MB w/8192kB Cache, CHS=30401/255/63, LBA48
empeg-flash driver initialized
smc chip id/revision 0x3349
smc9194.c:v0.12 03/06/96 by Erik Stahlman ([email protected])

SMC9194: SMC91C94(r:9) at 0x4008000 IRQ:7 INTF:TP MEM:6144b MAC 00:02:d7:28:0f:0
c
Partition check:
hda: hda1 < hda5 hda6 > hda2 hda3 hda4
hdb: hdb1 < hdb5 hdb6 > hdb2 hdb3 hdb4
RAMDISK: ext2 filesystem found at block 0
RAMDISK: Loading 320 blocks [1 disk] into ram disk... done.
VFS: Mounted root (ext2 filesystem).
empeg-pump v0.03 (19980601)
Press Ctrl-A to enter pump...VFS: Mounted root (ext2 filesystem) readonly.
change_root: old root has d_count=1
Trying to unmount old root ... okay
Freeing unused kernel memory: 4k initempeg init 0.8
I see this is a developer image!
Mounting proc
Mounting first music partition
Tried to mount /dev/hda4 as reiserfs but got error 19
Mounting second music partition
Remounting first music partition read-only
Remounting second music partition read-only
Press 'q' now to go into development mode. You Have Zero Seconds To Comply...
Starting player
Timezone: MST7MDT
Hijack: intercepting config.ini

hijack: removed menu entry: "Hard Disk Detection"
hijack: removed menu entry: "Serial Port Assignment"
kftpd: listening on port 21
khttpd: listening on port 80
Using non-standard cache size 118 (bonus 0Mb, adjustment 16)
player.cpp : 385:empeg-car 2.01 2004/07/06.
! tags.cpp : 61:Failed to open tags (0xc0041002).
Dead temp.sensor, status=0x00
Prolux 4 empeg car - 2.1434 Jul 5 2004
Vcb: 0x4076d000


............ I performed a CTRL-C to Exit ................

Dead temp.sensor, status=0x00
Restored terminal settings
Remounting first music partition read-only
Remounting second music partition read-only
Abnormal player termination
Player received SIGINT, user interruption
Switching to shell-player loop
Starting bash.
empeg:/empeg/bin#


It really just looks like the tags got hosed. Any ideas where to go from here?

Ross
Posted by: tanstaafl.

Re: builder_bigdisk_v3 now available - 01/08/2010 03:14

Originally Posted By: Ross Wellington
BTW, I think I recently found a problem with the file system getting too large.

Ross, this was "before your time" so to speak... but do you remember when the original empeg came out with a 2 GB hard drive, unless you wanted to spend the extra money and get the massive 4 GB model? I still remember the discussions about whether there was any reason to get the bigger hard drive. Who would need four whole gigabytes of MP3s? When I finally got my first empeg, I knew I was set for life because by then it came with a truly gigantic 10 GB drive. By today's standards I am still a piker with just 160 GB. smile

Now here you are, running more than 100 times the capacity of the largest empeg that was available when emplode was conceived and first written. I guess it wouldn't be too surprising to discover that you may have finally managed to outstrip the limits of the program.

11 years later, who woulda thought...

tanstaafl.
Posted by: peter

Re: builder_bigdisk_v3 now available - 01/08/2010 09:00

Originally Posted By: Ross Wellington
It really just looks like the tags got hosed. Any ideas where to go from here?

Good question. If you'd really exceeded the maximum database size, the player wouldn't be able to rebuild its own database on power-on. The fact that that rebuild works, does give you a potential way to get the player back:
Quote:
Ctrl-C to force quit player
# rwm
# exit

(The "rwm" command takes a while to run.) The player will then laboriously rebuild the database again, but this time should save it to disk afterwards and then be OK from then on.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 01/08/2010 10:05

Originally Posted By: peter
(The "rwm" command takes a while to run.)

If you wish to save 15-minutes of your remaining lifespan, then you could just do what "rwm" does by hand, with the extra flag to cause it to complete immediately:
Code:
mount -n -o remount,rw,nocheck /drive0
mount -n -o remount,rw,nocheck /drive1  ## only if it has two drives

I really ought to have had Hijack just force "nocheck" regardless..
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 03/08/2010 05:12

Hi,

You guys are AMAZING!!!

I tried the rwm command which still had the same problem (unable to see the Tags).

I tried the mount -n -o remount,rw,nocheck /drive0 & drive1

IT FIXED THE PROBLEM!!!!

Serial link ...........

hijack: removed menu entry: "Hard Disk Detection"
hijack: removed menu entry: "Serial Port Assignment"
kftpd: listening on port 21
khttpd: listening on port 80
Using non-standard cache size 118 (bonus 0Mb, adjustment 16)
player.cpp : 385:empeg-car 2.01 2004/07/06.
Dead temp.sensor, status=0x00
Prolux 4 empeg car - 2.1434 Jul 5 2004
Vcb: 0x4076d000


Emplode: Est. Free Space: 2.61GB Capacity: 465GB


AWESOME!

Time to play some more....


Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 01/01/2011 03:54

Hi,

Guess what? The problem returned recently - same issues. Symptoms...

1) It took a long time to rebuild the database on power-plug being applied. Does the same thing every time.

2) With front panel power-on after database rebuild, it has normal response, played fine, for hours, no memory errors

3) When Emplode is used to add/delete songs, couldn't get past rebuilding the database, Emplode times out (because the database is taking too long to "normally" build), core dumps and dies.

4) jEmplode does a similar thing.

5) Serial String shows that it can't find the tags.

6) FSCK can't fix it. Last FSCK rendered the 2nd drive useless after performing it. The drive was not visible afterwards.


Tried all of the fixes, couldn't make it work. The last FSCK I ran tried to fix the boot partition (remember that the -fay answers "yes" to all of the questions - might want to add a caution in the FAQ), and it totally hosed the second drive. There were lots of file type and structure errors that it automatically "fixed" for me - every inodes was fixed, seems like ALL of them. Probably killed the sector map, drive was unusable. Time to rebuild...

Since it looked like it was a problem with the file system reserve_cache again (although, I didn't see any nomem xxxx errors during playing it), and I couldn't get into Emplode to fix the config.ini, I gave up and thought I would try more reserve_cache next time.


Used the latest V508 big disk v6 builder and new v508 v2.01 player compatible software and rebuilt the drives. It is sure slick to be able to build and load new disks compared to earlier days - thanks Mark.

I am reloading the music database again. I am doing it in 25GB chunks and it is of course taking a while to do that. It will take (DAYS) to reload the 460GB+ database. I left the player cover off and allow about 1 hour of cool-down time for both my PC system and the Empeg between 25GB loads. The room temperature is around 70 degrees F. and the drives are barely warm during the load. I could probe them and log temperature, but they are running quite cool this way.

This time I will create a master set of drives.

I did adjust the Reserve_Cache to 18 in config.ini to see if that was the problem.



Still want to pursue SATA drives (both laptop for the cars and desktop for the home units). I know there is a lot of difference for spin-up and access time for the desktop drives. If I can get big enough laptop drives, I will use them. A few questions...

1) Does anyone know about the new Samsung SpinPoint MT2 1TB laptop SATA drive $99.00 USD? I thought I might start with one of those. Samsung drives in particular? I have always favored Western Digital Drives since I lost a 750GB to the dreaded firmware bug. Lost a lot of music database, good thing I had backed up onto my simulation system. Still like to recover the drive contents, does their firmware fix actually fix a drive that isn't recognized by DOS?

2)I read somewhere that the PATA to SATA adapters only address up to 500GB, is that true? If so, are there others available that exceed that limit?

3) Is the Empeg going to handle the 1TB file system? Will I need additional EDO RAM beyond 16MB to manage it? I do have an additional 32 MB I could install. Researched additional older and modern memory chips, not very feasible since that pin-out pretty much topped out at 4M x 16. The next bump in size included a larger package size and pin-out change. The 54 pin package wasn't very compatible with the 50 pin TSOP in the Empeg. Although, I'm not really satisfied yet.


Rory, weren't you trying out the SATA approach with a WD 250GB SATA drive? If so, how did that all turn out?

I hope you all had a great Holiday. Happy New Year (in an hour and 6 minutes here), and a prosperous, kinder 2011 to you all.

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 01/01/2011 04:01

Ooops,

The firmware bug I mention (I didn't curse), was with the Seagate 7200 RPM drives.

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 01/01/2011 13:38

Quote:
2)I read somewhere that the PATA to SATA adapters only address up to 500GB, is that true? If so, are there others available that exceed that limit?

That doesn't sound likely. There is nothing "magic" for PATA drives other than the "128GB barrier", beyond which a new set of R/W opcodes and protocols is used ("LBA-48").

I really expect that anything that works for, say 160GB, will work equally well up to at least 2TB. At 2TB, a 32-bit calculation limit comes into play for some devices/software, including the empeg.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 01/01/2011 17:49

Hi,

I couldn't see any reason why it wouldn't work up to 2TB either.

What do you think of the Samsung? It's 12.5mm high, but that shouldn't matter in the Empeg. Is it going to pull a "Seagate" on me?

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 01/01/2011 20:25

I've got no opinion, one way or the other, on Samsung drives.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 01/01/2011 20:43

Okay.

Thanks,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 05/01/2011 18:46

Originally Posted By: mlord
I've got no opinion, one way or the other, on Samsung drives.

I now do have an opinion, on Samsung Spinpoint HD103SJ (1TB) and HD204UI (2TB) drives: their onboard "power mangement" is crap.

Enable it, and the system crawls to a halt, because the drive unloads heads way too quickly, and on some settings the drives just spin down immediately after each command.

At least it can be controlled (with hdparm and similar), unlike with the WD Green drives.

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 06/01/2011 01:29

Hi,

Good to know Mark.

I have now reloaded 300+GB onto my new reference drive set.

Thanks,

Ross
Posted by: Roger

Re: builder_bigdisk_v3 now available - 06/01/2011 11:08

Originally Posted By: mlord
At least it can be controlled (with hdparm and similar), unlike with the WD Green drives.


Can you remind me how? This could be the problem I'm having with the new drive in my empeg...
Posted by: mlord

Re: builder_bigdisk_v3 now available - 06/01/2011 11:35

hdparm -B255 -S0 -K1 /dev/hda
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 06/01/2011 23:58

Hi,

I'm getting a little frustrated now.

I had the identical error occur again. The player will operate as normal until I power it down or open it with emplode. Then it times out rebuilding the database.

I had 335GB loaded onto it. Tried to load another 18GB onto it. It loaded the files, rebuilt the database and was waiting for the player to respond back.

Same symptoms, tried to fix with the mount -n -o remount,rw,nocheck for both drives, still can't find the tags.

Error:

! tags.cpp : 61: Failed to open tags (0xc0041002)



A few items of interest.

1) The original failure I had was with the drives loaded with 455GB and I was adding 3GB when it failed. I don't suspect a bad location on the drive because the database size is so different (335GB vs 455GB). It would have likely been different locations on the platters.

2) The drives were not overheating during the load process. I did in fact instrument the drives with the player cover removed and compared against ambient. Ambient is between 68 and 70 degrees F. The drive temperatures never exceeded 87 Degrees F.

3) I used a different host computer this time. Dedicated Ethernet switch (the empeg and host are the only agents on the a dedicated ports - no internet connected). Shielded Ethernet cabling. Host hard set for 10Base-T Half Duplex. IP Address hard coded, not broadcasted (no global DNS used). We have very solid power but everything powered on a UPS - no power anomalies logged. All of this configuration was consistent for this several days database load session.

4) Fresh load of V508 Software. Failed with V505 and V508 software - not likely the problem.

5) I had bumped the ReserveCache up to 18 in anticipation the player needed more player memory and caused the failure last time.


Some questions....

Assuming that there isn't enough player memory for the tag database...

1) Does emplode build the tag database then transfer it to the player or does the player build its own tag database based on the installed FID database?

2) Is there a way that I can see how big the tag database is in emplode and in the player?

3) Is there a possibility that I have a bad (corrupted), tag in the 18GB uploaded database that could cause the problem?

4) Should I remove some of the FIDS and rebuild the database on the player (which might remove the possible corrupted tags associated with them)? If that's a good idea, how should I do that?

4) Any other ideas?


Crawling back under my rock....

Thanks,

Ross
Posted by: Roger

Re: builder_bigdisk_v3 now available - 07/01/2011 07:37

Originally Posted By: Ross Wellington
Same symptoms, tried to fix with the mount -n -o remount,rw,nocheck for both drives, still can't find the tags.


"Failed to open tags" just means that the database didn't successfully rebuild. The database is held in three separate files: tags, playlists and database (or database3 for v3a11). "tags" is merely the first file that the player looks for.

Originally Posted By: Ross Wellington
1) Does emplode build the tag database then transfer it to the player or does the player build its own tag database based on the installed FID database?


The second. emplode merely tells the player to build the database. JEmplode, OTOH, has a mode where it'll build the database on the PC.

Originally Posted By: Ross Wellington
2) Is there a way that I can see how big the tag database is in emplode and in the player?


Not unless it's successfully built by the player -- it only exists on disk on the empeg.

Originally Posted By: Ross Wellington
3) Is there a possibility that I have a bad (corrupted), tag in the 18GB uploaded database that could cause the problem?


Possibly, but don't fixate on the "tags" file. It's merely a symptom of the empeg failing to rebuild the entire database. In fact, the "tags" file doesn't hold the tags. It holds the tag names, which is a fairly static list. The tags are in the "database" or "database3" file. See "Cached database", about two-thirds of the way down http://www.differentpla.net/content/2005/02/empeg-file-structures.

Originally Posted By: Ross Wellington
4) Should I remove some of the FIDS and rebuild the database on the player (which might remove the possible corrupted tags associated with them)? If that's a good idea, how should I do that?


You could try that. The quickest way is probably to "mv /drive1/fids /drive1/fids~". That'll remove half of your music in one stroke.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 07/01/2011 09:05

Hi Roger,

I have tried to rebuild the database several times in different ways.

Thanks for answering the questions. I just spent the last hour researching through this (including the link you gave me).

I also remember one of the last things I did after the last sync was to change the the ReserveCache from 18 to 20. I don't think this matters, but, I don't know either. I was looking for another way to just change Config.ini on the player in the var directory. I couldn't invoke vi or something to directly edit it without using the .reg program on the download site.

If the database (file) is deleted would it be able to build a successful database from scratch?

It's after 4:00AM here, I'll try removing 1/2 of the FIDS sometime later today, unless someone has any better ideas.

Thanks for the help.

Ross
Posted by: Roger

Re: builder_bigdisk_v3 now available - 07/01/2011 11:50

Originally Posted By: Ross Wellington
If the database (file) is deleted would it be able to build a successful database from scratch?


The player is capable of rebuilding the database completely from scratch. Run 'rwm', delete /empeg/var/{tags,playlists,database,database3}, Ctrl+D to restart the player. Wait. Ctrl+C to stop player. Run 'rom'.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 07/01/2011 17:07

Hi,

When I looked in var (I checked both hda & hdc), the only thing there is config.ini. I didn't see tags, playlists, or a database there, couldn't delete them. I checked /empeg/var, Drive0/var, and Drive1/var locations. See below:

q
Dead temp.sensor, status=0x00
Restored terminal settings
Remounting first music partition read-only
Remounting second music partition read-only
Player exited normally: 0
Switching to shell-player loop
Starting bash.
empeg:/empeg/bin# cd /drive0
empeg:/drive0# ls -la
total 521
drwxr-xr-x 5 0 0 4096 Aug 23 15:30 .
drwxr-xr-x 15 502 220 1024 Jan 26 2009 ..
drwxr-xr-x 2 0 0 471040 Aug 31 03:41 fids
drwxr-xr-x 2 0 0 49152 Aug 23 15:29 lost+found
drwxr-xr-x 2 0 0 4096 Aug 31 18:07 var
empeg:/drive0# cd var
empeg:/drive0/var# ls -la
total 12
drwxr-xr-x 2 0 0 4096 Aug 31 18:07 .
drwxr-xr-x 5 0 0 4096 Aug 23 15:30 ..
-rw-r--r-- 1 0 0 614 Aug 31 17:51 config.ini
empeg:/drive0/var#



I checked a couple of older Empeg drives....

-rw-r--r-- 1 0 0 1004 Feb 1 17:39 config.ini
-rw-r--r-- 1 0 0 978154 Feb 1 17:39 database
drwxr-xr-x 3 0 0 1024 Oct 27 2036 emptriv
-rw-r--r-- 1 0 0 37704 Feb 1 17:39 playlists
drwxrwxr-x 2 0 0 1024 Oct 25 2036 programs
-rw-r--r-- 1 0 0 376 Feb 1 17:39 tags
empeg:/drive0/var#

----------------------------------------------------------------

empeg:/drive0# ls -la
total 101
drwxr-xr-x 5 0 0 4096 Mar 14 2001 .
drwxr-xr-x 15 505 220 1024 Apr 1 2003 ..
drwxr-xr-x 2 0 0 73728 Apr 1 2009 fids
drwxr-xr-x 2 0 0 16384 Mar 14 2001 lost+found
drwxr-xr-x 2 0 0 4096 Apr 1 2009 var
empeg:/drive0# cd var
empeg:/drive0/var# ls -la
total 1068
drwxr-xr-x 2 0 0 4096 Apr 1 2009 .
drwxr-xr-x 5 0 0 4096 Mar 14 2001 ..
-rw-r--r-- 1 0 0 671 Apr 1 2009 config.ini
-rw-r--r-- 1 0 0 1051311 Apr 1 2009 database
-rw-r--r-- 1 0 0 19324 Apr 1 2009 playlists
-rw-r--r-- 1 0 0 376 Apr 1 2009 tags
empeg:/drive0/var#



On the older two empegs, Drive0/var has tags, database, playlist, etc... Programs is also missing from both of my drives (but this is an old drive).


Guess it didn't even write these files on my player.
Don't have a clue why not. I would think it would need these files to create the playlists.

I did check and there are fids on both drives in the fids directories, although, with 335GB of files, I would have expected to see a lot more fids in the fids directories.



I checked the partitions and noticed that hdc only has a 16MB swapfile - (/dev/hdc6) again. Mark and I had worked through that a long time ago. Either the 2nd drive doesn't need 64MB (I was under the impression that to use the drive in hda position it was needed), we missed it when we were debugging it the first time, or V508 didn't build it right. See below:


Disk /dev/hda: 255 heads, 63 sectors, 30401 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 11 88326 5 Extended
/dev/hda2 12 16 40162+ 83 Linux
/dev/hda3 17 21 40162+ 10 OPUS
/dev/hda4 22 30401 244027350 83 Linux
/dev/hda5 * 1 3 24034+ 83 Linux
/dev/hda6 4 11 64228+ 82 Linux swap
empeg:/bin# fdisk -l /dev/hdc

Disk /dev/hdc: 255 heads, 63 sectors, 30401 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hdc1 1 5 40131 5 Extended
/dev/hdc2 6 10 40162+ 83 Linux
/dev/hdc3 11 13 24097+ 10 OPUS
/dev/hdc4 14 30401 244091610 83 Linux
/dev/hdc5 1 3 24034+ 83 Linux
/dev/hdc6 4 5 16033+ 82 Linux swap
empeg:/bin#


I thought the swap-space was only used on hda6. Am I running out of swap space and is that causing the problem?


Obviously, I'm missing something here (vars and swap partition).


Mark or Roger do you know what is going wrong?

Thanks,

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 10/01/2011 05:14

Hi,

No ideas?

Ross
Posted by: Roger

Re: builder_bigdisk_v3 now available - 10/01/2011 07:40

Originally Posted By: Ross Wellington
No ideas?


What happens when you Ctrl+D and the player restarts and attempts to rebuild the database files? Don't forget to 'rwm' first.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 10/01/2011 08:09

Hi Roger,

Thanks for the reply.

I will try this tomorrow.

Ross
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 10/01/2011 08:48

Hi Roger,

I decided to do this now. It seemed to work.

This is the serial port string...



Before:

Hijack: intercepting config.ini

player.cpp : 385:empeg-car 2.01 2004/07/06.
! tags.cpp : 61:Failed to open tags (0xc0041002).



q
Restored terminal settings
Remounting first music partition read-only
Remounting second music partition read-only
Player exited normally: 0
Switching to shell-player loop
Starting bash.
empeg:/empeg/bin# rwm
empeg:/empeg/bin# logout
Shell exit
Starting player
Timezone: GB
Hijack: intercepting config.ini

player.cpp : 385:empeg-car 2.01 2004/07/06.
! tags.cpp : 61:Failed to open tags (0xc0041002).
Prolux 4 empeg car - 2.1434 Jul 5 2004
Vcb: 0x4086d000

---------------------------------------------------------------

After the rwm and Ctrl+D....

q
Restored terminal settings
Remounting first music partition read-only
Remounting second music partition read-only
Player exited normally: 0
Switching to shell-player loop
Starting bash.
empeg:/empeg/bin# cd /empeg/var
empeg:/empeg/var# ls -la
total 3628
drwxr-xr-x 2 0 0 4096 Sep 3 23:47 .
drwxr-xr-x 5 0 0 4096 Aug 23 15:30 ..
-rw-r--r-- 1 0 0 614 Aug 31 17:51 config.ini
-rw-r--r-- 1 0 0 3565946 Sep 3 23:46 database
-rw-r--r-- 1 0 0 121972 Sep 3 23:47 playlists
-rw-r--r-- 1 0 0 376 Sep 3 23:46 tags
empeg:/empeg/var# exit
logout
Shell exit
Starting player
Timezone: GB
Hijack: intercepting config.ini

player.cpp : 385:empeg-car 2.01 2004/07/06.
Prolux 4 empeg car - 2.1434 Jul 5 2004
Vcb: 0x4086d000

------------------------------------------------------------


Emplode sees it correctly as well.

Looks like it wasn't able to build the database as you had suspected. Don't know why exactly, glad you helped me through it though.

What's the best way to copy the drives (is there a command within the player to make a copy from one drive to the other)?

Thanks again,

Ross
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/01/2011 12:33

If you just want an exact bit-copy of one drive to another, then connect the second (slave) *unpartitioned* drive, and boot (from the first prepared drive).

Hit control^C to get a shell prompt, and do this:

cat /dev/hda > /dev/hdb

Come back and check on it a few days later. smile
Posted by: mlord

Re: builder_bigdisk_v3 now available - 10/01/2011 12:34

Note that the copying can be done MUCH (10-20X) faster if the two drives are removed from the empeg and connected to a faster Linux desktop system for copying.

And regardless of method, enabling the on-drive write cache will speed things up quite a bit as well:

hdparm -W1 -K1 /dev/hda /dev/hdb

Cheers
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 15/01/2011 04:31

Hey Tony,

This information might be good to add to the FAQ. The player lost the database & playlist files in /empeg/var and couldn't rebuild the database in emplode or when power was applied.

Roger recommended a CTRL + D to command the player to rebuild the database, which worked.

It might be good to show where the files should be (missing example and correct example) and how to see them.

Very good to know.

Ross
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 15/01/2011 22:39

The database rebuild entry is already in the FAQ here.

I didn't quite understand what you did that was different from that FAQ entry. What exactly is missing from that FAQ entry that fixed your particular problem?

The "Ctrl-D" thing is new to me. In fact, I'm doubtful that Ctrl-D even does anything to the player since it's the first I've heard of it. Are you sure that wasn't just a typo of Ctrl-C (break to the shell prompt)?
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 16/01/2011 04:19

Hi,

No, the CTRL-D is real. It commanded the player to rebuild the database directly (from scratch as Roger put it). What it provided was the building of the 2 files that were missing (the database and playlist files). Without it, I was stuck. I was grateful that there was a command that did this.

Maybe Roger or someone else can provide a better explanation than what I have. I only know about the effect of using it.

What is different with the CTRL-D vs. powering on the player database rebuild vs. emplode commanded database rebuild?

Are there other commands like the CTRL-D?

Ross
Posted by: Roger

Re: builder_bigdisk_v3 now available - 16/01/2011 06:02

Originally Posted By: tfabris
The "Ctrl-D" thing is new to me.


Ctrl+D is just a shortcut key for "exit". The complete sequence is:

- Quit the player by pressing Ctrl+C
- Mount the disks as read/write, so that the database can be written out, by using "rw" and "rwm".
- (Recommended) turn on swap, by using "swapon /swapfile".
- Press Ctrl+D to exit bash and restart the player.
- The player will rebuild the databases and write them to disk.
- Press Ctrl+C to exit the player.
- Remove swap, remount the disks "swapoff /swapfile; ro; rom"
- Press Ctrl+D to exit bash and restart the player.
Posted by: peter

Re: builder_bigdisk_v3 now available - 16/01/2011 11:28

Originally Posted By: Roger
Ctrl+D is just a shortcut key for "exit". The complete sequence is:

- Quit the player by pressing Ctrl+C
- Mount the disks as read/write, so that the database can be written out, by using "rw" and "rwm".
- (Recommended) turn on swap, by using "swapon /swapfile".
- Press Ctrl+D to exit bash and restart the player.
- The player will rebuild the databases and write them to disk.
- Press Ctrl+C to exit the player.
- Remove swap, remount the disks "swapoff /swapfile; ro; rom"
- Press Ctrl+D to exit bash and restart the player.

The player remounts the disks read-only itself, after database rebuild, if it finds them read/write on startup. Thats's how normal Emplode-mediated database rebuild works. Also, you only need the music partitions read/write ("rwm"), not the root filesystem too ("rw"). I thought (but am not quite sure enough to say that Roger's wrong) that the player did a swapoff on startup too, only re-enabling it for those operations (rebuild, fsck) that need it.

Peter
Posted by: mlord

Re: builder_bigdisk_v3 now available - 16/01/2011 11:36

Originally Posted By: Roger
Originally Posted By: tfabris
The "Ctrl-D" thing is new to me.

Ctrl+D is just a shortcut key for "exit".

Specifically, "Ctrl^D" is interpreted as "end of file" on the standard input (stdin) stream to the player process. The player is coded to interpret this as meaning "time to quit", the same as most other programs on *nix systems.

Cheers
Posted by: Roger

Re: builder_bigdisk_v3 now available - 16/01/2011 14:48

Originally Posted By: peter
The player remounts the disks read-only itself, after database rebuild, if it finds them read/write on startup. Thats's how normal Emplode-mediated database rebuild works. Also, you only need the music partitions read/write ("rwm"), not the root filesystem too ("rw"). I thought (but am not quite sure enough to say that Roger's wrong) that the player did a swapoff on startup too, only re-enabling it for those operations (rebuild, fsck) that need it.


Peter's correct here: you don't need to remount the disks read-only (because the player will do it). You also only have to "rwm", since /empeg/var is on the music partition (under /drive0).

I'm not sure about whether the player issues a swapoff. It almost certainly won't if you've manually mounted both swap partitions.
Posted by: Roger

Re: builder_bigdisk_v3 now available - 16/01/2011 14:49

Originally Posted By: mlord
The player is coded to interpret this as meaning "time to quit", the same as most other programs on *nix systems.


Not quite: bash treats Ctrl+D (eof) as time to quit. For the player, you need Ctrl+C (interrupt).
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 17/01/2011 04:33

So, Ctrl-D is the equivalent of typing "exit" at the bash prompt?
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 17/01/2011 04:38

It looks like the most of the differences between the advice given in this thread and what was already in the FAQ are cosmetic, such as:

- Ctrl-D instead of typing "exit"
- Deleting all the database files by hand before asking the player to rebuild them.

However, the one difference I see that might not be cosmetic is:

- swapon /swapfile

That means that you turn on swap before rebuilding the database. But here's my thinking as to why it's not in the FAQ:

The entry in the FAQ is intended to be a fix for when things go wrong, meant to be run once and then Emplode+Player would take care of things thereafter. But Emplode+Player don't turn on swap before rebuilding the database, so if you need to turn on swap to accomplish that, then something else is wrong. In other words, if the situation is dire enough to need swap to rebuild the database, you might get one good database rebuild from the shell prompt, but then the next time Emplode triggers a database rebuild, it will fail because swap isn't on.

Am I on the right track here?
Posted by: Roger

Re: builder_bigdisk_v3 now available - 17/01/2011 05:15

Originally Posted By: tfabris
So, Ctrl-D is the equivalent of typing "exit" at the bash prompt?


Yep.
Posted by: Ross Wellington

Re: builder_bigdisk_v3 now available - 17/01/2011 05:30

Hi,

Thanks for the information.

Tony, it looks like you got what you needed.

BTW, is there a way that I can check the size of the playlists to see if I have allocated enough ReserveCache size? I want to avoid the nomem-xxxx errors in the future by right-sizing the reserve based on the playlist or database.

Thanks,

Ross
Posted by: tfabris

Re: builder_bigdisk_v3 now available - 17/01/2011 18:56

Originally Posted By: Roger
Originally Posted By: tfabris
So, Ctrl-D is the equivalent of typing "exit" at the bash prompt?


Yep.


So what about turning on swap? Should I add that to the FAQ? What about the cases where Emplode+Player want to rebuild the database and they don't turn on swap?