Using compactflash cards in the Empeg

Hi everyone

I want to use my Empeg as image storage for pictures taken with my digital camera. I bought a Compactflash to IDE converter ( CF-IDE-44 from http://www.flashmemory.com.au/shop/shopdisplayproducts.asp?id=3&cat=FlashMemory+Disk+Drives), and I am now trying to get my Empeg to recognise an inserted compactflashcard as slave HDD at boot time.

My problem is, that during boot the Empeg only detects the flashcard (hdb) but NOT my master HDD (hda).
When i eject the compactflash card from the converter and boot again, the Empeg detects the HDD (hda) and continues booting.

It seems to me as if the insertion of the compactflashcard in the converter blocks the detection of my master harddrive ?

The compactflash(8Mb) comes straight from the digital camera and is FAT formatted - but this should not stop the two "HDD"s from being correctly detected at boot ?

Here is the disk detection part of the boot sequence WITH a compactflash card inserted in the converter:

empeg single channel IDE
Probing primary interface...
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
ide0: unexpected interrupt, status=0x80, count=1
ide0 at 0x000-0x007,0x038 on irq 6
hdb: SanDisk SDCFB-8, 7MB w/1kB Cache, CHS=245/2/32
empeg-flash driver initialized
smc chip id/revision 0x3349
smc9194.c:v0.12 03/06/96 by Erik Stahlman ([email protected])

SMC9194: SMC91C94(r:9) at 0x4008000 IRQ:7 INTF:TP MEM:6144b MAC 00:02:d7:22:07:60
Partition check:
hdb: hdb1
RAMDISK: Couldn't find valid RAM disk image starting at 0.
VFS: Cannot open root device 03:05
Kernel panic: VFS: Unable to mount root fs on 03:05


And here is the same part without the flashcard inserted:

empeg single channel IDE
Probing primary interface...
hda: TOSHIBA MK6014MAP, ATA DISK drive
hda: TOSHIBA MK6014MAP, ATA DISK drive
hda: TOSHIBA MK6014MAP, ATA DISK drive
hda: TOSHIBA MK6014MAP, ATA DISK drive
hda: TOSHIBA MK6014MAP, ATA DISK drive
hda: TOSHIBA MK6014MAP, ATA DISK drive
ide0 at 0x000-0x007,0x038 on irq 6
hda: TOSHIBA MK6014MAP, 5729MB w/0kB Cache, CHS=12416/15/63
empeg-flash driver initialized
smc chip id/revision 0x3349
smc9194.c:v0.12 03/06/96 by Erik Stahlman ([email protected])

SMC9194: SMC91C94(r:9) at 0x4008000 IRQ:7 INTF:TP MEM:6144b MAC 00:02:d7:22:07:60
Partition check:
hda: [PTBL] [776/240/63] hda1 < hda5 hda6 > hda2 hda3 hda4
RAMDISK: Couldn't find valid RAM disk image starting at 0.
EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended
VFS: Mounted root (ext2 filesystem).
.....


My Empeg is a MkIIa running the 2.00b11 firmware with Hijack 243

Does anyone have an idea of what might be wrong ?

Regards
Nicolai

Don't suppose you've got it set to one drive only in the Hard Disk Detection menu in Hijack do you?

Also does your adapter have any jumpers for master/slave?

- Trevor

Hi Trevor

Just checked the Hard Disk Detection menu in Hijack and it is set to two drives...
The adapter has a master/slave jumper which is set to slave...

In the meantime I discovered something else:
If I try to run the CF card as master and the HDD as slave they both get detected allright. My problem is, that I need to do it the other way around, as I do not have the Empeg kernel on the CF cards....

/Nicolai

Could it be a cable positioning issue?

First of all, I thought that those compact flash readers required driver software support. I didn't think you could just plug one in.

Next thing:

Even if you can get the thing to read, there is no advantage to using a compact flash card as the second hard disk because the empeg's software does not support removable drives.

For instance, the player stores its FIDs pretty much will-nilly on the drives and you cannot control which songs go where. You cannot load new tunes by dropping them on a CF card and inserting it. The player software simply doesn't support this method of operation and certain features of its design (because it assumes the second drive is a fixed disk and it will attempt to use it as such) will thwart any of your attempts to do this.

Also, you said you had it FAT-formatted. This will not work for the player. You -must- run the builder image on any new disk or else it will not have the right partitions.

Hi Tony

I don't want to use the CF cards for storing music.
My plan was to be able to transfer images captured with my digital camera from the CF card to the players harddrive.
I want to launch a script after boot (using the preinit method) to check to see if there is a CF card plugged in - and then cp the images to a directory on the harddrive.

I always take a lot of photos while I'm away on holiday, and using the Empeg as extended storage would make it possible for me to leave the laptop PC at home....

Paul: I tried switching positions on the cable - that is placing the HDD before/after the CF adapter - but with no luck.

Regards
Nicolai

I think your clue might be this line:
ide0: unexpected interrupt, status=0x80, count=1

I don't understand the error, and Google didn't help me too much, but If Mark Lord isn't to busy, he may understand it since it may be his code generating the error.

Tony: It looks like his CF reader will work fine, the kernel is seeing it properly as a normal IDE device, so no drivers needed.

Hi everyone

Now I'm confused:
I got hold of two other CFcards and while one of them showed the same symptoms (refused to coexist with my master HDD) the second one worked just fine (16 Mb Lexar CF):

empeg single channel IDE
Probing primary interface...
hda: TOSHIBA MK6014MAP, ATA DISK drive
hdb: LEXAR ATA_FLASH, ATA DISK drive
ide0 at 0x000-0x007,0x038 on irq 6
hda: TOSHIBA MK6014MAP, 5729MB w/0kB Cache, CHS=12416/15/63
hdb: LEXAR ATA_FLASH, 15MB w/1kB Cache, CHS=251/4/32
empeg-flash driver initialized

I am absolutely sure that the three CF cards are formatted the same way (FAT), as I have done this using my digicam.

Here are the results using the other two cards:

(64 Mb Sandisk CF):
empeg single channel IDE
Probing primary interface...
hdb: SanDisk SDCFB-64, ATA DISK drive
hdb: SanDisk SDCFB-64, ATA DISK drive
hdb: SanDisk SDCFB-64, ATA DISK drive
hdb: SanDisk SDCFB-64, ATA DISK drive
hdb: SanDisk SDCFB-64, ATA DISK drive
hdb: SanDisk SDCFB-64, ATA DISK drive
ide0: unexpected interrupt, status=0x80, count=1
ide0 at 0x000-0x007,0x038 on irq 6
hdb: SanDisk SDCFB-64, 61MB w/1kB Cache, CHS=490/8/32
empeg-flash driver initialized

(8Mb Sandisk CF):
empeg single channel IDE
Probing primary interface...
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
hdb: SanDisk SDCFB-8, ATA DISK drive
ide0: unexpected interrupt, status=0x80, count=1
ide0 at 0x000-0x007,0x038 on irq 6
hdb: SanDisk SDCFB-8, 7MB w/1kB Cache, CHS=245/2/32
empeg-flash driver initialized

I have ruled out Hijack role in this (I got the exact same results after having uninstalled Hijack), so I think it might be Sandisk's implementation of the IDE interface that might be different than Lexars.
It may be that the Sandisk cards cannot be slaves to another drive - I will try to figure that one out...

Thanks to everyone for your suggestions - this board is excellent....

By the way - do you know if I have to rebuild the kernel to get FAT support in there ?

Regards
Nicolai

By the way - do you know if I have to rebuild the kernel to get FAT support in there ?

It looks like you will. The .config file in my current tree contains "# CONFIG_FAT_FS is not set".

In reply to:

---

I thought that those compact flash readers required driver software support. I didn't think you could just plug one in.

---

That's the beauty of compact flash. It has three different interface types. One of them is called True IDE mode where the device emulates a standard IDE interface when certain pins are wired correctly. You can plug them straight into a PC if you want to. What you will need drivers for though is hot plugging/unplugging but this will also require buffer hardware which these cheapo adapters do not have.

Anyways this sounds very much like the problem where some drives don't like co-operating with each other.

This sounds like a problem I remember from dealing with the I-Opener, but it only happened with a Linux kernel newer than something (maybe 2.4.2) and it could be worked around with an argument to the kernel. If I can find my notes I will post them.

Hi Christian

I think you are right about the different type of compactflash.
Out of my 3 CF cards only one of them would coexist as slaves with my master harddrive. The ones that failed was both Sandisk cards - the one that worked was a Lexar card.

My only problem now is to get FAT support compiled into the kernel....

Regards
Nicolai

Hi Paul

Thanks for your response..

I found out that there are different types of compactflash types - regarding IDE/ATA compatibility.

Some cards are more compliant than other - 2 out of my 3 CF cards would not coexist as slaves with my master harddrive - while another one worked just fine...

Regards
Nicolai

Hi.

Just wanted to let you know that I had very similar experiences with my CF cards:
My SanDisk would not work as an IDE slave, my Lexar did. Also, my TRGpro handheld works smoothly with the Lexar, but has a lot of trouble working with the SanDisk one. My notebook also reads and writes the Lexar just fine (with a CF to PC-Card adapter), but to read the SanDisk one, I had to get a USB reader for it.

cu,
sven

dito

and to think that I always took SanDisk as the leaders of compact flash and heck all the other mini data store media!
Its from their site that I gleaned all of this info for some past project IIRC...

Cool. I want one.

For those who are stuck with MS-Win, you'll be suprised to discover that CF cards do NOT require a special driver under Linux, since they emulate a hard disk drive interface (IDE/ATA) rather well.

The reason for the problem reported here is because of this near perfect emulation.. Linux can sit there probing for a long time for a second (non-existant) drive when a CF card is on the bus (cable) by itself.

Back when this issue first surfaced (about 4-5 years ago), I put a quick hack into the "frozen" kernel to just skip the wait for a second drive under most conditions when a CF card was found, since they were NEVER paired up with another device at the time.

There was also a note left behind to "fix it properly" once the 2.2 kernel got released and the code freeze was lifted. Unfortunately, Linus took FOREVER to release 2.2 (like a year or more after the "freeze"), and by then I had given up maintaining the IDE subsystem for free. So the hack is still there.. ugh.

Anyway, I'll nuke it from the Empeg kernel for Hijack v262, available in 5-10 minutes from now.

Let me know if it behaves any better for you.

Cheers

Hi Mark

Thanks a lot for your effort with Hijack.

Unfortunately it didn't solve the problem with two of my CF cards not wanting to coexist as slaves to the master harddrive.
I still get the same messages at boot - I posted them here: http://empeg.comms.net/php/showthreaded.php?Cat=&Board=empeg_tech&Number=87739&page=&view=&sb=&o=

Perhaps it is the near perfect IDE emulation you mention, which might be better implemented on my working Lexar card than on my two Sandisk cards ?

Regards
Nicolai

Wow, it's good to have the "Linux IDE guy" on the payroll here, isn't it?

Mmm.. bummer.

I'd expect the Sandisk cards to work in that situation. Quite strange that they don't. I only have Lexar cards (and an IBM microdrive) here, so I cannot poke around easily to see what's special about them.

Wanna send me your card & adaptor?

Cheers

Well, I think it isn't too much of you to ask, with all the work you put into Hijack, so here you go Mark:



I have no clue on how to interface your Empeg with that jpeg though...

I'm receiving a 256 Mb Kingston CF card later this week, and I'm looking forward to see if their implementation of the IDE standard is better than Sandisks.
Perhaps they don't even produce their own cards but just put the Kingston label on a Lexar or Sandisk card (the "Viking" card on the picture gets identified by my Empeg as "LEXAR ATA_FLASH, ATA DISK drive" while a Canon card I also have reports "SanDisk SDCFB-8, ATA DISK drive").

Btw: Does anyone in this forum have a precompiled kernel with FAT support built in already ?
It does not need to be the latest version - it's just for testing if I can read data from my CF card onto the Empeg...
I read the FAQ about compiling the kernel, but I think my Linux/gcc kernel-compiling skills have faded a bit in the last couple of years

/Nicolai

I still have my crosscompiler here on one of the lab machines, I'll grab the B11 patch and toss in hijack, enable FAT, and post it. Should have it done in a few hours.

Note to self, saying this makes more calls come in.

I'll work on the kernel tomorrow if you don't have it yet. It got really busy this afternoon, plus Tony distracted me in the few moments I had free to find an image. (Not a huge distraction, I still wouldn't have gotten it compiled).

Tony distracted me in the few moments I had free to find an image.

Or, in this case, to search in vain for the image, leaving Tony to create the image himself.

And I just wanted to say that if you can get this working, it's a cool idea for a project. Sorry for making an incorrect initial assumption about the purpose of the flash card interface. Good luck.

Kernel for a Mk2 v2.00b11 with Hijack v262 and FAT support is here

- Trevor

Yippieeee! It works...
Upgraded the kernel, booted the Empeg and then did this:

empeg:/# rw
empeg:/# mount -t msdos /dev/hdc1 /mnt/cf
empeg:/# ls -al /mnt/cf/dcim/122canon/
total 540
drwxrwxr-x 2 0 0 4096 Apr 21 2002 .
drwxrwxr-x 3 0 0 4096 Apr 21 2002 ..
-rwxrwxr-x 1 0 0 542521 Apr 21 2002 img_2260.jpg
empeg:/#

Thanks a lot for the kernel Trevor...

Now I will start coding a shell script to look for a CF card at boot and then copy and verify the images to the harddrive...

I don't hope that Drakino or Tony got too much work done with compiling the kernel now that you beat them to it

Thanks to everyone in this thread - and in the forum - for your support...

Regards
Nicolai

If you've got the date command on your empeg then you could make it create a unique directory and then just recursively copy the entire CF card to the HD. That why you won't have any problems if the camera resets the file numbers for some reason.

#!/bin/sh
DIRECTORY = `date +%d%m%y-%k%M%S`
mkdir $DIRECTORY
cp --preserve --recursive /mnt/* $DIRECTORY

- Trevor

Trevor, what was the additional footprint due to te inclusion of FAT in the kernel?

I'm wondering if it is desirable for it to be included in Hijack by default or whether people who want this are going to have to compile their own binaries when upgrading Hijack.

Great idea, but I don't seem to have the date command on my Empeg.
Do you know where I can download additional binaries for the Empeg/StrongARM ?

/Nicolai

I was actually thinking that it might make more sense to enable module support in the kernel. IIRC, doing that doesn't increase overhead (until you insert modules, obviously). That way, people could add FAT support or anything else, just by having a new module made, rather than having to distribute a whole new kernel.

Am attaching the date binary. Don't remember where I dug it up, but there's been posted pointers to various tar-balls of utilities.

If nowhere else there's the debian arm distribution http://www.debian.org - just got to figure out which .deb file the wanted binary lives in...

I'm more than willing to compile a new kernel and modules for anybody that wants it but you'll have to test anything yourselves because I probably won't have the necessary hardware.

Might be an idea to ask Mark to turn on module support in his prebuilt kernels. If not I'll do them.

- Trevor

Looks like it adds about 32k to the compressed kernel, so it's not insubstantial. More reason for modules I believe although that will add some overhead.

What was the kernel size limit again? The empeg only has 1MB of flash all up remember

At first glance, module support would appear to make sense, but I have been thinking about it some more, and I suspect that it isn't worth it:

1) We'd have to define where the modules were kept on the filesystem. IIRC, they are usually in /lib/modules/version. This would have to live on a disk partition as it's size would be indeterminate, so a custom mount would be required. This isn't a big issue, but it'd be a pain to set up.

2) We'd also need the necessary modutils. insmod, which is symlinked by lsmod,rmmod and modprobe is 94k. This could probably live ok on the disc partition too, as long as it wasn't needed before pre-init. We'd also need /etc/modules.conf and /lib/modules/version/modules.dep (about 56k on my system, but it is build specific I believe)

3) What else do we really need in the kernel? I can't see any need for any other filesystem support - ext2 and FAT should cover it. Even if people are using the IDE bus in unexpected ways (eg this compactflash idea - way cool), chances are that anything they plug into it is going to be either ext2 or FAT. Maybe it'd be nice to have ext3 with it's journalling capabilites, but AFAIK this is only available in the 2.4.x linux trees, and hasn't been back-ported.
As for hardware, there isn't anywhere to plug any traditional hardware into. No PCI, no parallel, no USB master, no VESA, no AGP etc. All the current hardware (serial, LAN, USB slave, display board etc) is already controlled by the kernel anyway, so there's nothing to add here.
What does that leave that we might want?

just got to figure out which .deb file the wanted binary lives in...

Here's what I do:

1. Go to http://www.debian.org/distrib/packages
2. At the bottom of the page is the search window.
3. Type your binary name (e.g. 'date') into the "Search the contents" window.
4. Hit "Search". The results come back.
5. Top of the list, in this case, is bin/date, in base/shellutils

Now, if you click on the "base/shellutils" link, it'll take you to a page summarising dependencies, etc. Take a look at it. However, from here, it'll probably take you to x86 binaries. We want ARM binaries, so...

6. Go to http://http.us.debian.org/debian/dists/stable/main/binary-arm/
7. Click on (in this example) "base"
8. Grab the "shellutils_version.deb" file.

This is the .deb file containing the binary files we wanted (and probably more besides).

Now, to unpack the file, you'll either need a copy of 'ar' on your player, or a Linux/Cygwin installation handy:

1. ar p file.deb data.tar.gz > file.tar.gz
2. Send this file to the empeg (either using Hijack or ZModem). I generally put the .tar.gz files into /drive0, so that they're not overwritten by an upgrade.
3. Unpack it into the root.

Presto, a copy of date for your empeg.

Module support chews up a fair amount of RAM to hold the kernel exported symbol table. Dunno how much bigger that makes it.. maybe 64KBytes or so. I'd rather use that space as a music buffer.

By comparism, almost all of Hijack currently fits into about 64KB.

Cheers

Fair enough. Misinformation on my part.

This thing you're working on is pretty exciting. Is it possible to get one of those Flash Card interfaced IBM Microdrives and use it as a "Music Keg for the Empeg" ? :-)

Calvin

It would be interesting for those who want a small area to record things to with the microphone input, or for capturing live radio clips (albeit pretty short ones). Leaving a piece of flash RW would't seem to be as problematic as leaving a HD read-write. At least from the layman's point of view (i.e. mine).

Just a thought.

-Zeke

Well, just keep in mind that, contrary to common opinion, the Empeg's drives are "read write" the entire time they are running.. and the Empeg player software does write to them on the fly for shuffle lists, equalizer settings, and the INFO-SEEK tool data.

Pretty safe on the whole. I think sudden powerfail to a compactflash device is way more dangerous.

-ml

I sit corrected! I guess needing to set the drives RO at the end of terminal sessions led me to that belief. I'm a bit of a luddite on these matters...

-Zeke

Hi everyone

Again - thanks for all your help with my project.
Product support like this can't be beaten
Here's an update on the project:

I have now written the first (very raw) version of my script, which checks the existence of the CF card after boot (run via a preinit script), and then just copies its entire content to a directory under /drive0/images with the name YYYYMMDD-HHMMSS.

This way I get a complete backup of the CF card everytime i boot the Empeg.

The script then writes the current status of the "imagebank" and updates the log - all in a file called "statuslog".
I can the read this logfile using John Heathco's Viewer application - run from the Hijack menu.
The logfile looks something like this:

-------- status --------
Empeg currently holds:
images: 34
movies: 0
size: 16M
--------- log ----------
25 Apr 14:53
copied 13 img & 0 mov
24 Apr 17:21
copied 21 img & 0 mov

I made a directory called /cfcopy for the logfiles, this then has a subdirectory /cfcopy/images which is a link to /drive0/images.

Heres the script (first version - not necessarily well written ):

#!/bin/sh



#Checking for existence of CF card

if mount -t msdos -r -o noauto,nocheck /dev/hdc1 /mnt/cf

then 



  # Mounting hdd in RW mode

  /bin/rw

  /bin/rwm



  #Copying images to hdd

  imgdir=`/bin/date +%Y%m%d-%H%M%S`

  mkdir /cfcopy/images/$imgdir

  cp --preserve --recursive /mnt/cf/* /cfcopy/images/$imgdir



  #Updating status

  imgnumall=`ls -R /cfcopy/images | grep -c jpg`

  movnumall=`ls -R /cfcopy/images | grep -c avi`

  size=`du -Shc /cfcopy/images/* | grep total`

  echo "--------  status  --------" > /cfcopy/.tmp/status

  echo "Empeg currently holds:" >> /cfcopy/.tmp/status

  echo " images: $imgnumall" >> /cfcopy/.tmp/status

  echo " movies: $movnumall" >> /cfcopy/.tmp/status

  echo " size: $size" >> /cfcopy/.tmp/status

  echo "---------   log  ----------" >> /cfcopy/.tmp/status

  

  #Updating log

  imgnum=`ls -R /cfcopy/images/$imgdir | grep -c jpg`

  movnum=`ls -R /cfcopy/images/$imgdir | grep -c avi`

  echo `/bin/date "+%d %b %H:%M"` > /cfcopy/.tmp/log.new

  echo " copied $imgnum img & $movnum mov" >> /cfcopy/.tmp/log.new

  cat /cfcopy/.tmp/log.new /cfcopy/.tmp/log.old > /cfcopy/.tmp/log

  cp --force /cfcopy/.tmp/log /cfcopy/.tmp/log.old

  

  #Writing new status file

  cat /cfcopy/.tmp/status /cfcopy/.tmp/log > /cfcopy/statuslog

    

  # Cleaning up

  umount /dev/hdc1

  /bin/rom

  /bin/ro

  

else 



  echo "Copying failed..." 



fi

My next plan is to borrow some of the source code from the excellent viewer application, and make use of its capability of hacking into the Hijack menu.
This way I could do a lot of extra things in the code, for instance do a verification of the copying, launch the copy process itself from within the menu instead of running it at boot time and many other things...

And by the way - I got my new Kingston 256 Mb CF card yesterday - and it is working perfect!
At boot time it reports itself as a "TOSHIBA THNCF256MBA, ATA DISK drive".

Regards
Nicolai

What Mark was saying is that even though the music partitions are mounted read-only, other parts of the drive get written to during normal operation. You do need to set the music partitions (and the root partition) read-only when you've finished them, unless you want to corrupt your filesystems.

Sorry about that, my lab machine that had the crosscompiler died in an RAID bios glitch. I had forgotten to update things here.

Wow, this is really cool. You're basically making the empeg into a Digital Wallet. So, if you ever go on a road trip, you have all the storage you need! I guess it would work for those of you who take your empeg with you on planes too...

Please keep us posted as to how this comes along...