HD Upgrade woes...

I've done two previous hard drive upgrades, one was simply adding a drive and the second was replacing my primary 12GB IBM with a 30GB one. My latest try hasn't been as smooth as the prior.

Setting the stage: I have 2 drives in my RioCar now. A 20GB and a 30GB. Both IBM Travelstars. I am replacing the 20GB drive with a Fuji 30GB I bought from another board member.

After removing the old drives entirely, I plugged in the new Fuji into the primary slot. It would not find the drive. It took me awhile to find the jumper settings (a sticker was over them) but I realized that the drive was mailed to me with the jumper set at "Cable SEL". But even after setting it to Drive 0 (master), the RioCar would not find it until I plugged it into the secondary slot on the drive cable (again, jumpered as master).

This was beginning to sound a lot like the "bad hard drive cable" problem, and it is in fact one of the RioCars. But, it is was truely a bad cable, how would one explain that the 2 IBM's work with no problem. Is it possible that the IBM's have better connectors that are less likely to be affected by the bad batch of crimped hard drive cables?

Either way, I did manage to get the disk recognized, jumpered as a master, in the secondary slot. So I installed 2.00b11 developer. Now, the FAQ lists how to replace the primary drive, and that is what I was planning to do, but since I can't use the new drive on the primary slot, can I instead copy the FID's from drive 0 to drive 1 by entering:

cp -v /drive0/fids/* /drive1/fids (instead of command listed on the FAQ?)

Will this work? I don't see any reason it wouldn't, but I'm not Linux hack and I've got way too much music (20GB) to risk...

Any comments are welcome! I will contact support in the morning for a new cable unless someone can think of another possible culprit.

Sometimes I forget what resources I have available... I dug out my Mk.2 empeg (spare) and pulled the drive cable from that. Now everything seems to be working fine!

So odd that the IBM's were less effected by this.. I went back and forth, A and B and the IBMs were fine every time with the "bad" cable where as the Fuji's weren't.

I attached the errors I get while trying to copy files....

EDIT: (I hate bumping up my own threads when I'm the only one responding! ) Is there a work around other than this?

Hmmm. The way to do this with a full-featured Unix install would be:

cd /drive1/fids

find . -exec cp {} /drive0/fids/ \;

but I don't think that the empeg has a find command.

Oh -- if the drive you're copying to has no data in the fids directory (I've never seen a brand-new empeg drive), you could try

cd /drive0

rm -rf fids

cd /drive1

cp -R fids /drive0

Not sure if tar could process all the files in one go - probably, as I've seen it mentioned as a way to feed netcat (nc) for cloning. I'm just up, so the brain capacity is not yet up to speed, so I can't recall the exact syntax - but it involves a tar'ing of the data, piping it to another tar process that unpacks in the source directory...

Might be possible to do it with a bash script that loops over files and copies one at a time - but then again it might complain about the length of the list it has to loop over...

But what's so horrible in splitting the cp command into two? Put them on a single line separated by a ';' and the second will execute immediately after the first one is done, if you want unsupervised/one step copying of all the fids.

/Michael

Using tar has one nice side-effect - the original file modification dates/times are
preserved.

to move the files via tar you can use the following commands:

cd /drive0

rm -rf fids

cd /drive1

tar cf - fids | (cd /drive0; tar xvf -) 

(Effectively replace the 'cp -R fids /drive0' with the tar cf - ... line from the earlier post).

if you don't want tar to show each file as its restored leave the v out
of the xvf [i.e. tar xvf - become tar xf - ]
Note that the brackets '(' & ')' are critical in the above command - do not leave them out!

This is more or less what the Lazy bastards copying method does, except the pipe is replaced by a netcat in 'send mode' on the sending empeg and a netcar in 'receive mode' is then piped to the tar xvf - thus allowing you to move trees across machines.

Bitt's advice is good (and exactly what I'd suggest), but I thought I'd add some explanation to your error message.

Linux, like any other operating system, has limits on how many command-line arguments can be passed when starting a new process. The limits are large enough that you'd never type enough by hand, but it's easy to get the shell to do it for you. If you type something like

/drive1/fids/*

it turns that into as many arguments as there are matching files, i.e.

/drive1/fids/100 /drive1/fids/101 /drive1/fids/1000 /drive1/fids/1001 /drive1/fids/1010 /drive1/fids/1011 /drive1/fids/1020 

etc. Obviously, with the fids directory, this makes for a very long command (you just don't get to see it).

You can make the argument list shorter by changing to that directory, then using *, which expands to 100 101 1000 1001 1010 1011 1020 etc, but that may still be too long. Then you can split it into two copy commands:

cd /drive1/fids/

cp -p [0-4]* /drive0/fids

cp -p [5-9]* /drive0/fids

I hope it's obvious how to extend this if you need to do it in 3 or more batches. Note also that I used the '-p' flag to cp to preserve the last-modified times when doing the copy.

Thanks guys. I already started the /1* /2* /3* process, so I will keep with that until I get through all the alphabeta and numbers. There should be some note in the FAQ about this. I think with larger collections of music becoming more common, so will the "too many arguments" error.

While copying, I saw this one line that really stood out. It is attactched. One of the copy lines is way longer than others. What do you guys think it is? Is it okay?

PS - Did this thread get real W - I - D - E for anyone else?

That line looks like a display or buffering glitch in your terminal software (or conceivably in the player's terminal drivers). I don't think there's any problem.

Cool. Thanks Toby.

I accidentally did a /fids/*0 instead of /fids/0* .... now it is copying a bunch of files that I will probably be re-copying later.

Does the "cp" command automatically overwrite? I pray it doesn't auto append the filenames! :O

"man cp" is your friend - though if you don't have a Unix(-like) system to hand, you'll have to use one of the many Unix manuals on the Web.

First, fsck your drives like the messages are telling you to do!

ro

fsck /

fsck /drive0

fsck /drive1

..reboot..

Now just tell 'cp' to copy the whole directory:

rw

cp -auvfx /drive1/fids /drive0/fids

In reply to:

---

cp -auvfx /drive1/fids /drive0/fids

---

Er, won't that try to put a copy of fids from drive1 into /drive0/fids (since the last argument is a directory)?

ITYM

cp -auvfx /drive1/fids /drive0/

fsck on BOTH drives?

also, how do I (like "control-c" in DOS) terminate this cp to start over?

YEah, that's better!

hi.

Why don't you just try CTRL-C? What do you think where DOS got that combination from?

cu,
sven

Er, press Ctrl-C ?

Um... yeah. hehe thx.

So fsck on both discs? Even though Drive1 has all my mp3's on it?

I suggest that you follow the instructions from the FAQ, to be on the safe side.

I was following the FAQ, but got the "too many arguments" error message.... so I had to start manually doing the cp command.

EDIT: Thanks Roger, I thought you were refering to the drive upgrade FAQ. Thanks for the link!

No, he didn't link the drive upgrade guide just now. You were following the drive upgrade guide when you got your error message.

What he just linked was the section of the FAQ that tells you how to fix the "e2fsck" errors. He is suggesting you follow those instructions before trying to copy any files:

ro
umount /dev/hda4
umount /dev/hdc4
swapon /swapfile
fsck -fay /
fsck -fay /dev/hda4
fsck -fay /dev/hdc4
swapoff /swapfile
sync

(and then reboot the player by pulling its power cord)

"cp -auvfx /drive1/fids /drive0/" - That is working! Now, why is THAT working without "too many arguments" but the wildcard command didn't? (Sorry, I'm just trying to learn here...) If I could make a tiny request: could the telnet commands in the Linux have a little description here and there? For example, I assume the "-v" condition is "verify"? But I had no idea what I was telling to player to do in the fsck FAQ entry (I guess it worked though!)

cp -auvfx /drive1/fids /drive0/ is my fish.

Now, why is THAT working without "too many arguments" but the wildcard command didn't?

Because cp is told to do an archive copy (preserve time/dates, owner etc) of a directory - this includes, recursively, all subdirs and files inside the directory. The filenames are however not passed to cp on the command line, it digs them out itself, while running. It is the length of the command line (or rather, number of arguments) that cp choked on in the /* case, in this case it only has two arguments, source and destination directory, since it is also told to recurse into the directory it looks inside the directory and copies everything - without you having to spell out each file.

/Michael

Thank you! That makes sense now. Is there any drawback to this? Does anyone think it would be good to have this in the FAQ because this may become more common with larger hard drives being more common?

No drawback that I know of... cp -ax (some of the flags listed earlier are probably
redundant if you have a and x) is a well known method to clone/copy stuff. GNUs
cp (which is what's on the empeg) handles oddball stuff (links, pipes, devices etc)
better than some other cp:s included in other Unices though.

Yes, I think it would be good if the FAQ would be updated to recommend this
approach instead - Tony?

Edit:
Hmmmm... no redundancy in those switches after all:

-f, --force
remove existing destinations, never prompt
-u, --update
copy only when the SOURCE file is newer than the
destination file or when the destination file is
missing

these look somewhat contradictory. Not a problem if you have a blank destination though...

-v, --verbose
explain what is being done
-x, --one-file-system
stay on this file system

and -a equals -dpR :
-d, --no-dereference
preserve links
-p, --preserve
preserve file attributes if possible
-R, --recursive
copy directories recursively







/Michael

Yes, I think it would be good if the FAQ would be updated to recommend this
approach instead - Tony?

If someone can look at the Drive Upgrade guide right now and tell me, EXACTLY, letter for letter, PRECISELY what to change in the drive upgrade gude, then, if those participating in this thread can look at it and confirm that it will truly work correctly on all systems, then yes, I will change the FAQ.

From the drive upgrade instructions:

Copy the files from the second drive to the first drive:
cp -v /drive1/fids/* /drive0/fids (Enter)


I'd change the second line to

cp -avfx /drive1/fids /drive0 (Enter)


Now, to get a consensus on exactly which flags should be given might be tricky.
Minimalistically, you should in any case where you haven't done weird stuff like mounted some other partition inside the /drive1/fids directory or have a non-empty /drive0/fids, cope with the -rv options.

-avfx would be more robust, managing any weird file types, overwriting preexisting files, excluding any mounts, etc.

I played around a bit with this and can't seem to find a case where it doesn't do what I expect this to do; copy /drive1/fids and everything inside (except stuff on a partition mounted inside) to /drive0/fids, overwriting anything that already exists in the /drive0/fids directory (and below) with whatever has the same name in /drive1/fids.

Any one else?

/Michael

I'd go with Micheal's wording, possibly adding the u flag to cp, as Mark did.

I was about to make one small exception - I'd change the /drive0 at the end of the command to /drive0/ - which means it will fail if, for some reason, /drive0/ doesn't exist. But that's overkill, and unnecessary - if you'd somehow lost your /drive0 music partition, you'd be pretty stuck already.

For anyone reading this thread who is themselves thinking of upgrading their drive, I want to point out that the problem I had was only a result of me having so many files to copy. (AND a bad drive cable, but that is easy to take care of.) I have done 2 other drive upgrades without a hitch using the Upgrade Guide at RioCar.org. So once again, thanks Tony and everyone else who contributed to it! The solution that was suggested here took care of my problem and I assume that this will become part of the new "un-official" procedure because large hard drive swaps will be more and more common.

Of note: I think the "u" that I entered saved me a lot of time. I had already started the "1*" and "2*" copy process, so I think the "u" skipped over files that had already been copied? I say this because I thought I was going to have to re-copy a bunch of files, but it was surprisingly fast. All is well now!

Okay, so how is the u flag added, how is the different drive0 added, does everyone agree, etc., then I need an exact line of exact text that gets everyone's vote of approval. Then I will put it in there.

I just can't bend my brain to understand how the -f and -u flags are supposed work
together...

-f says to always remove existing target file in favor of source file
-u says only copy if source is newer than an existing target

anyone that can enlighten me?

After playing with cp for a while it seems that (at least on Debian x86 Linux) -u
beats -f, and if target file, newer than source file, exists, no copy is performed, no
matter in what order the flags are given.

For a freshly created disk (from factory or after applying the builder .upgrade)
which the FAQ procedure uses, neither -f nor -u should be necessary, as there are
no existing fid files in the target that we need to clobber or preserve. If you're
playing around having mounted a partition under /drive0 on the source drive you
probably also know enough to append 'x' on your own...

The /drive0/ instead of /drive0 is a good precaution, though if /drive0 is missing you
have something seriously wrong and will probably need to reapply a .upgrade file
anyway...

Tony, generally flags in Unix can be given separately, like '-f -a -q', or in one go
(some exceptions exist) like '-faq'

Considering the fact that the FAQ presumes a fresh drive, I'd say

cp -av /drive1/fids /drive0/ (Enter)

for the FAQ, with a small note that if you already have partially copied the fids,
change '-av' to '-avuf'. Or just point to this thread

Edit:

Bah, also given the fact that the FAQ presumes a fresh target - adding the -u
and -f flags will do no harm, and might be useful on other occasions - I'll change my vote to

cp -avuf /drive1/fids /drive0/ (Enter)

/Michael

But you've still got f AND u?

Duh, yeah... as -u always seems to override -f , -f could be removed...

Another try

cp -avu /drive1/fids /drive0/ (Enter)

/Michael

-u and -f together mean that the file will be copied if

• there is no existing file in the destination directory, or
• the existing file in the destination directory is older than the source file

but not if

• there is an existing file in the destination directory that's newer or the same age

Without the -f, the mere presence of the file in the destination directory will prevent the copy (and give an error message IIRC). Without the -u, any file may be overwritten, regardless of its timestamp.

The safest thing is to use -f and not -u - then even if a file has a corrupt timestamp or has been half-copied, it should still get overwritten. This is of course the slowest version, though.

If you're starting with a fresh, empty destination directory, neither of these flags makes any difference, anyway.