Database transfer

Posted by: Hollywood Justin

Database transfer - 18/09/2010 16:09

I purchased 2 newer, bigger (250 GB) hard drives for my empeg(s) and I want to transfer my MP3 collection from the two existing 20 Gig hard drives onto one of the 250's. I also have a Rio Car player that I could use if necessary as a cloning step to copy the songs and the data, but what I am really interested in is preserving things like structure including playlists and multiple song references to a single FID, play counts, ratings and all the stuff that is not stored within the ID3 tags. So my primary question centers around the ability to merge songs and FID pointers from 2 drives onto a single drive and then to copy the actual player database, leaving it in tact and utilizing it on the new drive. Are there any existing tools to allow me to do this, or is someone working on something that they'd like help programming/developing and testing? My secondary question has to do with manipulating the stats in the database. Are there any tools (like a GUI-based Jemplode) that would allow me to change information such as times-played & times-skipped in order to rate songs with multiple selections for the empathic add-on which utilizes this second field? I've tried changing the play count in Jemplode, but that data does not get synch'ed and saved onto the player.
Posted by: Shonky

Re: Database transfer - 19/09/2010 00:52

The FIDs for the songs are unique across the two drives assuming (they come from the same empeg of course). The playlist structure is all stored as FIDs too.

So simply merging them all on to one disk is a simple case of moving them over. The player software looks on both disks so you can happily move them back and forth between drives without issue.

I'd install the 250GB disk and get it up and running, perhaps installing a song or two to check. Then I'd have the first drive as a slave and move everything over. Then the remove the first drive, install the second and do the same thing. There shouldn't be any clashes. Then rebuild the database and you should be set to go.

Play counts and ratings are stored in the dynamic data partition of the first drive. As long as the new drive has dynamic data partition that is the same or bigger a simple "cp /dev/hdc? /dev/hda?" where the question mark is the appropriate partition number should preserve all of that.

There is no tool that I've seen for manipulating the dynamic data directly. Not to say it can't be done. I think we have all the structure information. Not an insignificant job to create a GUI to modify it though.
Posted by: Hollywood Justin

Re: Database transfer - 19/09/2010 04:56

Awesome! You have been a great help. Thanks a ton!
Posted by: mlord

Re: Database transfer - 19/09/2010 11:50

Originally Posted By: Hollywood Justin
what I am really interested in is preserving things like structure including playlists and multiple song references to a single FID, play counts, ratings and all the stuff that is not stored within the ID3 tags.

That stuff is collectively referred to as "the database", and copies of it are stored separately in the "fids" directories on the two drives.

So long as you copy everything from the two drives' fids directories to the new drive, and then rebuild the database (run emplode), it should all survive the move.

In addition, there is a special disk partition on drive0, which holds what the empeg sees as "dynamic data". This is stuff like the current playlist, equalizer settings, FM radio presets, track profiles, etc..

That dynamic data was not in your original list of things you wanted to preserve. But if you do want it, it should be as simple as using cat (on the Linux command line) to do a simple binary copy from the old drive0 to the new single drive.

Perhaps somebody with a better memory might kick in here with the exact partition names to supply as parameters for "cat".

Cheers
Posted by: Hollywood Justin

Re: Database transfer - 19/09/2010 16:30

Thanks. I guess one question I neglected to ask because I forgot how to do it: How do I get to the empeg's Linux prompt? Is it as simple as connecting to the serial port and connecting through telnet or is there some break code that I need to issue during the boot cycle in order to change the player from normal runtime mode to shell mode?
Posted by: BartDG

Re: Database transfer - 19/09/2010 17:16

Connect a serial cable to the serial port and use hyperterminal (or comparable). Also make sure you run a Developer version of the firmware.
Posted by: tfabris

Re: Database transfer - 20/09/2010 14:39

Originally Posted By: Hollywood Justin
I want to transfer my MP3 collection from the two existing 20 Gig hard drives onto one of the 250's.


All of the details of this exact procedure are listed in excruciating detail in the drive upgrade guide. Including the bit about how you can copy the FIDs from both of the old hard disks onto the same new hard disk, and including links about how to get to the shell prompt.
Posted by: Hollywood Justin

Re: Database transfer - 07/03/2012 23:58

I tried copying my FID's and dynamic data as per the Rio Car Disk Upgrade Guide FAQ instructions, although rather than using the developer version of the firmware, I used version 2, beta 13 with hijack included and it seemed to copy all of my files properly except that when I connected with emplode there were many warnings and I only had about 400 tunes out of what was supposed to be 5400. Also, the playlists were not all nested correctly. When I copied the original slave drive to the new master SSD, my computer went to sleep so I re-ran the cp command and there were no additional files copied so I thought all was okay. I am trying to merge two 30-gig drives onto a single 64-gig SSD.

Next I transferred the development kernel using the Rio upgrade wizard version 3, alpha 8 and when it booted it seemed to have erased my entire music collection because it displayed “Finding music…” so I unplugged it and installed version 2 of the development kernel using the Rio upgrade wizard and applied the v515.hijack.mk2.zImage using Tony's Empeg Logo Editor. All went well except when I swapped the old master out for the new master, I get a message on the empeg screen “Drives already built”. And when I tried to copy the FID’s from my original slave drive to my new SSD, Windows hyperterminal gave me a message that there were no files to be copied.

Does anyone have any recommendations as to which version of the kernel and hijack I need to use or any other suggestions in order to make this transfer work properly?
Posted by: Hollywood Justin

Re: Database transfer - 08/03/2012 08:17

Okay, I found the answer to my “Drives already built” problem in the FAQ under 'disk is already built', but I don't understand what initially caused the problem because I never copied any files onto that drive that I know of since I ran the disk-builder image v3. Perhaps the fids directory was created when I tried to copy the files using and older development kernel v2 or v2.01 when it told me that there were no files. Maybe that version of the firmware couldn't read the newer fs on my drives or there was some other major change to the way data is stored and retrieved between v2 and v2 beta 13? I ran v6 of the disk builder this time. And I also learned that version 2, beta 13 with the hijack kernel I was originally trying to copy files with is a developer kernel, or at least it claims to be in a terminal message.
Posted by: Hollywood Justin

Re: Database transfer - 08/03/2012 12:10

I copied dynamic data & config.ini to new SSD master disk. Rebooted and connected to emplode to show that the transfer worked perfectly & flawlessly! And there was MUCH rejoicing! HALLELUJAH!!!