Hi everyone,

I have a couple of questions for the group regarding a large mp3 collection (approx 2500 CDs) on a Linux machine. This is not exactly empeg related, though I use (and love) my empeg for mobile use. The entire mp3 collection about 260GB, so using the empeg for the whole thing is not an option. I'm struggling with the best way to manage the large collection.

Since I ripped and encoded these myself, nearly all of them are properly tagged, but some didn't get tagged. Since I started this a while back, many are tagged with v1 tags only.

The mp3s are arranged in files in an Artist/Album/##_Track.mp3 structure.

Here's what I want to do:

1. Create a list of all files on the server that don't have tags
2. Create a list of all files with v1 tags only

and then:

3. Tag the untagged files using the artist/album/track structure
4. Load all of the tag information (since everything is now tagged) into a mySQL database.

and maybe:

5. *Consider* re-tagging the v1 only files with v2 tags. I would definitely do this if I could do it automagically using the artist/album/etc. structure.

I know mp3tag studio does things like this, but it is a bitch with huge recursive directories, and I'd like a nice little unix program and shell script to do it, if possible.

Which brings me to my next questions:

I'm using mp3blaster for "home" playing which is a nice little text based mp3 player for the UNIX console. With this many files, the limits of filesystem management are becoming serious.

I want to be able to use data in a relational database to compose playlists rather than select files on the filesystem. This will also facilitate sophisticated searching, etc. For example, a big problem with file system management is having a file or group of files in multiple "genres" or something like that. I suppose one could use links, but that sucks for a variety of other reasons. The empeg handles this whole thing beautifully. It also seems to use a relational database to do it...

Ideally, I'm looking for a linux console player solution that uses a database to find files rather than a "file manager" that is customary.

Alternatively, I'm looking for server-side software that will allow me to manage the collection and some kind of client (Rio Receiver???) that plays the collection.

Does anyone know of any software that does things like this for home use? With a filesystem of this size, I have no interest in using Microsoft, and I don't think it would work anyhow. Using Linux, I have the whole shebang in one filesystem and can play 256kbit/sec mp3s while uploading/downloading on a 100bT connection -- all on an old Pentium 133. I fired up Samba and tried to share the directory with Windows, and it crashed my PIII 866 with 512MB so hard (after thrashing for 15 minutes) I needed to reset.

The JReceiver Java servlet server for the Rio Receiver seems that it might do all the database magic for me. If I get this running, are there other options for playing other than the Rio Receiver? What about configuration clients and searching clients that talk to the JReceiver database?

Can the Rio Receiver even support this number of files (probably about 40-45k files)?

What is everyone else doing?

Am I nuts? Are these questions appropriate for this forum?

Are there other Linux/UNIX collections out there? Is anyone using some kind of sql database for metadata on their collection?

OK, one last thought:

Once one has the information in a database, it seems to me that the "Album" or "Source" table could contain a field for "catalog number" so that I can arrange the physical CDs by part number. The same problems happen in a physical collection, and a part number-based inventory system seems to make sense to me. Then, I could search for "Bach" and "Prelude" and get a listing of all mp3s that match *and* the catalog number in my CD collection for the cds on which those tracks originally appeared. Since the CDs would be arranged sequentailly by part number, one could instantly find the desired CD. New CDs would be ripped, encoded, uploaded to the mp3 server, loaded into the database and tagged with the next sequential part number (supplied by the database) and then put away.

Whew!

Thanks!

Jim

The mp3s are arranged in files in an Artist/Album/##_Track.mp3 structure.

What I did was CDDBid/CDDBid-trackno.mp3 and get the metadata in a flat dbm until I learned more SQL.

The JReceiver project is a definite bonus for me. It can be a bit daunting at first, but for huge collections, its worth it.

I've got about 1600 cds, a quarter of them ripped (it takes a long time...) JReceiver doesn't require a Rio Receiver, but it helps You can use any mp3 streamer as a client. I'm not sure of the interface for a text-based system though.

I'm extremely happy with JReceiver. It might take a bit more oomph than a P133 with all the Java (I'm running it on a bargain basement $130 Duron-950), but its well worth it. Its being actively developed as well.


Once one has the information in a database, it seems to me that the "Album" or "Source" table could contain a field for "catalog number" so that I can arrange the physical CDs by part number. The same problems happen in a physical collection, and a part number-based inventory system seems to make sense to me. Then, I could search for "Bach" and "Prelude" and get a listing of all mp3s that match *and* the catalog number in my CD collection for the cds on which those tracks originally appeared. Since the CDs would be arranged sequentailly by part number, one could instantly find the desired CD. New CDs would be ripped, encoded, uploaded to the mp3 server, loaded into the database and tagged with the next sequential part number (supplied by the database) and then put away.


I was going to go with a part number as well, but settled on using storage boxes. I've got my CDs stored away in boxes, about 100 in each. When/if I need the original disc, I can scout for it in the box I've tagged it with. I still have to hunt for a CD out of 100, but I don't have to label each jewel case with a sticker or something. Yes, I'm that lazy. It's bad enough to have to rip all these CDs, you think I want to print up 1500+ stickers with serial numbers?!

How's your programming skills? Since you're playing on Linux, and asking for shell script-y type things, I'll suggest looking at perl, particularly the File::Find module (to recurse through your directories), and the MP3::Tag* modules (you could probably just install the MP3 bundle from CPAN). I wrote my own custom rip/encode/tag script with them, and they're pretty easy to use. With them, 1, 2, 3 and 5 become trivial:

use MP3::Tag::ID3v1;

use MP3::Tag::ID3v2;

my $mp3 = MP3::Tag->new($filename);

$mp3->get_tags();

if (exists $mp3->{ID3v2}) {

    # Put it in the database. There's mySQL modules

    # available on CPAN, as well.

}

elsif (exists $mp3->{ID3v1}) {

    # Convert tag to id3v2. Left as exercise for the 

    # reader. perldoc MP3::Tag::ID3v2

}

else {

    print "$filename : No ID3 tags found.\n";

    # Create id3v2 tag from $filename

}

The id3v2 tag spec is pretty simple -- you're just looking at TPE1 (artist), TALB (album), TIT2 (trackname), TRCK (track number), TCON (genre), TYER (year) and COMM (comment) at most. Except for the last three, everything you need in your id3v2 tag is encoded in your filename.

Sorry I can't give any suggestions on the sql -- I'm so far lucky enough to remain blissfully unaware of databases.

Cheers,

Thanks, this is a great suggestion. I haven't done much programming in years, but I'm not afraid of it. I haven't done anything in Perl, but I've been looking for an excuse to learn it :-)

Would you be willing to post your perl scrips for me to start with? Or, you could email them to me. I don't know how this board works for emailing, but I'm at jrcampbe [at] tiny [dot] net.

Thanks also for the comments on JReceiver. I will look into that further for a potential SQL solution.

I appreciate all of the thoughts!

Jim

In reply to:

---

Would you be willing to post your perl scrips for me to start with?

---

Sure, no problem. Err... once I sort out my network problems at home (I'm posting this from work).

Incidentally, I was doing a Debian install over the weekend (proud 2nd owner of a DEC Alpha used to render visual effects for Titanic), and I noticed in one of the audio sections that there were a couple of packages that look like they would fit the bill for what you want -- command line mp3 player, mySQL mp3 db utility, etc. I'll poke around again and see if I can find them again.

Cheers,

That mp3/SQL thing is called digitaldj. There's a webpage at http://nostatic.org/ddj/ddj.html, with full source code.

In reply to:

---

Would you be willing to post your perl scrips for me to start with?

---

Ah, finally. If you're still interested in the script, it should be an attachment to this...