Automatic MP3 identification

Tried it and it rocks!

$ sndref "Beck - Mellow Gold - 12 - Blackhole.mp3"
Processing Beck - Mellow Gold - 12 - Blackhole.mp3
Generating signature
Signature:
98a3    a436    fffe    f0fe    ec93    c612    b291    96c7
8e4a    68db    53ab    4314    193b    1285    06b2    0035
209d    200a    20b8    1be1    1e41    1f0c    226c    1f00
1ca0    1f44    1cae    1b5f    15ab    140a    0dc6    01ad
Querying server for a match for this signature
Matches found for Beck - Mellow Gold - 12 - Blackhole.mp3:

Title: Blackhole
Artist: Beck
Album: Mellow Gold
Genre: Pop

GPL signature library.

eTantrum

Paul

Neat, neat, neat, neat!!!!!!!

It's like a CDDB for your audio files, only it's better and it's not dependent upon the media format. What a great idea. Thanks for the link.

___________
Tony Fabris

Wow!

In reply to:

---

Title: Blackhole
Artist: Beck
Album: Mellow Gold
Genre: Pop

GPL signature library.

---

How do they get Beck as "pop"?!? Maybe Alternative pop but mostly Rock. This is something that rubs me wrong about automatic tags... they aren't always right.

Tom

Reg #2845: Mark 1 #00173, Mark 2 #119

This is something that rubs me wrong about automatic tags... they aren't always right.

Well, genre is subjective...

Keep in mind that what's rubbing you wrong isn't so much the automatic tag technology, it's the fact that the data comes from a database maintained by the general public. Whoever tried to tag that Beck album first is the one that chose the genre. I dunno if this new software allows updates, but the CDDB does so I'll bet this one does, too. So if you think it should be some other genre, you'd tell the software to update the database with a more correct one.

Despite that hassle, it's a lot better than a complete lack of tag information, and certainly better than having to look up the information manually. Still, I always carefully go through my own tags, whether I'm assigning them myself or I'm getting them automatically.

That's really the only thing keeping me from demanding that everybody fill out the lyric field in ID3v2 tags: You'd be depending upon some yokel to fill out the lyrics correctly. Then you'd download "Purple Haze" and the screen would scroll "'Scuse me while I kiss this guy."

___________
Tony Fabris

I never knew Jimi was gay... hehe

Frank van Gestel

I never knew Jimi was gay... hehe

http://www.kissthisguy.com

___________
Tony Fabris

$ sndref "Beck - Mellow Gold - 12 - Blackhole.mp3"

I was reading through the forums and came upon this post. It immediately made me call the last few of us remaining in the office into here to read it.

I work for eTantrum, the company that created this. :) Ironic that I should see a post about it. We just released our GPL'd code yesterday and it's already here. Rock on!

--plug alert, plug alert--

We support multiple formats too, MP3, Vorbis, WAV, etc. I recommend you all try it out and tell all your friends too. We have Linux and Win32 clients to demo the ID system.

We also have a full Media Organization application out there called the PMA (Personal Media Agent) with both Win32 and Linux versions. You can get this off our page. It "learns" what you listen to, and not only will fingerprint your music and get the tags from our system, but will recommend music you might be interested in, as well as providing "more info" on albums you have, such as full album info and cover art. Plus we have a kick ass player with a skinning system you've never seen the equal of. ;)

-- end plug alert --

Still, that's pretty DAMN cool that this made it into the empeg forums so quickly. :)

The code is on sourceforge under the project name Freetantrum, or you can head to www.freetantrum.org, or our company's page at www.etantrum.com.

Rock on empeg!

(O|||||O) / eTantrum :)

I work for eTantrum, the company that created this. :) Ironic that I should see a post about it. We just released our GPL'd code yesterday and it's already here. Rock on!

Nah, we know you and Xml are in cahoots and this was just a staged thing.

Regardless, it's a nifty idea. I plan on checking out the database spec now and pointing out the holes in it...

So you've been on this BBS for, what, at least a year now. WHY DIDN'T YOU TELL US WHAT YOU WERE WORKING ON?!?!?!

___________
Tony Fabris

So you've been on this BBS for, what, at least a year now. WHY DIDN'T YOU TELL US WHAT YOU WERE WORKING ON?!?!?!

I've been on this BBS since you were in diapers. ;) Nah, I've been on the board since nearly day 1... I guess I just don't have a lot to say most of the time. But I've read nearly every post on here. :)

Anyway to the company..

The stuff we've been working on has been real hush-hush, since we have companies possibly trying to acquire us, as well as competition on many different levels. So our big announcement was yesterday (of the opensourced GPL songprint library). We should be on the various opensource news media channels early next week. I think we already are on www.linuxtoday.com.

My stuff is in sp_identify.c, feel free to rip it a new a-hole. ;)

PS: we're DEFINITELY looking for constructive criticism about the service. We have forums set up on sourceforge I believe, feel free to use them. :)

(O|||||O)

ps, I still think the fact that this was on the empeg boards so quickly just rocks.

PS: we're DEFINITELY looking for constructive criticism about the service.

Well, first criticism is that it doesn't work on my box.

I think it's because I'm behind an authenticating firewall. The media agent never prompts me for my firewall ID or password, so I assume it just doesn't support that yet. This is a pretty normal thing- most internet apps I try won't work through my firewall until they've matured. Heck, ICQ didn't even work through my firewall until just a couple months ago.

___________
Tony Fabris

In reply to:

---

ps, I still think the fact that this was on the empeg boards so quickly just rocks.

---

Well, think about it... Sure your products have broad appeal to any music lover, but who needs this kind of functionality more than Empeg owners, with their gigs upon gigs of MP3's? This seems like very cool technology... You guys hiring? :)

---
080000554, soon to be in my 1999 Mustang

John took a look at the code and says he can make it a huge amount more efficient... beware, he'll be all over it :)

Hugo

In reply to:

---

eTantrum

---

Now this is another really useful piece of technology which won't run cos I'm doing something wrong.

So...

I try and './configure' and it throws a wobbler because there's no FFTW library installed, so I download that, make it and install it. So, I try './configure'ing Songprint again. Of course, I assumed that I had glib installed and I hadn't, so it failed again. Off I trotted to download glib, configured, made and installed it.

Back to Songprint, which failed because it said it still couldn't find glib. One quick 'ldconfig' later and all's well.

Several options, configurations and whatever later, it tells me that I need a new glibc (v.2.1.3). Off I trot to Freshmeat.net who say that the latest version is 2.1.2 (it isn't, and 2.1.3 is actually used on the page you click through to, but that one had me going for a few minutes). Of course, when I try to install that, it says that I need this, that and the other.

Oh well, I thought, it says I only really need it for Ogg and I'm not using that, so back to Songprint.

I finally get through to typing 'make' and am then thwarted with all sorts of problems with variable 'tval' in sp_identify.c I gave it up as a bad job and went to bed.

Like I said, it's bound to be my fault and I'm not blaming anybody here. I was tired last night and maybe I didn't read great swathes of the docs. I'll give it another go later.

Nick.


--
18Gb blue - s/n 080000299 (original queue position 8724)

Now this is another really useful piece of technology which won't run cos I'm doing something wrong.

We could use the input on your environment to improve the documentation and the FAQ. What distribution are you trying this out on?

-farmy
CIO / Co-founder
eTantrum.com

John took a look at the code and says he can make it a huge amount more efficient... beware, he'll be all over it :)

Oooo.... Since it's open-source, what about adding this functionality to Emplode? Or are you already thinking along these lines?

Just think- drop all your MP3s into Emplode and it could offer to just fill out the blank tags for you?

___________
Tony Fabris

This might happen, but only for tunes without tags. For ones with tags, the chances are that the existing tags are more accurate...

Hugo

Dont feel bad, I couldn't get it to compile either. Im missing the glib library. I went to download it and everything was in tarballs. I don't have a program that will decode tarballs for windows. Is there anyone who could email me a working win32 exe file?

Sean
[email protected]

Dont feel bad, I couldn't get it to compile either. Im missing the glib library. I went to download it and everything was in tarballs. I don't have a program that will decode tarballs
for windows. Is there anyone who could email me a working win32 exe file?

Yeah theres a .zip file, its the next file below songprint-1.0.tar.gz. Heres the shortcut:

http://download.sourceforge.net/freetantrum/songprint-1.0.tar.gz
http://download.sourceforge.net/freetantrum/songprint-1.0.zip

-farmy

In reply to:

---

What distribution are you trying this out on?

---

Hi, I would have replied privately, but your profile is blank .

It's a SuSE derivative. Installed a while back (6 months or so) from the latest distribution at the time. Since then it's been upgraded to 2.4.0-test4 kernel and various other odds and sodds.

E-mail me privately ([email protected]) if you want more info.

Cheers,

Nick.


--
18Gb blue - s/n 080000299 (original queue position 8724)

I don't have a program that will decode tarballs for windows.

You're kidding. Don't you use WinZip? Mine does tarballs just fine.

Is there anyone who could email me a working win32 exe file?

They have a full Win32 media agent program available for download at their site. Is that all you wanted, was to run it in Windows? If that's all, then why were you trying to do it the hard way by compiling the source code?

___________
Tony Fabris

I didnt even try winzip, I just assumed it wouldnt do tarballs. I wanted to mess with the source code and see how it works. I have a data structures class this semester and its been a while since ive had to code anything - the last compiler I used was borland 3.0 for dos. I have microsoft visual studio installed now and wanted to get used to the commands and how it works. I tried to download the media agent several times and it just stops halfway through the download for some reason.

It was the glib library that was on another site that came only in a tar file. I got the zip file from the tantrum site already. :-) Thanks for the help though.

Sean

> I work for eTantrum, the company that created this. :)

Any chance you can post a description of the algorithm? I'm having a hard time figuring it out from the source code.

I actually had an attempt at doing something like this some time ago. I used voc26 from SFS, and created coarse vocoder samples from the first 15s of a tune. The problems I immediately observed if I downloaded a tune from napster from several different people (of something I already owned of course ;-) ) were that tunes were of different amplitude, differing, or no leading silence, different frequency responses (maybe due to bit rate) and inverted wave forms. It seemed that some normalisation phase was required prior to generating a signature. How do you handle these kind of things?

Some links that might be of interest:

Tuneprint
A Patent
Srinivasan
Musclefish
Musclefish paper

Paul

John took a look at the code and says he can make it a huge amount more efficient... beware, he'll be all over it :)

Which is why we made it opensourced under the GPL. :)

Which part, BTW? The algorithm for generating the fingerprint of the audio is actually probably going to be outdated relatively soon. We've come up with an even better way to uniquely represent a stream of audio data, and we're in testing phases right now with it.

(O|||||O)

-farmy
CIO / Co-founder
eTantrum.com

Hi, boss. :P

(O|||||O)

I think it's because I'm behind an authenticating firewall. The media agent never prompts me for my firewall ID or password, so I assume it just doesn't support that yet. This is a pretty normal thing- most internet apps I try won't work through my firewall until they've matured. Heck, ICQ didn't even work through my firewall until just a couple months ago.

Yep, proxy/firewall support is on the list of things "to do". We just wanted to get this thing out there. I'll probably be adding that sometime next week. :)

(O|||||O)

I think it was that bit, he didn't look impressed with the way it was done and muttered something about a 10x speedup :)

Hugo

Could someone tell me what the signature actually represents?

Alex Lear

I tried to download the media agent several times and it just stops halfway through the download for some reason.

Ah, then go snag GetRight. The second most useful program on my hard drive, right behind UltraEdit.

___________
Tony Fabris

Yep, proxy/firewall support is on the list of things "to do". We just wanted to get this thing out there. I'll probably be adding that sometime next week. :)

Sweet. Let me know, eh?

By the way, when you do get the proxy support in there, make sure it supports authenticated proxies: i.e., ones where you have to enter your username and password. Most apps I've seen go through the following development growth:

- First release: No firewall support.
- Three months later: Firewall support, but only for simple proxies.
- Three months later: Support for authenticating firewalls.

I can never use the app until it reaches the third stage.

By the way, does it use a non-HTTP port for its communication? My firewall has all the ports blocked except for 80 and 81. For instance, when using the CDDB, I have to tell it to use the HTTP protocol to get the job done or it won't work.

___________
Tony Fabris

I used to have getright a long time ago, but I found out it had spyware built into it, so I deleted it. You can read about it at www.grc.com. Netscape smartdownload also sends info on everything you download.

Sean

I used to have getright a long time ago, but I found out it had spyware built into it, so I deleted it. You can read about it at www.grc.com.

Odd, I just read Steve's page there, and the name GetRight isn't even mentioned. He lists three other download utilities, but GetRight wasn't one of them.

For a second, you had me really panicked.



___________
Tony Fabris

>> I used to have getright a long time ago, but I found out
>> it had spyware built into it

> Odd, I just read Steve's page there, and the name
> GetRight isn't even mentioned

Apparently it contains Aureate according to Aureate's product list.

Paul

Apparently it contains Aureate according to Aureate's product list.

Oh, if that's all, then I don't care. Aureate is just ad-banner software (slightly unsecure ad-banner software I'll grant). The thing that scared me from Steve's page is the idea that GetRight might be sending my download habits back to Headlight Software. Apparently, it's not doing that, so I'm not worried now.

And Steve gives pretty good instructions on which Aurate file is insecure, and I can't find it on my system anywhere, so everything's cool.

___________
Tony Fabris

This is exactly the type of thing we were hoping the open source community could help us out with :)

Download and run the optout program he has on his website - it will look for certain files and let you know if you have spyware installed. I thought getright's ad engine was powered by aureate, but im not sure. Its been a while - I didnt mean to freak you out.

Sean

I thought getright's ad engine was powered by aureate, but im not sure.

I think the reason it's not a problem for me might possibly be that I've been a registered GetRight user since before they had the Aureate adware included with it. So every time I install a new version, it just skips the adware part.

___________
Tony Fabris

Cool! I might have to check into that then. I thought it just came with the program, and there was no way to get rid of it except to uninstall it, and go through and find the evil files on my hard drive. -nice run on sentence-

Sean

Guys@eTantrum,

I might be wrong, but you seem to have succumbed to temptation very often succumbed to, that is to default your configure templates to configuration you had (including the very latest versions of rather generic libraries), when older versions would probably do (since we are talking about char-mode application).

Could you either relax your build requirements (so that the program builds with, say RH 5.0 up and equivalent other distribs - if that is indeed possible), or provide statically built Linux binaries.

Your program seems great, but I was still not able to try it...

Cheers!

Dragi "Bonzi" Raos
Zagreb, Croatia
Q#5196, MkII#80000376, 18GB green

I might be wrong, but you seem to have succumbed to temptation very often succumbed to, that is to default your configure templates to configuration you had (including the very latest versions of rather generic libraries), when older versions would probably do (since we are talking about char-mode application).

Could you either relax your build requirements (so that the program builds with, say RH 5.0 up and equivalent other distribs - if that is indeed possible), or provide statically built Linux binaries.

What requirement that the SDK contains are you having trouble with? We use pretty standard build requirements, except for glib-1.2 (which has been around for at least a few years) and the fftw libraries (which our algorithm relies on). Other than that you shouldn't have any problems building it.

Or are you talking about getting the PMA running? Because that does require more recent libraries.

Let me know exactly what the problem is.

(O|||||O)

We use pretty standard build requirements, except for glib-1.2 (which has been around for at least a few years) and the fftw libraries (which our algorithm relies on). Other than that you shouldn't have any problems building it.

You are right, of course. I have some mixup between glib 1.0.6 and 1.2.3 (with both versions in /usr/lib, older one being default), with some apps requiring each (and without development version of 1.2.3 installed, apparently, so no glib-config). I extracted glib-config from 1.2.3 dev manually, but now of course glib test program does not run due to old includes. I will try installing 1.2.3 dev somehere else and tell config manually where are libs and includes.

It seems I will not be able to postpone that upgrade much longer....

UPDATE: I am a complete idiot, of course. All it took was to force-install glib-dev-1.2.3 (and the only other package that depended on earlier version, gtk). Now I am playing with sndref.

Dragi "Bonzi" Raos
Zagreb, Croatia
Q#5196, MkII#80000376, 18GB green

Edited by bonzi on 13/9/00 10:01 PM.

Okay, I'm not afraid to sound unkowledgable here. How the hell do I get this to work?

I downloaded that PMA program which is fairly interesting. I want to know how to do the ID tag thing. I tried clicking the "more info" or whatever, but it gave me a message saying something like "can't get info for user-defined song" or something. The help file doesn't help much in this area either.

I downloaded the songprint files but I don't have a compiler.

I'd really like to know how to do this.

Thanks

DiGNAN

I am not sure I can help here, since I haven't tried PMA, but only Linux version of 'song-fingerprinting' library (which does include a fully functional command-line demo program). The building process is completely straightforward (one also needs a fancy Fourier transform library available separately installed - see eTantrum's pages). My only problem was a mess my Linux machine is in. Once I got it somewhat in order (by force-installing development libraries (glib 1.2.3 and such) around for a year or two), it only took standard './configure; make; make install'. I made a fool of myself slightly by complaining they insist on using the most recent libraries before realizing this.

Do you have problems on Win or Linux?

The demo program fingerprints your WAV, MP3 or OggVorbis tune and contacts their server to try and identify it. If succesfull, it shows a list of possible matches (it guessed the tune correctly every time it found it, but there were different albums and sometimes genres). If not, it prints a long URL by which one can add a tune to their database. This, a little piece of Perl and a ID3 maintaining utility is all it takes to fill in one's missing tags (provided somebody already added the tune in question to eTantrum's database). However, since they don't take 'last submission is valid' approach like CDDB, but rather present you with all possibilities (as they should), the process is interactive.

BTW, I am now playing with different versions of the same tune (different albums, different encodings) to see how good are they in recognising them. So far, quite impressive.

Well, I don't think this really helped.....

Dragi "Bonzi" Raos
Zagreb, Croatia
Q#5196, MkII#80000376, 18GB green

Edited by bonzi on 17/9/00 10:58 AM.

Well, I guess you could say I'm having problems in Windows, but not really problems. As I said, I have no idea what you guys are talking about! I'm not a programmer, and I don't know why heck don't supply the program in a finished form! I just want something to do what it says it will do, and I'm not interested in compiling and all that stuff.

Also, you said you needed an "ID3 maintaining utility"? What is that? Does that mean you have to fill in the tags yourself even after they've been identified? Heck, I can do that myself! I know where all my songs are from, I'm just looking for an easy way to fill them out!

Again, correct any of my misguided assumptions if they are incorrect. I'm a little confused here...

DiGNAN

I'm not a programmer, and I don't know why heck don't supply the program in a finished form!

I thought they did. Isn't that what the Personal Media Agent is? I don't know because I can't get the PMA to function from behind my firewall.

Also, you said you needed an "ID3 maintaining utility"? What is that? Does that mean you have to fill in the tags yourself even after they've been identified?

I think that's only if you're using it in command-line mode under Linux or if you're integrating their code into your own. Isn't the Personal Media Agent supposed to handle that?

___________
Tony Fabris

Well, I guess you could say I'm having problems in Windows, but not really problems. As I said, I have no idea what you guys are talking about! I'm not a programmer, and I don't know why heck don't supply the program in a finished form! I just want something to do what it says it will do, and I'm not interested in compiling and all that stuff.

As Tony already answered (is this guy ever away from the keyboard for more than ten minutes :), PMA is supposed to be a fully functional demo GUI application. If you have problems with it, perhaps eTantrum's and our ClemsonJeep could help. I am playing on Linux at the moment, with their 'bare' library, so no help from me, I am affraid.

Also, you said you needed an "ID3 maintaining utility"? What is that? Does that mean you have to fill in the tags yourself even after they've been identified? Heck, I can do that myself! I know where all my songs are from, I'm just looking for an easy way to fill them out!

Again, PMA is supposed to do that (if I read description correctly). Command-line demo, on the other hand, is written in Unix tradition: it provides what guys@eTantrum do best - it identifies the tune, queries their database and prints results. There are numerous command-line utilities to modify ID3 tags (as are for ripping, for mp3 encoding, for cddb querying, for maintaining jour mp3 database etc). The user is expected to glue them together using shell/Perl/awk/Python/bubblegum/duct-tape/whatever, with or without GUI. If someone comes with particularly nice and usefull leashup and decides to publish it, it becomes more or less a standard. For example, grip, one of most popular all-in-one Linux rippers/encoders, is actually just a GUI shell for variety of rippers, encoders and taggers, and user can add their own. That is (original) Unix philosophy: write small tools that do one job, but do it right, and provide an easy way to combine them. However, nowadays Unix is also full of bloatware in Widows style.

Cheers, and keep us informed about advances with PMA!

Dragi "Bonzi" Raos
Zagreb, Croatia
Q#5196, MkII#80000376, 18GB green

Okay, then it goes back to what I said earlier. I thought PMA was supposed to do all this, but it just plain doesn't!

Here's the message I get when I highlight "Would?" by Alice In Chains:

In reply to:

---

"Would" is a user-defined song.
eTantrum.com cannot give detailed information about user-defined songs.

---

This is exactly what it gives me for EVERY SINGLE SONG I TRY!! Now, I might be mistaken about which button does what, so the "More Info" button may not be what I'm supposed to press for this function. However, the help file says this:

In reply to:

---

The More Info Button

This is where you can find out all the information on the song that you are currently listening to. Artist, Track name, Album it was released on, and Lyrics (if available). This will also be where you can use the "sounds like" feature to find similar sounding music.

---

That sure sounds like it to me. Although it doesn't say anything about filling out the tags it'self. It looks more like this program is informational. Like it will tell you the information and make you go change it. I don't know, it may be just me, but I think this program is trying to be too many things.

DiGNAN

I had the same problem you describe. Every time I hit more info, I got the exact same error. Then one time, I was on-line, and ran PMA. I did nothing with it because my cell phone rang, and I answered the call. As I was talking on my cell, I watched PMA slowly (one every five seconds or so) go through my MP3s, and change the Album from the correct one (Aqualung) to something else (I forget what) that was still the correct song, but not the album I ripped the CD from.

I assume it takes a while of inactivity to activate this feature?

I also know what Dignan was saying about the finished form. I would like to use the command line version of the program (not the full-blown player), just the identifier. I am away from my compilers, etc. so I can't compile the source.

It would be nice if they (someone) could supply a pre-compiled binary of just the command line version (sndref.)

-Trevor

-----
Mk 2, Green 12GB 080000349

I also experienced this. However, I still had some problems. Here's what I did.

I loaded one song, the Beatles' "Across the Universe". I knew it had absolutely no tag info in it. To my suprise, it filled out some info on it. I appologize for my earlier confusion, but there was no information saying the program did this.

Anyway, it only filled out the artist and track name fields. Of course, I wanted all the fields. So I tried something else. I removed the song from the playlist and closed the program. I then renamed the song from "Beatles - Across the Universe.mp3" to "song.mp3". Then I removed all tag info. Then I opened PMA and loaded the song. This time it just sat there, with the word "song" sitting in the track name field. I played the entire song about 3 times. Nothing.

What's up?

I also have yet to see a song get all it's fields completed. That's including "Loser" by Beck which, I assume, xml was able to get.

WHAT'S UP?

ps-honestly, I don't want another MP3 player. You should stick to making a program the just tags songs at first. After that works, you should expand.

DiGNAN

I was trying this program tonight and up pops a virus warning from Norton Antivirus.

This is the only place I can figure it came from.

Anyways it had infected my notepad.exe file with the W32.HLLW.Qaz.A virus.
Anybody else seen this or know if it can be transmitted by email?

Win98, IE4 Outlook Express 4.

Thanks all!

P.S. I was playing a song on the PMAs player when it happened.

#170... I got SN: 080000101 12 gig Blue!

I was trying this program tonight and up pops a virus warning from Norton Antivirus.

This is the only place I can figure it came from.

Er, if you downloaded the PMA from our website, it is impossible for it to have a virus inside of it. It must have been infected when you installed it, which means you contracted the virus from somewhere else.

I'd recommend booting from a clean floppy and running a virus check on every file in your system.

Let me know if you find any problems or anything.

(O|||||O)

Just a quick note, more info about that virus can be found here.


Donato

Sorry for any scare... I have just dug up some info and this is a fairly new worm... which was only detected recently due to my DAT update. Sorry to cause any confusion.

Though I can't remember installing anything else in the last couple of months on that particular computer... oh well.

Later, and sorry for any confusion ClemonJeep.

#170... I got SN: 080000101 12 gig Blue!

I agree with DiGNAN17. This is a really cool concept and I'm very encouraged that it is working so well in identifying the songs. You don't need extra bells and whistles to make people want to use this tool. And you certainly don't want to make it unneccissarily difficult to use for its primary purpose by adding more features. And you really don't want to be doing things in the background without the user asking for it and without telling the user what you are doing and why. The sndref command line version for Linux sounds like exacly what I want. No more no less. How about a Windows version like that? And post some binaries. ClemsonJeep, I know you are reading this thread, but you haven't addressed any of the recent comments.

I'll third list. I installed it, and the first thing it did was ask for demographic information including pay rate. For someone who doesn't live in the USA this is relatively meaningless, and I always suspect anything that tries to get this kind of information from me - it smells too much like targetted advertising. I quit it and deinstalled it immediately.

Therefore I haven't tried out the program at all. I don't think it's unreasonable for the program to continue working if I try to cancel that dialog, albeit without whatever benefit that information might provide to it. I also want to know why it needs to know that information before I'll fill it in.

Suspiciously yours,

Paul

Save the whales. Feed the hungry. Free the mallocs.

I'll third list. I installed it, and the first thing it did was ask for demographic information including pay rate. For someone who doesn't live in the USA this is relatively meaningless, and I always suspect anything that tries to get this kind of information from me - it smells too much like targetted advertising. I quit it and deinstalled it immediately.

I assume you are now talking about the PMA. The demographic information for one is optional, and for two is simply gathered for use with our algorithms for song recommendation. To be completely honest, most of that stuff isn't even used; the PMA was technology that was already WELL under development when I started working for eTantrum, so I can't say I know a lot about what that info was for.

I *do* know that the ZIP code *IS* used with our Concert recommendation engine, but that really doesn't apply to overseas people as we only provide concert information for US clients.

Therefore I haven't tried out the program at all. I don't think it's unreasonable for the program to continue working if I try to cancel that dialog, albeit without whatever benefit that information might provide to it. I also want to know why it needs to know that information before I'll fill it in.

All of that information *should* be optional. If it is not, then that is a bug which I'll look into. I'll do a fresh install tomorrow and see if this is the case.

Either way, this thread was began with discussion of the SongPrint SDK that was released. The PMA is an application that was created months before we even decided to release an opensource version of our client.

Okay. Any more questions, feel free to ask. :) I'll be sure to look into that bug.

Thanks!

(O|||||O)

Okay first off, I think we have some confusion as to what we are discussing here.

We have two *totally* different products at the moment available for download.

We have the PMA, which is a very large application used for media organization, identification, and recommendation.

We have the SongPrint SDK, which is simply the identification part from above.

That being said, I will try to respond to your message as best I can. :)

You don't need extra bells and whistles to make people want to use this tool. And you certainly don't want to make it unneccissarily difficult to use for its primary purpose by adding more features.

I assume you are talking about the PMA. The PMA is *not* designed to simply be an identification tool. It is a very large application that provides what I said above - a VERY smart media "Jukebox" (although I hate to use that term)... it just also happens to identify music. :)

The sndref command line version for Linux sounds like exacly what I want. No more no less.

Okay, now to clear up some other points.

The sndref "utility" really isn't a utility. It is simply a linux program created to USE the SongPrint SDK and demonstrate its ability, in a nice easy command line interface. It's meant for people to compile and run and say "Wow, this kicks ass! I want to add this SDK into my YYY media program!" (Insert the media application they are working on in place of YYY)

So this brings me to my next reply..

How about a Windows version like that? And post some binaries.

There *is* a windows test program included in the archive. It performs the exact same thing as the linux "sndref" command line utility, except it has a simple GUI. Since we aimed the SDK at developers, we decided to just give the source and the project files for the test programs, to let the developer compile it themselves.

I can understand why you would want a windows binary, though... especially if "Average Joe User" just wants to see the technology in action.

I will compile up the latest version along with all the files you need for the Win32 "sndref" and post it on this board tomorrow. Sound good? :)

Whew. Okay. I'm going to bed now. I have to wake up early. :)

Seriously, thanks for the comments. I definitely appreciate all feedback, especially from you fellow empeg guys. :)

(O|||||O)

All of that information *should* be optional. If it is not, then that is a bug which I'll look into. I'll do a fresh install tomorrow and see if this is the case.

Whenever a piece of software or a web site asks me for demographic data, e-mail addresses, or personal address/phone information, I always fill out the fields with filthy garbage. The PMA was no exception. They can't invade your privacy if you don't let them.

(Of course, this doesn't apply if they need my e-mail address to verify an online purchase, etc., but most programs/sites that prompt for this data don't require it to be accurate.)

___________
Tony Fabris

I assume you are talking about the PMA. The PMA is *not* designed to simply be an identification tool. It is a very large application that provides what I said above - a VERY smart media "Jukebox" (although I hate to use that term)... it just also happens to identify music. :)

Okay, a few things. I knew PMA had extra stuff in addition. My complaint was two-fold: if I can't have an identification-only program, why do I have to have all that additional stuff? The SongPrint thing is NOT for the average user and you don't say much to that fact.

What's the problem sith developing a simple utility that identifies songs and fills out the tags? And remember, simple means simple setup, too (no compiling or libraries nonsense seen anywhere). In other words, for "Average Joe Consumer".

And please read all our troubles with identifying songs using PMA, since you say that it can and we have found that it can't.

I don't mean to sound irritated, I just think this is an incredible idea from what I've read! But for some reason, I am completely unable to see it!

anxiously awaiting your post tomorrow...

DiGNAN

Mmmm, but this is how eTantrum want to make their money; quote from the website:

---- snip!

eTantrum's Dynamic Auditing System provides a powerful solution for real-time usage and demographic reporting. This technology generates customized reports on:

- Aggregate, anonymous consumer usage data

- Point-of-use behavioral analysis based on media usage coupled with;
Demographic data (Sex, Age, and Zip code)

- This accurate and timely market research information is available in real-time and offers significant data that influences decisions made by the entertainment industry from pre-production through product release and distribution.

*eTantrum's technology is able to generate real-time demographic and usage information without compromising the user's name, email address, physical address, or any other personally identifiable information.

----

I don't see this as a problem personally, but I know people who would; however "non-identifiable" the source is, there's usually a problem a year down the line when someone finds out it's been logging ip addresses (not saying this will happen to eTantrum, but it seems to be pretty common in statistic gathering software!)

Hugo

And please read all our troubles with identifying songs using PMA, since you say that it can and we have found that it can't.

When you add songs into the PMA, it automatically starts identifying them in the background. If the song is not found, it will return "User-Defined" as the "File type". If the song is found in our database, it will return "eTantrum ID" for the file type. To get moreinfo, the song has to be found in our database.

We have about 60,000 signatures in our database right now, and the number keeps growing daily.

Attached is a freshly compiled SongSigDemo.exe along with the two DLL files you need to run it. Keep in mind this is just a simple test program just to demonstrate the technology. It doesn't have functionality to go back and modify the ID3 tags in a MP3 file.

If you have any problems, questions, or requests, please feel free to msg here or privately.

My email is [email protected].

--addition---

Apparently we can't attach files that big to posts. Here is a link to where you can grab it.

http://www1.etantrum.com/~rjohnston/SongprintDemo-1.0.zip

----

(O|||||O)

I don't see this as a problem personally, but I know people who would; however "non-identifiable" the source is, there's usually a problem a year down the line when someone finds out it's been logging ip addresses (not saying this will happen to eTantrum, but it seems to be pretty common in statistic gathering software!)

I can definitely tell you that we do not log (or ever will log) IP addresses. I've seen the logs myself. :)

Buuuut then again, why should you believe me? :)

(O|||||O)

I thought they did. Isn't that what the Personal Media Agent is? I don't know because I can't get the PMA to function from behind my firewall.

The PMA is far from done. :) We still have a pantload of stuff to implement/fix inside of it. It's been kinda put on the backburner since we've been pouring our entire effort into the Opensourced Songprint SDK recently.

I think that's only if you're using it in command-line mode under Linux or if you're integrating their code into your own. Isn't the Personal Media Agent supposed to handle that?

Nope. Well, not currently. The whole point of the technology is to *NOT* rely on ID3 tag data (which is inherently incorrect). Now, a feature where the PMA would go and REPLACE your ID3 tags with OUR tag data, that would be something that could be implemented, and we just hadn't thought of yet.

(O|||||O)

Thanks for clearing this up a little.

Then how do you explain my "Across the Universe" thing? When the filename was "BEATLES -- Across the Universe.mp3", when it was loaded into the player it eventually filled out the track name field and the artist field. The others were blank. And there was no tag info on the file to start with.

Then I renamed the file "song.mp3" just to test it. It didn't find it, and left the word "song" in the track name field.

Then there's the test I ran with Beck's "Loser". In the very first post in this thread, the xml found the info for an arbitrary song off of Beck's Mellow Gold album, the same one that "Loser" is on. But when I loaded "Loser", it didn't find it. Instead it did the same as the Beatles tune, and put the filename, sans-extention, in the track name field.

I'm still figuring out how this thing works. also, is it possible to return info to the database using PMA? How does it know if we don't want it to fill out our tags for us.

DiGNAN

Then how do you explain my "Across the Universe" thing? When the filename was "BEATLES -- Across the Universe.mp3", when it was loaded into the player it eventually filled out the track name field and the artist field. The others were blank. And there was no tag info on the file to start with.

Then I renamed the file "song.mp3" just to test it. It didn't find it, and left the word "song" in the track name field.

Then there's the test I ran with Beck's "Loser". In the very first post in this thread, the xml found the info for an arbitrary song off of Beck's Mellow Gold album, the same one that "Loser" is on. But when I loaded "Loser", it didn't find it. Instead it did the same as the Beatles tune, and put the filename, sans-extention, in the track name field.

I'm still figuring out how this thing works. also, is it possible to return info to the database using PMA? How does it know if we don't want it to fill out our tags for us.

Er.. When you say "tags" what exactly are you referring to? ID3 tags? What application are you referring to? The PMA or this test program? You need to be a bit more specific. :)

The PMA will do this in this order:

1) Parse the filename (Artist, Title, Album, etc) into the correct places
2) Parse the ID3 tag info (if any)
3) Attempt to identify the song with the song fingerprinting technology.

It will override any of the above "tags" with each of its predecessors.

In the PMA, if you do NOT have a "eTantrum ID" under the File status column, that song did NOT get identified with the songprint technology. It is relying on #2 or #1 from above.

The PMA does not have the option to enter song information if it is not found in our database (like the songprint SDK does). Once again, the PMA is older technology and the songprint SDK is newer.

My recommendation would be if you are just trying to see our identification system, use the program I linked to above. It will tell you if the songs you have are in our database, and if they aren't, you can add the information via a web page.

(O|||||O)

Okay, that's what I figured, despite the fact that on your website somewhere, it specifically says that PMA doesn't rely on the filename for identification. You should probably ammend that.

And yes, I'm using the PMA program.

And it still doesn't explain the Beck thing. This is a song that I'm guessing is in the database yet doesn't get recognized. But whatever.

DiGNAN

Okay, that's what I figured, despite the fact that on your website somewhere, it specifically says that PMA doesn't rely on the filename for identification. You should probably ammend that.

It doesn't rely on that. :) It will fall back on that if other identification methods fail, but in a perfect world, our database would contain every signature for every song ever created. Then we wouldn't need filenames or ID3 tags.

We're just in the process of building our database up right now, so we've got a.. er.. while.. to go. :)

And it still doesn't explain the Beck thing. This is a song that I'm guessing is in the database yet doesn't get recognized. But whatever.

Its possible. Each fingerprint is relatively unique. We perform some analysis on them ("alignment") which figures out what fingerprints are matched to what song data. It just might happen that you have an iteration that hasn't been aligned yet. If you use the SongPrintDemo-v1.0 program, it allows you to enter in the data to be submitted to our servers. Right now there is a lead time on when they get added, but we're fixing that right now so they are added automatically.

(O|||||O)

Anyone else tried this? Any successes, failures, comments?

Feel free to reply to this with your opinions!

(O|||||O)

Thanks for the Windows binary, ClemsonJeep. I actually would like to play with the source code, and I was glad to see that MS C++ project files were supplied in the source code zip file. My problem was that it required the glibc and fftw .lib files which had to be compiled as well and of course did not come with project files. Any chance you or anyone else could post those .lib files? Thanks again.

Well, what is it you are trying to compile? Because you are going to need the header files for glib and fftw as well as the lib files. This is why we recommend you going to their site and downloading the source. If you can't do this for some reason let me know and I'll try to get the correct files together.

(O|||||O)

I did go to the site and download the source for glibc and fftw, and I now have the header files, but I can't compile the windows .lib files because there is no project file for them. I think that all I need now is the fftw.lib and glibc.lib files. Then I will be able toy with and compile the songprint demo.

Well, well, I tried out the songprint finally, and yes, it does work! fantastic! It did recognize the Beck song. It didn't recognize "Across the Universe" and "Would?" (by Alice In Chains). So I submitted the info to the database.

However, because it recognized the Beck song, it leads me to believe that the PMA software just does not work in this aspect of the program. It just didn't recognize any of the songs at all.

All you need is to make a simple, minimalist piece of software that finds the information and fills it out for you. Nothing more and nothing less. Yes, I would pay for this kind of software. And I tell you, I wouldn't pay for something like PMA. too cluttered, like WMP7 has become...

But great job otherwise! I was very impressed once I finally saw my song ID'd by your system! I'm looking forward to anything you guys come out with in the future!

DiGNAN

I want the same thing, and I am willing to code it, but I need these .lib files. Or instructions on how to create them. And another thing that would be nice would be an easy, automated way to submit new signature-tag pairs, without having to do it interactively via the web. I (and many others) have tons of MP3s with perfectly good tags on them that I would love to contribute to your database, but I am not willing to spend the time to do it manually. If you are worried about bad tags overwriting good tags, perhaps you only allow automatic submition for tags that don't yet exist in the database at all.

Edited by sc400 on 21/9/00 10:47 PM.

I want the same thing, and I am willing to code it, but I need these .lib files. Or instructions on how to create them. And another thing that would be nice would be an easy, automated way to submit new signature-tag pairs, without having to do it interactively via the web. I (and many others) have tons of MP3s with perfectly good tags on them that I would love to contribute to your database, but I am not willing to spend the time to do it manually. If you are worried about bad tags overwriting good tags, perhaps you only allow automatic submition for tags that don't yet exist in the database at all.

We actually have a tool created such as this. It is called the gathertool and sofar it's used internally and been released to a few people to gather info about their song collections and send it to us. It basically uses the tag information from the MP3s or whatever as basic information and submits the data to our servers. We then do a process called "alignment" to make sure that the tag data is actually correct for that fingerprint before it is actually added to our production servers.

Last I checked the person working on the gathertool was working on a public version of it, but I'll check into that and contact you. If there isn't a public version out, I'm sure I could get a private copy to you if you want to do that. :)

(O|||||O)

heck, I have 3000 files in my collection. chances are you don't have info for at least 2000 of them. probably 2500.

I've got pretty obscure music (so heck, maybe nobody else will ever access the stuff I provide ) but what the heck, can't hurt right?

DiGNAN

any news on this? I also have a ton of music that I would like to rename/reorganize. Since CDDB doesn't have year my search by year won't work unless I correct the tag.

Also track number would be f****ng great! I have several Live albums that got processed without track number and It's just not the same!

John

Track number would be hard since there is no track number tag.

DiGNAN

Ermm...i definately could be wrong on this, but in MPTagger, there is a field for track number in the ID3v2 tags....


|| loren.cox
|| 080000446

There is track number availabe in both V1 and V2 tags. In V2, it's a specific field. In V1, you can optionally use the last byte of the "Comment" field for the track number (in binary, not in ascii, that's why you sometimes see V1 comments with a little "block" character at the end).

___________
Tony Fabris

Track number would be hard since there is no track number tag.

It would also be difficult to maintain this kind of information because any given song can be on multiple albums, all with different track numbers. That is to say, the same exact song with the same fingerprint can occur on multiple albums with different track numbers. So it is a relative impossibility to keep track of this kind of information.

As an UPDATE:

I'm working on getting the GatherTool ready for public release. It will contain Win32 console and Win32 GUI interfaces, along with a Linux console interface. AFAIK it will be closed-source right now, but be open later on when we can clean up the code a bit. It's only just a frontend to the Songprint library anyway.

Ya'll will be the first to know about it when I am through with my work. :)

(O|||||O)

Awesome!

You know, I think I'd like PMA better too if it didn't look so cluttered. Heck, I think it would work great if it were as streamlined as the songprint program you gave us! And I'd pay for it! (if it work)

DiGNAN

You know, I think I'd like PMA better too if it didn't look so cluttered. Heck, I think it would work great if it were as streamlined as the songprint program you gave us! And I'd pay for it! (if it work)

Yeah, I agree. Unfortunately I wasn't around for the PMA design, otherwise I would have started yelling about feature bloat and the fact that it is so very cluttered. I like a nice simple interface personally. I believe the PMA will be redesigned from ground up, and since I'll be on the design team, this will be one of our prime things to keep in mind.

The GatherTool is still in progress, BTW.. I'll keep you posted.

(O|||||O)

Cool beans. Way to go!

DiGNAN

Yes, track numbers will change since a song may appear on more than one album, but the album name also changes and we keep track of that. It is no different for track numbers. If a tag is representing album A, it will contain album A's album name and the track number the the song is at ON THAT ALBUM. Then you have to ask, well what is the correct album to use for a song that appears on multiple albums? The first one released, I guess. The space is there for a track number, to not attempt to include it (as well as the year) would be foolish. If someone decides they don't like the track number to be in there tags, I'm sure it could easily be batch removed with one of the many tag editors.

Great program... :)

Has any one managed to get it cross-compiled to run on the empeg itself...? I had a quick go and got a binary that ran but crashed when actually identifying a track... I won't post the error just yet as I will have another go at cross compiling it later once I have set up the environment a little better, but I was wondering if anyone else has had any more luck...

I was intending on running it over various badly tagged tracks on the empeg and having a script modify the *1 fids directly and then rebuilding the database... Can anyone see any other problems with this idea (assuming I can get the songprint cross-compiled)...?

Cheers

Kim

Has any one managed to get it cross-compiled to run on the empeg itself...? I had a quick go and got a binary that ran but crashed when actually identifying a track... I won't post the error just yet as I will have another go at cross compiling it later once I have set up the environment a little better, but I was wondering if anyone else has had any more luck...

You won't be able to get it to run on the actual empeg box. The algorithm that is used to calculate the fingerprint for the song uses floating point mathematics and the ARM processor has no floating point unit. :)

(O|||||O)

ps: for those of you waiting on more about the library, our company is going through some interesting times. Downsizing to say the least. Work has been put on "pause" on this project to work on more important projects.

In reply to:

---

You won't be able to get it to run on the actual empeg box. The algorithm that is used to calculate the fingerprint for the song uses floating point mathematics and the ARM processor has no floating point unit. :)

---

It's true that the arm has no floating point unit, but the Linux kernel has a natty little FP emulation library. So you can run code containing FP, it's just rather slow... =)

Jazz
(List 112, Mk2 12 gig #40. Mk1 4 gig #30. Mk3 1.6 16v)

Yes, I had thought this would be the problem... my next move (which I still haven't had the chance to try yet) is to compile it with libfloat (ftp://ftp.netwinder.org/users/s/scottb/libfloat/) and see if I can get it working with that... I don't really care how slow it is if it works... I can always leave it running overnight...

Do you think it will work with this...? I will give it a go as soon as I get a chance and work life butts out for a while...

Cheers

Kim

Just something I'd like to mention to you all. I emailed the guy who made Tag Studio to tell him about the Songprint stuff. He's going to look into it, but thought it was very cool and might add it to his program eventually.

Pretty cool stuff.

DiGNAN

Picking up on the PMA side of things, just how does it decide which tracks to recommed? Has anybody else used this and do they have any comments on its accuracy/taste :-)

Gavin

Are you still working on this?

Sean

Not actively. The service is still running and will probably do so for the time being... but my company went the way of the dot com do do, and a majority of employees (including the entire dev team) have been laid off.

Any updates to it are purely done for the fun of it now. :)

(O|||||O)

Im sorry to hear that you got laid off. Who has the database, and will it keep running? Im trying to get the author of Dr. Tag to include the ability to ID songs using songprint individually. Do you think something like that would be possible?

Sean

Yeah, I was talking to the Tag Studio guy about it too. he said he was interested but probably will not be as much, with curent events being what they are.

DiGNAN
Ob-La-Di, Ob-La-Da, etc.

Well, keep in mind, the technology is VERY stable, and we still are looking for partnerships/licensing options. I still am working on the project, helping a few of the old employees maintain the servers and so on. It just isn't as high of a priority as it was 6 months ago.

We have more than quintupled the size of our database in the past few months, and we still continue to have new fingerprints added daily.

(O|||||O)

Err, ummm, how to put it tactfully... Does your company, should it reach the, ehm, end of its lifespan, plan to opensource the technology and put collected database in public domain, or do you, hopefully, consider such discussions premature?

Anyway, good luck!



Dragi "Bonzi" Raos
Zagreb, Croatia
Q#5196, MkII#80000376, 18GB green

I would appreciate it if you could email me a compliled win32 exe. I still havent figured out how to compile the thing. I guess im not getting the environment settings right in visual c++.

Sean

They are very premature. We still have several companies interested in acquiring the technology and/or licensing it... but these kind of contracts take forever to go through.

If anything along those lines happens, I'll let ya'll know.

(O|||||O)

I put together an ugly little shell script that will call songprint and add the tag to the MP3. I tested it in Cygwin, but it should work in any unix environment. It looks to see if you have an id3v1 tag and if so, preserves the year (because SongPrint doesn't provide that info). If SongPrint returns multiple albums, it uses the one that was already there, if it was in the list SongPrint returned, otherwise it uses the first one returned. That may have been confusing, sorry. It doesn't overwrite the old tag, just adds another one after it. It is exteremely slow due to the fact that accessing the SongPrint database is extremely slow (but low CPU utilization). It took me about 8 hours to id 2858 songs, of which 1214 were recognised. Thats about 42%, a little lower than I was expecting. And my music is not all that weird. ClemsonJeep said in September of last year that there were 60,000 songs in the database and growing daily, so maybe others will have a higher percentage than I did. You can periodically run tagall to tag any new mp3s you have aquired and/or attempt to tag any mp3s that were previously unrecognized. It will not attempt to re-tag mp3s already tagged by this tool. This will make subsiquent calls to 'tagall' faster, depending on how many songs it recognized the first time around. It uses the comment field to tell if it was already succesful for a given mp3.

And here is 'tagall'

Cool. It's wonderful to see that people still find our tool useful... We had a new version of the library in development that was 200% better than the one that is in production now, but alas, will probably never see the light of day.

You don't happen to be (or know any) a VC with deep pockets? ;-)

(O|||||O)

cool scripts. Too bad the only linux platform I have is the empeg. I have located another program, freeamp, that uses technology developed by relatable to id the songs.

Sean

Whats a VC? Negative on the deep pockets.

Terminator, thanks for the tip, I'll check out freeamp.

VC=Venture Capital, companies whose sole business is investing in other companies.

___________
Tony Fabris

Venture Capitalist - they are non existant these days. The library freeamp is using is pretty big. You can get more info at http://www.musicbrainz.org
I just checked their server stats - they have over 250,000 songs in their library,
Sean

Edited by Terminator on 26/03/01 06:58 PM.

So, ClemsonJeep, is there any songs currently being added to the database? Why not do The Right Thing and put the database and server-side software in the public domain and let this cool technology flourish? You guys had your chance to commerialize it. It didn't work out. If you continue to keep it secret, some other company will just write there own version of it. It sounds like the FreeAmp people already have. But I don't know anything about FreeAmp, only what Terminator was saying.

Anyone who loves Linux but mostly runs Windoze for multiple reasons (like me), needs to check out Cygwin. Its basically a bash prompt for Windows. Complete with all the great shell tools normally availible at a unix prompt that only Bill and God know why were never included in any version of Windows.

Okay, I just looked at the FreeAmp, Relatable and MusicBrainz websites. The quiestion I have that I wasn't able find the answer for on any of the websites is, Is there a tool that can leverage this large database to fix your ID3 tags for you? If not, is there a tool that will just output the correct information, like SongPrint?

Run free amp, hit the my music button and I think free amp will search the whole computer for mp3s. When its done, go to the relatable tab and create a profile. Then hit start signituring. It will look up everything, and I THINK it tags everything that doesnt have an id3 tag with a ID3v1 tag. If you go to the musicbrainz page, you can download the source for the client.

Sean

Edited by Terminator on 26/03/01 07:25 PM.

Cygwin is really cool! It looks and works just like linux. Maybe I will be able to try out your shell scripts after all.

Sean

Yeah, Relatable was "competition" for our company when we were around last year. I'm not sure if they are profitable or not, but at least they are still around. :)

(O|||||O)

Well, we are still in the process of talking to a couple companies about selling the IP of our company, so the technology can still evolve. If those bomb out, I'll be the first one to post the code for everything to sourceforge. :)

And yes, the database still evolves. It was designed to work with little-to-no human interaction. If you tag a song that isn't in our database, it attempts to figure out information from ID3 and the filename itself, justifies it all, then sends the data to our server which goes through a process we call "alignment" which makes sure we get little to no duplicate or bad data. Thus the database is still growing.

(O|||||O)