mp3tofid 5.00

Posted by: pim

mp3tofid 5.00 - 01/03/2011 20:20

Previous version of mp3tofid was 4.03, released in 2007, not announced on this forum.

Compared to 4.03, version 5.00 has three changes:

  • Makefile adapted to current Linux distros
  • fixed crashing on scanning flac files with embedded artwork
  • Support for multiple music directories


Support for multiple music directories now allows you to separate your mp3tofid playlist symlinks from your actual music directory which might otherwise confuse other applications. Support for multiple music directories slightly changes the syntax and introduces two new options.

-f | --fids-directory directory

The fids directory used to be the second non-option argument. Now it is a mandatory option argument

--main-mp3-dir-index number

This option is useful only when you have multiple music directories on different filesystems. mp3tofid uses device and inode numbers to track tunes and works best when it knows which of your music directories contains the bulk of your music. By default it assumes the first one is the most important, but if it would the second one, you would use --main-mp3-dir-index 2

All non-option arguments are now assumed to be music directories and will be scanned for tunes or playlist links. The contents of each music directory will be concatenated in the root empeg playlist. Identically named subdirectories will not be merged, they will just appear twice.

As always, mp3tofid can be found here.

Pim
Posted by: julf

Re: mp3tofid 5.00 - 02/03/2011 14:54

Great! many thanks!
Posted by: wfaulk

Re: mp3tofid 5.00 - 02/03/2011 16:14

I really want to use mp3tofid, but I have a greater desire for my list of artists to be in correct alphabetical order and albums to be in release order, which currently requires that I sort semi-manually. Right now, what I do is enter the artist name for a playlist in the artist field, corrected for alphabetization ("Beatles, The", e.g.) and the release year in the appropriate field for album playlists, sort on the appropriate fields in emplode, and sync that way.

I assume there's still no way for mp3tofid to do that, correct?

I've been meaning to modify mp3tofid to do that for ages now, and I just haven't gotten around to it.
Posted by: pim

Re: mp3tofid 5.00 - 02/03/2011 21:37

Well. there's the -I|--intelligent-sort option that will make "The Beatles" sort as "Beatles". But sorting is always alphabetic, based on the directory or file name. Mp3tofid makes no assumption on the meaning of the directory depth. One level may be artists and the next level albums, but for mp3tofid these are just directories.

You could create a tree of symlinks like
/linksdir/Beatles, The/1966 Revolver -> /realdir/The Beatles/Revolver
and scan both directories.

Your music would appear twice, once in the order you like, and again in the order you dislike.

Pim
Posted by: tonyc

Re: mp3tofid 5.00 - 08/03/2011 01:59

Thanks much for the update, Pim. I don't really have a need for any of the new features, but it's good to see mp3tofid still getting some love.

I started hacking it a while back to add m3u playlist support, but then I decided that writing a script to translate an m3u playlist into a bunch of symlinks was easier.
Posted by: wfaulk

Re: mp3tofid 5.00 - 08/03/2011 20:45

Originally Posted By: pim
sorting is always alphabetic, based on the directory or file name

I'm not sure if you're talking about mp3tofid or the empeg itself. If the former, that's what I meant by modifying it. If the latter, I'm talking about this empeg feature:
Originally Posted By: FAQ
If you wish for the player's menu screen to show the playlists in position-order instead of alphabetically, there is an option in the "Configure Player" screen in emplode, on the Advanced tab. Turn off the "Sort playlist menu by title" option

I've had my empeg sorting by playlist and not alphabetically pretty much since day one.
Posted by: pim

Re: mp3tofid 5.00 - 22/03/2011 20:09

Originally Posted By: wfaulk
I'm not sure if you're talking about mp3tofid or the empeg itself.
I'm talking about the way mp3fid sorts directory contents. I'm pretty sure there's nothing in the player software that could alter the sort order in runtime.
Quote:
I've had my empeg sorting by playlist and not alphabetically pretty much since day one.
I don't see how you could modify mp3tofid to do that. mp3tofid was designed to run from the command line, without any user interface.
Posted by: Daria

Re: mp3tofid 5.00 - 21/06/2011 02:25

Originally Posted By: pim
Originally Posted By: wfaulk
I'm not sure if you're talking about mp3tofid or the empeg itself.
I'm talking about the way mp3fid sorts directory contents. I'm pretty sure there's nothing in the player software that could alter the sort order in runtime.
Quote:
I've had my empeg sorting by playlist and not alphabetically pretty much since day one.
I don't see how you could modify mp3tofid to do that. mp3tofid was designed to run from the command line, without any user interface.


I assume that's a dynamic data partition setting, so mp3tofid would be useless.
Posted by: sirkingchase

Re: mp3tofid 5.00 - 02/08/2011 01:15

Originally Posted By: tonyc

I started hacking it a while back to add m3u playlist support, but then I decided that writing a script to translate an m3u playlist into a bunch of symlinks was easier.


Would you mind sharing the script?

Also does mp3tofid 5.0 have m3u support? In the -l argument it has --ignore-m3u as an attribute. I can not get it to add m3u playlists, it seems to ignore them always.
Posted by: pim

Re: mp3tofid 5.00 - 15/12/2011 15:17

Here's the snippet of code that handles .m3u files.

Code:
/* m3u playlists */
else if (!progopts.ignorem3u && endswith(childpath, ".m3u"))
{
        /* TODO */
}


Handling .m3u files was intended from the very start but never got any further than the option to ignore them.

Pim
Posted by: Daria

Re: mp3tofid 5.00 - 15/12/2011 16:40

semi-related, given forked-daapd and vlc are both GPL, their itl parsers could be stolen. if I get time, I may look harder at that as well as their m3u parsers. but at the moment, time? hah.
Posted by: pim

Re: mp3tofid 5.00 - 23/01/2012 20:41

I've been thinking about stealing the smart playlist code from xbmc.

Pim
Posted by: julf

Re: mp3tofid 5.00 - 29/01/2013 07:59

Originally Posted By: pim
Support for multiple music directories now allows you to separate your mp3tofid playlist symlinks from your actual music directory which might otherwise confuse other applications. Support for multiple music directories slightly changes the syntax and introduces two new options.

-f | --fids-directory directory

The fids directory used to be the second non-option argument. Now it is a mandatory option argument


I finally got around to upgrading my automatic sync scripts from using 4.03 to 5.0, but I can't quite figure it out...

My mp3's live in /home/music/empeg/mp3s, and I want to generate the files to be rsynced to the empeg into /home/music/empegfids. When I run

mp3tofid --fids-directory /home/music/empegfids /home/music/empeg/mp3s

I get the fid files in /home/music/empegfids/drive0 and /home/music/empegfids/drive1, but the file fids are symbolic links referring to /home/music/empeg/mp3s. I understand playlists consist of symbolic links, but shouldn't the original files end up in the fids directory at least once?
Posted by: Roger

Re: mp3tofid 5.00 - 29/01/2013 09:25

Originally Posted By: julf
the file fids are symbolic links referring to /home/music/empeg/mp3s. I understand playlists consist of symbolic links, but shouldn't the original files end up in the fids directory at least once?


Playlists are not symbolic links, in the traditional sense. They are files containing a list of the FIDs contained therein. They don't appear as symlinks in the filesystem.

So, assuming that you're seeing concrete FIDs for the playlist files, and symlinks for the MP3 files, this seems like sensible behaviour. You merely need to ensure that rsync resolves the symlinks before copying them to the empeg. That way you'll get concrete music files and concrete playlist files.
Posted by: julf

Re: mp3tofid 5.00 - 29/01/2013 10:10

Originally Posted By: Roger
Playlists are not symbolic links, in the traditional sense. They are files containing a list of the FIDs contained therein. They don't appear as symlinks in the filesystem.


Ah, yes, it's all slowly coming back to me now - clearly Alzheimer's is slowly creeping in.

If I remember correctly, mp3tofid takes directories with symbolic links and massages them into playlists.

Quote:
So, assuming that you're seeing concrete FIDs for the playlist files, and symlinks for the MP3 files, this seems like sensible behaviour. You merely need to ensure that rsync resolves the symlinks before copying them to the empeg. That way you'll get concrete music files and concrete playlist files.


Right - the *1 files are "real", and the *0 files are symlinks. I am a bit concerned about the "loadfrom" fields in the *1 files - are they used for anything? They point to file paths in my source directory that is not replicated on the empeg.
Posted by: Roger

Re: mp3tofid 5.00 - 29/01/2013 15:07

Originally Posted By: julf
Right - the *1 files are "real", and the *0 files are symlinks.


I'd also expect the *0 files for playlists to be "real" as well.

Originally Posted By: julf
I am a bit concerned about the "loadfrom" fields in the *1 files - are they used for anything? They point to file paths in my source directory that is not replicated on the empeg.


They're not used for anything on the empeg -- they're ignored (although they do take up RAM); maybe mp3tofid uses them for tracking missing files?
Posted by: julf

Re: mp3tofid 5.00 - 30/01/2013 06:19

Originally Posted By: Roger
I'd also expect the *0 files for playlists to be "real" as well.

Hmm... Out of 20681 *0 files, 18702 are links and 1680 actual data.
I only have one user-defined playlist at the moment, with 107 tracks in it.

Originally Posted By: Roger
Originally Posted By: julf
I am a bit concerned about the "loadfrom" fields in the *1 files - are they used for anything? They point to file paths in my source directory that is not replicated on the empeg.


They're not used for anything on the empeg -- they're ignored (although they do take up RAM); maybe mp3tofid uses them for tracking missing files?


So removing them would be a good thing in order to preserve precious RAM. Do we know what fields are actually required by the player? Mp3tofid seems to put in at least these:

bitrate=vs205
codec=mp3
ctime=1359445255
duration=170344
length=4380748
loadfrom=/home/music/empeg/mp3s/all/Ewan MacColl/The Real MacColl/04 Haughs O'cromdale.mp3
offset=1034
rid=b5b66ffe5ff299144ab2b1a5e34d6a78
samplerate=44100
title=04 Haughs O'cromdale
trailer=104
type=tune
Posted by: pim

Re: mp3tofid 5.00 - 25/02/2013 20:28

mp3tofid tries to create tag files (*1) as complete as possible. for backwards compatibility, and adds some extra tags for tracking, such as "loadfrom".

Bu only the most essential tags are compiled into the database.

As far as I can tell, the player reads tag files only during rebuilding the database. During playing, it reads the database only.

I believe an mp3tofid generated database is smaller than a player generated database, and thus should consume less RAM rather than more.

Pim
Posted by: julf

Re: mp3tofid 5.00 - 26/02/2013 09:38

Originally Posted By: pim
I believe an mp3tofid generated database is smaller than a player generated database, and thus should consume less RAM rather than more.


Excellent! Appreciated!
Posted by: elperepat

Re: mp3tofid 5.00 - 01/04/2014 17:47

Hello!

I just gave my empeg a second life with a 128GB kingspec SSD. I now want to try mp3tofid.


First, I couldn't download from the official website because I get a
Code:
Forbidden

You don't have permission to access /~pim/empeg/mp3tofid/ on this server.


I did find mp3tofid in a git repo. I was able to compile it after fixing a few defines in bswap.h. Lines 11, 14 and 18 were trowing errors because there was a missing space after #define.

Anyway, I got it to compile but now I want to download rsync for the empeg and can't find it anywhere. The help file points to pim's own server which is not working anymore. Anyone has rsync for empeg?

Thanks
Posted by: mlord

Re: mp3tofid 5.00 - 02/04/2014 00:47

http://rtr.ca/rsync.bin

(from the mp3tofid package I had archived here)
Posted by: Daria

Re: mp3tofid 5.00 - 03/04/2014 14:28

if you send a pull request with your fixes, i'll merge it
Posted by: elperepat

Re: mp3tofid 5.00 - 04/04/2014 12:37

Thanks Mark, I tried it, but I got the error:
Code:
./rsync: error in loading shared libraries: libresolv.so.2: cannot open shared object file: No such file or directory

when I try to run it.

For the record (and others with failing memory like mine):

The linux version on the empeg is debian potato. If you get such error, it is because the application is missing a library. Make a search in google for "name_of_the_missing_library" (in my case, libresolv.so.2) and add "potato deb" to your search. This should tell you which package contains the missing file. In my case, it's libc6_2.1.3-20.deb. Now, get the package: Go to potato archive and find the package. In my case, it was in "Base" category. Extract the package using ar:
Code:
ar vx libc6_2.1.3-20.deb
then extract the resulting data.tar.gz using
Code:
tar -xzvf data.tar.gz
. In the extracted files, you should find the library. For me, the file "/lib/libresolv-2.1.3.so" was in /lib. You can notice there's a symlink (/lib/libresolv.so.2) pointing to that library. After transferring the library to the empeg via ftp, you have to recreate the symlink there too with ln -s libresolv-2.1.3.so libresolv.so.2.

There you have successfully installed the missing library on the empeg.
Posted by: mlord

Re: mp3tofid 5.00 - 04/04/2014 18:48

Looks like you need more than just the rsync component. Here is the entire mp3tofid package that I have here, which includes rsync and libresolv.

Cheers
Posted by: elperepat

Re: mp3tofid 5.00 - 05/04/2014 13:32

Great, thanks!

One more question. I'm trying to create a mp3tofid.sh script that would run when I dock the empeg at home. I created the script on my computer with gedit (on a mint linux box) and transfered it to the empeg via ftp. I changed the executable flag (chmod +x mp3tofid.sh) but when I try to execute it (with /usr/bin/mp3tofid.sh or ./mp3tofid.sh) I get
Code:
bash: ./mp3tofid.sh: No such file or directory
. Any idea?

Here is the script I'm trying to use.


Also, I tried to run nano directly on the empeg, but I can't run it: I get:
Code:
BUG IN DYNAMIC LINKER ld.so: dynamic-link.h: 57: elf_get_dynamic_info: Assertion `! "bad dynamic tag"' failed!


Any idea what I do wrong?

Again, thank you for your time. I really appreciate it.
Posted by: mlord

Re: mp3tofid 5.00 - 05/04/2014 15:22

I don't know what's wrong with the shell script, but you can telnet to the empeg if need be for easier test / debug of stuff like that.

To use telnet, first ensure you have the current Hijack kernel installed, and then these lines in /empeg/var/config.ini:
Code:
[hijack]
ktelnetd_port=23


Edit: Oh, there is no /usr/bin/ on my empeg.
Posted by: mlord

Re: mp3tofid 5.00 - 05/04/2014 15:27

You should probably also insert the full pathname of any external commands to be executed from the shell script, such as the full path of "rsync" (eg. "/rsync") rather than merely "rsync".
Posted by: mlord

Re: mp3tofid 5.00 - 05/04/2014 15:31

Btw.. where in Quebec are you located? I'm in Ottawa (Nepean, actually).
Posted by: elperepat

Re: mp3tofid 5.00 - 16/04/2014 20:21

I'm from Alma, near Lac-St-Jean, 200km North of Quebec.

I'm now able to run my mp3tofid.sh script. Turns out filezilla was changing my CR to CR/LF in all the files sent to the empeg via ftp.


The only thing left to sort out is why rsync fails when the script is ran via ;@EXEC_ONCE but is succesful via shell. The error I get is:
Code:
rsync: failed to connect to 10.215.0.105: Network is unreachable
.

I read that the network is slow/sometimes won't get up properly. I sleep 5 and mess with ifconfig eth0 up in my script, but it doesn't seem to help...

Any idea?
Posted by: mlord

Re: mp3tofid 5.00 - 17/04/2014 10:44

You could put the EXEC_ONCE command onto the Hijack menu, and run it from there after the network is fully up.
Code:
## In the config.ini file:

[hijack]
;@MENUEXEC RunRsync /bin/rsync ... &>/dev/null

After adding that to config.ini on the empeg, you should then see a "RunRsync" menu entry on the Hijack menu (long press of knob).

Or have the script background itself and poll for networking before starting the rsync.
Posted by: Daria

Re: mp3tofid 5.00 - 22/04/2014 17:56

Originally Posted By: dbrashear
if you send a pull request with your fixes, i'll merge it


Thanks, btw. Merged!