mp3tofid 3.00 - now for Windows too

Posted by: pim

mp3tofid 3.00 - now for Windows too - 13/10/2002 16:44

I have just released mp3tofid version 3.00 , to be found here as usual.

There are three major new features
  • support for (and defaulting to) the new player directory structure, ie. fids named /driveX/fids/_YYYYY/ZZZ. Use "-o" to revert to the old structure.
  • intermediate database for storing inode to fid relationship, resulting in a much smaller player database
  • a Windows port (actually a cygwin port)


Until now, it seemed mp3tofid only worked for me. Previous versions seemed to produce much too large databases for the player to handle. The database size would depend on how inode numbers were distributed on the filesystem containing the source MP3's. In this version, this should no longer be of influence.

The intermediate database is stored as .../drive0/var/mp3tofid. Don't lose this database. A new one will be created if it is lost, but all tunes will get new fid numbers resulting in a total re-upload if you run rsync again.

As a result of its location, the mp3tofid database will be copied to the player. It serves no purpose on the player, but it might be handy to have it stored as a backup there.

Pim
Posted by: dmuench

Re: mp3tofid 3.00 - now for Windows too - 13/10/2002 19:11

So far, it looks great. My 600 meg database has dropped to 229k. I haven't synced it all to my empeg yet, I'll try that tomorrow. Thanks again.
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 14/10/2002 21:18

i'm getting this error when using rsync
skipping non-regular file "drive0/fids/_00000/8f0"

any help?
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 15/10/2002 02:46

It's probably a dangling symlink. Have you checked with ls -l?

You can also use symlinks (written by Mark Lord) to check for bad symlinks.

Another possibility is that the rsync server does not have permission to read the file the symlink is pointing to.
The rsync server usually runs as nobody on Unix or SYSTEM on Windows, so the target should be world-readable.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 15/10/2002 08:04

its because i ran mp3tofid w/ drive letters (e:/mp3 e:/empegfids)instead of using /cygdrive/e/. its been syncing since last night.

another question.... i'm looking at the fids directory.... theres a blank files that aren't connected to files everywhere. is that by design?


Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 15/10/2002 09:37

i ran mp3tofid w/ drive letters (e:/mp3 e:/empegfids)instead of using /cygdrive/e/.

Ok, now I understand you're using cygwin. I was not aware windows-type paths worked at all in cygwin ...

theres a blank files that aren't connected to files everywhere

You mean zero-sized files? There should not be any.

You'll see three types of files:
- tag files, ending with "1". These are small (unix) text files.
- playlists, ending with zero. These are small binary files.
- symlinks, ending with zero. They point to you mp3's.

symlinks can only be seen as such by cygwin apps. Native windows apps see them as shortcuts, but they are twisted. Dont try to edit the shortcut properties. True shortcuts do not appear as symlinks on cygwin, but as plain files with a .lnk extension.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 15/10/2002 18:12

ok, i was wondering what the tag files were. now i know. but this poses another problem. is there anyway for mp3tofid to re-create the tag files if i modify them on the hdd? this was the whole point of me getting this to work (aside from the knowledge =).
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 16/10/2002 15:01

ok, i was thinking this over today. if mp3tofid can check the modified date versus the last time mp3tofid was run, if its a match, delete the tag file and regenerate.

also, can there be a switch to check for id3v2 tags only? i see that mp3tofid is scanning my huge compilation cds wholy, and i was wondering why?
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 16/10/2002 16:01

if mp3tofid can check the modified date versus the last time mp3tofid was run, if its a match, delete the tag file and regenerate

This is basically the algorithm:
1. read all existing fids into memory
2. for each fid, do numerous tests to see whether it still matches the source tune, otherwise delete from memory
3. scan the source mp3 tree, skipping all tunes that have valid fids in memory
4. completely empty the fids directories
5. regenerate all fids and databases from memory
6. set modification date of tune fids and tune tag fids to their source file's modification date (this speeds up rsync a lot)

also, can there be a switch to check for id3v2 tags only?

It is libid3tag's behaviour to look for id3v2 tags and drop back to id3v1 if no id3v2 tags at all are found.

i see that mp3tofid is scanning my huge compilation cds wholy, and i was wondering why?

I could not find a reliable way to calculate duration and bitrate from an mp3 using libmad, other than by decoding each frame header, essentially reading the whole file. It's only decoding the headers of the frame, not the frame itself, so this is an i/o bound operation. On my system, it took 110 minutes to scan 72 GB of MP3's. (It took 30 hours to upload using rsync in one single run).

Still, it only has to be done once; on next runs, only new or changed tunes get scanned, and only new or changed fids are uploaded by rsync.

If you have a huge amount of tunes on your player, adding a single album using mp3tofid+rsync might take less time than it takes emplode+player just to rebuild the database.

But speed was not really the most important issue for me to write mp3tofid. I just did not want to keep track of all changes and apply these changes to my player manually.

I keep changing things to my mp3 collection, like
- changing id3 tags
- re-ripping badly ripped cd's
- re-encoding using better encoders or encoding options
- renaming tunes and directories
- deleting tunes and directories.
- adding tunes, of course
Whatever I do, mp3tofid and rsync will see the changes and apply them to the player automatically.

If you are going to upload a huge amount of tunes a first time using rsync, you may consider to mke2fs your fid filesystems before doing so (don't forget to save your config.ini file). Uploading to a freshly created, empty fs is much faster than overwriting old stuff. You will also get rid ot the huge fids directories that stay huge otherwise, even if you delete their contents.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 16/10/2002 23:49

ok, let me clarify my predicament. i noticed yesterday that one of my tags aren't capitalized right (Dj instead of DJ). i guess the tests doesn't check for capitalization changes.

but otherwise, i'm enjoying the mp3tofid/rsync otherwise. can't wait til ignore as child and "other" file types gets implemented. thanks for developing it.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 17/10/2002 08:00

i noticed yesterday that one of my tags aren't capitalized right (Dj instead of DJ). i guess the tests doesn't check for capitalization changes.

Well, the windows filesystem is case insensitive, even under cygwin. So if you only change capitalization, mp3tofid will still be able to open the file, and conclude it did not change. But if you edit the id3 tag inside the file as well, the modification date ought to change, and mp3tofid would notice. Perhaps your id3 tag editor resets the modification date after its change?

can't wait til ignore as child

I know what you mean. The other day, my player randomly played a 0db sine wave off my test cd ...

and "other" file types gets implemented

Which other types in particular? WMA is out, because of its closed nature. OGG and FLAC should be possible, I guess.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 17/10/2002 09:00

let me try this again. yeah, your right. mp3tagstudio doesnt modify the date. it works, just user error.

as for unsupported files, i need to sync up my wav files so i can do my test tones also. i just dont believe in mp3 test tones..

and how about this for a feature.... the ability to put a jemplode only soup file in a directory (probably in var), and let mp3tofid interpret it and convert to a playlist.
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 17/10/2002 09:34

argh. check out what i did. i took off the "keep original date/time" toggle, and applied a tag renaming scheme to my whole collection. mp3tofid catches this, and rescans. there are many mp3s that stay the same, but just had the modified date changed.

when i try to rsync, it starts to do the whole collection again. i guess it only checks if theres a change in the modified date. i know this is not mp3tofid's fault, but isnt there a switch to make rsync make sure that nothing was changed?
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 17/10/2002 10:07

isnt there a switch to make rsync make sure that nothing was changed?

You shouldn't need to worry; rsync will detect the files did not change; both ends are exchanging checksums. But both ends will need to read the whole file, so it will be slower. As a result, the fids on the player will get the modification time of the source fids, without actually transferring their contents.

Pim
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 17/10/2002 10:14

as for unsupported files, i need to sync up my wav files

As wav files do not contain tags, I would have to add code that takes the title from the filename. This would be useful for mp3 too, as this is what emplode does now if a title tag is not found. I'll put that on my todo list.

the ability to put a jemplode only soup file in a directory

Uhm, now's the time to shamefully admit I have never tried to use jempeg ...

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 18/10/2002 16:40

question.. i'm going to start using a preinit script to run rsync on startup, so i need to know if its safe to run it while the player app is still up? and if it eventually gives up if it doesnt find a network connection (i.e. in the car).

is there any sure way to know that i even have a network connection? somewhere in /proc maybe? so i can make the script conditional.

also, even though the database and playlist is functional, everytime i run emplode after rsync is done, it has to add/change something (litte red cross). i was thinking of doing a file compare, so you can implement what happens to the database/playlist generation code. or you can do it yourself.
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 19/10/2002 10:44

ok, another feature request. i have different size hard drives... and i currently use the -2 50 switch. i need a celing on the smaller drive so i dont go over capacity. but until the time comes, i want to split up the mp3s evenly.
Posted by: dmuench

Re: mp3tofid 3.00 - now for Windows too - 19/10/2002 13:54

I just finished rsyncing the new fids and all to my empeg, and it works great now. No problems at all. I love the program, thanks so much for writing it.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 20/10/2002 11:57

is there any sure way to know that i even have a network connection?

If you get your IP from a DHCP server, you can see with ifconfig that you did not get an IP number.Get ifconfig from a Debian image.

somewhere in /proc maybe? so i can make the script conditional.

you can see in /proc/empeg_power whether you're on AC power or on car power.

even though the database and playlist is functional, everytime i run emplode after rsync is done, it has to add/change something

I noticed that too. I think it's because of some timestamp, not because of something in the database. I tested this in an earlier mp3tofid version; the databases created by emplode and by mp3tofid were 100% identical. With current versions they are not, because mp3tofid is leaving some tags out of the database that do not have to be there. This makes the database, and thus the player's memory usage, significantly smaller.

Pim
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 20/10/2002 12:10

I have different size hard drives... and i currently use the -2 50 switch. i need a celing on the smaller drive so i dont go over capacity

In order to prevent unnecessary upload, it has to be predictable for any single file on which drive it will end. So I can't dynamically put the the file on the drive with the most diskspace like emplode does.

If it turns out one drive is getting full, then just change the percentage (ie "-2 51" or "-2 49"). You can check the diskspace on your host pc using du -L -s .../empegfids/drive?

I guess this makes it hard to fill both drives to the last byte, but you can't have everything in life ...

Pim
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 20/10/2002 17:07

i'm going to start using a preinit script to run rsync on startup, so i need to know if its safe to run it while the player app is still up

I guess it would be safe to run rsync as a daemon while the player app is running, and drives are mounted read-only.

I would guess it is not safe to write to fids and databases while the player app reads them.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 21/10/2002 00:01

i have found that running rsync while the player is running is incredibly slow even while the player's on standby. i'm guessing the player app is running at a higher cpu priority and all memory is filled with song data.

which leads me to ask... is there anyway aside from hitting q in terminal mode to kill the player? i've tried to just use kill -9 pid but it always restarts.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 21/10/2002 11:45

is there anyway aside from hitting q in terminal mode to kill the player?

I wouldn't know. Perhaps it is best to ask again outside the context of this thread, and get a wider audience. I'd be interested too...

Pim
Posted by: Daria

Re: mp3tofid 3.00 - now for Windows too - 21/10/2002 11:57

It's evil, but you can:
-move the binary out of the way
-kill "player"
-init has nothing to respawn
-don't forget to move it back or you will be sad later
Posted by: jaharkes

Re: mp3tofid 3.00 - now for Windows too - 21/10/2002 12:57

You could use kill -STOP <playerpid> which should freeze the player. Run rsync and then kill -9 <playerpid>, and the player will restart and reload the new databases. Although the player is a threaded process, so I'm not sure whether you can just kill -STOP the lowest player pid, or if you have to stop all of them individually.
Posted by: dmuench

Re: mp3tofid 3.00 - now for Windows too - 21/10/2002 18:15

I guess I spoke too soon.

I just added a new album and re-ran mp3tofid, and it acted as if I had changed all the files - rescanned every file, and generated a new fid for every song. I was suspicious, so I ran rsync with the dry run option, and sure enough, it wanted to delete everything on the player and start over again.

I am sure this is something to do with me using XFS on my server, but I doubt that XFS randomly reassigns inode numbers whenever it feels like it. I am wondering if I am overflowing anything with the high inode numbers (the latest songs I just added all have inodes in the 373 million range).
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 23/10/2002 03:33

Can you reproduce this, using a subset of your collection and a test fid dir?

I doubt that XFS randomly reassigns inode numbers whenever it feels like it

It's unlikely, but not impossible. This is easily detected. Just type ls -li in an mp3 dir, reboot and look again. smbfs on Linux turned out to give different results on each boot.

I am wondering if I am overflowing anything with the high inode numbers (the latest songs I just added all have inodes in the 373 million range)

There's nothing that can overflow anymore (at least not sooner than with other tools). A tune is now uniquely identified by a string built by the major and minor number of the device holding the filesystem containing the tune and the inode number of the tune. This string is used as a key to look up the fid number in a gdbm database. using -d dimMsS debug options you can follow all gdbm activity.

In theory, a 64-bit inode number (yours still look like 32-bit) could overflow the sprintf() function, but then still it would come up with a unique string.

So I can think of a number of explanations for this behaviour:
- your filesystem does not have persistent inode numbers
- you somehow lost the mp3tofid database
- you used the -n switch to force generating a new mp3tofid database
- the major/minor number of the filesystem changed caused by fdisk or lvm operations
- a weird bug

If you did lose the mp3tofid database, by accident or by using the -n switch, try restoring it from the player. This is why it is backed up there. Restore it, delete the fids on your host PC and run mp3tofid again. mp3tofid will have to scan everything again, but your fid numbers will be back to what they were.

Pim
Posted by: dmuench

Re: mp3tofid 3.00 - now for Windows too - 23/10/2002 14:59

I'll do some testing. I didn't delete the database or use -n though. And I am using EVMS, but one of it's features is persistent major/minors. I'll let you know what I find out.
Posted by: dmuench

Re: mp3tofid 3.00 - now for Windows too - 23/10/2002 15:14

A lightbulb just went off in my head. Permit me to place my foot in my mouth.

I still have to test, but I am sure it's the major/minor issue. What I said is true, one of the features of an EVMS native volume is persistent major/minors. However, the volume that my mp3s are on is not a native, but a compatibility volume (dynamic major/minors) because I haven't had time to convert it yet. And I am pretty sure I did something to cause it's minor number to shift between mp3tofid runs.

So that must be the cause. Can I suggest to add a very visible section to the README about this, and I'd mention LVM volumes and EVMS compatibility volumes specifically. EVMS native volumes won't have this problem.

Thanks..
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 24/10/2002 07:22

here's what i've discovered about the windows version. if i modify a file using winamp (or anything else for that manner), mp3tofid immediately recognizes it as a new file (i see 1 empty fids in the summary screen), and deletes the old fid. not sure why.

but the weird thing is... rsync says it only wrote < 10000 bytes, but took 10 seconds to complete after the initial scan. i dont understand whats happening in rsync i guess.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 24/10/2002 07:41

here's what i've discovered about the windows version. if i modify a file using winamp (or anything else for that manner), mp3tofid immediately recognizes it as a new file (i see 1 empty fids in the summary screen), and deletes the old fid. not sure why.

This all depends on how your id3 tag editor works. If it writes out a new file, deletes the old one and renames the new file to the old name, then you have created a new file with a new inode number. If it just opens the file in read/write mode and makes the changes inside the file, then the inode number remains the same.

i dont understand whats happening in rsync i guess

You can increase rsync's verbosity by placing the --verbose on the command line multiple times.

Pim

Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 24/10/2002 09:50

Thanks for your suggestion. I added the major/minor stuff in the lookup key as to enable support for cross-filesystem symlinks and mountpoints within your mp3 tree.

It did not occur to me that major/minor numbers could change at all until I read your report.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 09/11/2002 11:22

i've been noticing that some mp3s that i have in the collection shows as invalid format when played. i decided to erase and re-upload using emplode, and it worked fine. could a bad parse of the tag cause this?

also, have you noticed that when the rsync algorithm is used, it takes longer than just doing the whole file. I guess its just the weak processor of the empeg making the checksums thats slowing it down. If i edit an id3v2 tag, I get around 350k/sec regular, and 700k w/ the --whole-file switch.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 10/11/2002 19:54

If i edit an id3v2 tag, I get around 350k/sec regular, and 700k w/ the --whole-file switch

The problem with id3v2 tags is that they are at the beginning of the file, rather than at the end, like id3v1 tags. If you insert a id3v2 tag, or when its size increases, the mp3 frames are shifted to a different location in the file. As far as I can tell, rsync cannot detect this and compares the whole file, only to find out it has to transmit everything again, because none of the file parts are identical.

Pim
Posted by: tms13

Re: mp3tofid 3.00 - now for Windows too - 11/11/2002 11:12

In reply to:

If you insert a id3v2 tag, or when its size increases, the mp3 frames are shifted to a different location in the file. As far as I can tell, rsync cannot detect this and compares the whole file, only to find out it has to transmit everything again, because none of the file parts are identical.


If that happens, your rsync isn't working correctly. The rsync protocol is designed to find common parts of files so that it can cope with this kind of thing - otherwise it would also fail on e.g. tar files when a file is added near the beginning. (Of course, compressed tar files are a different matter...)
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 11/11/2002 12:35

so anyway, regarding the fact that some files are being considered invalid format... here are the two tag files that were generated for the same file... one by emplode and one for mp3tofid.

emplode
artist=New Energy
bitrate=fs192
codec=mp3
ctime=1037042767
duration=410540
length=9895462
offset=42492
samplerate=44100
title=Energetic Wave (Nu-Nrg Original Mix)
type=tune
year=2002

mp3tofid
type=tune
artist=New Energy
bitrate=fs192
codec=mp3
ctime=1034559218
duration=410514
frames=15715
length=9895462
loadfrom=/cygdrive/e/mp3/2002/1002/new_energy-energetic_wave_nu-nrg_original_mix.mp3
offset=1540
samplerate=44100
title=Energetic Wave (Nu-Nrg Original Mix)
year=2002

parsing error in mp3tofid?
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 11/11/2002 13:24

You're right, I should have just tried it instead of trying to understand the sources.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 11/11/2002 14:31

here's an anomoly. try to just change the datestamp of one of the mp3s. when you rsync, it still appears to transmit the whole file. nothing changed at all, except the stamp. i'm going to do a --verbose 4 times and capture the log. hopefully, you can understand whats happening.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 11/11/2002 15:42

offset=42492

That's it. mp3tofid assumes it can find the start of the first mp3 frame in the first chunk it reads.
The chunksize is 40960 (#define INPUT_BUFFER_SIZE (5*8192)) in scanmp3.c.

Your mp3 seems to have a much larger id3v2 tag than I then the ones I tested with.

The lazy workaround to this bug is to increase the buffersize and recompile.

Unfortunately I will not be able to help, as I am packing for a three week trip to Asia.

Pim
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 11/11/2002 17:39

If timestamp and/or size are different on each side, rsync has to check checksums of each side. This might be very slow if the empeg is involved, like you have seen in the case of editing id3 tags.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 12/11/2002 00:14

but the point is, emplode saw the offset at 1540. its the same track so... in theory, shouldn't all the parameters found be the same in both tag files?

edit: whoops. i'm dumb. it was mp3tofid that saw the offset at 1540. i have found that the files that were failing were because of un-standard hidden id3v2 tags, which i promptly erased. everything is fine now.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 12/11/2002 04:58

It's a bug nonetheless. I'll try to fix it when I return.

The other notable differences are seen in ctime and duration. The ctimes are inherently different, as mp3tofid uses the ctime of the source file, while emplode uses the ctime of the fid file on the player. The calculated duration is different too, but then I have yet to see two programs that come up with the same duration in milliseconds. I could be wrong, and libmad could be wrong, but emplode might as well be wrong. Then again, the player copes well with wrong duration tags, as I have seen during development, when mp3tofid durations were way off.

But beware of different offset or length. They have to match exactly.

Pim
Posted by: image

Re: mp3tofid 3.00 - now for Windows too - 12/11/2002 08:06

i've found a bug in your compile of rsync. it seems to have the rolling checksum broken. with the --stats switch, if you only transfer one file, Matched data:will always be the same size as the block-size. even if there is no difference in the file. not really sure if its just a display error or not.
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 12/11/2002 10:01

You could try the debian version (2.3.2) to see whether there's any difference.
My experience with the debian version is that it locks up when transferring/comparing huge numbers of tunes (>14000 in my case).

Of course, the problem you are seeing might also be caused by cygwin's rsync.
For testing, it would be nice to have a replacement for that too.

I'd be interested to see what you have found out. Now's the time to shut down my computers and leave for the airport.

Pim
Posted by: dmuench

Re: mp3tofid 3.00 - now for Windows too - 12/11/2002 10:54

It's been working great since. I've got a feature suggestion for when you get back from your trip though - I've got some music (holiday music) that I really don't want on my empeg. Can I suggest either a command line switch to exclude a directory(s) from mp3tofid scanning, or maybe a flag file we could place in a directory that would signal mp3tofid to ignore the directory (Maybe ".no_empeg" or something)? Either would be nice, I can see plusses and minuses for either solution. Thanks..
Posted by: smu

Re: mp3tofid 3.00 - now for Windows too - 24/11/2002 11:09

Hi.

Couldn't you use an md5 hash over the mountpoint _name_ instead of the partitions major/minor? By that, mp3tofid wouldn't recreate the whole database if the drive and/or partition changes its position, which can happen quite often on some systems (like mine: I often add/remove harddiscs or controllers, but the mount point for partitions always stays the same because I mount them by volume labels).

cu,
sven
Posted by: pim

rsync > 2.3.2 broken - 04/12/2002 19:02

i've found a bug in your compile of rsync. it seems to have the rolling checksum broken.

Yes, it appears rsync 2.5.5 on the empeg will always transfer a file, even if only the timestamp is different. This is a bug that supposedly was in 2.5.4 but has been fixed in 2.5.5. But the problem also appears to be there in version 2.4.6, which can be found here. It's also there in the latest CVS version.

The only version that does not appear to have this bug is the 2.3.2 debian version. Unfortunately, this version has other problems.

I did all my testing against a linux x86 box with rsync 2.5.5.

I'm not sure what causes this bug. Maybe an ARM alignment problem, or a gcc bug. I can't find anything in the rsync mailing archives either.

Pim
Posted by: pim

Re: mp3tofid 3.00 - now for Windows too - 04/12/2002 19:25

Couldn't you use an md5 hash over the mountpoint _name_ instead of the partitions major/minor?

I don't think there's a fast and portable way to discover the mountpoint name. major and minor numbers are easy, though, as they are returned in the stat() data.

Thinking about it, I might just as well have ommitted the mountpoint data, unless it is different from the mountpoint data of the root of the mp3 tree. This would solve almost anyones problems, and it would still work in the rare cases where mountpoints exist within the mp3 tree.

Pim
Posted by: image

Re: rsync > 2.3.2 broken - 04/12/2002 23:13

welcome back from asia btw. how was the trip?

as for rsync... i'm content w/ 2.3.2. don't really see a problem with it, and i was able to speed up the upload of a slightly modified mp3 file (id3 tag mods) by putting the checksum block size to 100000 bytes. it minimizes the transmittion of checksums/packets.

i was wondering if it would be possible to "import" a current fid/file structure on the empeg. there is a bug that got me by surprise. mp3tofid segfaults when you change operating systems (xp to 2k in my case). i deleted everything EXCEPT the mp3tofid association file, and still segfaulted. finally gave up, deleted the mp3tofid file, and started anew. but my point is... if you took all the tag files (______1) from the empeg, couldnt you parse them and rebuild the associations, depending on the fact that the import_location field is filled (mp3tofid and jemplode use that field and emplode seems to have omitted it). the only limitation is in windows based systems that used jemlode, you'd have to convert "c:\" to "/cygdrive/c".

speaking of jemplode, i'd really like to be able to have the hash value of an mp3 calculated and put into the tag file just like jemplode. i know that you don't use jemplode... but i use it to do the stuff that mp3tofid can't do (yet).
Posted by: pim

Re: rsync > 2.3.2 broken - 05/12/2002 08:29

welcome back from asia btw. how was the trip?

Thanks, I had a great time visting Bangkok, Hong Kong, Macau and Guangzhou.

as for rsync... i'm content w/ 2.3.2.

Lucky you. Mine kept locking up. Not during large fresh transfers, but during small changes inside 70 gigabyte of MP3's.

i was able to speed up the upload of a slightly modified mp3 file (id3 tag mods) by putting the checksum block size to 100000 bytes. it minimizes the transmittion of checksums/packets

You'll be transferring at least one 100.000 byte block then. Did you time different sizes as well?

i was wondering if it would be possible to "import" a current fid/file structure on the empeg

That's going to be hard, as plain empeg fids do not contain the "loadfrom" tag.

mp3tofid segfaults when you change operating systems (xp to 2k in my case)

mp3tofid should not segfault, but you lose anyway. mp3tofid since version 3 uses not only the inode number, but also the major and minor number of the filesystem to uniquely identify a file. Doing so, mp3tofid will handle the rare case of mountpoints within the mp3 tree, which could allow for duplicate inode numbers.

If you copy your mp3's to a different computer, every mp3 file will probably have different inode numbers and the filesystem containing them will probably have different major/minor device numbers. So all fid numbers will be invalid, and will have to be recalculated. And new fid numbers means yet again a new whole rsync transfer. Sorry, but this is how it's designed.

Pim

Posted by: pim

Re: mp3tofid with jemplode hashes - 05/12/2002 08:48

i'd really like to be able to have the hash value of an mp3 calculated and put into the tag file just like jemplode

Is that a plain md5sum hash? Should be doable, as mp3tofid reads the whole file anyway.

Pim
Posted by: image

Re: rsync > 2.3.2 broken - 05/12/2002 09:29


You'll be transferring at least one 100.000 byte block then. Did you time different sizes as well?


i tried a 500,000 block, but it seemed to take longer. didn't really want to to test it too much. i'll try it tonight.


That's going to be hard, as plain empeg fids do not contain the "loadfrom" tag.

If you copy your mp3's to a different computer, every mp3 file will probably have different inode numbers and the filesystem containing them will probably have different major/minor device numbers


i know that the plain empeg doesn't use the tag. but the tags already generated have them, as well as jemplode. to recover from an inode change, can't you parse the existing tags, find out the new inode/major/minor#, and repopulate the mp3tofid file, assuming that the file structure was the same.... (or just mass replace the path to wherever the files are).

Is that a plain md5sum hash? Should be doable, as mp3tofid reads the whole file anyway.

its a crc32 check on only the mp3. exclude the tags.
Posted by: pim

Re: stopping the player from the shell/hijack - 04/02/2003 14:52

is there anyway aside from hitting q in terminal mode to kill the player? i've tried to just use kill -9 pid but it always restarts.

You can gracefully stop the player with "kill -INT <player-pid>"
and restart it with "kill -HUP <bash-pid>". Or use killall from
debian to automatically find the pid.

Another way to stop and start the player is to use hijack:
http://player-ip/?NODATA&SERIAL=q will stop the player,
http://player-ip/?NODATA&SERIAL=exit will start it.

You would need wget or something to do this from the player,
but there might be a more direct way to have hijack emulate
serial input. Anyone?

Pim
Posted by: wfaulk

Re: stopping the player from the shell/hijack - 04/02/2003 15:03

Yeah. You should be able to:

echo "q" > /proc/empeg_notify
Posted by: pim

Re: stopping the player from the shell/hijack - 04/02/2003 16:26

echo "q" > /proc/empeg_notify]

Hmm... Does that work for you? Not for me, running
hijack v310. Would be cool, though.

Pim

EDIT: Oops, forgot to add


[output]
notify=1



to config.ini
Posted by: wfaulk

Re: stopping the player from the shell/hijack - 04/02/2003 16:44

I didn't know that was a requirement. It's how empegVNC sends button presses to the player.
Posted by: pim

Re: stopping the player from the shell/hijack - 05/02/2003 14:51

Well, it did not work, I made a post about it and then read the FAQ.

Wrong order ...

But now it appears not to work even with the mentioned addition
to config.ini. Did you try this? Of couse, you need to install emptelnetd
to try.

Pim
Posted by: wfaulk

Re: stopping the player from the shell/hijack - 05/02/2003 14:57

Okay, no, I didn't try it.

All I know is that it works for button presses, et al. I'd try it here, but I can't attach my empeg to the network at work.