daapd on empeg (iTunes music sharing support)

Posted by: mcomb

daapd on empeg (iTunes music sharing support) - 30/09/2005 20:20

OK, I put way more time into this then I ever intended so hopefully someone beside me will find it usefull. I've got mt-daapd running on the empeg. This will allow any computer on your network running iTunes to stream music from your empeg. For now I'm not going to bother with an installer or detailed directions, but here is the gist of it...

Instructions
  • 1. You need a kernel compiled with Multicast support. Either roll your own or grap this one from me which includes hijack, multicast, and ext3 support.
  • 2. Reserve some memory for mt-daapd by adding a ReserveCache line to the appropriate spot in your config.ini. A value of 40 seems to work well enough for my empeg with 62 gigs of music running v3a11 (I made some changes to mt-daapd to cut down a lot on it's memory requirements so it shouldn't require a memory upgraded player). If you don't give it enough memory your empeg will probably still show up under iTunes, but when you click on it you will never get a list of mp3s.
  • 3. While you're in config.ini you might want to add a menuexec line as well, something like:
    Code:
    ;@AC ;@MENUEXEC daapd exec /drive0/bin/mt-daapd/mt-daapd -c /drive0/bin/mt-daapd/mt-daapd.conf
    


  • 4. Grab this tar file and copy it to your player.
  • 5. Extract it like so
    Code:
    tar -xf empeg-daapd.tar /


    This will extract to /drive0/bin/mt-daapd
  • 6. mt-daapd needs to build a database of your mp3s, this takes a long time the first time so don't do it with the player software running. Run
    Code:
    /drive0/bin/mt-daapd/mt-daapd -c /drive0/bin/mt-daapd/mt-daapd.conf


    After the first time it just updates the existing database so it is much faster.
  • 7. This will run in the background and you'll know it is done when "empeg" pops up under iTunes as a shared drive.

Caveats
  • 1. This will remount your /drive0 as read/write automatically for a few minutes every time you start it. It has to in order to update it's database. On my empeg with 62 gigs of music this takes about 5 minutes with the player running. Do not unplug your empeg until it shows up under iTunes or you risk file system corruption and will have to do an fsck the next time you want to load music. It will go back to read only when it is complete.
  • 2. I'm not sure if this works on players that haven't been fid sifted. I don't have one here to try.
  • 3. I'm not sure what will happen if this hits an audio file that isn't an mp3. I think it will skip it and if it doesn't it is unlikely iTunes will play it.
  • 4. The empeg is slow when the player is running. Don't be surprised if it takes a few minutes for tracks to show up in iTunes after you click on the shared playlist.
  • 5. This keeps it's database which is on /drive0 open at all times so you won't be able to sync with it running. I may fix this at some point.
  • 6. Don't futz with the location of the files, just leave everything where it is as some of the paths are hardcoded.


If you want the source code you can grab it here. This is based on mt-daapd version 0.2.3.

-Mike
Posted by: sein

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 04:57

Awesome! Will try this tonight
Posted by: tfabris

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 05:20

Wow.

Wow, wow, wow.

How much to do the same for the Rio Central?
Posted by: sein

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 06:56

Quote:
Caveats
  • 1. This will remount your /drive0 as read/write automatically for a few minutes every time you start it. It has to in order to update it's database. On my empeg with 62 gigs of music this takes about 5 minutes with the player running. Do not unplug your empeg until it shows up under iTunes or you risk file system corruption and will have to do an fsck the next time you want to load music. It will go back to read only when it is complete.


It would be really cool if mt-daapd had a --readonly flag which would not scan for new files and not update any data on the harddrive, just take its database as gospel. That way it would start up faster and I could still unplug the empeg without a second thought.

I only really sync my 'peg every few weeks and its not too much hassle to run it manually to regenerate its database when I do.
Posted by: Daria

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 09:15

Quote:
Wow.

Wow, wow, wow.

How much to do the same for the Rio Central?

How much does a Central cost?
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 14:03

Quote:
It would be really cool if mt-daapd had a --readonly flag which would not scan for new files and not update any data on the harddrive, just take its database as gospel.


#1 on the to do list, and actually I can do it without the --readonly flag. All I need to do is see if the empeg's database has a newer modification time then daapd's. If it doesn't then there is no reason for daapd to do it's scan. I may try to get to that sometime next week.

-Mike
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 14:06

Quote:
How much to do the same for the Rio Central?

I suspect that this version would run on a central with little if any change, but I don't really know much about them. The first question would be if the central has multicast support enabled in it's kernel and if not has anybody figured out how to get a replacement kernel on it?

-Mike
Posted by: sein

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 15:39

Quote:
#1 on the to do list, and actually I can do it without the --readonly flag. All I need to do is see if the empeg's database has a newer modification time then daapd's.


Now that just makes sense, nice one
Posted by: peter

Re: daapd on empeg (iTunes music sharing support) - 01/10/2005 17:07

Quote:
The first question would be if the central has multicast support enabled in it's kernel and if not has anybody figured out how to get a replacement kernel on it?

Looks like the Central defconfig doesn't have multicast. Getting new kernels on is easy: you just replace /zimage (or /vmlinux, or some such file). You should of course start from the Central kernel sources, not stock 2.4, not least because IIRC we had to patch ReiserFS to get it to work on ARM. (Yeah yeah, but ext3 hadn't been invented in those days.)

Peter
Posted by: pgrzelak

Re: daapd on empeg (iTunes music sharing support) - 03/10/2005 09:59

Wow!!! Extremely cool!

I have never used iTunes before, but was thinking of putting it on the player for the Pod People. Question - can iTunes be used to delete (or otherwise mangle) files on the empeg? I assume that it cannot, since the partitions are read only for the duration (except for that rebuild period), but I am just being paranoid. Thanks!

And thanks for the work in this!!!
Posted by: jdandrea

Re: daapd on empeg (iTunes music sharing support) - 03/10/2005 10:39

First of all, Mike - thank you! Excruciatingly wonderful that you did this.

Next, Paul - good question WRT iTunes. From what I can tell, it is not possible to delete files. For instance, if I tap into my wife's iTunes feed over the home network, I can't seem to modify or delete anything on her end. Perhaps there's a way to do it, but I haven't noticed yet.
Posted by: JBjorgen

Re: daapd on empeg (iTunes music sharing support) - 03/10/2005 11:43

This is pretty cool. I'll try it out as soon as you've got it updating its database only when the empeg database changes as noted above.
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 03/10/2005 15:41

Quote:
From what I can tell, it is not possible to delete files.

Yeah, what he said. iTunes won't let you make changes to a remote share even if the empeg would. It doesn't give you the ability to delete, edit, or even download files, just stream them. Of course when Paul feeds daapd 200 Gigs of mp3s I'm not sure it will work anyway

-Mike
Posted by: drakino

Re: daapd on empeg (iTunes music sharing support) - 04/10/2005 21:36

I got it working per your exact instructions, and it's running great.

I also went ahead on mine and set it to start automaticially with an SH script:
Code:

#!/bin/sh
(/bin/sleep 30; /drive0/bin/mt-daapd/mt-daapd -c /drive0/bin/mt-daapd/mt-daapd.conf) &


And Hijack wise, I added an option to kill the server.
Code:

;@AC ;@EXEC_ONCE /drive0/bin/mt-daapd.sh
;@AC ;@MENUEXEC Kill_dappd kill -9 `cat /drive0/bin/mt-daapd/mt-daapd.pid`



Thanks for the work on this.
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 15:31

Quote:
And Hijack wise, I added an option to kill the server.

And that is actually working? I tried something similar and it never worked (it killed the one thread, but the rest wouldn't die off). Anyway, you might want to double check that.

-Mike
Posted by: Mataglap

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 16:48

Quote:
Code:

;@AC ;@MENUEXEC Kill_dappd kill -9 `cat /drive0/bin/mt-daapd/mt-daapd.pid`




There's a serious application problem if you need to kill -9 / SIGKILL it instead of the default kill -15 / SIGTERM. This comment isn't meant as a slam against all of the hard work that's been done, this seems like an insanely cool thing for iTunes users, but it's really bad unixishness.

--Nathan
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 17:07

Quote:
There's a serious application problem if you need to kill -9 / SIGKILL it instead of the default kill -15 / SIGTERM.

In my testing a regular SIGTERM would terminate that thread, but neither SIGTERM or SIGKILL would kill all threads belonging to md-daapd. That is probably a bug, not sure if it is one I introduced or if it was already there. Never had a chance to look into it farther, but that is why I mentioned it to Drakino.

-Mike
Posted by: drakino

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 17:29

Quote:
And that is actually working?


It seems to be. When I run that, the web interface stops working, and any iTunes clients stop playing music. I can't say for sure it killed all the threads, but it appears to do what I need.

I noticed this way didn't work to have it disappear from iTunes gracefully, but didn't look into it much. I was assuming it probably wanted to write some final things to the drive before stopping.
Posted by: wfaulk

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 17:34

Well, if you didn't give it a SIGKILL (that is, signal 9), it might have the opportunity to do that.
Posted by: Mataglap

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 17:47

Quote:
In my testing a regular SIGTERM would terminate that thread, but neither SIGTERM or SIGKILL would kill all threads belonging to md-daapd. That is probably a bug, not sure if it is one I introduced or if it was already there. Never had a chance to look into it farther, but that is why I mentioned it to Drakino.

-Mike


Sure. I'd be really surprised if your porting it to the empeg had much if any effect on the signal handlling.

For the non-unixy folks, SIGKILL is basically a bullet in the head of a process, and will make just about everthing (other than zombie threads -- so much for a coherent metaphor) go away RIGHT NOW. SIGTERM means "Shutdown gracefully, please, and make sure to clean up after yourself," which includes making sure all related threads stop cleanly too, and saying goodbye to all of the other processes it's talking to, like iTunes.

--Nathan
Posted by: wfaulk

Re: daapd on empeg (iTunes music sharing support) - 05/10/2005 17:54

Zombie processes are dead, it's just that nobody has noticed them yet. If they didn't stop moving because of being disemboweled with that chainsaw, the bullet to the head isn't going to do anything. Actually, they ought to be called vampire processes since the only way to kill them for good is to kill all of their forebears.
Posted by: drakino

Re: daapd on empeg (iTunes music sharing support) - 07/10/2005 14:24

Quote:
Well, if you didn't give it a SIGKILL (that is, signal 9), it might have the opportunity to do that.


I forgot to confirm I was seeing the same issues as Mike was. Anything other then a SIGKILL doesn't work, at least not from the hijack menu. I haven't sat down to see exactly why, as to me, this doesn't bother me. Sure, the empeg is a Linux running box, but I see it more as an appliance. As long as the empeg recovers properly from this, and does what I expect when I say stop the process, I'm happy. And thus far doing a SIGKILL is not showing any problems.
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 18:32

OK, I've released a new version with only two changes:

1. I've added a 30 second sleep before it does anything to allow the empeg time to startup in case this is started from hijack with EXEC_ONCE.

2. It no longer rescans your drives every time. It only perform the scan if the empeg's /drive0/var/playlists file has been updated more recently then it's database. This means your drive is only remounted read write for a second in order to write a process ID file unless you've just updated the music on your empeg in which case it does a scan for new music. I'm assuming people want the pid file bit left in so that it can be killed from hijack (see Drakino's post above for an example how).

I've created a web page with instructions (mostly just the first post from this thread) and downloads here.

-Mike
Posted by: JBjorgen

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 18:49

Cool. Sounds like it's time to try it if I can get the working empeg back from my wife...
Posted by: wfaulk

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 18:55

Are the timestamps accurate? Weren't there a lot of people for whom the clock was not?
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 19:05

Quote:
Quote:
Well, if you didn't give it a SIGKILL (that is, signal 9), it might have the opportunity to do that.


I forgot to confirm I was seeing the same issues as Mike was. Anything other then a SIGKILL doesn't work, at least not from the hijack menu. I haven't sat down to see exactly why, as to me, this doesn't bother me. Sure, the empeg is a Linux running box, but I see it more as an appliance. As long as the empeg recovers properly from this, and does what I expect when I say stop the process, I'm happy. And thus far doing a SIGKILL is not showing any problems.


FWIW, I've decided I agree with Drakino and have added instructions for doing a kill -9 to my install page. The drives are mounted read only at that point so it's not like daapd would be doing anything other than quitting other threads and saying goodbye to iTunes clients gracefully. A SIGKILL is effective and harmless in this situation even if it isn't exactly proper. Of course the source is up on my page if anybody wants to do a proper fix.

-Mike
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 19:06

Quote:
Are the timestamps accurate? Weren't there a lot of people for whom the clock was not?


Seem to be on mine. They don't actually have to be correct, just accurate relative to each other.

-Mike
Posted by: wfaulk

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 19:07

Yeah, but I thought I remembered people's clocks resetting, and, if that's the case, they wouldn't be accurate relative to each other. I could be completely making stuff up, though.
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 24/10/2005 19:55

Quote:
Yeah, but I thought I remembered people's clocks resetting, and, if that's the case, they wouldn't be accurate relative to each other. I could be completely making stuff up, though.

I guess I will deal with that if anybody reports it then. I can always add an option to force the check to happen every time if someone needs it.

-Mike
Posted by: sein

Re: daapd on empeg (iTunes music sharing support) - 25/10/2005 02:39

Quote:
Yeah, but I thought I remembered people's clocks resetting...

Has anyone got an ntpclient working on the 'peg?
Posted by: Roger

Re: daapd on empeg (iTunes music sharing support) - 25/10/2005 12:14

Quote:
Quote:
Yeah, but I thought I remembered people's clocks resetting...

Has anyone got an ntpclient working on the 'peg?


Someone undoubtedly has, but what I do is have ntpdate run every time I drop to the shell.

For the magic I use, check out http://www.differentpla.net/~roger/empeg/car/files/, and grab base.tar.gz and configs.tar.gz. They need a modicum of configuring, but I like 'em.
Posted by: mcomb

Re: daapd on empeg (iTunes music sharing support) - 25/10/2005 15:04

There is also an rdate binary that I use floating around if that is your preferred method of time syncing.

-Mike