Empload sever (was New player App)

Hello,
I've just completed a server (empeg side) for empload /emptool clients. As suggested by someone in this forum, I wrote the server by sniffing ethernet packets (via Ethereal), studying the commands in client_protocol.cpp from emptool and implementing the required responses. It correctly sends and receives fids( tunes, playlists, config.ini, etc), rebuilds /empeg/var/database and /empeg/var/playlists files, sends back progress reports, and responds to all known commands. It is designed to be a companion to rioplay, enabling tune and playlist downloading. It's almost finished, but there are a couple of things that I don't understand...

When a new fid (say 0x320) is created, e.g. a new tune, three files are created:
320 -- the mp3 data file
321 -- a text file that contains the tags for the tune (e.g. title, bitrate, etc)
32f -- a 28 byte file with 7 integers 1,14,0,0,0,0,0

What is the purpose of the ??f file? The real empeg player app does not store the file in the /drive0/fid directory. It might be an entry for the dynamic database...


Next question
During the empload <-> empeg startup negotiation, empload requests the dynamic database. I just send a null file and empload doesn't seem to mind. But I believe the dynamic database contains information about when and how many times a song was played. Can someone tell me where I can read this data off my empeg and the format of the dynamic database? As far as I can tell, emptool does not use the dynamic database, so there is no source code for this.

Last question.
Is there an easy way to determine the last fid (actually the highest) on a drive? Is it wrong to include extra 0xff at the end of the database file?


Thanks,

-Scott

In protocolclient.cpp, look at

STATUS ProtocolClient::RetrieveDatabases(TuneDatabaseObserver *db_observer)

specifically the references to dynamic_buffer (which is of type dynamic_data)

I'd tell you to look at the jEmplode source too, but I believe there are some problems in my port of the dynamic data code

ms

The real empeg player app does not store the file in the /drive0/fid directory. It might be an entry for the dynamic database...

Indeed. The stock player spots those "special" (*F) fids, and writes to the dynamic database instead of the filesystem.

But I believe the dynamic database contains information about when and how many times a song was played. Can someone tell me where I can read this data off my empeg and the format of the dynamic database?

The dynamic database lives on the scratch partition. We've never given out the layout of the scratch partition (so that we could change it behind the scenes if we wanted -- it's bad enough that we're stuck with that static database format now everyone reads/writes it) but having got this far it shouldn't be hard for you to reverse-engineer it. The scratch partition layout will not change for 2.0 final, but it's not ideal and so might well change in some post-2.0 release.

Is there an easy way to determine the last fid (actually the highest) on a drive?

No. Currently you must readdir() both fid directories.

Is it wrong to include extra 0xff at the end of the database file?

Nothing will break, but you'd be wasting valuable player memory (and marginally decreasing search speed).

Peter

I'm a little confused at the idea behind this empeg-side server. Would this be a web/java version of (j)emplode that resides on the empeg instead of having to install (j)emplode on multiople pc's?

If not, could you please give an explination about the purpose/application of this server?

AOL

No.

It is designed as a replacement for part of the player software. The bit it replaces is the bit that (j)emplode talks to when managing the database.

The end aim being to not have to run the player software at all.

The reasons for wanting to do this is so that you can end up with a complete open source replacement for the player software and therefore add new features (such as OGG) without waiting for Hugo, Rob, Peter, Roger, Toby et al

That's how I understand it anyway.

Race?

Ooooohhh!

Race?

Ooh, ooh, competition is good! Maybe now we'll get some of those "not yet backported" features onto our beloved empegs!

In reply to:

---

Race?

---

Now thats what I like to hear

Can't wait for 2.0 final and beyond.

I've got $5 on the new guy.

I've got $5 on the new guy.

Given the historical precedent, I'll take the other side of that one.

Many have tried, and once everyone sees how tough it is to actually make these things happen, they realize waiting for the empeg guys to do it (despite how often their priorities don't give them much car player time) is the best way to go.

In fact this thread hasn't changed anything. Plans have been afoot for a while. The alpha team are blowing cobwebs from their serial ports as I speak.

Rob

So, I assume you're still not looking for bribes from or to new alpha testers?

Rob, I've been meaning to ask this for awhile. What the hell is the need for an "alpha team" nowadays when the product is now EOL? I get the feeling that there are features that have been "in the can" for months and years now that are held back because they haven't been fully tested. Are our empegs controlling missiles or space shuttles? I understand that the point of solid testing procedure is to get release-quality code so that consumers don't call in with stupid questions because they're running beta versions. But empeg users have been running beta versions *forever* and really, we're used to a few quirky things not working.

Like last year some time, I can't remember if it was 2.0b7, b11, or b13, but one of those.... Some stupid USB bug held up the entire release for god-knows-how-long. I use Ethernet, and couldn't give a damn about USB. Why couldn't it just be released with the caveat "if you run USB, you may experience problems."

I guess I just don't see the point anymore in holding these things back from release now that the product has been discontinued, and isn't going to be a mass-market product. Since we're kinda just a branch hanging off of the tree now (with the other current Rio products being at the trunk) why can't we get exprimental stuff without the hassle of a big test cycle to slow progress down even more? We already pay the penalty of having to wait for a back-port, and now we need to let the alpha team hang onto it for even longer?

I think the whole alpha test phase made sense back in the day, but nowadays, I just wish we could abandon the alpha and just go to wide beta testing, with maybe quarterly releases instead of yearly ones. Those who "can't stand the heat" of running the new version can go back to their last-known-good. Simple, yes?

I couldn't agree more, where I work we pay our testers A LOT, they aren't very tech savy and basically document steps to break the software. I'm assuming you pay your testers.... why not save SB some money and give it to us to test for free.

The alpha team get a release for a few days before you get it, assuming it works out OK. We want a few focussed reports and discussions of bugs in a new drop, not 3000 people all jumping in at once. They also have direct access to the bug database, which is not about to become available to the world at large.

The team may also get to play with features that we have no intention of releasing widely yet. They're under NDA, you're not

In practice the team haven't had anything since Beta 13 - until tonight.

Rob

The alpha team are all customer volunteers. An EOL product doesn't get to use extremely expensive QA resources, so the effort put in by the alpha team is much appreciated.

I'm assuming you pay your testers.... why not save SB some money and give it to us to test for free.

No, the alpha testers are not paid. They are just a sub-set of the bbs people who get things that are a bit buggier than the normal beta releases.

There have been alpha releases you would *not* have wanted -- one of them actually killed all of the songs on the player. I think Tony Fabris and... who was it? N6Mod? got nailed by that one.

Not all of the bugs fixed by the alpha team are benign annoyances. There are really good reasons that the guys@empeg alpha test this stuff before releasing it to to the general population of the bbs.

tanstaafl.

Ah.... I see.... Need anymore alpha testers?

I'd sign an NDA. Just send it over... It's not like this one would at all interfere with my work in distributed filesystems, or authentication

I think this is one of those "Don't call us, we'll call you" type things

Yeah, probably, but a man's got to try. I don't even know if they have my number

I'd sign an NDA too....

I think this is one of those "someone has to die before someone new can join" things. That's why we don't publish a list of team members

Rob

In practice the team haven't had anything since Beta 13 - until tonight.

Well, I'm glad to hear that. We'll see how things work out this time. I've just been hoping that as the necessity for SonicBlue to officially be involved in the empeg car product diminishes, we'd start seeing more frequent and less "officially blessed" and rigorously-tested releases. It's not that I don't want quality code, it's more that I think that some of the best innovations on the player were things that were kinda thrown in and initially had bugs in them (the seek tool comes to mind.)

I just hope we don't have to wait another six months between releases, and I thought a change in philosophy and focus might be appropriate now that the product doesn't need to officially be supported as much anymore.

Yeah. If there is some disasterous problem with the alpha build then you'll only have a few people needing help to get it fixed. Having 2000-3000 people all complaining about how it nuked their entire player wouldn't be too great!

- Trevor

The alpha team wasn't SB's idea, it was an empeg creation. The car player software strategy has always been in the hands of the Cambridge team, and we can release what we want when we want - as long as the official projects also get done. Since taking over all of Audio for SB we have a lot of official projects right now.

Anyway, in a few days I'll release some information and you'll be able to see why the 2.0 milestone had effectively stagnated car player development, and why getting past that to 3.0 betas is going to help plenty. You might just get that horribly unreliable firmware you crave a little sooner than you realise

Rob

What do I care if it nukes my whole player? That's why I have a spare, an extra just to break again and again. Just use that for testing for a while, carry my "real" empeg with me so if it becomes unusable due to testing I can still listen, and move right along with life.

I know that the alpha team has been around since the beginning. I would just much rather have the features out there earlier in more peoples' hands. If you can't handle the bugs, or are stupid enough to nuke your player and not have a backup, you'll be less likely to use the newer builds. In my experience, a small "alpha" team doesn't do nearly as much as a larger "wide beta" approach.

In my experience, a small "alpha" team doesn't do nearly as much as a larger "wide beta" approach.

I think you're missing the point: They do both. In that order.

What if we discount such remarks from people on the alpha team?

Yeah. And in my experience, releasing it earlier to the larger group earlier on makes the *beta* releases *final* quality in a shorter period of time. Granted it takes a little more effort to manage the bugs coming in and such, but I think it's worth it.

Hell, why not just have automated nightly builds?

Cool.. yn0t_ just volunteered to manage all of the bugs from everyone.

Several comments -- Managing bug reports/release management is a really big timesink... I can say this both from my professional work as well as my jEmplode work, which is far less complex than the entire Empeg project. The Alpha team is setup to catch the obvious things quickly and it saves everyone a lot of time. Imagine if everyone was reporting the same bug (which happens ... a lot)? What a pain -- especially when they're doing this out of the goodness of their hearts. Optimizing that process is essential to all of us continuing to get releases.

Also, like Rob said, the Alpha team is under NDA, so we can see stuff you can't So in some cases it's not feasible from a marketing standpoint to deploy to a wide beta without revealing competitive info.

Also, there was something at the bottom of the NDA about souls and first borns. I didn't really understand it -- I think it was written in "proper" English, or something. But apparently the Empeg team feeds off of the lifeblood of the Alpha team and would actually wither and die without us. Anyway, I just signed it (in blood).

ms

Oops .. I think there was something about not talking about fight club on there too.

The Empeg alpha team is paid in free orgasms when emails come out with headers like

From: [email protected]
Subject: [empeg-caralpha] [ANNOUNCE] v<insert number greater than existing release> released

Managing bug reports/release management is a really big timesink

I got a sucker^H^H^H^H^H^Hvolunteer to help with this for OpenAFS; I was doing it all myself until like 6 months ago, and as we gained momentum it got to be a huge pain.

I'm not sure it's smart for me to keep doing it instead of developing, but I don't think we have a choice just now.

Also, like Rob said, the Alpha team is under NDA, so we can see stuff you can't

Hmmm, see, all the testers keep giving themselves away, takes a lot of the challenge out of figuring who to bump off!

The Empeg alpha team is paid in free orgasms

Damn, I missed that part in the NDA. Where do I download those again?

Yeah. And in my experience, releasing it earlier to the larger group earlier on makes the *beta* releases *final* quality in a shorter period of time. Granted it takes a little more effort to manage the bugs coming in and such, but I think it's worth it.

We're not talking about Beta releases. They're not of known Beta quality until the Alpha team have tested them, irrespective of the filename.

So let's put it another way. I don't have time to manage a mass ALPHA programme and the consequences that will arise from it. So, we can continue to use my system or we can discontinue new releases entirely.

OK class?

Imagine if everyone was reporting the same bug (which happens ... a lot)?

The S Series Beta Programme with about 40 participants reported 334 bugs, of which 30 were found to be valid and unique. The Programme Coordinator filtered out all the junk, and the development team got to see just the valid reports - which were all subsequently fixed.

SB Beta Programme Coordinators work full time to manage two to three simultaneous programmes, with probably no more than a couple of hundred participants in total. This is with Beta quality software which has already been through a QA alpha regression.

Rob

So let's put it another way. I don't have time to manage a mass ALPHA programme and the consequences that will arise from it. So, we can continue to use my system or we can discontinue new releases entirely.

OK class?

Why do I get the urge to say "Yeah! put that in your pipe and smoke it you ingrates"...

So let's put it another way. I don't have time to manage a mass ALPHA programme and the consequences that will arise from it. So, we can continue to use my system or we can discontinue new releases entirely.

Hey, relax, I'm just trying to share my experience and see if there's a better way to do things. If the finite resource in question is your time managing things, then I guess there's no better way to do it in this case.

Sorry I asked.

Why do I get the urge to say "Yeah! put that in your pipe and smoke it you ingrates"...

Funny, I don't consider myself an ingrate for trying to share a new perspective on things. On the contrary, I think it's good to occasionally challenge existing methods and see if there's a better way. Clearly in this case it looks like there isn't a better way (short of sharing the source with us and letting us do our own releases!) so I stand corrected.

Oh yeah, and ingrates don't spend a significant amount of their free time reporting bugs, helping new users, and developing 3rd party software. Actually, an ingrate wouldn't have cared enough to bother thinking about such a thing in the first place.

Funny, I don't consider myself an ingrate

Of course not...most of us share the same eagerness. I should have made it clear that that was tongue-in-cheek. My apologies, sir!

I think this is one of those "someone has to die before someone new can join" things. That's why we don't publish a list of team members

Well, two of your members just blabbed who they were, so... anyone want to get a spot on the alpha team?

I suppose this means I should be watching my back from here on out

So, can any of you alpha guys tell us... is OGG there?

Uh...they're under NDA's. They couldn't tell us if they wanted to.

Read this post. No idea if it's in the release given to the alpha team though.

- Trevor

... yeah -- what he said ...

I just sent a private message, then noticed he hasn't posted again.

Sad that there's no way to tell when someone last read.

I miss "finger".

Peter,
Thanks for all your help, I now understand the dynamic database and how it's used. I have a few more questions... )

I'm planning on adding a few new "tags" and I'm wondering the best way to do this. I've added two new tags in a tune fid: "starttime" and "endtime". They are used to set where to begin and end playback of a tune on a time scale. An example: The song "St. Robinson and his Cadillac dream" by the Counting Crow's is 15 minutes long, and is composed of two songs separated by 8 minutes of silence. By setting the endtime, I can stop the playback at the end of the first segment. I have the feature working on my empeg server using libMAD for mp3 playback, but I am wondering how I allow the user to make changes to this tag on my 3rd party player?

Given that the /drive0 partition is mounted read only, like the empeg. I don't want to remount it RW just to make changes to the fid | 0x1 file. I could allocate another scratch partition, and write out the changes to the fid | 0x1 file, (like the dynamic database), and then update the real fid | 0x1 file when I rebuild the database. But how would you do it?

-Scott

PS I'v attach a picture of my 3rd party touch screen player. Eventually, I will release it on sourceforge.net...

So, does this mean someone owes me $5?

i thought it was for ogg implementation? there isnt any evidence that the player supports it.

Hmm. You're right. I bet I could have an ogg-using version (instead of a MAD-using version) in a small number of hours, but I doubt it's worth $5 to do so.