Rebuilding DataBase speed

Is there anyway to increase the Rebuidling database speed after you upload a song?

I have 2500 songs, for about 15 gigs of space used.
IM Using Emplode 2.10

You can have fewer songs.

Alternatively, you could try to convince Mike to work some more on the fast database rebuild feature of Jemplode. He had a version working, but there were problems with it so he disabled the feature until he could get to the bottom of those problems. (This is all from fuzzy memory, someone correct me if I'm wrong.)

Your recollection is correct, but I'm unsure what the problems were. I kept using a jar with the feature and it's serving me well, better than waiting multiple hours for a database rebuild that never finished.

You may want to try to fidsift your directories. That should help with the rebuild speed.

Paul - what sort of time are you getting? I mean, I only have 9500 tracks at the moment and I have fidsifted so mine is pretty fast - full rebuild is only a few minutes, but you have the record for number of tracks, don't you?

Um, my database rebuild times really don't apply. At least not with emplode. I have too many fids - well beyond the limits - and it crashes emplode each time. It works, but I have to manually rebuild the database each time I load more music...

Maybe I pulled it too quickly ... There seemed to be quite a few complaints that it wasn't consistently writing the database, tags, etc files back to the Empeg and I could never figure out why that was happening. Because it was so much code to implement this feature, I decided to pull it. But I can bring it back if people want ... Maybe someone will be able to figure out why the writes are sometimes failing.

ms

My player went from a 17 minute rebuild to an 8 minute one after fidsift. 22,000~ songs.

why the writes are sometimes failing.

Without ever having looked at the code involved, I assume you are writing stuff back with ftp. Is it possible that the sync is timing out because you aren't talking to the player for a while and the player is restarting? That is what was happening with my ftp upload patch before I added the (ugly) hack that basically pings the player every so often.

-Mike

Mmm.. I wonder where the timeout is happening? The kftpd code I have in the Hijack kernel does not have any explicit connection timeouts. So it must be either the client-side, or perhaps the TCP/IP stack (dubious).

cheers

Hmmm...

A thought.

I have always been blaming the fact that my syncs were failing based on the number of fids. But my problem sounds exactly the same: through emplode (not jemplode), all of the track are copied up and it starts to rebuild the database. It does not happen immediately and emplode times out. The database build occurs on the player, it reboots - missing the database / playlists / tags files. I also notice that I fidsifted my player, but all the new files are still in the fids directory.

A connection?

Yeah, I've noticed that as new fids are uploaded, the player software does not create any new fids subdirs as needed -- it will use the existing subdirs, but if a new fid requires a subdir that does not yet exist, the player will not create it.

I originally had code in Hijack to work around this, but JEmplode got too confused by it (used wrong paths), so I weakened the algorithm some.

The tunes still get picked up (no worries), but you can re-run fidsift at any time to maintain the structure -- much faster than the initial run, too.

Cheers

Greetings!

Oh, I know that fidsift will grab them again - rerun fidsift after every upload. I am just wondering if it is somehow confusing the sync database build process itself.

I wonder where the timeout is happening

It isn't ftpd timing out it is the player software which is sitting there thinking it is in the middle of a sync wondering why nobody is talking to it (since we are uploading fids via ftp instead).

Technically the player doesn't have to be in sync mode at this point, but there are some special fids that must be uploaded through the player software and I wasn't going to rewrite jemplode to change the order that it uploads things to delay those.

-Mike

Bump. With Jemplode getting some serious tuneups lately, maybe it could be made to work with fidsift now?

EDIT: Or possibly having the option to run the program via a HiJack option?

Is fidift the new fid layout with _0000 style folders? If so, jEmplode should already upload into those folders. Someone submitted a patch for it a long time ago (mcomb maybe?? sorry whoever I'm dissing right now ). Right now if Use Hijack When Available is true and you have a /empeg/fids0/_00000 folder, then it should automatically upload into the appropriate directory. Is it not doing that or is fidift something different entirely?

Nope, that's what fidsift does! I must have not been around the boards when it was discovered, but it looks like Mark may have removed the function of fidsift to automagically run on syncs because of an older issue with Jemplode? Or maybe he has since reinstated it.

Quote:
Is it possible that the sync is timing out

Hey thanks to Brad for bumping this thread ... I don't think I noticed this post by Mike before. And yes, you are exactly right. I just did a test now that pc rebuild is back in and in a long rebuild the player IS restarting. I'm splitting your PingProgressListener code out so I can use the same method during a rebuild. Thanks!