Emplode 2.0-b7 Too CPU Bound on Sync

I don't know if you'd classify this as a bug or not, but when syncing from an SMB mounted drive, Emplode uses 100% of the CPU and almost none of the network bandwidth. I figured it'd be just the opposite.

I'm using XP, everything over 100BT switch (except empeg on 10BT, of course). It's an 800mhz P3.

Just seems odd that it would need so much cpu for what would seem to be a primarily network based operation.

Also, we need a cancel on syncing ... though I doubt this is possible. It seems crappy that I have to kill the app whenever anything bad goes wrong while syncing up.

g

I originally was copying files from our server to my RioCar like this :

Server 100BT -> 100BT Switch -> 100BT card in my PC -> back out same 100BT card -> same switch -> 10BT RioCar

I too was getting basically 100% CPU. So I copied a large number to a local drive and synce from there. Same deal 100% CPU. Didn't seem to be a huge (if any at all) speed increase).

Just did a 260Meg transfer from my local drive in about 600 seconds =~ 430kb/sec

Using Bulletproof FTP thru Hijack I was getting around 700kb/sec. Any suggestions to get the speed up?

Running Win98 + 2.0b7 + Hijack v180.

FWIW using the Hijack FTP client I get pretty much spot on 500kb/sec (as longer as the player isn't playing) through a 10BT crossover cable at home.

The protocol is riddled with latency issues. These can eat CPU without using much network bandwidth. With Ethernet, the situation is improved, but there's still latency in certain places.

Also, did you have tracing turned on? It'd have warned you at startup if you had, so you probably didn't. If you did, there's a shedload of data going to disk, and that can introduce a bunch of CPU usage -- every line needs to be sprintf'ed, which eats CPU.

Anyway, I'll see if I can get the source profiler working, and see if there's anywhere obvious that emplode is spending all of its time.

Just as a quick comparison, JEmplode used comparitively little CPU time and was limited by the network throughput of SMB it seems.

Greg

2.0b11 with Emplode under WindowsXP uses 100% CPU while performing an fsck after hitting the max mount count. I am running Hijack v226 so just to be sure I will test with the original kernel.

On another note, any chance of emptool ever getting the code that fixes corrupt database issues? I gave up on emptool + linux after frequently getting into states that only emplode would fix. (This is generally after a sync fails, only emplode can rebuild the database without crashing.)

-Kit

...100% CPU while performing an fsck...

Yeah, I noticed that myself today. I'll take a quick look. Medium term, I plan on getting the source profiler to work, which should give me some hotspots to look at.

any chance of emptool ever getting the code that fixes corrupt database issues?

I don't know. If you have specific things that can cause it to crash, report them in the Bug Reports forum. They'll get looked at.

which should give me some hotspots to look at.

Had a quick look at this today, while attempting to lift some code for another project.

It's not CPU-bound -- it's sorta-kinda busy-waiting for a background thread. This will affect the speed, but not appreciably. I'll see if I can figure a way to remove the busy-loop. Don't hold your breath, though.

Thanks Roger - you rock!

FITNR.

If anybody cares for the technical details, check out http://www.differentpla.net/~roger/devel/background_threads/, in particular background #3 (what emplode used to do) and background #4 (what it does now).

The link isn't working right now?

The link isn't working right now?

Roger is some kind of muppet and has linked to our intranet webserver?

Peter

Yeah, what he said. Fixed.

Fascinating article, Roger. Gonna have to remember that one for when I need it later.

http://www.differentpla.net/~roger/devel/background_threads

3a. The observers called from the background thread notify an event, and the foreground thread's message loop sits in MsgWaitForMultipleObjects?

AFAICS, this is just like 3 but without the busy-waiting. Or have I missed something?

Peter

No, not really.