Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Page 1 of 2 1 2 >
Topic Options
#252322 - 22/03/2005 21:06 Dead Mk2?
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Heh, did that thread title, combined with this user name, get your attention?

Thought I'd cash in on Meatballman's other thread popularity...

My player isn't dead, it's not even very ill. At least I don't think it is. But it's done something strange a few times recently.

The player app freezes up mid-song. The visuals freeze and the music stops. Almost as if someone had pressed the pause key, but without the on-screen indication of a pause.

But pressing and holding the top button still puts it into standby mode (so I can pull it from the dash and reboot it without a speaker pop). After the reboot, all works well and it even seems to remember which song it was on and remember the correct point to continue playing again.

Details:

It only started doing this recently, I made no changes to the player between the time it worked fine and the time it started doing this.

I didn't add any bad songs to the player, as far as I know. The fact that it works fine after a reboot seems to indicate it's not a bad song (if it were a bad song it'd re-cache the same songs after a reboot).

It only seems to do it after a certain amount of time playing in the car. 30-60 minutes or so.

It does not seem to be temperature-related, it still happens when I've got the cooling fan turned on. When I pull it from the sled, it doesn't feel unusually hot (it's been much hotter and ran fine).

Once, it did it while GPSApp was running in the foreground. GPSApp continued to run for quite a while, but eventually it also froze.

It does not do this on AC power, only DC power in the car. So I am unable to monitor its serial port output during the failure.

It does not do this while sitting on the passenger seat and playing from a cig-lighter adapter into a tape-input adapter in a 4wd pickup truck on a bumpy ride up to the slopes.

I tried adding [hijack] stalk_enabled=0 to the config.ini, no fix to the behavior.

I checked for the "missed interrupts" message on the serial port when it was on home power, no missed interrupts reported.

I have not seen any indications of hard disk trouble. I mean, other than this if you count this as an indication.

Software version is 2.0 final with Hijack 413.

Any other ideas?
_________________________
Tony Fabris

Top
#252323 - 22/03/2005 21:13 Re: Dead Mk2? [Re: tfabris]
pgrzelak
carpal tunnel

Registered: 15/08/2000
Posts: 4859
Loc: New Jersey, USA
Can you force DC power using one of Mark's docks and watch the serial port there? The fact that it does not do it on AC, even when on the seat, makes me think that it is sled / wiring related. Is your sled easy to remove, to look for loose wires on the loom (especially the serial link ones)?

Also, if not already there, turn on the HDD indicator, to see if you are getting any starved thread conditions.

Additional: Any hardware modifications? Especially on the memory end of things. Does your memory test run cleanly?


Edited by pgrzelak (22/03/2005 21:16)
_________________________
Paul Grzelak
200GB with 48MB RAM, Illuminated Buttons and Digital Outputs

Top
#252324 - 22/03/2005 21:23 Re: Dead Mk2? [Re: tfabris]
SE_Sport_Driver
carpal tunnel

Registered: 05/01/2001
Posts: 4903
Loc: Detroit, MI USA
Jiffies.
_________________________
Brad B.

Top
#252325 - 22/03/2005 21:30 Re: Dead Mk2? [Re: pgrzelak]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Good ideas, Paul, I'll try those.

No hardware mods to the memory, but I've got the lit-button hack and I've added a fan which is running off the 3v rail.
_________________________
Tony Fabris

Top
#252326 - 22/03/2005 21:38 Re: Dead Mk2? [Re: tfabris]
pgrzelak
carpal tunnel

Registered: 15/08/2000
Posts: 4859
Loc: New Jersey, USA
In a way, it almost sounds like a memory leak, building until the player app can no longer allocate and crashes. That would explain why the GPSapp would still run for a while but eventually crash too. No explaination of why only on DC power, though. Unless there is something in your config.ini tied to DC vs. AC. I am also thinking it might be a bad memory hit, but (again) no explaination why only on DC.

I really wish you could capture a console log. It really seems like the player app is going down - hard. Or getting blocked by a resource request that cannot be met.

Does it happen with a vanilla config.ini?
_________________________
Paul Grzelak
200GB with 48MB RAM, Illuminated Buttons and Digital Outputs

Top
#252327 - 22/03/2005 22:26 Re: Dead Mk2? [Re: pgrzelak]
mcomb
pooh-bah

Registered: 31/08/1999
Posts: 1649
Loc: San Carlos, CA
Quote:
In a way, it almost sounds like a memory leak


Thats the same thought I had. Maybe it only happens on DC because Tony only has GPSapp running on DC and GPSapp is leaking mem? Or the player is at fault, but it only shows up on DC because there is more spare memory running on AC without GPSapp?

-Mike
_________________________
EmpMenuX - ext3 filesystem - Empeg iTunes integration

Top
#252328 - 22/03/2005 22:32 Re: Dead Mk2? [Re: mcomb]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
I have "Launcher" running on DC, and choose GPSapp or emphatic or whatever as needed. So they don't take up RAM when not running. But it's happened without those things running before.

Right now, I'm running it forced-DC with a serial terminal open, not happening yet. Will try other suggested diagnostics later if this one doesn't pan out.
_________________________
Tony Fabris

Top
#252329 - 23/03/2005 13:50 Re: Dead Mk2? [Re: tfabris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14478
Loc: Canada
Standby for a new "Tony Edition" general release of Hijack.

Cheers

Top
#252330 - 25/03/2005 01:38 Re: Dead Mk2? [Re: mlord]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Wow, cool, Mark, thanks!

I have no idea what you're planning, but I guess you've got some ideas?

More information on this topic came out the last couple of days:

- I had it happen when running in AC mode once. While it was connected to a cigarette lighter 12v adapter riding on the passenger seat of the pickup truck on its way to the slopes. So DC power is not the factor I originally thought it would be.

- I noticed that the player app does not actually freeze. All that freezes is the playback of the song, but with no pause indicator. I can open the player's main menu and futz around with it. Pressing pause/unpause and hush/unhush on the remote does nothing.

Does any of that help?
_________________________
Tony Fabris

Top
#252331 - 25/03/2005 01:45 Re: Dead Mk2? [Re: tfabris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14478
Loc: Canada
Hijack v419 has the "Tony features" you need, I think.

cheers

Top
#252332 - 25/03/2005 01:51 Re: Dead Mk2? [Re: mlord]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
/me grabs anxiously...

Thank you!!!!!

In case it really is a hard disk fault, is one of the popups a hard disk trouble indicator? Version note just said "various" hard/soft faults.
_________________________
Tony Fabris

Top
#252333 - 25/03/2005 01:56 Re: Dead Mk2? [Re: tfabris]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
I see my question was answered in an adjacent thread. THANK YOU!
_________________________
Tony Fabris

Top
#252334 - 30/03/2005 19:25 Re: Dead Mk2? [Re: tfabris]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
I haven't seen the problem happen since the new kernel was installed.

It's a weird feeling, actually HOPING for a hardware flaw, just so I can see a new message.
_________________________
Tony Fabris

Top
#252335 - 05/04/2005 12:37 Re: Dead Mk2? [Re: tfabris]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Interesting follow-up.

I had the problem happen again recently. I was driving home, talking on the cell phone, and I'd had the player in standby mode. The player had booted into standby mode, since it had been in standby mode in the office.

I finished the phone conversation, woke the player, and it started playing some music. After only a few bars of the song, it did that freeze. Visuals froze, music stopped. Note that the time-dither effect of the visuals was still running, but the visuals didn't "move".

No error messages on the screen at all.

It's possible that the error happened when it was trying to fill the cache, and therefore it happened when the player was booted in standby mode, and therefore not visible on the deactivated VFD display. Sigh.

I tried something, too. I tried pressing down-down-down to get it to reshuffle a new playlist, and it gave me the hourglass and then locked up the user interface. I couldn't press/hold the top button to put it into standby any more. However when I got home and shut off the ignition, it correctly went into standby. When I plugged the player back in indoors, it resumed the playlist it had been playing prior to my down-down-down attempt. (In other words, the down-down-down thing did nothing at all to it.)
_________________________
Tony Fabris

Top
#252336 - 05/04/2005 21:28 Re: Dead Mk2? [Re: tfabris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14478
Loc: Canada
Mmm.. if it's a disk error, the message will not appear until after the read's timeout and fail mutiple tries (lousy error handling for this situation, I know..).

I can't think of anything else it might be, but..

Top
#252337 - 10/04/2005 16:12 Re: Dead Mk2? [Re: mlord]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Okay, here's something fun and interesting.

The player was playing a song. The song played fine all the way through.

Just before the end of the song, I decided I wanted to hear it a second time. So I pressed the "Left arrow" button on the faceplate. The song started over from the beginning as expected.

Then, about two bars into the second replay of the song, it did the same freeze. No error messages on the screen. At any time, even prior to this happening.

Now, here's my thinking. If this had been hard disk trouble and the freeze happens when it tries to play a song that didn't get fully cached, you'd think that this song would already be in the player's cache, right? Since I already heard the whole song and had simply started it over.

So I'm wondering if we're still looking at disk trouble here? Or maybe something else?
_________________________
Tony Fabris

Top
#252338 - 10/04/2005 16:39 Re: Dead Mk2? [Re: tfabris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14478
Loc: Canada
Dunno. The player s/w tries to handle caching internally, and exactly how that works is a mystery to anyone without access to the source code. It would make sense for them to toss the data after it's been sent to the audio jacks.

Top
#252339 - 10/04/2005 17:33 Re: Dead Mk2? [Re: tfabris]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4172
Loc: Cambridge, England
Quote:
Now, here's my thinking. If this had been hard disk trouble and the freeze happens when it tries to play a song that didn't get fully cached, you'd think that this song would already be in the player's cache, right?

Only if you are certain that the cache was not refilled at any point during the song's first play. If it had been, already-played blocks except the first few would have been discarded.

Peter

Top
#252340 - 10/04/2005 22:08 Re: Dead Mk2? [Re: peter]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Ah, so I'm misunderstanding the way the cache works. It doesn't always keep the current song in memory. If it does read-ahead during the song, then it makes room by discarding already-played portions of the current song. Makes sense.

So OK, we're still looking at hard disk trouble possibly then?
_________________________
Tony Fabris

Top
#252341 - 11/04/2005 00:37 Re: Dead Mk2? [Re: tfabris]
SonicSnoop
addict

Registered: 29/06/2002
Posts: 531
Loc: Triangle, VA
Have another drive to just throw in and see if it still does it?
_________________________
-D Modifying and Tweaking is a journey, not a destination................................ MKIIa : 60gig - 040103286 - Blue - v2 + PCATS tuner MKIIa : 20gig - 040103260 - Blue - v3a8 + Mark Lord Special Edition Cherry Dock

Top
#252342 - 11/04/2005 00:50 Re: Dead Mk2? [Re: SonicSnoop]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Good question, good idea. I don't have an extra drive with sufficient capacity (40gb) to trade into it. The freeze happens so intermittently that I might go a week or more before seeing it, so I also don't want to go down to one of my 10gb spares because that would leave me with a paltry subset of my true collection.

But I wonder if there is a tool, either on the player or that I could add to the player, that will nondestructively test the hard disk sectors?

I know so little about the linux tool set, for all I know the right parameters to FSCK would do it, and I'd have no idea.
_________________________
Tony Fabris

Top
#252343 - 11/04/2005 06:32 Re: Dead Mk2? [Re: tfabris]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
Quote:
But I wonder if there is a tool, either on the player or that I could add to the player, that will nondestructively test the hard disk sectors?


Assuming that the fsck program on the player supports it, you can pass the '-c' switch to fsck.ext2, which will cause a read-only block test. If you pass it twice, you get a non-destructive read-then-write test.
_________________________
-- roger

Top
#252344 - 11/04/2005 10:26 Re: Dead Mk2? [Re: Roger]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4172
Loc: Cambridge, England
Quote:
Assuming that the fsck program on the player supports it, you can pass the '-c' switch to fsck.ext2, which will cause a read-only block test. If you pass it twice, you get a non-destructive read-then-write test.

Fsck does this by running the "badblocks" program, which you can also run directly. But badblocks looks for individual sectors with hard errors, which is not the problem you have -- or at least, not the main problem. Your experience with the song that wouldn't re-play says that a sector which read right the first time, failed the second time. Running badblocks on AC power won't diagnose whatever it is (heat? vibration? bad solder joint?) that makes the whole drive sporadically cease to exist (from the player's point of view) when in the car.

Peter

Top
#252345 - 11/04/2005 10:54 Re: Dead Mk2? [Re: peter]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14478
Loc: Canada
Quote:
badblocks looks for individual sectors with hard errors, which is not the problem you have -- or at least, not the main problem. Your experience with the song that wouldn't re-play says that a sector which read right the first time, failed the second time.


That's really hard to know for sure. It could easily be a bad sector in the current *physical* track that's mucking things up, outside of the sectors actually needed to play the song.

A bad block scan is what needs to be done. To begin with, that could be as simple as "cat /dev/hda >/dev/null" -- and then sit back and wait for 14 hours.

Cheers

Top
#252346 - 11/04/2005 13:03 Re: Dead Mk2? [Re: mlord]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Quote:
To begin with, that could be as simple as "cat /dev/hda >/dev/null" -- and then sit back and wait for 14 hours.

Cool, I'll give this a shot. Would it be just as effective if I do it without mounting the drives Read Write?
_________________________
Tony Fabris

Top
#252347 - 11/04/2005 13:04 Re: Dead Mk2? [Re: tfabris]
pgrzelak
carpal tunnel

Registered: 15/08/2000
Posts: 4859
Loc: New Jersey, USA
It should be. It is a read check, and this will go through the entire drive as a device file instead of as a filesystem, so it should not matter.
_________________________
Paul Grzelak
200GB with 48MB RAM, Illuminated Buttons and Digital Outputs

Top
#252348 - 11/04/2005 13:23 Re: Dead Mk2? [Re: mlord]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4172
Loc: Cambridge, England
Quote:
That's really hard to know for sure. It could easily be a bad sector in the current *physical* track that's mucking things up, outside of the sectors actually needed to play the song.

But surely it would have to be an intermittently bad sector, or it would have failed the first time it played the track too? For a song to get a few seconds in and then stop, it must have blocked the audio thread, which means a read() of the next 64K of MP3 data -- exactly the same read which had been issued the first time the track was cached -- has blocked. But yeah, a bad-blocks scan will at least help eliminate some types of disk failure as suspects. If we're lucky it'll show up heat-related problems too.

Peter

Top
#252349 - 11/04/2005 13:54 Re: Dead Mk2? [Re: peter]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
So if I did the "cat" to dev/null command, and there was something wrong, would I expect to see error messages at the console?

Because I'd be doing it with the player software turned off, so the VFD wouldn't be showing, so I wouldn't see hijack hardware error messages (if any. But I've never seen hijack messages on this player yet anyway).
_________________________
Tony Fabris

Top
#252350 - 11/04/2005 14:04 Re: Dead Mk2? [Re: tfabris]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4172
Loc: Cambridge, England
Quote:
So if I did the "cat" to dev/null command, and there was something wrong, would I expect to see error messages at the console?

I think so, yes. Most "disk going away" error conditions end up with a "lost interrupt" message going to the console. There shouldn't be any that retry indefinitely.

Peter

Top
#252351 - 12/04/2005 00:18 Re: Dead Mk2? [Re: tfabris]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14478
Loc: Canada
Yuh, you'll see "read error" messages for every bad sector, on the serial port.

I think my driver will retry each error a few times (3?) before giving up and going on to the next.

Cheers

Top
Page 1 of 2 1 2 >