Hijack v490 and hdparm-8.6+

Hijack v490 is now released.

New in this version, is support for the SG_IO ioctl() call for use with SAT Passthru commands. This means that the empeg kernel now supports nearly all of the functionality of modern versions of hdparm.

In conjunction with the v490 release, I am attaching here a pre-built binary of hdparm-8.6+, with all functions enabled. This is a pre-release snapshot of the upcoming hdparm-8.7 code.

Nearly all hdparm functions should now work with Hijack v490 or later, including the --make-bad-sector and --repair-sector flags (tested), as well as all of the security functions (untested), such as --security-erase (aka. "low-level format").

Enjoy

As an example of "why you care", let's say that your player has been reporting a media error like this:

empeg:/empeg/bin# cat /dev/hda > /dev/null
hda: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
hda: read_intr: error=0x40 { UncorrectableError }, LBAsect=115459753, sector=115459753
end_request: I/O error, dev 03:00 (hda), sector 115459753
cat: /dev/hda: Input/output error

This bad sector can be fixed in-situ with this command:

hdparm --repair-sector 115459753 /dev/hda

---------------------------------------------------

Or perhaps you've used smartctl to check for problems, and seen a report such as this:

./smartctl.bin -l error /dev/hdc
smartctl version 5.33 ...
...
Error 0 occurred at disk power-on lifetime: 3887 hours (161 days + 23 hours)
When the command that caused the error occurred, the device was in a vendor specific state.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
51 01 01 00 00 43 a1 Error: UNC, IDNF, AMNF at LBA = 0x01430000 = 21168128

Commands leading to the command that caused the error were ...


The "UNC, IDNF, AMNF at LBA = 0x01430000 = 21168128" line indicates a bad sector (UNCorrectable, ID Not Found, Address Mark Not Found) at 21168128. which can now be repaired with this quick command:

hdparm --repair-sector 21168128 /dev/hdc


Enjoy.

Made of pure awesome.

FAQ entry on Smartctl has been updated to link this thread.

Is there any way of finding out which file sector 115,459,753 belonged to, so that it can be deleted or re-transferred? Like ioctl(FIBMAP) but in reverse? I'm sure I've used one at some point, but can't remember what it was called or even whether it was actually on Linux or not...

Peter

Hijack v491 is now available.

This version has two changes from v490:

-- Fixed version number: v490 was accidently labeled v499.
-- Supports long command timeouts for the new SG_IO SAT mechanism,
so that things like --security-erase won't time out early.

Cheers

No, not yet.

hdparm-8.7 will have built-in FIBMAP support when it is released, but even then it will not do reverse lookups.

I may write a program to try doing that though, which could be useful on empegs (but not likely on regular Linux boxes, because it would simply take too long to find the file).

EDIT: heck, just a shell script wrapper around hdparm-8.7 could do it rather easily, though slowly.

There's no reverse mapping mechanism, so it's basically a matter of just doing FIBMAP on each inode in turn, and comparing, until a match is found. One of the uglier aspects of filesystem forensics on Linux today.

Cheers

is there a way to have it fix all errors at once? or do one have to enter the command for each bad sector?

could be nice to have an "auto repair" option in hijack.. and just leave the player until it's done.. maybe some output on the display on the progress

A simple shell script wrapper is all you need -- have it run smartctl to get the error list, and then invoke hdparm --repair-sector once for each reported bad sector, then repeat until no more exist.

who can make such a script? i have both smartctl and hdparm in /tmp

Here you go, tested/fixed version is inline/attached here:

Code:

#!/bin/bash
export PATH="$PATH:/:/tmp"

## Before running this script, you must first have done these commands once:
##
##      smartctl -S on /dev/hdX
##      smartctl -t long /dev/hdX
##      ## wait for it to finish, according to the instructions it gives you


dev=$1
if [ ! -b "$dev" ]; then
        echo "$0: bad/missing device path parameter (eg. /dev/hda)"
        exit 1
fi

function mend_sector(){
        dev=$1
        lba=$2
        echo -n "$dev: Examining LBA $lba: "
        hdparm --read-sector $lba $dev &>/dev/null
        if [ $? -ne 5 ]; then
                echo "not bad."
        else
                echo -n "needs repair: "
                echo hdparm --yes-i-know-what-i-am-doing --repair-sector $lba $dev
                hdparm --yes-i-know-what-i-am-doing --repair-sector $lba $dev
        fi
}

function mend_bad_sectors(){
        dev=$1
        while [ 1 ]; do
                read lba
                [ "$lba" == "" ] && break
                [ $((lba)) -gt 0 ] && mend_sector $dev $lba
        done
}

## Fix currently known errors:
smartctl -l error $dev | grep 'Error:.*LBA = 0x[^ ]* = [0-9][0-9]*$' | sed -e"s/^.* //" | mend_bad_sectors $dev
smartctl -l selftest $dev | grep 'read failure ' | sed -e"s/^.* //" | mend_bad_sectors $dev

Just upload that file like any other executable, do the requisite chmod 755 command on it, and then try it out.

Cheers

thanks... i will give it a try in an hour, it's just doing the lon test on dev/hdc

Okay, good. That (the long test) is a requirement before running the script.

Also, after the long test completes, please post the output
of this command for us here: smartctl -a /dev/hdc

thanks.

the output:

smartctl version 5.33 [arm-empeg-linux-gnu] Copyright (C) 2002-4 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model: ST9808211A
Serial Number: 3LF3D3HA
Firmware Version: 3.05
User Capacity: 80,026,361,856 bytes
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 6
ATA Standard is: ATA/ATAPI-6 T13 1410D revision 2
Local Time is: Sat May 31 14:17:27 2008 /usr/local/armtools
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 121) The previous self-test completed having
the read element of the test failed.
Total time to complete Offline
data collection: ( 426) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off supp
ort.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
No General Purpose Logging support.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 84) minutes.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_
FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x0a0f 060 042 034 Pre-fail Always -
151238706
3 Spin_Up_Time 0x0003 095 094 000 Pre-fail Always -
0
4 Start_Stop_Count 0x0032 099 099 020 Old_age Always -
1421
5 Reallocated_Sector_Ct 0x0033 071 071 036 Pre-fail Always -
1195
7 Seek_Error_Rate 0x000f 079 060 030 Pre-fail Always -
94641337
9 Power_On_Hours 0x0032 096 096 000 Old_age Always -
3727
10 Spin_Retry_Count 0x0013 100 100 034 Pre-fail Always -
0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always -
635
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always -
635
193 Load_Cycle_Count 0x0032 090 090 000 Old_age Always -
20269
194 Temperature_Celsius 0x0022 031 066 000 Old_age Always -
31 (Lifetime Min/Max 0/2)
195 Hardware_ECC_Recovered 0x001a 060 042 000 Old_age Always -
151238706
197 Current_Pending_Sector 0x0012 002 001 000 Old_age Always -
1918
198 Offline_Uncorrectable 0x0010 002 001 000 Old_age Offline -
1918
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always -
0
200 Multi_Zone_Error_Rate 0x0000 100 253 000 Old_age Offline -
0
202 TA_Increase_Count 0x0032 100 253 000 Old_age Always -
0

SMART Error Log Version: 1
Warning: ATA error count 0 inconsistent with error log pointer 4

ATA Error Count: 0
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 0 occurred at disk power-on lifetime: 3887 hours (161 days + 23 hours)
When the command that caused the error occurred, the device was in a vendor sp
ecific state.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
51 01 01 00 00 43 a1 Error: UNC, IDNF, AMNF at LBA = 0x01430000 = 21168128

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
40 20 fe f9 6f 00 00 f6 12d+10:16:25.350 READ VERIFY SECTOR(S)
00 20 fd f9 6f 00 00 f6 00:00:43.016 NOP [Reserved subcommand]
00 20 fb f9 6f 00 00 f6 00:00:40.968 NOP [Reserved subcommand]
00 20 f9 f9 6f 00 00 f6 00:00:38.920 NOP [Reserved subcommand]
00 20 f7 f9 6f 00 00 f6 00:00:36.872 NOP [Reserved subcommand]

Error -1 occurred at disk power-on lifetime: 34824 hours (1451 days + 0 hours)
When the command that caused the error occurred, the device was in an unknown
state.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
19 43 f7 51 01 00 00 Error:

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
00 19 f7 20 0a 76 0d 2f 00:14:28.352 NOP [Reserved subcommand]
00 18 f7 20 ce 74 0d ef 00:14:11.968 NOP [Reserved subcommand]
00 18 f7 20 97 74 0d af 00:14:11.968 NOP [Reserved subcommand]
00 18 f7 20 8e 74 0d 6f 00:14:11.968 NOP [Reserved subcommand]
00 18 f7 20 31 98 0d 2f 00:14:11.968 NOP [Reserved subcommand]

Error -2 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was in a vendor sp
ecific state.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
d0 00 6f 16 f7 51 01

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
76 cf 6f 16 f7 20 0a 02 22d+22:06:06.861 SEEK [RET-4]
74 cf 2f 16 f7 20 ce 02 22d+16:20:57.485 SEEK [RET-4]
74 cf ef 15 f7 20 97 02 22d+15:20:53.005 SEEK [RET-4]
74 cf af 15 f7 20 8e 02 22d+15:11:03.181 SEEK [RET-4]
74 cf 6f 15 f7 20 87 02 22d+15:03:24.429 SEEK [RET-4]

Error -3 occurred at disk power-on lifetime: 3721 hours (155 days + 1 hours)
When the command that caused the error occurred, the device was active or idle
.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 d0 ef 12 f7 Error: UNC at LBA = 0x0712efd0 = 118681552

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
20 00 02 cf ef 12 f7 00 6d+09:37:51.899 READ SECTOR(S)
20 00 02 cf af 12 f7 00 6d+09:37:51.888 READ SECTOR(S)
20 00 02 cf 6f 12 f7 00 6d+09:37:51.883 READ SECTOR(S)
20 00 02 cf 2f 12 f7 00 6d+09:37:51.879 READ SECTOR(S)
20 00 02 cf ef 11 f7 00 6d+09:37:51.873 READ SECTOR(S)

Error -4 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was in an unknown
state.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
00 00 00 43 a1 01 00

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
d0 00 0d 00 00 40 00 8a 40d+09:21:05.071 [RESERVED]
cf 00 0d 00 00 00 02 81 40d+04:43:38.927 [RESERVED]
cf 00 0d 00 00 00 02 76 40d+04:43:38.863 [RESERVED]
cf 00 0d 00 00 00 02 6f 40d+04:43:38.799 [RESERVED]
cf ff 0c 00 00 00 02 1e 40d+04:43:38.735 [RESERVED]

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA
_of_first_error
# 1 Extended offline Completed: read failure 90% 3726 101
617328
# 2 Short offline Completed: read failure 90% 3337 101
517771
# 3 Short offline Completed: read failure 90% 3337 101
517771
# 4 Short offline Interrupted (host reset) 90% 3337 -
# 5 Short offline Completed: read failure 90% 3336 176
31400
# 6 Short offline Completed: read failure 90% 3106 101
458623

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

empeg:/tmp#

the script says:

bad/missing device path parameter (eg. /dev/hda


have to go to work, but back in like 12 hours

Sorry, but I really do need to see the output without wrap-around mangling of the lines!!!

Thanks.

what about the script error?

the output was from hyberterminal.. if i do the command in putty it freezes

i have tried to figure out how to disable the "wrap" thing.. but no luck

Nevermind.. I found some sample output elsewhere with google.

New version of the script is attached to this posting.

still same error from script

You need to tell it what drive you're running it on.

thanks

i have placed both smartctl and hdparm in /tmp

but the scripts says that it cand find smartctl...

i have tried to change smartctl in the script to /tmp/smartctl.bin with no luck so far

fixed... i changed it to: ./smartctl.bin

the attached script are corrected and requires:

smartctl.bin , hdparm.bin and the script are located in /tmp and are chmodded to 755

./smartctl -S on /dev/hdX and
./smartctl -t long /dev/hdX

must be run before the script (replace X with either a or c if you have two drives)

so to repair a two drive setup, connect via serial and:

1. hit ctrl+c
2. change dir to /tmp
3. enter these commands in sequence:

./smartctl -S on /dev/hda
./smartctl -t long /dev/hda
./smartctl -S on /dev/hdc
./smartctl -t long /dev/hdc
./fix_bad_sectors.sh /dev/hda
./fix_bad_sectors.sh /dev/hdc

(remeber to see how long it takes to complete each of the long tests before you move on to the next command)

No, that's just wrong. smartctl should be named smartctl, not smartctl.bin. Ditto for hdparm.

The .bin is there on the original files simply so that they don't get mangled when downloading on non-Linux systems, and to make it obvious that they are supposed to be binary (not text) files to browsers and the like.

I generally rely upon the receiving gray matter to rename them properly.

So folks, get rid of the .bin parts of the filenames when installing them. And ignore the corrupted script from that last posting by Boelle.

-ml

so i should just rename them to smartctl and hdparm?

Is there a particular reason why you keep persisting with this drive? Get it replaced...

yes there is: marks script told me that there where no bad sectors... just have to cut and paste the output

empeg:/tmp# ./56062.sh /dev/hda
empeg:/tmp# ./56062.sh /dev/hdc
/dev/hdc: Examining LBA 21168128: not bad.
/dev/hdc: Examining LBA 118681552: not bad.
/dev/hdc: Examining LBA 101618857: not bad.
/dev/hdc: Examining LBA 101617328: not bad.
/dev/hdc: Examining LBA 101517771: not bad.
/dev/hdc: Examining LBA 101517771: not bad.
/dev/hdc: Examining LBA 17631400: not bad.
/dev/hdc: Examining LBA 101458623: not bad.
empeg:/tmp#

Did the script repair those for you on an earlier run?
It must have, because they *were* bad.

Just to make sure you haven't completely busted the working script I posted earlier, try this command:

hdparm --read-sector 101517771 /dev/hdc

Duh, like, YES.

just to be as sure as a "monkey" im doing another diskbuild and then i will use your script as provided, it was just me that was a bit confused about the renameing of the files smartctl and hdparm

hey, i'm just a beginner.... and slow at learning (as a monkey) :-)

On a (slight) tangent: is it worth me grabbing some 2.5" PATA drives now, while they're still available?

I've got two empegs, and both seem to be fine, but I don't want to be caught out when the disks die, and there are no replacements available.

That's when you replace them with CompactFlash cards, assuming those are still available. I think they're a safe bet yet for the next 3 years, at least.

Cheers

*wanders off to check the prices on 64GB CF cards again*

Version 8.7 of hdparm is now available from Sourceforge.net.

A binary built for the empeg is attached to this posting.

cheers

after another diskbuild and loading ver. 3 alpha 11 + newest hijack, i did these commands (i used the script mark made, not the "screwed up" one i did post here):

./smartctl -S on /dev/hda
./smartctl -t long /dev/hda
./smartctl -S on /dev/hdc
./smartctl -t long /dev/hdc
./fix_bad_sectors.sh /dev/hda
./fix_bad_sectors.sh /dev/hdc

the output is still the same:

empeg:/tmp# ./fix_bad_sectors.sh /dev/hda
empeg:/tmp# ./fix_bad_sectors.sh /dev/hdc
/dev/hdc: Examining LBA 21168128: not bad.
/dev/hdc: Examining LBA 118681552: not bad.
/dev/hdc: Examining LBA 101619881: not bad.
/dev/hdc: Examining LBA 101618857: not bad.
/dev/hdc: Examining LBA 101617328: not bad.
/dev/hdc: Examining LBA 101517771: not bad.
/dev/hdc: Examining LBA 101517771: not bad.
/dev/hdc: Examining LBA 17631400: not bad.
/dev/hdc: Examining LBA 101458623: not bad.
empeg:/tmp#

i will give it another go with the new hdparm, but i dont know how i do delete the one i have on the player...

but doing hdparm --read-sector ******** /dev/hdc replacing ****** with the lba numbers listed only returns that reading those sectors is succeeded

Okay, the drive is good, for now.

And there's no need to run it again with the newer hdparm -- the features you are using have not changed in the new version.

Cheers

a big thanks for all the help, one last note is that the display shows: pump hda5 648:2 during boot (just before it says loading visuals etc.)

Oh.. Yeah, I meant to get rid of that message from Hijack.
New version coming soon.

Cheers

is there a chance for a diskcheck / repair thing in hijack? so one just choose that in the menu? maybe along with some progrss info on the display? of course it should only be there is it is in public interest and not just for a single person

Too big/complex for hijack itself, though anyone who wants to could write a script/program to call the fsck program from the Hijack menu -- it's user-extensible, you know.

yep, but what good would come out if let a "monkey" write a script?

Well, if the monkey were to write it, post it here with questions and corrections, and repeat until done, I'd say you'd get a more knowledgeable monkey, which would be good, no?

i was just trying to make a joke on myself. It's me that is the monkey

That fact wasn't lost on me. I'm just encouraging you to evolve.

As it turns out, hdparm-8.7 had broken -r and -a flags, so here is hdparm-8.8 (for the empeg) with those issues fixed.

Cheers