I recently received a critical alert email from my FreeNAS box with the following error:
Device: /dev/ada3, Self-Test Log error count increased from 0 to 1
Device: /dev/ada3, 1 Currently unreadable (pending) sectors
Rather glad to know the email alerts I setup is working reliably, but looks as though I might have a few bad sectors on one of my drives.
The following commands resolved the error without resulting in any downtime.
The drive in question was /dev/ada3, so first login to a shell on your FreeNAS box as root and run a SMART long self-test (Replace adaX with your corresponding device).
smartctl -t long /dev/adaX
After the test has finished (It might take a few hours) view the results.
smartctl -a /dev/adaX
From the results remember the sector size and the location of the faulty sector (LBA_of_first_error).
In my case my sector size was 512 and LBA_of_first_error was 3082982872.
To correct the SMART error we will zero out the bad sector(s) on the drive, but first we need to permit access to drive.
sysctl kern.geom.debugflags=16
Now zero the sector stated in self-test results out.
Replace of=/dev/adaX, bs=512, seek=3082982872 with values relevant to your drive.
dd if=/dev/zero of=/dev/ada3 bs=512 count=1 seek=3082982872
Re-run the SMART report command to check if the ‘Current_Pending_Sector’ is now showing 0.
smartctl -a /dev/adaX
To check the ZFS file system integrity run a scrub of the pool, replace poolX with the pool name the drive is under (list pools with ‘zpool list’).
zpool scrub poolX
Finally check the output of the scrub to ensure there are no known data errors.
zpool status -v poolX