When I first started tinkering with FreeNAS I installed it on a spare MicroServer just to try it out. The system only had 4GB of RAM installed, so I opted to create a UFS volume over a ZFS volume, as the latter requires at least 8GB of RAM according to the system requirements.
After some success with getting the UFS volume to play nicely with Time Machine on OS X, I took the plunge and ordered a set of Kingston 16GB (2x8GB) DDR3 1333Mhz with ECC support for £119.56 at the time of writing, the link for Amazon UK can be found here.
After the RAM arrived, I installed into the system and destroyed the old UFS volume. Suddenly I was hit with an issue, any subsequent attempts of attempting to create a fresh ZFS volume would result in failure. Several disk wipes, reboots and volume creation attempts later I was left scratching my head. My gut feeling was telling me that FreeNAS was struggling to wipe the disks (Also checking the logs helped). After googling I was able to wipe the disks after entering sysctl kern.geom.debugflags=0x10.. Little did I know this would be the command that would cause my future problems. After the command executed I was able to wipe the disks and create my new ZFS volume.
This ran for about 4 weeks perfectly, until I upgraded the system from 9.2.1.5 to 9.2.1.6. After the upgrade the system rebooted and I was suddenly presented with a amber alert status. Also my ZFS volume had disappeared, and dsmeg was spitting out errors about my mirror being corrupt (see below).
freenas kernel: GEOM: ada1: the secondary GPT table is corrupt or invalid.
freenas kernel: GEOM: ada1: using the primary only — recovery suggested.
freenas kernel: GEOM: ada2: the secondary GPT table is corrupt or invalid.
freenas kernel: GEOM: ada2: using the primary only — recovery suggested.
freenas kernel: GEOM_MIRROR: Device mirror/system launched (2/2).
freenas kernel: GEOM: mirror/system: corrupt or invalid GPT detected.
freenas kernel: GEOM: mirror/system: GPT rejected — may not be recoverable.
After a few hours goggling I determined it wasn’t anything with the disks that had failed, whilst numerous forum posts pointed to that. Instead it was linked to how I overcame the issue of wiping the disks whilst attempting to delete my old UFS volume. Turns out I hadn’t rebooted the FreeNAS system upon creating the ZFS volume, hence the firmware upgrade had nothing to do with my problems. Instead it was a disk mounting issue that was persistent since the very beginning, I just wasn’t aware of it until that moment.
I deleted the ZFS volume using the FreeNAS GUI and attempted to create a new one in its place, which repeatedly kept failing with no success. In desperation I also attempted to create UFS volumes, but that also resulted in similar errors and failure.
This post significantly helped me with figuring out what was going on. After running gmirror forget, clear and remove I was able to fix my disk creation issue.
To ensure I didn’t run into the original issue I followed information from this post to ensure my disks were wiped correctly.
Finally I was successfully able to create a fresh ZFS volume and start afresh. This time making sure the volume was able to be successfully mounted after a reboot.