TrueNAS Scale data recovery

My TrueNAS Scale server recently started throwing the following zfs module kernel panic 1 on boot:

PANIC at space_map.c:405:space_map_load_callback()

What follows are the steps I took to retrieve the data from the corrupted zfs pool.

Getting setup

$ apt install zfsutils-linux
$ zpool import -o readonly=on -f Seagate4TB -R /mnt

Pool file systems are now available readonly under /mnt. In my case, I had a file system that I was using for NFS storage. Volumes have to be accessed a different way - see below.

Using a new pool

In my case, the data I really needed was inside a zvol which had an XFS filesystem on it. Attempting to mount that directly from the readonly pool resulted in an XFS error due to it being read-only. The solution was to copy the zvol to a new pool.

The new pool needs to reside somewhere with sufficient free space to hold data exported from the readonly pool. I used a spare external USB hard disk I had lying around. Instead of using the USB disk as a block device for the pool, I created a sparse file on the USB disk and used that for the pool’s storage.

$ dd if=/dev/zero of=/external-usb/testpool bs=1 count=0 seek=100G
$ zpool create testpool /external-usb/testpool
$ zfs send Seagate4TB/Fedora-home | zfs receive testpool/Fedora-home
mount /dev/zvol/testpool/Fedora-home /mnt2