I have an ubuntu 22.04 system on hetzner cloud arm machines that keeps experiencing zfs errors, and I am a bit stumped as to what is the problem.
zpool status -v pool: encpool state: ONLINEstatus: One or more devices has experienced an error resulting in data corruption. Applications may be affected.action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A scan: scrub repaired 0B in 00:02:09 with 0 errors on Sun Feb 4 20:53:31 2024config: NAME STATE READ WRITE CKSUM encpool ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdd ONLINE 0 0 0 sda ONLINE 0 0 0errors: Permanent errors have been detected in the following files:<0xe16d>:<0x0>It usually shows up as above. I did zfs scrub encpool and when it completed first time it was showing the same error. Then I reran it and now it shows as OK:
zpool status -v pool: encpool state: ONLINE scan: scrub repaired 0B in 00:01:56 with 0 errors on Sun Feb 4 21:02:48 2024config: NAME STATE READ WRITE CKSUM encpool ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdd ONLINE 0 0 0 sda ONLINE 0 0 0errors: No known data errorsThis has now happened a few times already. scrub sometimes helps immediately, sometimes after 2 runs.
sdx disks are hetzner cloud volumes, which should themselves be redundant ( every block of data is stored on three different physical servers ).encpool is encrypted and compression is turned on, but not dedup. it's configured as raidz1.
zpool list -vNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTencpool 59.5G 48.1G 11.4G - - 53% 80% 1.00x ONLINE - raidz1-0 59.5G 48.1G 11.4G - - 53% 80.8% - ONLINE sdb - - - - - - - - ONLINE sdd - - - - - - - - ONLINE sda - - - - - - - - ONLINEzfs versionzfs-2.1.5-1ubuntu6~22.04.2zfs-kmod-2.1.5-1ubuntu6~22.04.1