I number of months back I switch to `zfs` and `syncoid`. I've just discovered that the backups I had set are out of date, even though I was under the impression I had configured `syncoid` correctly.
I was hoping someone could give me some insight into what's going wrong here.
Firstly, here's my configurations:
The production machine running applications:
[storage/services] frequently = 0 hourly = 12 daily = 7 monthly = 3 yearly = 0 recursive = yes autosnap = yes # autosnap based on the policy above autoprune = yes # autoprune to delete old backups outside this policyThe backup machine for just mirroring/backups:
[storage/backups] frequently = 0 hourly = 0 daily = 7 monthly = 3 yearly = 0 recursive = yes autosnap = no # autosnap not needed, inherited from the back up source autoprune = yes # do delete old snapshots stored here based on policy aboveThe syncoid command is then ran via a cronjob from production to the backup:
su zfs-send -c "\ syncoid \ --recursive \ --no-sync-snap \ --create-bookmark \ --no-rollback \ --no-privilege-elevation \ storage/services \ zfs-recv@10.3.14.223:storage/backups/node24/services"I also make use of manual snapshots on the production machine when performing updates to services. So Before I run a version update, I will create a manual snapshot to revert to, if it goes badly. I have a feeling this might be what's causing the issue.
As of now here are the snapshots I can see under a particular dataset on production:
storage/services/daemon@init 1.46G - 65.3G -storage/services/daemon@v1.6.21-to-v1.6.31 2.27G - 66.3G -storage/services/daemon@v1.6.31-to-v1.6.32 3.49G - 39.4G -storage/services/daemon@v1.6.32-to-v1.6.33 1.24G - 37.0G -storage/services/daemon@pre-manual-digging-around 1.23G - 37.9G -storage/services/daemon@1.6.33-to-1.6.35 1.03G - 37.9G -storage/services/daemon@1.6.33-to-1.6.36 13.5M - 37.9G -storage/services/daemon@1.6.33-to-1.6.37 13.7M - 37.9G -storage/services/daemon@v1.6.33-to-v1.6.38 1.24G - 37.9G -storage/services/daemon@v1.6.33-to-v1.6.39 985M - 37.9G -storage/services/daemon@autosnap_2024-01-01_00:00:01_monthly 129M - 41.0G -storage/services/daemon@autosnap_2024-02-01_00:00:01_monthly 318M - 57.1G -storage/services/daemon@v1.6.39-to-v1.6.42 9.86G - 68.8G -storage/services/daemon@autosnap_2024-03-01_00:00:15_monthly 1.31G - 41.0G -storage/services/daemon@v1.6.42-to-v1.6.49 1.31G - 460G -storage/services/daemon@autosnap_2024-03-20_00:00:01_daily 1.29G - 460G -storage/services/daemon@v1.6.42-to-v1.6.50 288M - 464G -storage/services/daemon@autosnap_2024-03-21_00:00:02_daily 15.6M - 464G -storage/services/daemon@autosnap_2024-03-22_00:00:02_daily 92.3M - 468G -storage/services/daemon@autosnap_2024-03-23_00:00:01_daily 271M - 474G -storage/services/daemon@v1.6.41-to-v1.6.51 1.60G - 474G -storage/services/daemon@v1.6.42-to-v1.6.52 13.7M - 474G -storage/services/daemon@autosnap_2024-03-24_13:45:13_daily 852K - 474G -storage/services/daemon@autosnap_2024-03-25_00:00:01_daily 810K - 474G -storage/services/daemon@autosnap_2024-03-25_23:00:01_hourly 1.42M - 474G -storage/services/daemon@autosnap_2024-03-26_00:00:01_daily 0B - 474G -storage/services/daemon@autosnap_2024-03-26_00:00:01_hourly 0B - 474G -storage/services/daemon@autosnap_2024-03-26_01:00:15_hourly 916K - 474G -storage/services/daemon@autosnap_2024-03-26_02:00:15_hourly 938K - 474G -storage/services/daemon@autosnap_2024-03-26_03:00:15_hourly 938K - 474G -storage/services/daemon@autosnap_2024-03-26_04:00:01_hourly 916K - 474G -storage/services/daemon@autosnap_2024-03-26_05:00:01_hourly 916K - 474G -storage/services/daemon@autosnap_2024-03-26_06:00:01_hourly 884K - 474G -storage/services/daemon@autosnap_2024-03-26_07:00:01_hourly 884K - 474G -storage/services/daemon@autosnap_2024-03-26_08:00:01_hourly 938K - 474G -storage/services/daemon@autosnap_2024-03-26_09:00:01_hourly 3.26M - 474G -storage/services/daemon@autosnap_2024-03-26_10:00:01_hourly 6.04M - 475G -And then on the backup:
storage/backups/node24/services/daemon@init 1.46G - 65.3G -storage/backups/node24/services/daemon@v1.6.21-to-v1.6.31 2.27G - 66.3G -storage/backups/node24/services/daemon@v1.6.31-to-v1.6.32 3.49G - 39.4G -storage/backups/node24/services/daemon@autosnap_2024-01-01_00:00:02_monthly 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-02-01_00:00:01_monthly 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-01_00:00:02_monthly 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-20_00:00:02_daily 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-21_00:00:01_daily 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-22_00:00:02_daily 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-23_00:00:02_daily 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-24_00:00:01_daily 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-25_00:00:02_daily 0B - 37.0G -storage/backups/node24/services/daemon@autosnap_2024-03-26_00:00:02_daily 0B - 37.0G -The backups of the snapshots seem to have stopped from the v1.6.31-to-v1.6.32 update on the daemon service (diffs of 0bytes from then). And when I take a copy of the backup to my local machine to look, it does seem to have stopped then.
From my understanding my configuration should be staying in sync with the production machine, even if I perform manual recursive rollbacks.
Would be very appreciative if someone could point me to where I'm going wrong.
Thanks so much.