Unexpected backup data loss on re-adding legacy backup folder/account

[bentolor]

Hi @joeyates

thanks to ZFS and zfs-auto-snapshots and furthermore having them also enabled on the relevant volume, I'm quite relaxed. Otherwise I'd have been frustrated. I'm not sure what I did wrong. So it's at least unexpected for me from a user perspective.

What was I trying to accommplish

As mentioned in #191 : Restore and old, decomissioned IMAP account backup to a new IMAP account

What did I do?

1. I added the new target [email protected] account via imap-backup setup
2. I readded the legacy backup account to ~/.imap-backup/config.json (see below)
3. I executed imap-backup migrate [email protected] [email protected] -v
4. I interrupted the session as many " INFO -- : Local metadata for folder '/srv/backups/imap/myoldemailadress_acme.bar/Myfolder" is currently stored in the version 2 format. scrolled over my screen
5. I restarted the command and let it finish this time
6. I re-executed the command above several times, as I realized no migration happened.
7. A short moment of panic…

   {
      "username": "[email protected]",
      "password": "foo",
      "local_path": "/srv/backups/imap/myoldemailadress_acme.bar",
      "server": "acme.bar"
    }

What was the result

• Target account kept to be empty
• /srv/backups/imap/myoldemailadress_acme.bar folder content was purged. 6.9G of backup became a solely empty folder structure with no backup files any longer in it.

What did I expect

Ideally: The target account contains my backup.

I suspect the missing folder inclusion list might be the culprit. At least, as all my accounts are in "keep all emails"-mode, I would expect my local copy not to be purged.

Did I miss something important?

[bentolor]

Update: I redid the exercise, but this time with "folder_blacklist": true for the source account. It doesn't seem to have changed for the good. I'm a little puzzled.

I only get output in the following form:

D, [2024-01-17T15:25:52.309892 #1232144] DEBUG -- : [Folder] 0 to migrate
I, [2024-01-17T15:25:52.310234 #1232144]  INFO -- : Local metadata for folder '/srv/backups/imap/MyAccount/Folder' is currently stored in the version 2 format.

Migrating to the version 3 format...

Ähem... I start to feel getting lost. How'd I achieve my plan? Where is my thought error??

[joeyates]

Hi @bentolor

I'm sorry you've run into this problem. I suspect that the recent work on speeding up large backups may have introduced a bug in handling the conversion of old backups.

If everything is working correctly, you shouldn't need to change any configuration to get this working.

I'll have a look.

[joeyates]

@bentolor I've release v14.5.1 with a fix for this.

[bentolor]

Thanks @joeyates for the awesome and quick fix! Tested this right away and f82fbae obviously fixed this. Thanks!