excessive vault writing

[mwang]

Hi,

I've my ~/Documents folder backed up by QRecall (v2.x) every time there is any change to the file system. When inspecting my backup data today I found frequent backups (every 10 min. or so) even when I'm not around my computer (when sleeping e.g.). Turns out Enpass is writing to my two vaults every 10 min. even when no change has been made and other synced devices (my notebook and android phones, e.g.) are either shut down or sleeping.

My guess is Enpass is constantly checking the cloud sync store (Google Drive in my case) and writing sync status (or something in that nature) to the vaults. This is causing excessive vault writing and backups, however, which not only causes unnecessary wearing on SSDs, but also makes it difficult to tell exactly when real changes were made to the vaults (when did I change the password for XXX website recently, e.g., or when did I move an entry from vault A to B).

It's also filling up my backup disk space much more quickly than necessary. Because Enpass re-encrypts a vault every time, it's completed different from one version to the next. QRecall (or any other backup software for that matter) can't put its delta-sync algorithm to work and has to backup the whole thing every time. Granted the vaults aren't large (less than 2MB in total for the moment), but 2MB every 10 min. would translate into 288MB per day.  My whole QRecall backup archive for ~/Document has accumulated less than 18GB after 4+ years; my two Enpass vaults alone would beat that in 3 months.

I've since added a 21-min. delay between backup jobs to mitigate the issue. It's a band-aid solution at best, however, and when I'm busy working, I really prefer more frequent backups.

If I'm right about Enpass writing sync status data (or anything of the sort) to the vault, could you please consider writing it to another file (with a name I can easily exclude from backup) instead? Thanks!

 

[abhibeckert]

Exclude the Enpass vault from entirely from your QRecall backups and backup the Auto Backups directory (you can find the location in Enpass settings).

Auto Backups are only written when changes have been made, and it's not immediate so you're given a chance to roll several changes into a single backup. For me, there have only been seven backups in the last month.

Wearing out your SSD isn't a concern - a good quality SSD can handle around 150GB of writes per day and still last a decade or so. It's only a concern for old or cheap drives.

Also... it's a good policy to have a local backup, perhaps to an external HDD, in addition to your cloud backup. Since it's faster and cheaper, you can backup more frequently and include things like your Enpass vault. Also, restoring from a local disk is a lot quicker than restoring from the cloud.

Edited April 14, 2019 by abhibeckert

[Ankur Gupta]

Hi @mwang,

Thanks for writing to us.

Enpass desktop version checks for changes on cloud every 10 minutes and store the sync status in .sync folder in ~/Documents/Enpass. Also Enpass stores some temporary file here used from syncing your vault with cloud vault.

You can exclude easily exclude the folder "~/Documents/Enpass/.sync" from backups because it is just like caches which will be created next time you will connect to sync. Alternatively you can follow the suggestion

8 hours ago, abhibeckert said:

Exclude the Enpass vault from entirely from your QRecall backups and backup the Auto Backups directory (you can find the location in Enpass settings).

 

 

Please let me know if it solves your issue.

Thanks.

[mwang]

12 hours ago, abhibeckert said:

Exclude the Enpass vault from entirely from your QRecall backups and backup the Auto Backups directory (you can find the location in Enpass settings).

Auto Backups are only written when changes have been made, and it's not immediate so you're given a chance to roll several changes into a single backup. For me, there have only been seven backups in the last month.

That's exactly the first alternative I looked into when I discovered the issue 2 days ago. 

The problem is, how Auto Backups (for simplicity and clarity, I'll call them AB from now on, to distinguish with backups made with other software such as QRecall) are done is not documented (is it?). In line with your observation, I have only 5 backups so far, with the most recent one dated 2019-04-10, meaning changes made after that haven't been backed up yet, including one I made just for test more than 24 hours ago. I don't know what you think, but that's not sound backup strategy to me.

Two other problems: first, AB files are rarely used, so you don't really know how good they are until you need them. This is true for all backups so we don't need to get too hung up over it. But it does mean, to me at least, backing up the original, whenever possible, is better than backing up the ABs. Second, Enpass rolls all vaults into one AB file, and all vaults are wiped and restored when you restore from AB. That's not very convenient. These are nonetheless minor quibbles. 

12 hours ago, abhibeckert said:

Wearing out your SSD isn't a concern - a good quality SSD can handle around 150GB of writes per day and still last a decade or so. It's only a concern for old or cheap drives.

Sorry if I mislead you on the importance of SSD wearing. I do have 10-year-old SSDs still going strong (while the one in my newest Macbook died on me after a little more than one year), so I know it's not a big issue, though I still consider it a good practice to minimize unnecessary writing. The main issues, as described in my original post, are the accelerated (and, again, unnecessary) growth of my backup archive, and losing one of QRecall's best features: making it easy to see the history of file changes.

12 hours ago, abhibeckert said:

Also... it's a good policy to have a local backup, perhaps to an external HDD, in addition to your cloud backup. Since it's faster and cheaper, you can backup more frequently and include things like your Enpass vault. Also, restoring from a local disk is a lot quicker than restoring from the cloud.

Well, my old SSDs are for this purpose exactly: fast and frequent backups. They are too old for front line duties, but they are good backup drives. But since they are old, they are small. I do have HDDs backing up other things: photos, music, and so on.

To be clear, I do have a rather elaborate backup strategy, utilizing multiple applications, media, devices, and locations (including cloud). Thanks for your kind suggestions all the same.

[mwang]

5 hours ago, Ankur Gupta said:

You can exclude easily exclude the folder "~/Documents/Enpass/.sync" from backups because it is just like caches which will be created next time you will connect to sync.

Sorry if I didn't make myself clear: I don't back up the ".sync" folder, nor (for the moment) the "Backups" folder. Just the vault files (.enpassdb & .json) and the various .enpassattach files. Among them, only the vault.enpassdb file is changed constantly (every 10 min.).

5 hours ago, Ankur Gupta said:

Alternatively you can follow the suggestion

    ... (quoting abhibeckert's suggestions)

Please let me know if it solves your issue.

As explained in my previous post, no.

[Ankur Gupta]

Hi @mwang,

I have just checked and found that vault.enpassdb is getting changed every 10 minutes whereas it shouldn't be. We're currently investigating the issue, and if possible we will try to fix it in the upcoming release. We appreciate your patience and co-operation.

Meanwhile, I suggest you use the folder '~/Documents/Enpass/Backups' for now.

Thanks.

[mwang]

Thanks! I'll wait. Could you please tell me how Enpass's Auto Backup works? Sometimes a change is backed up quickly, and yet sometimes hours have gone by and the newest changes are still not backed up. Thanks!