haumea/zrepl: reduce snapshot count #447
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,7 +22,7 @@ | |
| filesystems."rpool/safe<" = true; | ||
| snapshotting = { | ||
| type = "periodic"; | ||
| interval = "30m"; | ||
| prefix = "zrepl_snap_"; | ||
| hooks = [ { | ||
| # https://zrepl.github.io/master/configuration/snapshotting.html#postgres-checkpoint-hook | ||
|
|
@@ -31,27 +31,52 @@ | |
| filesystems."rpool/safe/postgres" = true; | ||
| } ]; | ||
| }; | ||
|
|
||
| # The current pruning setup is an exponentially growing scheme, at both sides. | ||
| pruning = { | ||
| keep_sender = [ | ||
| { type = "not_replicated"; } | ||
| { | ||
| type = "grid"; | ||
| regex = "^zrepl_snap_.*"; | ||
| grid = lib.concatStringsSep " | " [ | ||
| "1x1h(keep=all)" | ||
| "1x1h" | ||
| "1x2h" | ||
| "1x4h" | ||
| # "grid" acts weird if an interval isn't a whole-number multiple | ||
| # of the previous one, so we jump from 8h to 24h | ||
|
There was a problem hiding this comment. Not sure if it's worth trying to explain the weirdness. I base it not on actual experience but on definition in their docs – and how such a model behaves then when running continuously. |
||
| "2x8h" | ||
| "1x1d" | ||
| "1x2d" | ||
| "1x4d" | ||
| "1x8d" | ||
| # At this point we keep ~10 snapshots spanning 8--16 days (depends on moment), | ||
| # with exponentially increasing spacing (almost). | ||
| ]; | ||
| } | ||
| ]; | ||
| keep_receiver = [ | ||
| { type = "grid"; | ||
| regex = "^zrepl_snap_.*"; | ||
| grid = lib.concatStringsSep " | " [ | ||
| "2x1h(keep=all)" | ||
| "2x1h" | ||
| "2x2h" | ||
| "2x4h" | ||
| "4x8h" | ||
| # At this point the grid spans 2 days by ~13 snapshots. | ||
| # (See note above about 8h -> 24h.) | ||
| "2x1d" | ||
| "2x2d" | ||
| "2x4d" | ||
| "2x8d" | ||
| "2x16d" | ||
| "2x32d" | ||
| "2x64d" | ||
| "2x128d" | ||
| # At this point we keep ~29 snapshots spanning 384--512 days (depends on moment), | ||
| # with exponentially increasing spacing (almost). | ||
| ]; | ||
| } | ||
| ]; | ||
|
|
@@ -71,18 +96,26 @@ | |
| }; | ||
|
|
||
| jobs = [ | ||
| # Covers 20240629+ | ||
| (defaultBackupJob // { | ||
| name = "rsyncnet"; | ||
| connect = { | ||
| identity_file = "/root/.ssh/id_ed25519"; | ||
| type = "ssh+stdinserver"; | ||
| host = "zh4461b.rsync.net"; | ||
| user = "root"; | ||
| port = 22; | ||
| }; | ||
| }) | ||
| /* rsync.net provides a VM with FreeBSD | ||
| - almost nothing is preserved on upgrades except this "data1" zpool | ||
| $ scp ./zrepl.yml [email protected]:/usr/local/etc/zrepl/zrepl.yml | ||
| # pkg install zrepl | ||
| # service zrepl enable | ||
| # service zrepl start | ||
| */ | ||
|
|
||
| /* Covered 2024: 0212 -- 0629 | ||
| (defaultBackupJob // { | ||
| name = "hexa"; | ||
| connect = { | ||
|
|
@@ -93,6 +126,7 @@ | |
| port = 22; | ||
| }; | ||
| }) | ||
| */ | ||
| ]; | ||
| }; | ||
| }; | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # [email protected]:/usr/local/etc/zrepl/zrepl.yml | ||
| # zrepl main configuration file. | ||
| # For documentation, refer to https://zrepl.github.io/ | ||
| # | ||
| global: | ||
| logging: | ||
| - type: "stdout" | ||
| level: "error" | ||
| format: "human" | ||
| - type: "syslog" | ||
| level: "info" | ||
| format: "logfmt" | ||
|
|
||
| # mostly from https://blog.lenny.ninja/zrepl-on-rsync-net.html | ||
| jobs: | ||
| - name: sink | ||
| type: sink | ||
| serve: | ||
| type: stdinserver | ||
| client_identities: [ haumea ] | ||
| recv: | ||
| placeholder: | ||
| encryption: off | ||
| root_fs: "data1" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, I expect we need have the
regex =part here as well.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or maybe not. Their examples set doesn't have it:
https://zrepl.github.io/configuration/prune.html#pruning-policies
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should drop this line anyway. In case the remote end is down, we probably want to keep pruning the sender to reduce the risk of running out of space. Also it might reduce the time to sync up when the receiver becomes reachable again.