Skip to content

Commit 0810d99

Browse files
jpartlowjpartlow
committed
(PE-39352) Update backup/restore plans for hac database
PE 2023.7.0 added the host-action-collector service and the backup/restore plans need to be made aware of the pe-hac database. This patch adds a version test, best on the addition of pe_version check of /opt/puppetlabs/server/pe_build in the peadm::get_peadm_config task. If >= 2023.7, then pe-hac is added to the defaults returned by the various recover_opts functions that define which resource backup/restore should wrangle. The patch does not attempt to version test the user's custom override hash of selections, on the assumption that they know, or should be allowed to specify whatever is needed. If pe_version is not found (damaged cluster, missing from peadm_config.json), the patch defaults to assuming earliest version, which will skip pe-hac. This favors the plans running without error at the cost of possibly missing pe-hac in a newer installation. Alternately, we could reverse this and incur failures in damaged older clusters requiring use of $custom to work around. Finally there is an edge case it does not address which is the case of newer pe versions with pe-hac restoring tarballs that were intended to include all databases, but which were created with peadm predating this patch. These will fail the pg-restore pe-hac commands, since pe-hac won't be in the backup, and again require use of $custom to work around.
1 parent 66567dc commit 0810d99

16 files changed

+372
-52
lines changed

‎functions/amend_recovery_defaults_by_pe_version.pp

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
function peadm::amend_recovery_defaults_by_pe_version (
2+
Hash $base_opts,
3+
Peadm::Pe_version $pe_version,
4+
Boolean $opt_value,
5+
) {
6+
# work around puppet-lint check_unquoted_string_in_case
7+
$semverrange = SemVerRange('>= 2023.7')
8+
case $pe_version {
9+
$semverrange: {
10+
$base_opts + {
11+
'hac' => $opt_value,
12+
}
13+
}
14+
default: {
15+
$base_opts
16+
}
17+
}
18+
}

‎functions/migration_opts_default.pp

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
function peadm::migration_opts_default () {
2-
{
1+
function peadm::migration_opts_default (
2+
Peadm::Pe_version $pe_version,
3+
) {
4+
$base_opts = {
35
'activity' => true,
46
'ca' => true,
57
'classifier' => true,
@@ -9,4 +11,5 @@ function peadm::migration_opts_default () {
911
'puppetdb' => true,
1012
'rbac' => true,
1113
}
14+
peadm::amend_recovery_defaults_by_pe_version($base_opts, $pe_version, true)
1215
}

‎functions/recovery_opts_all.pp

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
function peadm::recovery_opts_all () {
2-
{
1+
function peadm::recovery_opts_all (
2+
Peadm::Pe_version $pe_version,
3+
) {
4+
$base_opts = {
35
'activity' => true,
46
'ca' => true,
57
'classifier' => true,
@@ -9,4 +11,5 @@ function peadm::recovery_opts_all () {
911
'puppetdb' => true,
1012
'rbac' => true,
1113
}
14+
peadm::amend_recovery_defaults_by_pe_version($base_opts, $pe_version, true)
1215
}

‎functions/recovery_opts_default.pp

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
function peadm::recovery_opts_default () {
2-
{
1+
function peadm::recovery_opts_default (
2+
Peadm::Pe_version $pe_version,
3+
) {
4+
$base_opts = {
35
'activity' => false,
46
'ca' => true,
57
'classifier' => false,
@@ -9,4 +11,5 @@ function peadm::recovery_opts_default () {
911
'puppetdb' => true,
1012
'rbac' => false,
1113
}
14+
peadm::amend_recovery_defaults_by_pe_version($base_opts, $pe_version, false)
1215
}

‎functions/validated_pe_version_for_backup_restore.pp

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Verify that *pe_version* string is a valid SemVer.
2+
# If not, warn, and return "0.0.0" as a permissive default.
3+
function peadm::validated_pe_version_for_backup_restore(
4+
Optional[String] $pe_version,
5+
) {
6+
# work around puppet-lint check_unquoted_string_in_case
7+
$semverrange = SemVerRange('>=0.0.0')
8+
case $pe_version {
9+
# Validate that the value is a SemVer value.
10+
$semverrange: {
11+
$pe_version
12+
}
13+
default: {
14+
$msg = @("WARN")
15+
WARNING: Retrieved a missing or unparseable PE version of '${pe_version}'.
16+
The host_action_collector database will be skipped from defaults.
17+
|-WARN
18+
out::message($msg)
19+
'0.0.0'
20+
}
21+
}
22+
}

‎plans/backup.pp

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,12 @@
3535
getvar('cluster.params.compiler_hosts'),
3636
)
3737

38+
$pe_version = peadm::validated_pe_version_for_backup_restore(getvar('cluster.pe_version'))
39+
3840
$recovery_opts = $backup_type? {
39-
'recovery' => peadm::recovery_opts_default(),
40-
'migration' => peadm::migration_opts_default(),
41-
'custom' => peadm::recovery_opts_all() + $backup,
41+
'recovery' => peadm::recovery_opts_default($pe_version),
42+
'migration' => peadm::migration_opts_default($pe_version),
43+
'custom' => peadm::recovery_opts_all($pe_version) + $backup,
4244
}
4345

4446
$timestamp = Timestamp.new().strftime('%Y-%m-%dT%H%M%SZ')
@@ -55,6 +57,8 @@
5557
'activity' => $primary_target,
5658
'rbac' => $primary_target,
5759
'puppetdb' => $puppetdb_postgresql_target,
60+
# (host-action-collector db will be filtered for pe version by recovery_opts)
61+
'hac' => $primary_target,
5862
}.filter |$key,$_| {
5963
$recovery_opts[$key] == true
6064
}

‎plans/restore.pp

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
# try to load the cluster configuration by running peadm::get_peadm_config, but allow for errors to happen
3535
$_cluster = run_task('peadm::get_peadm_config', $targets, { '_catch_errors' => true }).first.value
3636

37-
if $_cluster == undef or getvar('_cluster.params') == undef {
37+
if $_cluster == undef or getvar('_cluster.params') == undef or getvar('_cluster.pe_version') == undef {
3838
# failed to get cluster config, load from backup
3939
out::message('Failed to get cluster configuration, loading from backup...')
4040
$result = download_file("${recovery_directory}/peadm/peadm_config.json", 'peadm_config.json', $targets).first.value
@@ -59,11 +59,13 @@
5959
getvar('cluster.params.compiler_hosts'),
6060
)
6161
62+
$pe_version = peadm::validated_pe_version_for_backup_restore(getvar('cluster.pe_version'))
63+
6264
$recovery_opts = $restore_type? {
63-
'recovery' => peadm::recovery_opts_default(),
65+
'recovery' => peadm::recovery_opts_default($pe_version),
6466
'recovery-db' => { 'puppetdb' => true, },
65-
'migration' => peadm::migration_opts_default(),
66-
'custom' => peadm::recovery_opts_all() + $restore,
67+
'migration' => peadm::migration_opts_default($pe_version),
68+
'custom' => peadm::recovery_opts_all($pe_version) + $restore,
6769
}
6870
6971
$primary_target = peadm::get_targets(getvar('cluster.params.primary_host'), 1)
@@ -97,6 +99,8 @@
9799
'activity' => [$primary_target],
98100
'rbac' => [$primary_target],
99101
'puppetdb' => $puppetdb_postgresql_targets,
102+
# (host-action-collector db will be filtered for pe version by recovery_opts)
103+
'hac' => $primary_target,
100104
}.filter |$key,$_| {
101105
$recovery_opts[$key] == true
102106
}
@@ -203,7 +207,7 @@
203207
if getvar('recovery_opts.orchestrator') {
204208
out::message('# Restoring orchestrator secret keys')
205209
run_command(@("CMD"/L), $primary_target)
206-
cp -rp ${shellquote($recovery_directory)}/orchestrator/secrets/* /etc/puppetlabs/orchestration-services/conf.d/secrets/
210+
cp -rp ${shellquote($recovery_directory)}/orchestrator/secrets/* /etc/puppetlabs/orchestration-services/conf.d/secrets/
207211
| CMD
208212
}
209213
# lint:endignore

‎spec/fixtures/peadm_config.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
{
2+
"pe_version" : "2023.7.0",
23
"params": { "primary_host": "primary", "primary_postgresql_host": "postgres" }
34
}

‎spec/fixtures/peadm_config.no_pe_version.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"params": { "primary_host": "primary", "primary_postgresql_host": "postgres" }
3+
}

‎spec/functions/amend_recovery_defaults_by_pe_version_spec.rb

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
require 'spec_helper'
2+
3+
describe 'peadm::amend_recovery_defaults_by_pe_version' do
4+
it 'just returns the base opts if version < 2023.7' do
5+
is_expected.to run.with_params({}, '2023.6.0', true).and_return({})
6+
end
7+
8+
it 'adds hac if version >= 2023.7' do
9+
is_expected.to run.with_params({}, '2023.7.0', true).and_return({ 'hac' => true })
10+
end
11+
12+
it 'adds hac false based on opt_value' do
13+
is_expected.to run.with_params({}, '2023.7.0', false).and_return({ 'hac' => false })
14+
end
15+
end

‎spec/functions/migration_opts_default_spec.rb

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
require 'spec_helper'
2+
3+
describe 'peadm::migration_opts_default' do
4+
it 'returns pre 2023.7 defaults' do
5+
is_expected.to run.with_params('2023.6.0').and_return(
6+
{
7+
'activity' => true,
8+
'ca' => true,
9+
'classifier' => true,
10+
'code' => false,
11+
'config' => false,
12+
'orchestrator' => true,
13+
'puppetdb' => true,
14+
'rbac' => true,
15+
},
16+
)
17+
end
18+
19+
it 'returns 2023.7+ defaults with hac' do
20+
is_expected.to run.with_params('2023.7.0').and_return(
21+
{
22+
'activity' => true,
23+
'ca' => true,
24+
'classifier' => true,
25+
'code' => false,
26+
'config' => false,
27+
'orchestrator' => true,
28+
'puppetdb' => true,
29+
'rbac' => true,
30+
'hac' => true,
31+
},
32+
)
33+
end
34+
end

‎spec/functions/recovery_opts_all_spec.rb

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
require 'spec_helper'
2+
3+
describe 'peadm::recovery_opts_all' do
4+
it 'returns pre 2023.7 defaults' do
5+
is_expected.to run.with_params('2023.6.0').and_return(
6+
{
7+
'activity' => true,
8+
'ca' => true,
9+
'classifier' => true,
10+
'code' => true,
11+
'config' => true,
12+
'orchestrator' => true,
13+
'puppetdb' => true,
14+
'rbac' => true,
15+
},
16+
)
17+
end
18+
19+
it 'returns 2023.7+ defaults with hac' do
20+
is_expected.to run.with_params('2023.7.0').and_return(
21+
{
22+
'activity' => true,
23+
'ca' => true,
24+
'classifier' => true,
25+
'code' => true,
26+
'config' => true,
27+
'orchestrator' => true,
28+
'puppetdb' => true,
29+
'rbac' => true,
30+
'hac' => true,
31+
},
32+
)
33+
end
34+
end

‎spec/functions/recovery_opts_default_spec.rb

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
require 'spec_helper'
2+
3+
describe 'peadm::recovery_opts_default' do
4+
it 'returns pre 2023.7 defaults' do
5+
is_expected.to run.with_params('2023.6.0').and_return(
6+
{
7+
'activity' => false,
8+
'ca' => true,
9+
'classifier' => false,
10+
'code' => true,
11+
'config' => true,
12+
'orchestrator' => false,
13+
'puppetdb' => true,
14+
'rbac' => false,
15+
},
16+
)
17+
end
18+
19+
it 'returns 2023.7+ defaults with hac' do
20+
is_expected.to run.with_params('2023.7.0').and_return(
21+
{
22+
'activity' => false,
23+
'ca' => true,
24+
'classifier' => false,
25+
'code' => true,
26+
'config' => true,
27+
'orchestrator' => false,
28+
'puppetdb' => true,
29+
'rbac' => false,
30+
'hac' => false,
31+
},
32+
)
33+
end
34+
end

0 commit comments

Comments
 (0)