$PATH being ignored by autorestic

[ilium007]

autorestic does not seem to have access to $PATH when calling restic or ssh binaries. I am running a hook script after a Proxmox backup and I have output '$PATH' and $(which ssh) and the ssh binary is in the correct path. It seems that autorestic is ignoring the system $PATH env variable. While I was able to use the --restic-bin option and hard code the restic path (also in an accessible $PATH) it is now falling over on ssh.

Expected behavior

Environment

• OS: Debian 12
• Version: [e.g. 22]

Hook script that calls autorestic (and produces output below):

#!/bin/bash

PVE_NODE="$(hostname)"
LOG_FILE="/mnt/ext_usb_01/$PVE_NODE.log"

if [ "$1" == "job-start" ]; then
  printf "$(date) PVE hook script start ($PVE_NODE) - $(pveversion)\n" | tee $LOG_FILE
  printf "PATH: $PATH\n" | tee -a $LOG_FILE
  printf "WHICH SSH: $( which ssh)\n" | tee -a $LOG_FILE
fi

if [ "$1" == "job-end" ]; then
  printf "PATH: $PATH\n" | tee -a $LOG_FILE
  printf "WHICH SSH: $( which ssh)\n" | tee -a $LOG_FILE
  /usr/local/bin/autorestic -c /root/.autorestic.yml --restic-bin /usr/local/bin/restic backup --verbose --all 2>&1 | tee -a $LOG_FILE
  printf "$(date) PVE hook script finish ($PVE_NODE)\n" | tee -a $LOG_FILE
fi

Output:

Thu May  9 23:21:22 UTC 2024 PVE hook script start (pve01) - pve-manager/8.2.2/9355359cd7afbae4 (running kernel: 6.8.4-2-pve)
PATH: /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
WHICH SSH: /usr/bin/ssh
PATH: /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
WHICH SSH: /usr/bin/ssh
Using config:    /root/.autorestic.yml
Using lock:      /root/.autorestic.lock.yml


    Backing up location "ext_usb_01"

Backend: storage0
> Executing: /usr/local/bin/restic backup --tag ar:location:ext_usb_01 /mnt/ext_usb_01/dump/
Fatal: unable to open repository at sftp:honeyeater-autorestic:/mnt/storage0/xxx/pve01/: exec: "ssh": executable file not found in $PATH

ext_usb_01@storage0:
Fatal: unable to open repository at sftp:honeyeater-autorestic:/mnt/storage0/xxx/pve01/: exec: "ssh": executable file not found in $PATH
exit status 1

Error: 1 errors were found
Thu May  9 23:21:30 UTC 2024 PVE hook script finish (pve01)

[ilium007]

I re-wrote the script in Perl and same thing happens:

I have re-written the script in Perl and the same thing happens:

Script:

root@pve01:~# cat /etc/vzdump-hook.pl
#!/usr/bin/perl


use strict;
use warnings;


print "GUEST HOOK: " . join(' ', @ARGV). "\n";


my $phase = shift;
my $mode = shift;
my $vmid = shift;


if ($phase eq 'job-start') {


    print "Job is starting\n";
    my $cmd = qq[printf "\$(date) PVE hook script start pve01 - \$(pveversion)\n" | tee /mnt/ext_usb_01/pve01.log];
    system $cmd;


} elsif ($phase eq 'job-end') {


    print "Job is ending\n";
    my $cmd = qq[/usr/local/bin/autorestic -c /root/.autorestic.yml --restic-bin /usr/local/bin/restic backup --verbose --all 2>&1 | tee -a /mnt/ext_usb_01/pve01.log];
    system $cmd;


}


exit(0);

Same error:

root@pve01:~# cat /mnt/ext_usb_01/pve01.log
Fri May 10 01:32:10 UTC 2024 PVE hook script start pve01 - pve-manager/8.2.2/9355359cd7afbae4 (running kernel: 6.8.4-2-pve)
Using config:      /root/.autorestic.yml
Using lock:     /root/.autorestic.lock.yml




    Backing up location "ext_usb_01"


Backend: storage0
> Executing: /usr/local/bin/restic backup --tag ar:location:ext_usb_01 /mnt/ext_usb_01/dump/
Fatal: unable to open repository at sftp:honeyeater-autorestic:/mnt/storage0/winter/pve01/: exec: "ssh": executable file not found in $PATH


ext_usb_01@storage0:
Fatal: unable to open repository at sftp:honeyeater-autorestic:/mnt/storage0/winter/pve01/: exec: "ssh": executable file not found in $PATH
exit status 1


Error: 1 errors were found
root@pve01:~#

[ilium007]

If I remove hard coded paths to the --restic-bin in the autorestic command it fails... It seems /usr/local/bin/autorestic is found without hardcoding path but when autorestic calls 'restic' it fails on PATH as well.

autorestic does not have access to $PATH

tail: /mnt/ext_usb_01/pve01.log: file truncated
Fri May 10 01:38:43 UTC 2024 PVE hook script start pve01 - pve-manager/8.2.2/9355359cd7afbae4 (running kernel: 6.8.4-2-pve)


Using config:      /root/.autorestic.yml
Using lock:     /root/.autorestic.lock.yml


    Backing up location "ext_usb_01"

Backend: storage0
> Executing: restic backup --tag ar:location:ext_usb_01 /mnt/ext_usb_01/dump/

ext_usb_01@storage0:
exec: "restic": executable file not found in $PATH

Error: 1 errors were found

[ilium007]

Again - if I run the Perl script manually from the command line and pass in job-end it runs fine with no PATH issues.

root@pve01:~# /etc/vzdump-hook.pl job-end
GUEST HOOK: job-end
Job is ending
Using config:      /root/.autorestic.yml
Using lock:     /root/.autorestic.lock.yml




    Backing up location "ext_usb_01"


Backend: storage0
> Executing: /usr/local/bin/restic backup --tag ar:location:ext_usb_01 /mnt/ext_usb_01/dump/
using parent snapshot 08c763d1


Files:           3 new,     0 changed,     0 unmodified
Dirs:            0 new,     3 changed,     0 unmodified
Added to the repository: 128.855 MiB (128.804 MiB stored)


processed 3 files, 268.868 MiB in 0:37
snapshot 73360270 saved




  Forgetting for location "ext_usb_01"


For backend "storage0"
> Executing: /usr/local/bin/restic forget --tag ar:location:ext_usb_01 --keep-weekly 4 --keep-monthly 3
Applying Policy: keep 4 weekly, 3 monthly snapshots
keep 2 snapshots:
ID        Time                 Host        Tags                    Reasons           Paths
---------------------------------------------------------------------------------------------------------
925ca5ed  2024-05-09 22:03:17  pve01       ar:location:ext_usb_01  weekly snapshot   /mnt/ext_usb_01/dump
                                                                   monthly snapshot
73360270  2024-05-10 01:45:42  pve01       ar:location:ext_usb_01  weekly snapshot   /mnt/ext_usb_01/dump
                                                                   monthly snapshot
---------------------------------------------------------------------------------------------------------
2 snapshots


remove 1 snapshots:
ID        Time                 Host        Tags                    Paths
---------------------------------------------------------------------------------------
08c763d1  2024-05-09 23:47:34  pve01       ar:location:ext_usb_01  /mnt/ext_usb_01/dump
---------------------------------------------------------------------------------------
1 snapshots


[0:00] 100.00%  1 / 1 files deleted


Done
Done
root@pve01:~#

[ilium007]

So is the issue in Proxmox, autorestic or my script?