IP address changes are not detected by the parallels-iso builder.

[hc-github-team-packer]

This issue was originally opened by @ladar in hashicorp/packer#11431 and has been migrated to this repository. The original issue description is below.

---

Overview of the Issue

The Parallels DHCP is a bit temperamental. As such, a handful of things can cause it to assign guests new IP addresses during a reboot. What triggers the address changes varies, depending on the DHCP client configuration inside the guests, but for several of the distros I work with, applying operating system updates and/or changing the hostname before the reboot, both cause problems. Devuan 4.0 appears to be doing this consistently.

Reproduction Steps

Install a basic Devuan 4.0 server environment. The IP address will sometimes change between when the installer finishes, and when the guest boots for the first time, but packer seems to detect this change. Unfortunately if the IP changes after packer has connected via SSH, it won't. Rather it will continue trying to connect to the previous IP address until the SSH timeout expires, and the build fails. Running these commands seemed to trigger the problem consistently for me.

# Update, and then install some stuff.
export DEBIAN_FRONTEND=noninteractive
export DEBCONF_NONINTERACTIVE_SEEN=true
apt-get --assume-yes -o Dpkg::Options::="--force-confnew" update
apt-get --assume-yes -o Dpkg::Options::="--force-confnew" upgrade
apt-get --assume-yes -o Dpkg::Options::="--force-confnew" dist-upgrade
apt-get --assume-yes install vim net-tools mlocate psmisc

# Change the hostname.
printf "$(openssl rand -base64 12 | tr '[:upper:]' '[:lower:]' | tr -d '/' | tr -d '+' ).localdomain\n" > /etc/hostname

# Clear out the existing automatic ifup rules.
sed -i -e '/^auto/d' /etc/network/interfaces
sed -i -e '/^iface/d' /etc/network/interfaces
sed -i -e '/^allow-hotplug/d' /etc/network/interfaces

# Ensure the loopback, and default network interface are automatically enabled and then dhcp'ed.
printf "allow-hotplug eth0\n" >> /etc/network/interfaces
printf "auto lo\n" >> /etc/network/interfaces
printf "iface lo inet loopback\n" >> /etc/network/interfaces
printf "iface eth0 inet dhcp\n" >> /etc/network/interfaces
printf "dns-nameserver 4.2.2.1\n" >> /etc/network/interfaces
printf "dns-nameserver 4.2.2.2\n" >> /etc/network/interfaces
printf "dns-nameserver 208.67.220.220\n" >> /etc/network/interfaces

# Adding a delay so dhclient will work properly.
printf "pre-up sleep 2\n" >> /etc/network/interfaces

# Ensure a nameserver is being used that won't return an IP for non-existent domain names.
printf "nameserver 4.2.2.1\nnameserver 4.2.2.2\nnameserver 208.67.220.220\n" > /etc/resolv.conf

# Reboot onto the new kernel (if applicable).
sed -i -e '/gethostname/d'   /etc/dhcp/dhclient.conf

# Schedule a reboot, and let the script exit normally.
(sleep 30 ; /sbin/reboot) &
echo "Rebooting in thirty seconds..."

Packer version

Tested using version 1.7.7 and 1.7.8.

Simplified Packer Template

You can duplicate my process precisely, and try building the Devuan 4.0 image using the current commit (presumably by the time you look at this I'll have a workaround in HEAD):

git clone https://github.com/lavabit/robox.git && cd robox 
git checkout -b packer-parallels-bug 50b8ac2e09a451df42e740a16af7f49a9265b8f2
./robox.sh box generic-devuan4-parallels

You can also skip the last command, and examine the packer templates/bash scripts manually. The configuration for the box in question is inside the generic-parallels.json config file.

Operating system and Environment details

macOS High Sierra v10.13.6
Parallels Desktop v16.5.0
Packer v1.7.8

Log Fragments and crash.log files

I'm including a small portion of the log, condensed to show the most important parts. Specifically when packer first detects the IP. The log will then skip to when the guest reboots. I only included the first two connection errors, as the same error repeats until the timeout is reached, and the build aborts.

If you scroll down past the log, you'll see the guest IP changed from 10.211.55.216 to 10.211.55.217 ... and that Parallels can provide the updated IP address information.

2021/12/03 14:35:10 packer-builder-parallels-iso plugin: [INFO] Waiting for SSH, up to timeout: 1h0m0s
2021/12/03 14:35:10 ui: ==> generic-devuan4-parallels: Waiting for SSH to become available...
2021/12/03 14:35:10 packer-builder-parallels-iso plugin: Found MAC address for NIC: net0 - 001C42CDF7A3
2021/12/03 14:35:10 packer-builder-parallels-iso plugin: [DEBUG] Error getting SSH address: IP lease not found for MAC address 001C42CDF7A3 in: /Library/Preferences/Parallels/parallels_dhcp_leases
2021/12/03 14:35:16 packer-builder-parallels-iso plugin: Found MAC address for NIC: net0 - 001C42CDF7A3
2021/12/03 14:35:16 packer-builder-parallels-iso plugin: [DEBUG] Error getting SSH address: IP lease not found for MAC address 001C42CDF7A3 in: /Library/Preferences/Parallels/parallels_dhcp_leases
2021/12/03 14:35:21 packer-builder-parallels-iso plugin: Found MAC address for NIC: net0 - 001C42CDF7A3
2021/12/03 14:35:21 packer-builder-parallels-iso plugin: [DEBUG] Error getting SSH address: IP lease not found for MAC address 001C42CDF7A3 in: /Library/Preferences/Parallels/parallels_dhcp_leases
2021/12/03 14:35:26 packer-builder-parallels-iso plugin: Found MAC address for NIC: net0 - 001C42CDF7A3
2021/12/03 14:35:26 packer-builder-parallels-iso plugin: [DEBUG] Error getting SSH address: IP lease not found for MAC address 001C42CDF7A3 in: /Library/Preferences/Parallels/parallels_dhcp_leases
2021/12/03 14:35:32 packer-builder-parallels-iso plugin: Found MAC address for NIC: net0 - 001C42CDF7A3
2021/12/03 14:35:32 packer-builder-parallels-iso plugin: Found lease: 10.211.55.215 for MAC: 001C42CDF7A3, expiring at 1638565529, leased for 1800 s.

<snipped>

2021/12/03 14:41:58 ui:     generic-devuan4-parallels: Rebooting in thirty seconds...
2021/12/03 14:41:58 ui error: ==> generic-devuan4-parallels: + sleep 30
2021/12/03 14:42:28 ui error: ==> generic-devuan4-parallels: + /sbin/reboot
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] 31 bytes written for 'stdout'
2021/12/03 14:42:28 [INFO] 75 bytes written for 'stderr'
2021/12/03 14:42:28 [INFO] RPC client: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] RPC endpoint: Communicator ended with: 0
2021/12/03 14:42:28 packer-provisioner-shell plugin: [INFO] 31 bytes written for 'stdout'
2021/12/03 14:42:28 packer-provisioner-shell plugin: [INFO] 75 bytes written for 'stderr'
2021/12/03 14:42:28 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [DEBUG] Opening new ssh session
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [DEBUG] starting remote command: rm -f /tmp/script_8566.sh
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] RPC client: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] RPC endpoint: Communicator ended with: 0
2021/12/03 14:42:28 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [DEBUG] Opening new ssh session
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [DEBUG] starting remote command: rm -f
2021/12/03 14:42:28 packer-builder-parallels-iso plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] RPC client: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] RPC endpoint: Communicator ended with: 0
2021/12/03 14:42:28 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2021/12/03 14:42:28 [INFO] (telemetry) ending shell
2021/12/03 14:42:28 [INFO] (telemetry) Starting provisioner shell
2021/12/03 14:42:28 ui: ==> generic-devuan4-parallels: Pausing 2m0s before the next provisioner...
2021/12/03 14:44:28 ui: ==> generic-devuan4-parallels: Provisioning with shell script: scripts/devuan4/floppy.sh
2021/12/03 14:44:28 packer-provisioner-shell plugin: Opening scripts/devuan4/floppy.sh for reading
2021/12/03 14:44:28 packer-provisioner-shell plugin: [INFO] 162 bytes written for 'uploadData'
2021/12/03 14:44:28 [INFO] 162 bytes written for 'uploadData'
2021/12/03 14:44:28 packer-builder-parallels-iso plugin: [DEBUG] Opening new ssh session
2021/12/03 14:44:28 packer-builder-parallels-iso plugin: [ERROR] ssh session open error: 'EOF', attempting reconnect
2021/12/03 14:44:28 packer-builder-parallels-iso plugin: [DEBUG] reconnecting to TCP connection for SSH
2021/12/03 14:44:43 packer-builder-parallels-iso plugin: [ERROR] reconnection error: dial tcp 10.211.55.216:22: i/o timeout
2021/12/03 14:44:43 packer-provisioner-shell plugin: Retryable error: Error uploading script: dial tcp 10.211.55.216:22: i/o timeout
2021/12/03 14:44:45 packer-provisioner-shell plugin: [INFO] 162 bytes written for 'uploadData'
2021/12/03 14:44:45 [INFO] 162 bytes written for 'uploadData'
2021/12/03 14:44:45 packer-builder-parallels-iso plugin: [DEBUG] Opening new ssh session
2021/12/03 14:44:45 packer-builder-parallels-iso plugin: [ERROR] ssh session open error: 'client not available', attempting reconnect
2021/12/03 14:44:45 packer-builder-parallels-iso plugin: [DEBUG] reconnecting to TCP connection for SSH
2021/12/03 14:45:00 packer-builder-parallels-iso plugin: [ERROR] reconnection error: dial tcp 10.211.55.216:22: i/o timeout
2021/12/03 14:45:00 packer-provisioner-shell plugin: Retryable error: Error uploading script: dial tcp 10.211.55.216:22: i/o timeout

$ prlctl list -f
UUID                                    STATUS       IP_ADDR         NAME
{69d061ca-4c35-4ed4-8eaf-45159a1247a0}  running      10.211.55.217   generic-devuan4-parallels

[ladar]

Replacing /etc/dhcp/dhclient.conf with the value send dhcp-client-identifier = hardware; is a workaround, to keep the IP from changing. But doesn't actually fix the underlying issue documented above.