Modernize vagrant environment so it is compatible with current Macs (M1)

[obrienad]

The current deployment and development setup uses Vagrant and Virtualbox with x86 Ubuntu images (18.04) which used to make developing or deploying on an M1 Mac essentially impossible.

However, there are newer dependencies available that may allow us to make the project functional on M1 Macs, specifically:

• There are beta releases of VirtualBox that are M1-compatible
• There are newer Ubuntu ARM-based VM images available--20.04 onward--that we can try using

So there now seems to be a possible path toward M1 compatibility, albeit one that depends on a beta version of VirtualBox. The alternative options are:

1. Provisioning an x86-based build box, either for this project or for multiple projects, that allows M1 users to deploy and/or develop on the project. This would be a shared resource across multiple team members that would need to be maintained.
2. Switch the application to use containers and docker compose. This would probably need to apply to the deployment as well; it would theoretically be possible to continue building AMIs with Ansible and deploying to simple EC2 instances (because that code isn't architecture-specific), but then the development environment would be worryingly different from the deployment environment.

Given the above, we should make a timeboxed attempt to allow the existing vagrant + Virtualbox development environment to run on M1 Macs (while preserving existing functionality for x86 users).

If I had an M1 and were doing this, I would probably attempt something like the following process (I'm envisioning this card as a timebox, so we don't need to finish up every single task in this list, but we should do enough that we feel reasonably confident that there aren't any hidden blockers that we don't know about):

1. Consider the VMs (there are 3) individually.
2. Start with the database VM because that is a pretty simple setup that runs vanilla Postgres, so I would expect that all the dependencies would be fairly well tested on ARM processors. Currently, PostGIS is being built from source, but I think that's only for versioning reasons -- feel free to jettison that if we can get a more modern version from the OS packages now (we probably can).
3. Assuming the database VM can be provisioned on M1 successfully, next attempt the otp VM, which runs OpenTripPlanner (OTP). OTP is a Java application, and it probably has the worst combination of complexity, age, and low maintenance of any dependency we use in the app, so if we're going to encounter total blockers I would expect to find them here (but I still think we should start with the Postgres VM because I expect that to be the easiest, which should help us isolate OTP-specific issues from more general Virtualbox-on-ARM issues).
   Note: Reach out to me at this stage for a copy of what should be in the otp_data folder to successfully provision the OTP VM. The generation of this data is somewhat complicated and runs in a separate environment that is already M1-compatible, so you don't need to learn how to create the data to complete the task at hand.
4. Assuming those both can be provisioned, finish up with the app VM. It has the most going on because it handles the frontend and Django, but I don't have any reason to think that either of those should be problematic to run on M1s.
5. Assuming provisioning seems to be working individually, try to operate all the VMs at the same time and give the (hopefully) working app a thorough once-over to make sure all functionality works as expected.
6. Adjust READMEs and add in any configuration settings or auto-detection needed in order to run on both architectures.

[rajadain]

The M1/M2 Mac ARM 7.0 Beta Technical Preview is a work in progress that will take some time to complete. The eventual goal is to run typical modern Intel/AMD x86 32 and 64 bit OS's on an ARM M1/M2 Mac.
The devs are continuing development on the Beta. They also plan to have an ARM on ARM Virtualbox hypervisor eventually.
No dates for expected final development of these goals have been or will be released.

https://forums.virtualbox.org/viewtopic.php?t=107344

In a way the ARM64 package "slipped out", and it's not expected to work reliably. The implementation isn't complete yet (which is what you saw), and in top of that the performance is known to be extremely low. It isn't anywhere near production ready, we know. This will not change for VirtualBox 7, and the "Technology Preview" marker will stay for the foreseeable future, indicating that it won't be supported at all.

https://forums.virtualbox.org/viewtopic.php?f=15&t=106919&p=523035#p523069

---

Doesn't look like VirtualBox for M1 will be available anytime soon.

[rajadain]

UTM, which is a GUI wrapper around qemu, does support having ARM guests on M1 machines. Don't think it quite has Vagrant support yet: hashicorp/vagrant#12518 utmapp/UTM#3718.

Seeing how far I can get with the UTM / qemu approach today.

[rajadain]

Making some progress with the vagrant-qemu plugin which allows using qemu as a provider, and the https://app.vagrantup.com/perk/boxes/ubuntu-20.04-arm64 box which is built for the qemu provider.

Running into issues with network and folder sharing support, which seem not yet available with qemu: ppggff/vagrant-qemu#40 (comment), https://github.com/ppggff/vagrant-qemu#todo.

[rajadain]

I have not been able to fix the network and file sharing support, and have also been unable to find an alternative to qemu or that vagrant-qemu plugin. Think we should pull the plug on this and just use Intel machines for now.

[ddohler]

Surfacing an offline discussion: we're going to move forward with plan B here, which is spinning up a shared x86-based build environment for this and other vagrant-based projects.