Skip to content

Installation instructions

Jump to bottom
vingman edited this page Apr 20, 2021 · 28 revisions

Table of Contents

Installing dependencies

  • Follow AaronTools installation or simply pip install "AaronTools[extra]". Be sure to also create your $AARONLIB directory and add any necessary environmental variables, as described here (Linux/Mac) or here (Windows).
  • Jinja2: pip install Jinja2 for script template handling (only need to run this command if not using "[extra]" flag to install AaronTools via pip)
  • Paramiko: pip install paramiko for file transfer tasks and communicating with a remote HPC
  • Matplotlib: pip install matplotlib (optional for most tasks, but required to use AaronJr plot command)
  • FireWorks: pip install FireWorks Needs to be installed on both the computer that will be running AaronJr and the HPC responsible for running the jobs.
  • MongoDB: You can either set up locally or via a cloud provider. Details on how to connect using AaronJr are below.

Connecting with a remote HPC

Set up SSH public key authentication and ensure you can SSH into the HPC without providing a password. Do not set a passphrase when creating your key pair (if this is an issue for you, let us know and we will add support for password authentication or public key with passphrase authentication).

Mac/Linux

Follow these instructions, for example: On your local computer: 

$ ssh-keygen
$ ssh-copy-id -i ~/.ssh/id_rsa [email protected]
If ssh-copy-id does not work, you can add your public key to the remote computer's list of authorized keys manually. 
$ scp ~/.ssh/id_rsa.pub [email protected]:/home/user/.ssh/add_this_key.pub
$ ssh [email protected]
$ cat ~/.ssh/add_this_key.pub >> ~/.ssh/authorized_keys
$ rm ~/.ssh/add_this_key.pub

Windows

Follow these instructions to if using OpenSSH and PowerShell.

Follow these instructions if using PuTTY.

Connecting to the FireWorks database server

If you installed MongoDB yourself, you may want to go through the security checklist to enable authentication, review access control options, and create an admin user. You may wish to only allow connections from the HPC; if so this page and this page should be helpful.


1. Create a user for the FireWorks database, or ask your system administrator to do so for you. For example, using the mongo shell to add a user to a database named 'fireworks' (the mongo command may require additional command line options to authenticate properly): 

$ mongo
use fireworks
db.createUser(
  {
    user: "myuser",
    pwd: passwordPrompt(),  // or cleartext password
    roles: [
       { role: "readWrite", db: "fireworks" }
    ]
  }
)
You can name the FireWorks database whatever you wish, but note that 'fireworks' will be assumed in any examples provided in this wiki.


2. The command lpad init provided by the FireWorks package can be used to generate the file needed to connect. Alternatively, you can use lpad init --uri to connect with a URI string (this is often the easiest method when using a cloud provider for database storage). Save this file to ~/.fireworks/my_launchpad.yaml on both your local machine and the HPC. You may want to restrict permissions to this file if password authentication is used, as the password is stored in plain text. Consider using certificate-based authentication if this is unacceptable. 

$ cat ~/.fireworks/my_launchpad.yaml
host: mongodb://myuser:[email protected]:27017/fireworks
uri_mode: true

$ chmod 600 ~/.fireworks/my_launchpad.yaml
$ ssh [email protected]
% mkdir ~/.fireworks
% exit
$ scp ~/.fireworks/my_launchpad.yaml [email protected]:~/.fireworks/my_launchpad.yaml
If you are using your local computer as the MongoDB host and you do not have a permanent IP address or DNS lookup name registered, this bash script may be helpful: 
ssh_client_ip=$(echo $SSH_CLIENT | awk '{ print $1  }')
echo "host: mongodb://username:password@${ssh_client_ip}:27017/fireworks" > $HOME/.fireworks
echo "uri_mode: true" >> $HOME/.fireworks/my_launchpad.yaml
On the remote HPC, save this script to a file (making sure you edit with the correct username, password, and database name) chmod 600 /path/to/script.sh, and add source /path/to/script.sh to your ~/.bashrc or similar. It will update the YAML file with your local machine's IP address every time you log in to the remote host.


3. Ensure you can connect to the FireWorks launchpad by running lpad get_wflows on both the machine that will be running AaronJr and the HPC. It may be necessary to first run lpad reset if the database has never been used before.

Clone this wiki locally