GW Landscape - Compas module

GW Landscape Compas module for running Compas jobs from the web.

Project Structure

There are two "projects" contained in this repository - one for the Django project (backend), and one for the React project (frontend).

The frontend is in gwlandscape_compas/src/react/ and the backend is in gwlandscape_compas/src/

Project Setup

Requirements before you start

• Python 3.12+ with the virtualenv module installed
• Node Version Manager (NVM) installed
• In some instances npm run relay may raise an error that requires the watchman package to be installed.
• COMPAS (refer to the docs to install it)
• Redis server (refer to the docs)

Python/Django setup

Set up the virtual environments for GW Landscape (don't forget you'll need to create a ):

cd gwlandscape_compas/src/
virtualenv -p python3.12 venv # or whatever version of python you have installed > 3.12
source venv/bin/activate
pip install -r requirements.txt
cd $COMPAS_ROOT_DIR
pip install . # install compas_python_utils, needed for plots
cd gwlandscape_compas/src
python development-manage.py migrate

MySQL

If you want to use actual data, you will need to install and configure mysql as well. This is due to the production server using mysql and therefore difficulties in importing a mysql data dump into sqlite

• Install mysql-client and mysql-server for your distro

sudo apt-get install mysql-client mysql-server

• Install the mysqlclient python package (note - you may need to install additional system packages for this to work)
• Create schema in mysql : gwlandscape_compas
• Create mysql user and grant it all privileges on that schema
• Generate a data dump from production (this is left as an exercise to the reader i.e., get someone with production access to do it for you)
• Import the data dump with

mysql -u <username> -p gwlandscape_compas < <production_dump.sql>

• Add a local settings file at src/gw_compas/local.py and add a DATABASES= section defining the mysql connection (see environemnt.py in that directory for an example)
• Migrate the database

# Inside the venv
python development-manage.py migrate

Jobcontroller

In order to fetch results or file lists you will need to have access to the production job controller. To do this, add a JOB_CONTROLLER_JWT_SECRET= field to the local.py settings file. This JWT_SECRET can be generated by Lewis

Frontend

Set up the node environments:

cd gwlandscape_compas/src/react/
nvm install $(cat .nvmrc)
nvm use $(cat .nvmrc)
npm install
cp .env.template .env

Running the project

You will need to run in four separate terminals - one for the django host, one for the react dev-server, one for redis and one for celery.

django

cd gwlandscape_compas/src
. venv/bin/activate
python development-manage.py runserver 8000

react

cd gwlandscape_compas/src/react
npm use
npm run dev

Redis

In many cases, redis will be running as a service, so this will not be required

redis-server

Celery

cd gwlandscape_compas/src/
celery -A gw_compas.celery worker -l INFO

Graphene/Relay

Any time changes are made to the graphene schema in python, you need to rebuild the GraphQL schema for javascript

This also needs to be executed once before running the project for the first time

cd gwlandscape_compas/src/
. venv/bin/activate
python development-manage.py graphql_schema

Any time changes are made to the javascript GraphQL queries, you need to rebuild the GraphQL query files. This is not automatic, and may require stopping the node server, generating the GraphQL files, then starting the node server again.

This also needs to be executed once before running the project for the first time

cd gwlandscape_compas/src/react
nvm use
npm run relay

Accessing the project

Once the project is running, you should be able to visit http://localhost:3000/ to open the project in a browser.

Building for production

The docker configuration in the docker directory can be used to build and run the project in production.

The production build uses .env and .env.production files to determine runtime configuration and secrets, rather than src/gw_compas/local.py. The src/react/.env template will also have to be updated when building for production.

cd gwlandscape_compas/docker
sudo docker-compose up --build -d