SimHarness is a Python-based harness that wraps a SimFire environment to generate effective wildfire mitigation strategy responses via reinforcement learning (RL). Through an easy-to-use API, SimHarness can train one, or multiple, intelligent agents within the SimFire Simulation with a variety of different RL algorithms defined by RLlib.
Documentation is available here.
Conceptual workflow for training an RL model using SimHarness within the SimFire environment.
Table of Contents
SimHarness is a Python repository designed to support the training of RLlib RL algorithms within simulated disaster environments defined by SimFire. SimHarness takes as input an instance of the SimFire Simulation class, such as SimFire's FireSimulation, as the training environment. The Simulation object provides an API that allows SimHarness to move agents around the simulated environment and interact with it by placing mitigations. The FireSimulation agents represent firefighters moving through an environment as a wildfire spreads, placing mitigations such as firelines to limit the spread of the fire within the area.
The SimHarness training loop functions similarly to a traditional RL training loop, except
it expects the passed-in environment to be a child class of Simulation as opposed to a
gymnasium environment. Simulation is currently a class
within the SimFire package, but is expected to be moved to a separate,
non-disaster-specific package in the future. The simulated environment outputs training
signals such as observations and rewards to the SimHarness agent(s) which use the
observations to predict optimal actions. The actions produced by the model provide both
movement and interaction information. Movements are how the agent is traversing
across the environment, such as [nothing, up, down, left, right]. Interactions are how
the agent is changing the environment itself. In the case of SimFire, this can be
[nothing, fireline, wetline, scratchline]. These actions are relayed back to the
simulated environment, which then affects the overall disaster scenario simulated by the
environment.
*Note: SimHarness has only been tested on Ubuntu 18.04 and Python 3.9.18
-
Clone the repository.
git clone https://github.com/mitrefireline/simharness.git cd simharness -
Setup Pyenv Virtual Environment
curl https://pyenv.run | bashexport PYENV_ROOT="$HOME/.pyenv" command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH" eval "$(pyenv init -)"
sudo apt update sudo apt install \ build-essential \ curl \ libbz2-dev \ libffi-dev \ liblzma-dev \ libncursesw5-dev \ libreadline-dev \ libsqlite3-dev \ libssl-dev \ libxml2-dev \ libxmlsec1-dev \ llvm \ make \ tk-dev \ wget \ xz-utils \ zlib1g-dev
pyenv install 3.9.18 pyenv local 3.9.18 -
Install poetry
curl -sSL https://install.python-poetry.org | python3 -Note: Don't forget to add
poetryto your path.export PATH="$HOME/.local/bin:$PATH"
poetry config virtualenvs.in-project true poetry env use 3.9.18 -
Install required packages
poetry shell poetry install --only main
Note: To re-enter the environment after this step, run
poetry shell.
SimHarness provides a number of tutorials within the tutorials section of the
documentation. Documentation is available here.
SimHarness is released under the Apache 2.0 license.
If you use SimHarness for your work, please cite our white paper with the following BibTex entry.
@misc{tapley2023simharness,
title={Reinforcement Learning for Wildfire Mitigation in Simulated Disaster Environments},
author={Alexander Tapley and Marissa Dotter and Michael Doyle and Aidan Fennelly and Dhanuj Gandikota and Savanna Smith and Michael Threet and Tim Welsh},
year={2023},
booktitle={"Proc. of the Tackling Climate Change with Machine Learning Workshop"},
series={NeurIPS},
howpublished={\url{https://arxiv.org/abs/2311.15925}}
}This project contains content developed by The MITRE Corporation. If this code is used in a deployment or embedded within another project, it is requested that you send an email to [email protected] in order to let us know where this software is being used.
Copyright ©2022-2023 The MITRE Corporation. ALL RIGHTS RESERVED. Approved for Public Release; Distribution Unlimited. Public Release Case Number 22-3261.
