Add AMReX as a git submodule (AMReX-Astro#762)

.gitmodules

@@ -1,3 +1,7 @@
+[submodule "amrex"]
+	path = external/amrex
+	url = https://github.com/AMReX-Codes/amrex.git
+
 [submodule "Microphysics"]
 	path = external/Microphysics
 	url = https://github.com/starkiller-astro/Microphysics.git

CHANGES.md

@@ -7,7 +7,8 @@
    * If a derived variable has multiple components, all components are now
      added to plotfiles. Previously only the first component was used. (#758)
 
-   * We have updated our workflow when it comes to Castro's microphysics.
+   * We have updated our workflow when it comes to Castro's dependencies.
+
      Previously Castro shipped with it a minimal set of microphysics that
      allowed basic problem setups like Sedov to compile, and more advanced
      setups (like ones that include nuclear burning) required downloading
@@ -26,7 +27,16 @@
      of Castro. The developer team ensures that the version of Microphysics
      that you obtain this way is consistent with the current version of Castro.
      Then, you can keep up to date with the code mostly as normal, except now
-     using git pull --recurse-submodules instead of git pull. (#760)
+     using git pull --recurse-submodules instead of git pull.
+
+     Similarly, AMReX is now maintained as a git submodule rather than as an
+     external standalone installation. If you use the same git submodule command
+     as above, you'll obtain AMReX. As with Microphysics, you may opt to
+     rely on your own installation of AMReX by setting the AMREX_HOME
+     environment variable. However you are then responsible for keeping it
+     in sync with Castro; if you use the submodule, then you'll get the version
+     of AMReX that we have tested to ensure compatibility with the current
+     version of Castro. (#651, #760, #762, #765)
 
    * The names of the conserved state variables in C++ (Density, Xmom, etc.)
      have been changed to match the names in Fortran (URHO, UMX, etc.).

Docs/source/getting_started.rst

@@ -12,66 +12,40 @@ Getting Started
 Downloading the Code
 ====================
 
-Castro is built on top of the AMReX framework. In order to run Castro 
-you must download two separate git modules. First, make sure that git
+Castro is maintained as a repository on GitHub, and can be obtained
+via standard git clone commands. First, make sure that git
 is installed on your machine—we recommend version 1.7.x or higher.
 
 
-#. Clone/fork the AMReX repository from the AMReX-Codes
-   github page (https://github.com/AMReX-Codes/amrex/). To
-   clone via the command line, simply type::
-
-       git clone https://github.com/AMReX-Codes/amrex.git
-
-   Alternately, if you have a github account with your
-   machine’s SSH-keys registered, you can do::
-
-       git clone [email protected]:AMReX-Codes/amrex.git
-
-   This will create a directory called amrex/ on your machine.
-   You will want to periodically update AMReX by typing::
-
-       git pull
-
-   in the ``amrex/`` directory.
-
-   Note: active development is done on the ``development`` branch in
-   each repo, and merged into the ``master`` branch periodically.  If
-   you wish to use the Castro ``development`` branch, then you should
-   also switch to the ``development`` branch for AMReX.
-
-#. Set the environment variable, ``AMREX_HOME``, on your
-   machine to point to the path name where you have put AMReX.
-   You can add this to your ``.bashrc`` as::
-
-       export AMREX_HOME=/path/to/amrex/
-
-   where you replace ``/path/to/amrex/`` will the full path to the
-   ``amrex/`` directory.
-
-#. Clone/fork the Castro repository from the same
-   github organization as above, using either HTTP access::
+#. Clone/fork the Castro repository from the AMReX-Astro GitHub
+   organization, using either HTTP access::
 
        git clone --recursive https://github.com/AMReX-Astro/Castro.git
 
-   or SSH access if you have it enabled::
+   or SSH access if you have an SSH key enabled with GitHub::
 
        git clone --recursive [email protected]:AMReX-Astro/Castro.git
 
-   As with AMReX, development on Castro is done in the
-   ``development`` branch, so you should work there if you want
-   the latest source.
-
    The ``--recursive`` option to ``git clone`` is used to ensure
    that all of Castro's dependencies are downloaded. Currently this
-   requirement is for the Microphysics repository from the starkiller-astro
-   organization on GitHub. This adds the equations of state, reaction
+   requirement is for the AMReX mesh refinement framework, which is
+   maintained in the AMReX-Codes organization on GitHub, and the
+   Microphysics repository from the starkiller-astro organization.
+   AMReX adds the necessary code for the driver code for the simulation,
+   while Microphysics adds the equations of state, reaction
    networks, and other microphysics needed to run Castro. If you forget
    to do a recursive clone, you can rectify the situation by running
    the following from the top-level of the Castro directory::
 
        git submodule update --init --recursive
 
+#. Development on Castro (and its primary dependencies, AMReX and
+   Microphysics) is done in the ``development`` branch, so you should
+   work there if you want the latest source. The Castro team runs
+   nightly regression testing on the ``development`` branch, so bugs
+   are usually found and fixed relatively quickly, but it is generally
+   less stable than staying on the ``master`` branch.
+
 #. We recommend setting the ``CASTRO_HOME`` environment
    variable to point to the path name where you have put Castro.
    Add the following to your .bashrc::
@@ -82,24 +56,26 @@ is installed on your machine—we recommend version 1.7.x or higher.
 
        git pull --recurse-submodules
 
-   The recommended frequency for doing this is monthly, if you want the
-   stable version of the code; we issue a new release of the code at the
-   beginning of each month. For more frequent updates, you may work on
-   the ``development`` branch, at the risk of a less stable codebase.
+   The recommended frequency for doing this is monthly, if you are on the
+   stable ``master`` branch of the code; we issue a new release of the code
+   at the beginning of each month.
 
-#. (optional, for developers) If you prefer, you can maintain Microphysics
-   as a standalone repository rather than a git submodule. To do so, you can
-   clone it from GitHub using::
+#. (optional, for developers) If you prefer, you can maintain AMReX and
+   Microphysics as standalone repositories rather than as git submodules.
+   To do so, you can clone them from GitHub using::
 
+       git clone https://github.com/AMReX-Codes/amrex.git
        git clone https://github.com/starkiller-astro/Microphysics.git
 
    or via SSH as::
 
+       git clone [email protected]:/AMReX-Codes/amrex.git
        git clone [email protected]:/starkiller-astro/Microphysics.git
 
-   Then, set the ``MICROPHYSICS_HOME`` environment variable to point to
-   the ``Microphysics/`` directory, and Castro will look there instead
-   of in its local ``external/Microphysics/`` subdirectory.
+   Then, set the ``AMREX_HOME`` environment variable to point to the
+   ``amrex/`` directory, and the ``MICROPHYSICS_HOME`` environment
+   variable to point to the ``Microphysics/`` directory. Castro will
+   look there instead of in its local ``external/`` subdirectory.
 
 Building the Code
 =================

Docs/source/releases.rst

@@ -2,23 +2,28 @@
 Castro Releases
 ***************
 
-This outlines the procedure for doing the month Castro release.
+This outlines the procedure for doing the monthly Castro release.
 
 Castro uses submodules for dependencies, this means that, at a
-minimum, we must update the Microphysics submodule monthly when we
-issue new releases. The release for Microphysics must be done
-first. Then navigate to the Microphysics directory, checkout the new
+minimum, we must update the AMReX and  Microphysics submodules monthly when we
+issue new releases. The releases for AMReX and Microphysics must be done
+first. Then navigate to each submodule directory, checkout the new
 tag, and then from the top-level directory of Castro do a "git add" on
-the Microphysics directory to store the new tag. So, for example, at
+the ``external/`` directory to store the new tags. So, for example, at
 the beginning of March 2020 we would first issue the ``20.03`` tag on
-Microphysics, then do::
+Microphysics, and wait for AMReX to release a ``20.03`` tag, then do::
 
-   cd $CASTRO_HOME/Microphysics
+   cd $CASTRO_HOME/external
+   cd amrex
    git pull
    git checkout 20.03
    cd ..
-   git add Microphysics
-   git commit -m "Update Microphysics to release 20.03"
+   cd Microphysics
+   git pull
+   git checkout 20.03
+   cd ..
+   git add -u .
+   git commit -m "Update AMReX and Microphysics to release 20.03"
 
 Then we can proceed with issuing our own release.
 
@@ -60,4 +65,4 @@ the Castro development branch in the same way, replacing the git
 checkout statement with the latest commit hash on the Microphysics
 development branch. A git submodule always tracks a specific
 commit/tag on the target repo -- it is not configured to automatically
-track a particular branch.
+track a particular branch. A similar process applies for AMReX.

Exec/Make.Castro

@@ -1,4 +1,3 @@
-AMREX_HOME ?= /path/to/amrex
 CASTRO_HOME ?= /path/to/Castro
 
 # radiation needs hypre
@@ -9,6 +8,12 @@ BLAS_LIBRARY ?= -lopenblas
 
 TOP := $(CASTRO_HOME)
 
+# Microphysics is a git submodule of Castro. By default
+# we assume it is in the external/ directory.
+# The user may override this with their own installation
+# of Microphysics as long as they set the environment variable
+# MICROPHYSICS_HOME.
+
 MICROPHYSICS_HOME ?= $(TOP)/external/Microphysics
 
 # Check to make sure that Microphysics actually exists,
@@ -18,12 +23,27 @@ ifeq ("$(wildcard $(MICROPHYSICS_HOME)/Make.Microphysics)","")
   $(error Microphysics has not been downloaded. Please run "git submodule update --init" from the top level of the code)
 endif
 
-# the Castro microphysics locations
+# Locations within Microphysics
 EOS_HOME ?= $(MICROPHYSICS_HOME)/EOS
 NETWORK_HOME ?= $(MICROPHYSICS_HOME)/networks
 COND_HOME ?= $(MICROPHYSICS_HOME)/conductivity
 VISC_HOME ?= $(MICROPHYSICS_HOME)/viscosity
 
+# AMReX is a git submodule of Castro. By default
+# we assume it is in the external/ directory.
+# The user may override this with their own installation
+# of AMReX as long as they set the environment variable
+# AMREX_HOME.
+
+AMREX_HOME ?= $(TOP)/external/amrex
+
+# Check to make sure that AMReX actually exists,
+# using an arbitrary file that must exist to check.
+# Throw an error if we don't have AMReX.
+ifeq ("$(wildcard $(AMREX_HOME)/Tools/GNUMake/Make.defs)","")
+  $(error AMReX has not been downloaded. Please run "git submodule update --init" from the top level of the code)
+endif
+
 # default integrator
 INTEGRATOR_DIR ?= VODE90