diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6d8705cfa..ca071dead 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -187,7 +187,7 @@ jobs: shell: bash -l {0} run: | mkdocs build --site-dir build/html - ford docs/udales-docs-software.md + ford ford.md - name: Upload docs artifact uses: actions/upload-artifact@v7 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2f2173370..57dae60e7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # How to contribute -In general, you can contribute by reporting an issue or by directly contributing to the source code. For the latter, clone the repository, make your changes, and create a pull request (PR) with a **clear description** of your changes -- if you are unfamiliar with creating PRs, please see [this guide](https://guides.github.com/activities/forking/#making-a-pull-request) first. Please also read the [development notes](./DEVELOP.md) before opening an issue or create a PR. For specific instructions on how to report a bug or how to submit a feature request, please see below: +In general, you can contribute by reporting an issue or by directly contributing to the source code. For the latter, clone the repository, make your changes, and create a pull request (PR) with a **clear description** of your changes -- if you are unfamiliar with creating PRs, please see [this guide](https://guides.github.com/activities/forking/#making-a-pull-request) first. Please also read the [development notes](https://udales.github.io/u-dales/udales-development/) before opening an issue or creating a PR. For specific instructions on how to report a bug or how to submit a feature request, please see below: - [How to contribute](#how-to-contribute) - [Report a bug](#report-a-bug) @@ -9,7 +9,7 @@ In general, you can contribute by reporting an issue or by directly contributing ## Report a bug -Before creating bug reports, please check if similar issue have already been reported [here](https://github.com/uDALES/u-dales/issues). If none exist please create a new issue and include as many details as possible using the required template. +Before creating bug reports, please check if similar issues have already been reported [here](https://github.com/uDALES/u-dales/issues). If none exist please create a new issue and include as many details as possible using the required template. ## Request a new feature diff --git a/DEVELOP.md b/DEVELOP.md index c775007d0..f3869059d 100644 --- a/DEVELOP.md +++ b/DEVELOP.md @@ -2,7 +2,7 @@ ## Set up -Install all required packages for uDALES described in the [prerequisites section](https://udales.github.io/u-dales/udales-getting-started/#prerequisites-when-not-using-singularity), plus optionally [Graphviz](https://graphviz.org/) for generating graphs in the code viewer. E.g. installing all the required packages using Ubuntu's APT: +Install all required packages for uDALES described in the [prerequisites section](https://udales.github.io/u-dales/udales-installation/#prerequisites), plus optionally [Graphviz](https://graphviz.org/) for generating graphs in the code viewer. E.g. installing all the required packages using Ubuntu's APT: ```sh sudo apt update && sudo apt install -y gfortran libopenmpi-dev openmpi-bin libnetcdf-dev libnetcdff-dev graphviz @@ -16,43 +16,26 @@ conda env create -f environment.yml Then activate with `conda activate udales`. -If a shared virtual environment already exists at `../.venv/`, you can -also use that directly for Python tooling and tests, for example: +For the Python tooling (pre-processing and the Python package tests), create the project virtual environment with: ```sh -../.venv/bin/python -m unittest tools/python/tests/test_namelist.py +./tools/python/setup_venv.sh common ``` -The Python radiation tooling also requires the compiled View3D executable and -the `directshortwave_f2py` wrapper. The simplest setup path is: +This creates the environment at `tools/python/.venv`, installs all dependencies, and builds the compiled preprocessing extensions (View3D and the f2py modules). See [tools/python/README_VENV.md](https://github.com/uDALES/u-dales/blob/master/tools/python/README_VENV.md) for details. -```sh -PYTHON_BIN=/opt/pbs/python/bin/python3 ./tools/python/setup_venv.sh -``` - -Use another interpreter only if it provides the matching Python development -headers needed by f2py. - -New Linux/WSL setups now default to `../.venv/`, matching the Windows -setup script. If you already have a repo-local `.venv/`, the setup script will -keep using it until you migrate manually. - -## Installation +## Building for development -To install uDALES on Linux, macOS, and WSL, use the following commands from the command prompt: +Building the model is described in the [installation guide](https://udales.github.io/u-dales/udales-installation/); the same instructions apply for development. In addition, developers will usually want a `Debug` build, which enables runtime checks and floating-point exception trapping: ```sh -mkdir -p build/release -pushd build/release -cmake ../.. +mkdir -p build/debug +pushd build/debug +cmake -DCMAKE_BUILD_TYPE=Debug ../.. make ``` -To know more about build options, please see [build/default options](https://udales.github.io/u-dales/udales-getting-started/#build-defaultsoptions). - -## Running - -A uDALES simulation needs to be executed from a directory containing all required input files. Examples of experiments and required inputs are in the `examples` directory. To run a uDALES simulation you need to specify the number of cpus ``, the path to the build file `` and the simulation configuration file `` and execute the simulation with the following command: +For quick testing without the wrapper scripts, the solver can be run directly from a directory containing all input files: ``` sh mpiexec -n @@ -60,15 +43,19 @@ mpiexec -n ## Testing -Please refer to [Test docs](https://github.com/uDALES/u-dales/blob/master/tests/README.md). +Tests are dispatched from a manifest with `tests/run_tests.py` (suites are defined in `tests/test_suites.yml`). Please refer to the [test docs](https://github.com/uDALES/u-dales/blob/master/tests/README.md) for the test layout and execution contracts. ## Documentation +The user documentation is built with [MkDocs](https://www.mkdocs.org/) (Material theme) and the software docs with [FORD](https://github.com/Fortran-FOSS-Programmers/ford); both are installed by the conda environment above. To build: + ```sh mkdocs build --site-dir build/html -ford docs/udales-docs-software.md +ford ford.md ``` +For live preview while editing, use `mkdocs serve`. + ### Examples input plots To create domain plots of the examples, run the following from your command line (requires MATLAB): @@ -77,7 +64,7 @@ To create domain plots of the examples, run the following from your command line matlab -nosplash -nodesktop -r "cd('tools/examples'); plot_blocks(''); quit" ``` -where `` is e.g. `201`. Plots are then saved in their respective example folders. +where `` is e.g. `201`. Plots are then saved in their respective example folders. ### Examples outputs and plots @@ -93,7 +80,6 @@ Then, to create a sample plot for case `102` run the following from your command matlab -nosplash -nodesktop -r "cd('tools/examples'); plot_fielddump_slice('102','u','y',32,1); quit" ``` - ## Versioning This project uses [semantic versioning](https://semver.org/). diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md deleted file mode 120000 index 44fcc6343..000000000 --- a/docs/CONTRIBUTING.md +++ /dev/null @@ -1 +0,0 @@ -../CONTRIBUTING.md \ No newline at end of file diff --git a/docs/DEVELOP.md b/docs/DEVELOP.md deleted file mode 120000 index c1906c09b..000000000 --- a/docs/DEVELOP.md +++ /dev/null @@ -1 +0,0 @@ -../DEVELOP.md \ No newline at end of file diff --git a/docs/cluster_workflows.md b/docs/cluster_workflows.md index a52c852c0..95afbe6d8 100644 --- a/docs/cluster_workflows.md +++ b/docs/cluster_workflows.md @@ -85,14 +85,15 @@ Use the gather wrapper to collect outputs after the run: ## Python Environment -Use the repo-local virtual environment created by `setup_venv.sh`: +Create the project virtual environment with the setup script, which also +builds the preprocessing tools (View3D and the f2py extension modules): ```bash -tools/python/.venv +./tools/python/setup_venv.sh icl ``` -When activating that environment on the cluster, load the matching Python module -first so the runtime libraries are available: +When activating the environment on the cluster, load the matching Python +module first so the runtime libraries are available: ```bash module load Python/3.13.1-GCCcore-14.2.0 @@ -101,7 +102,7 @@ source tools/python/.venv/bin/activate Use the same Python module for repo Python workflows on the cluster. In particular, the `f2py`-based extensions in this repository are expected to be -built and run with the same Python runtime rather than whichever `python3` +built and run with the same Python runtime environment above rather than whichever `python3` happens to be first on `PATH`. ## Interactive Analysis @@ -129,14 +130,14 @@ job environment. In particular: `PATH` after loading modules - batch-style output redirection patterns may fail interactively even when they work inside submitted jobs -- Codex sandboxed sessions can add another layer of difference: a launcher - failure seen inside the sandbox may be a sandbox socket restriction rather - than a real cluster-side problem +- sandboxed agent sessions (Codex, Claude Code) can add another layer of + difference: a launcher failure seen inside the sandbox may be a sandbox + socket restriction rather than a real cluster-side problem So for interactive debugging: - prefer reproducing the environment from the repo wrappers -- if a login-node MPI launch fails inside Codex, retry it outside the sandbox +- if a login-node MPI launch fails inside an agent sandbox, retry it outside the sandbox before treating it as a solver or cluster configuration issue - keep the launcher invocation minimal - avoid changing MPI launcher behavior and output handling unless you have diff --git a/docs/index.md b/docs/index.md index 704a5a116..71b85e173 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,12 +1,67 @@ -# uDALES Docs +# uDALES -This is the documentation for uDALES, an open-source multi-physics microscale urban modelling tool, capable of performing large-eddy simulation (LES) of urban airflow, heat transfer, and pollutant dispersion. This documentation contains user's guides and general reference documentation. +uDALES is an open-source multi-physics microscale urban modelling tool, capable of performing large-eddy simulation (LES) of urban airflow, heat transfer, and pollutant dispersion. It resolves buildings with a conservative immersed boundary method, includes a full urban surface energy balance with radiation, vegetation, and moisture, and scales to large domains on HPC systems. See the [model overview](udales-model-overview.md) for what uDALES can do. | | | | |:--:|:--:|:--:| | ![uDALES Temperature](./assets/images/T_5fps.gif) | ![uDALES Momentum](./assets/images/M_5fps.gif) | ![uDALES Vertical Velocity](./assets/images/w_5fps.gif) | -For a quick introduction and instructions on installing and running uDALES, see the "Installation and First Run" page: [Installation and First Run](./udales-installation.md). +
-For contributor guidance on using agents in this repo, see: -[Using Agents](./udales-agents.md). +- :material-download:{ .lg .middle } **Installation** + + --- + + Build uDALES on Linux, macOS, WSL, or HPC clusters. + + [:octicons-arrow-right-24: Installation and first run](udales-installation.md) + +- :material-map-marker-path:{ .lg .middle } **Workflow overview** + + --- + + The end-to-end simulation workflow: set up, pre-process, run, and post-process. + + [:octicons-arrow-right-24: Workflow overview](udales-workflow.md) + +- :material-tune:{ .lg .middle } **Simulation inputs** + + --- + + Configure your simulation via the `namoptions` input parameters and generate urban geometries. + + [:octicons-arrow-right-24: Input parameters](udales-namoptions-overview.md) + +- :material-play:{ .lg .middle } **Example simulations** + + --- + + Ready-to-run example cases, from neutral flow over cubes to full urban energy balance setups. + + [:octicons-arrow-right-24: Example simulations](udales-example-simulations.md) + +- :material-chart-line:{ .lg .middle } **Post-processing tutorials** + + --- + + Load and analyse uDALES output with the `udbase` class: fields, statistics, and facet data. + + [:octicons-arrow-right-24: Tutorials](udales-udbase-tutorial.md) + +- :material-language-python:{ .lg .middle } **Python package** + + --- + + The next-generation pre- and post-processing toolchain, currently under active testing. + + [:octicons-arrow-right-24: Python package](udales-python-package.md) + +
+ +## Citing uDALES + +When using uDALES or data generated by uDALES, please cite the model papers — see [How to cite](udales-how-to-cite.md). A list of publications using uDALES is available in the [publication list](udales-pub-list.md). + +## Contributing + +uDALES is developed openly on [GitHub](https://github.com/uDALES/u-dales). See the [developer documentation](udales-development.md) and [contribution guide](udales-contributing.md) to get involved. diff --git a/docs/javascripts/mathjax.js b/docs/javascripts/mathjax.js new file mode 100644 index 000000000..0525879da --- /dev/null +++ b/docs/javascripts/mathjax.js @@ -0,0 +1,16 @@ +window.MathJax = { + tex: { + inlineMath: [["\\(", "\\)"]], + displayMath: [["\\[", "\\]"]], + processEscapes: true, + processEnvironments: true + }, + options: { + ignoreHtmlClass: ".*|", + processHtmlClass: "arithmatex" + } +}; + +document$.subscribe(() => { + MathJax.typesetPromise(); +}); diff --git a/docs/references.bib b/docs/references.bib index d989b52a0..f92cb00fc 100644 --- a/docs/references.bib +++ b/docs/references.bib @@ -1,9 +1,53 @@ +@article{Wilson2026, + author = {Wilson, C. E. and Shonk, J. K. P. and Bohnenstengel, S. I. and Paschalis, A. and van Reeuwijk, M.}, + title = {Parameter investigation for urban surface-energy balance: A large-eddy simulation study}, + journal = {Q. J. Roy. Met. Soc.}, + volume = {152}, + number = {777}, + pages = {e70154}, + year = {2026}, + doi = {10.1002/qj.70154} +} + +@article{Fellini2026, + author = {Fellini, S. and Majumdar, D. and Salizzoni, P. and van Reeuwijk, M.}, + title = {Three-dimensional pollutant dispersion in tree-lined urban canyons: Combined wind-tunnel and LES analysis}, + journal = {Atmos. Env.}, + volume = {367}, + pages = {121748}, + year = {2026}, + doi = {10.1016/j.atmosenv.2025.121748} +} + @article{Owens2025, - url={https://dx.doi.org/10.2139/ssrn.5125973}, - year = {(awaiting publication)}, author = {Owens, S. and Beckett, O. and Acred, A. and van Reeuwijk, M.}, - title = {Large-Eddy Simulation of a Diurnal Cycle in a Coastal Urban Environment.}, - journal = {Building and Environment} + title = {Large-Eddy Simulation of a Diurnal Cycle in a Coastal Urban Environment}, + journal = {Building and Environment}, + volume = {281}, + pages = {112999}, + year = {2025}, + doi = {10.1016/j.buildenv.2025.112999} +} + +@article{Wilson2025, + author = {Wilson, C. and Shonk, J. K. P. and Bohnenstengel, S. I. and Paschalis, A. and van Reeuwijk, M.}, + title = {Microscale to neighbourhood scale: Impact of shading on urban climate}, + journal = {Building and Environment}, + volume = {275}, + pages = {112721}, + year = {2025}, + doi = {10.1016/j.buildenv.2025.112721} +} + +@article{Reeuwijk2025, + author = {van Reeuwijk, M. and Huang, J.}, + title = {Multi-scale Analysis of Flow over Heterogeneous Urban Environments}, + journal = {Bound.-Lay. Met.}, + volume = {191}, + number = {11}, + pages = {47}, + year = {2025}, + doi = {10.1007/s10546-025-00941-w} } @article{Majumdar2025, diff --git a/docs/udales-agents.md b/docs/udales-agents.md index 2fbd9c4f3..fa76332a6 100644 --- a/docs/udales-agents.md +++ b/docs/udales-agents.md @@ -1,7 +1,13 @@ # Using Agents -This page describes how we use Codex agents for uDALES development work. -It is aimed at contributors working on this repository. +This page describes how we use AI coding agents — currently OpenAI Codex and +Anthropic Claude Code — for uDALES development work. It is aimed at +contributors working on this repository. + +!!! note "Improved agent instructions coming" + A revised, expanded set of agent instructions is under development on the + [`performance` branch](https://github.com/uDALES/u-dales/tree/performance) + and will supersede parts of this page once merged. ## When to use an agent @@ -22,7 +28,7 @@ Use an agent when the task benefits from: Project-specific instructions for agents live in: -- `AGENTS.md` (repo-level guidance) +- `AGENTS.md` (repo-level guidance, read by both Codex and Claude Code) - `.github/skills/udales-exec/` (cluster execution runbook skill) - `.github/skills/udales-detect/` (environment detection skill) diff --git a/docs/udales-architecture.md b/docs/udales-architecture.md new file mode 100644 index 000000000..aeb29eb41 --- /dev/null +++ b/docs/udales-architecture.md @@ -0,0 +1,118 @@ +# Code architecture + +This page is a tour of the uDALES source code for developers who are about to modify the solver: where things live, how the Fortran modules are organised, and what happens — in order — during one timestep. + +## Repository layout + +- `src/` — the Fortran MPI solver (one flat directory of `mod*.f90` files plus `program.f90`). +- `tools/python/` — the current Python tooling: `udbase` (postprocessing), `udprep` (preprocessing), `udgeom` (geometry), plus tests and example notebooks. +- `tools/matlab/` — legacy MATLAB postprocessing and geometry helpers. +- `tools/IBM/` — immersed-boundary preprocessing utilities. +- `tools/SEB/` — surface energy balance preprocessing utilities. +- `tools/syntheticInflow/` — synthetic-inflow generation tooling. +- `tools/View3D/` — external view-factor calculator (git submodule) used for radiation preprocessing. +- `tools/*.sh` — build and run wrappers (`build_executable.sh`, `hpc_execute.sh`, ...). +- `examples/` — runnable example cases (`001`, `002`, ..., `999`) with reference inputs. +- `tests/` — unit, system, integration and regression tests; see `tests/README.md`. +- `docs/` — this documentation (MkDocs). +- `2decomp-fft/` — the 2DECOMP&FFT parallel decomposition/FFT library (git submodule). + +## Module organisation + +Everything in `src/` compiles into a single executable whose entry point is `program.f90`. The modules fall into a few natural groups. + +### State and configuration + +- `modglobal.f90` — global constants, grid dimensions and indices (`ib:ie`, `jb:je`, `kb:ke`), halo widths, switches, and run-mode constants; `initglobal` derives grid/timestep settings. +- `modfields.f90` — declares and allocates the 3-D prognostic and diagnostic field arrays (`initfields`). +- `modmpi.f90` — MPI layer: communicators, rank/neighbour info, reduction helpers. + +### Startup and input + +- `modstartup.f90` — reads the `namoptions` namelists (`readnamelists`), initialises 2DECOMP (`init2decomp`), sanity-checks settings (`checkinitvalues`), and reads initial/restart fields and profiles (`readinitfiles`). +- `readinput.f90` — generic readers for the sparse (i,j,k)-indexed input file formats. +- `initfac.f90` — reads the facet input files (facet geometry, types, properties). + +### Dynamics and numerics + +- `modadvection.f90` — advection schemes for momentum and scalars. +- `modsubgrid.f90` / `modsubgriddata.f90` — subgrid-scale diffusion (Vreman, Smagorinsky, one-equation TKE models) and its variables. +- `modforces.f90` — remaining momentum sources: body forces, Coriolis, large-scale forcing, nudging, mass-flow correction. +- `modpois.f90` — the FFT-based Poisson solver for the pressure correction. +- `modtstep.f90` — adaptive timestepping and the third-order Runge–Kutta integration (Wicker & Skamarock, 2002). + +### Physics + +- `modthermodynamics.f90` — diagnostic thermodynamics (liquid water, buoyancy-related quantities). +- `modEB.f90` — the facet surface energy balance. +- `vegetation.f90` — sparse vegetation: momentum drag, canopy energy balance, scalar deposition. +- `modchem.f90` — NO/NO2/O3 null-cycle chemistry, applied once per full timestep. +- `modscalsource.f90` — point/line/planar scalar sources. +- `modpurifiers.f90`, `heatpump.f90` — air purifiers and heat-pump exhaust. +- `modsurfdata.f90` — surface-model variables. + +### Boundaries and immersed boundary method (IBM) + +- `modboundary.f90` — all domain boundary conditions (except immersed boundaries), top-of-domain damping, and halo updates. +- `modibm.f90` / `modibmdata.f90` — immersed-boundary forcing: wall shear (`ibmwallfun`), zero normal velocities (`ibmnorm`), floor treatment (`bottom`). +- `modwallfunctions.f90` — wall-function fluxes (`wfuno`, `wfmneutral`) used by the IBM and facet code. +- `modinlet.f90` / `modinletdata.f90` — turbulent inflow generation by rescale–recycle (Lund et al.). +- `moddriver.f90` — driver simulations: writing and replaying precursor inlet planes. +- `modtimedep.f90` — time-dependent boundary values and forcings. + +### Output and monitoring + +- `modsave.f90` — restart files (`writerestartfiles`). +- `modfielddump.f90` — instantaneous 3-D field output (`fielddump*.nc`). +- `modstatsdump.f90` / `modstatistics.f90` — time- and space-averaged statistics output. +- `modstat_nc.f90` — low-level NetCDF writing routines used by the dump modules. +- `modchecksim.f90` — monitors Courant/Péclet numbers and divergence during the run. +- `tests.f90` — in-solver test routines dispatched via special `runmode` values. + +## Life of a timestep + +`program.f90` is short and readable; it is the best map of the solver. It has three phases. + +**Startup.** In order: `initmpi`, `readnamelists`, `init2decomp`, `checkinitvalues`, `initglobal`, then the `init*` routines of the other modules (`initfields`, `initboundary`, `initthermodynamics`, `initsubgrid`, `initdriver`, `initpois`), the IBM/facet setup (`readfacetfiles`, `initibm`, `createmasks`, `calcfluidvolumes`), the initial fields (`readinitfiles`, `createscals`), and finally the output and add-on modules (`initstatsdump`, `initEB`, `inittimedep`, `initfielddump`, `boundary`, `init_vegetation`, `createpurifiers`, `init_heatpump`). + +**Time loop.** The loop is `do while ((timeleft>0) .or. (rk3step < 3))`, and — important — **one pass through the loop is one RK3 substep**, not one full timestep. Each substep, every process adds its contribution to the tendency arrays (`up`, `vp`, `wp`, `thlp`, ...), and `tstep_integrate` applies them at the end. The ordered calls, with the module implementing each: + +1. `tstep_update` (`modtstep`) — advance `rk3step` (1→2→3→1...); at the start of substep 1 choose a new `dt` from the CFL and diffusion-number limits (if `ladaptive`). +2. `timedep` (`modtimedep`) — update time-dependent forcings. +3. `advection` (`modadvection`) — advection tendencies (includes the predicted pressure-gradient term). +4. `shiftedPBCs` (`modforces`) — shifted periodic boundary conditions. +5. `subgrid` (`modsubgrid`) — subgrid-scale diffusion tendencies. +6. `bottom` (`modibm`) — surface layer at the domain floor. +7. `coriolis`, `forces`, `lstend`, `nudge` (`modforces`) — Coriolis, remaining body forces, large-scale forcings, top-cell nudging. +8. `ibmwallfun` (`modibm`) — immersed-boundary wall shear (via `modwallfunctions`), then `periodicEBcorr` (`modforces`). +9. `masscorr` (`modforces`) — correct the predicted velocity for the prescribed mass flow. +10. `ibmnorm` (`modibm`) — set velocity components normal to immersed boundaries to zero. +11. `EB` (`modEB`), `vegetation_forcing` (`vegetation`), `heatpump` (`modheatpump`), `scalsource` (`modscalsource`) — facet energy balance, vegetation, heat pump, and scalar-source terms. +12. `fixuinf2`, `fixuinf1` (`modforces`) — free-stream velocity controllers. +13. `grwdamp` (`modboundary`) — gravity-wave damping near the top of the domain. +14. `poisson` (`modpois`) — solve the Poisson equation for the pressure correction and project the velocity field to be divergence-free. +15. `purifiers` (`modpurifiers`). +16. `tstep_integrate` (`modtstep`) — the RK3 update: `u0 = um + rk3coef * up` with `rk3coef = dt/(4 - rk3step)`, then reset the tendencies to zero; on substep 3, copy `u0` into `um` (chemistry via `modchem` is also applied here, once per full step). +17. `halos` (`modboundary`) — exchange halo data between neighbouring pencils. +18. `checksim` (`modchecksim`), `fielddump` (`modfielddump`), `statsdump` (`modstatsdump`) — monitoring and output. +19. `boundary` (`modboundary`) — apply domain boundary conditions to the updated fields. +20. `thermodynamics` (`modthermodynamics`) — diagnostic thermodynamic fields. +21. `writerestartfiles` (`modsave`) — restart output when due. + +**Finalisation.** `exitfielddump`, `exitstatsdump`, `exit_heatpump`, `exitmpi`. + +When adding a new physical process, the usual pattern is: a module with an `init*` routine called from the startup phase, and a forcing routine called from the loop that adds to the tendency arrays *before* `poisson`/`tstep_integrate`. + +## Field arrays and grid + +uDALES uses a staggered grid: scalars (`thl`, `qt`, `sv`, pressure) live at cell centres, and each velocity component on the cell face in its own direction (see the [fields tutorial](udales-fields-tutorial.md) for the exact layout). + +Each prognostic variable exists at three time levels (declared in `modfields.f90`): + +- `um`, `vm`, `wm`, `thlm`, ... — the field at the previous full timestep (the RK3 base state); +- `u0`, `v0`, `w0`, `thl0`, ... — the current state, updated after every RK3 substep; +- `up`, `vp`, `wp`, `thlp`, ... — the accumulated tendencies for the current substep. + +Physics and dynamics routines only ever *add* to the `*p` arrays; `tstep_integrate` combines them as `u0 = um + rk3coef*up` and zeroes them again. + +Arrays are allocated per MPI rank over the local pencil `(ib:ie, jb:je, kb:ke)` (all starting at 1) extended by halo (ghost) cells of width `ih`, `jh`, `kh` — e.g. `up(ib-ih:ie+ih, jb-jh:je+jh, kb:ke+kh)`. The halo widths are set in `modglobal.f90` based on the widest advection stencil in use (up to 3 cells in x and y). The domain is split among MPI ranks with a 2-D pencil decomposition, and halo cells are filled by 2DECOMP&FFT exchange routines — see [parallelisation](udales-2decomp.md) for the decomposition and its constraints on grid loops. diff --git a/docs/udales-boundary-conditions.md b/docs/udales-boundary-conditions.md index 8c3e57120..074113e24 100644 --- a/docs/udales-boundary-conditions.md +++ b/docs/udales-boundary-conditions.md @@ -77,4 +77,33 @@ Determined by `iwallmoist`. Possible values: ## Scalars -TBC +Only relevant when `nsv > 0`. + +### Top + +Determined by `BCtops`. Possible values: + +- 1: constant flux given by `wsvtop`. +- 2: constant value given by `sv_top`. + +### Bottom + +NB: only relevant if the bottom of the domain is not covered by floor facets. Determined by `BCbots`. Possible values: + +- 1: zero flux. + +### x + +Determined by `BCxs`. Possible values: + +- 1: periodic +- 2: inflow-outflow, fixed profile +- 3: inflow-outflow, inflow given by time-varying profile from precursor simulation +- 4: inflow-outflow, fixed profile imposed only in a narrow strip around the domain's central y-plane; used for demonstration/testing purposes rather than production use. + +### y + +Determined by `BCys`. Possible values: + +- 1: periodic +- 2: inflow-outflow, fixed profile diff --git a/docs/udales-contributing.md b/docs/udales-contributing.md new file mode 100644 index 000000000..ea38c9bff --- /dev/null +++ b/docs/udales-contributing.md @@ -0,0 +1 @@ +--8<-- "CONTRIBUTING.md" diff --git a/docs/udales-development-notes.md b/docs/udales-development-notes.md new file mode 100644 index 000000000..614bde058 --- /dev/null +++ b/docs/udales-development-notes.md @@ -0,0 +1 @@ +--8<-- "DEVELOP.md" diff --git a/docs/udales-development.md b/docs/udales-development.md new file mode 100644 index 000000000..cf56528ac --- /dev/null +++ b/docs/udales-development.md @@ -0,0 +1,18 @@ +# Developing uDALES + +uDALES is developed openly on [GitHub](https://github.com/uDALES/u-dales). Bug reports, feature requests, and pull requests are all welcome — see [how to contribute](udales-contributing.md) for the process. + +Development starts from the `master` branch, which is the bleeding-edge version of the model; users running simulations should instead use the [latest release](https://github.com/uDALES/u-dales/releases), as described in the [installation guide](udales-installation.md). + +This section collects the developer-facing documentation: + +- [Code architecture](udales-architecture.md) — how the solver is organised: repository layout, module responsibilities, and the exact sequence of calls in a timestep. Start here before modifying the solver. +- [Extending uDALES](udales-extending.md) — recipes for common tasks, each traced through a working example in the code: adding a namelist option, an output variable, a statistic, an example case, or a test. +- [Testing and CI](udales-testing.md) — the test suites, how to run them locally, and what continuous integration checks on every pull request. +- [Development notes](udales-development-notes.md) — setting up a development environment, building in debug mode, and building this documentation. +- [How to contribute](udales-contributing.md) — reporting bugs, requesting features, and opening pull requests. +- [Parallelisation](udales-2decomp.md) — how the 2-D pencil domain decomposition with 2DECOMP&FFT works, and what it means for code that loops over the grid. +- [Using agents](udales-agents.md) — conventions for working on this repository with AI coding agents. +- [Fortran API reference](udales-software-docs.md) — API documentation of the Fortran source, generated with [FORD](https://github.com/Fortran-FOSS-Programmers/ford). + +The model source lives in `src/`, the pre/post-processing tools in `tools/`, example cases in `examples/`, and the tests in `tests/` (see the [test docs](https://github.com/uDALES/u-dales/blob/master/tests/README.md) for the layout). uDALES uses [semantic versioning](https://semver.org/). diff --git a/docs/udales-driver-simulations.md b/docs/udales-driver-simulations.md index 9224dabf9..8b2dc4f42 100644 --- a/docs/udales-driver-simulations.md +++ b/docs/udales-driver-simulations.md @@ -26,7 +26,7 @@ It is necessary to first have run a simulation following the above instructions. - The driven simulation must use the same number of cores as the precursor simulation. - It is not necessary to apply a forcing to the driven simulation due to the enforced inlet-outlet boundary conditions. - `BCxm = 3` is the current index to enforce the required inlet-outlet boundary conditions. Boundary conditions in the x-direction for other prognostic variables will be overwritten by this and therefore do not need to be set. -- The \*driver\* files from the precursor simulation must be copied from its output directory to the experiments directory of the driven simulation. For example, if the precursor is 001 and the driven simulation is 002 and you are working from the top uDALES directory: `cp outputs/001/*driver* experiments/002/`. +- The \*driver\* files from the precursor simulation must be copied from its output directory to the experiments directory of the driven simulation. For example, if the precursor is 001 and the driven simulation is 002 and you are working from the top uDALES directory: `cp outputs/001/*driver* experiments/002/`. This can be automated with the helper script `tools/link_driver_files.sh `, which symlinks the \*driver\* files instead of copying them. - `driverstore` must be equal to or less than the number of timesteps saved in the \*driver\* files (equivalent value of `driverstore` in the precursor simulation). - `driverjobnr` must equal the job number of the corresponding precursor simulation. Following the above example: `driverjobnr = 001`. - `runtime` must equal `(driverstore-1)*dtdriver` seconds or less where `dtdriver` is from the precursor simulation. If this time limit is exceeded the simulation will be stopped as there will be no more data available to determine the inlet. diff --git a/docs/udales-example-simulations.md b/docs/udales-example-simulations.md index 953bdcf31..a9ca09343 100644 --- a/docs/udales-example-simulations.md +++ b/docs/udales-example-simulations.md @@ -73,7 +73,7 @@ Table 1: Main setups of urban case studies and reference example simulation(s). | Surface and top scalar BCs | energy balance | yes | 201 | | Output | instantaneous fields | no | 001, 101, 102, 201, 949 | | Output | time-averaged fields | no | 201, 950 | -| Output | space-averaged (along x,y) profiles | no | 002, 101, 102, 201, 501, 950 | +| Output | space-averaged (along x,y) profiles | no | 002, 101, 102, 201, 949, 950 | @@ -323,7 +323,7 @@ iadv_thl = 2 #### Isothermal BC for temperature -The temperature on the facets is set by the values in `Tfacinit.f90` (295 K in this case). These temperatures do not change as the energy balance is not used as a default (`lEB = .false.`). A fixed temperature at the the top of the domain (`thl_top`) is set by +The temperature on the facets is set by the values in `Tfacinit.inp.101` (295 K in this case). These temperatures do not change as the energy balance is not used as a default (`lEB = .false.`). A fixed temperature at the the top of the domain (`thl_top`) is set by ```fortran &BC @@ -471,7 +471,7 @@ Here `lreadscal` is necessary as we are running a warmstart. The position, strength and standard deviation of emission of the source are set by: ```fortran -&INP +&INPS xS = 4 yS = 8 zS = 3 @@ -627,9 +627,9 @@ nnudge = 64 ### 949 -![blocks.501](./assets/images/geom_949.png) +![blocks.949](./assets/images/geom_949.png) -This simulation is similar to `101` but does not contain any scalars. It is used as the precursor simulation for simulation `949`. +This simulation is similar to `101` but does not contain any scalars. It is used as the precursor simulation for simulation `950`. #### Driver/precursor simulation @@ -648,7 +648,7 @@ iplane = 256 ### 950 -![blocks.502](./assets/images/uDALES_geom.png) +![blocks.950](./assets/images/uDALES_geom.png) #### Prerequisites simulation with external forcing @@ -704,7 +704,7 @@ runtime = 100. dtmax = 1. ``` -and we want to set `dtmax` = `dtdriver` (`501`) to avoid interpolation of the driver snapshots. +and we want to set `dtmax` = `dtdriver` (`949`) to avoid interpolation of the driver snapshots. All boundary conditions (momentum, temperature) are set to inflow-outflow by: diff --git a/docs/udales-extending.md b/docs/udales-extending.md new file mode 100644 index 000000000..a0b58ba23 --- /dev/null +++ b/docs/udales-extending.md @@ -0,0 +1,132 @@ +# Extending uDALES + +This page collects practical recipes for common developer tasks: adding a namelist option, adding an output field, adding a statistic, adding an example case, and adding a test. Each recipe is derived by tracing a real, working example through the code, with file and function references, so you can use that example as a template rather than starting from scratch. See the [Fortran API reference](udales-software-docs.md) for generated source documentation and the [namoptions overview](udales-namoptions-overview.md) for the full input parameter list. + +## Adding a namelist option + +Template: the `lydump` switch (`&OUTPUT`, y-averaged statistics output) is a simple, working logical option you can copy directly. + +1. **Declare the variable and its default** in `src/modglobal.f90`, e.g. `logical :: lydump = .false.` (declared alongside the other output switches around line 200). Pick a sensible default that preserves current behaviour when the option is absent from `namoptions`. +2. **Add it to the relevant namelist group** in `src/modstartup.f90`, subroutine `readnamelists`. There are two places to touch: + - the `use modglobal, only : ...` import list at the top of `readnamelists` (`lydump` is imported there alongside the other `OUTPUT`-group switches) + - the `namelist/OUTPUT/ &` declaration itself (`lydump` is listed there together with `ltdump, lytdump, lxydump, lxytdump, lmintdump, ...`) + If you are adding a wholly new namelist group instead of extending an existing one, also add a `read (ifnamopt, YOURGROUP, iostat=ierr)` block (with the same `iostat` error-handling pattern used for `OUTPUT`, `BC`, etc.) and a `rewind (ifnamopt)` after it. +3. **Broadcast it from rank 0 to all MPI ranks**, still in `readnamelists`: `call MPI_BCAST(lydump, 1, MPI_LOGICAL, 0, comm3d, mpierr)`. Match the MPI type to the Fortran type (`MPI_LOGICAL`, `MPI_INTEGER`, `MY_REAL`, or `MPI_CHARACTER` with `len(...)` for strings). +4. **Use the value** wherever it is needed. `lydump` is consumed in two ways that are both worth knowing about as patterns: + - a sanity check in `checknamelistvalues` (also in `modstartup.f90`): the value is pulled in via `use modglobal, only : ..., lydump, lytdump, ...` and checked, e.g. `if ((lydump .or. lytdump) .and. (nprocx > 1)) then ... stop 1`. + - the actual behaviour, in `src/modstatsdump.f90`, which imports it with `use modglobal, only : dt, lydump, lytdump, ...` at module scope and gates both the NetCDF setup in `initstatsdump` and the sampling/writing in `statsdump` behind `if (lydump) then ... end if`. +5. **Document the new option** in `docs/udales-namoptions-overview.md`, adding a row to the table for the namelist group it belongs to (name, default, possible values, description, unit). + +Checklist: + +- [ ] Declare in `src/modglobal.f90` with a safe default +- [ ] Add to the `use modglobal, only : ...` list and the `namelist/GROUP/` statement in `readnamelists` (`src/modstartup.f90`) +- [ ] Broadcast with `MPI_BCAST` in the same subroutine +- [ ] Consume the value at its point of use, importing it with `use modglobal, only : ...` in that module +- [ ] Add a row to `docs/udales-namoptions-overview.md` + +**Do not rename existing namelist keys or groups.** Live, user-facing namelist names (e.g. `idriver`, `&DRIVER`) must stay stable across releases — case files in the wild reference them by name. Renaming an existing option is a separate, explicitly-approved task, not something to bundle into feature work. + +## Adding a variable to fielddump + +Template: `src/modfielddump.f90` already handles this generically — adding a new instantaneous 3D output field only means adding one `case` branch, using an existing one (e.g. `case('mc')` for the IBM cell mask, or `case('di')` for divergence, which is computed in-place) as a model. + +How it works: the namelist option `fieldvars` (declared in `src/modglobal.f90`, read in the `&OUTPUT` group in `modstartup.f90`) is a comma-separated string of two-character field codes, e.g. `'u0,v0,w0,th,s1'`. In `initfielddump`, `nvar = (LEN(trim(fieldvars))+1)/3` counts the fields (each code plus its separator is 3 characters), then a loop over `n=1,nvar` does `select case(fieldvars(3*n-2:3*n-1))` to pick out each two-character code in turn. Each `case` branch does two things: calls `ncinfo(ncname(n,:), 'ncvarname', 'long description', 'units', 'gridtag')` to describe the NetCDF variable, and points a module-level pointer at the data with `pfields(n)%point => somefield(ib:ie,jb:je,kb:ke)`. The generic `fielddump` subroutine later copies every `pfields(n)%point` into the output buffer and writes it via `writestat_nc` — it has no per-variable logic, so you never need to touch it. + +There are two nearly-identical `select case` blocks in `initfielddump`: one for `lhalos` (ghost cells included) and one for the normal case. `lhalos` is hard-coded `.false.` (see the assignment right above the `MPI_BCAST` calls), so in practice only the non-halo branch (starting `else` around the `do n=1,nvar` loop that includes `case('u0')`, ..., `case('mc')`, `case('di')`) is exercised — new fields normally only need adding there, and to the `lhalos` branch too if you want the option kept consistent (some existing fields, like `tx`/`ty`/`tz`/`hf`/`mu`/`mv`/`mw`/`mc`/`di`, are only implemented in the non-halo branch). + +The grid stagger tag (last `ncinfo` argument, e.g. `'tttt'`, `'mttt'`, `'ttmt'`) records which grid (cell-centre `t` vs. staggered `m`) each of the x, y, z, time dimensions uses — copy the tag from a field with the same staggering as yours (a cell-centred scalar like pressure uses `'tttt'`; a u-velocity-staggered flux like `tau_x` uses `'mttt'`). + +Checklist: + +- [ ] Confirm the field you want to output already exists as a module-level array (e.g. in `modfields.f90` or wherever it's computed) and is reachable via a `use` statement in `modfielddump.f90` +- [ ] Add a `use modsomething, only : yourfield` import in `initfielddump` if it isn't already imported +- [ ] Add a `case('xx')` branch (pick an unused two-character code) in the non-halo `select case(fieldvars(3*n-2:3*n-1))` block, with `ncinfo(...)` and `pfields(n)%point => yourfield(ib:ie,jb:je,kb:ke)` +- [ ] Mirror the branch in the `lhalos` block only if halo output for this field is wanted +- [ ] Add the new code to the `fieldvars` row's list of labels in `docs/udales-namoptions-overview.md` +- [ ] No changes needed in `fielddump` or `exitfielddump` — they iterate over `pfields` generically + +## Adding a statistic to xytdump + +Template: `pxyt` (domain-mean pressure) in `src/modstatsdump.f90`. It is the simplest xy/time-averaged statistic because it is a plain spatial-then-time average with no correlation/variance term to subtract — a good starting point before tackling flux or variance statistics. + +`xytdump.xxx.nc` holds statistics that are averaged over x, y, *and* time (as opposed to `xydump`, which is only x/y-averaged, or `tdump`, which is only time-averaged). The pipeline for `pxyt` runs through three stages in `statsdump`: + +1. **Instantaneous xy-average** (`if (lxydump .or. lxytdump) then` block, module-level variable `pxy`, declared `real, dimension(kb:ke+kh) :: pxy` inside `modstatsdump.f90`): `call avexy_ibm(pxy(kb:ke+kh), pres0(ib:ie,jb:je,kb:ke+kh), ib,ie,jb,je,kb,ke,kh, IIc(...), IIcs(kb:ke+kh), .false.)` averages the instantaneous `pres0` field over x and y at each sample. +2. **Running time-average** (`if (lxytdump) then` block further down): `pxyt(kb:ke+kh) = (pxyt(kb:ke+kh)*(tstatsdumpp-tsamplep) + pxy(kb:ke+kh)*tsamplep)*tstatsdumppi` accumulates `pxy` into the persistent, module-level array `pxyt`. `pxyt` is declared in `src/modfields.f90` (`real, allocatable :: pxyt(:)`) and allocated/zeroed alongside the other `xyt` accumulators (`allocate(pxyt(kb:ke+kh))`, `pxyt=0.`) in the same allocation block as `uxyt`, `vxyt`, etc. +3. **NetCDF definition and write**, both in `initstatsdump`/`statsdump`: + - in `initstatsdump`, inside `if (lxytdump) then`, `call ncinfo(ncstatxyt(6,:), 'pxyt', 'Pressure', 'm^2/s^2', 'tt')` registers the variable at index 6 of the `nstatxyt`-sized `ncstatxyt` array (`ncstatxyt` is declared in `modfields.f90` and allocated `allocate(ncstatxyt(nstatxyt,4))` in `initstatsdump`); + - in `statsdump`, inside the final `if (lxytdump) then ... if (myid == 0) then` block, `varsxyt(:,6) = pxyt(kb:ke)` copies it into the write buffer, which is then written in one call: `call writestat_1D_nc(ncidxyt, nstatxyt, ncstatxyt, varsxyt, nrecxyt, khigh-klow+1)`. + +Note the index (`6`) must be consistent between the `ncinfo` call and the `varsxyt(:,6) = ...` assignment, and `nstatxyt` (currently `23`, set in the module header alongside `nstaty`, `nstatt`, etc.) must be bumped if you add a new slot rather than reusing one. + +Checklist (for `xytdump`; the same pattern applies to `tdump`/`ncstatt`/`varst`, `xydump`/`ncstatxy`/`varsxy`, etc., with `t`/`xy` suffixes swapped accordingly): + +- [ ] If accumulating in time, add a persistent array (e.g. `real, allocatable :: myvarxyt(:)`) in `src/modfields.f90`, and allocate + zero it in the same block as `uxyt`, `vxyt`, ... +- [ ] Compute the instantaneous spatial average with `avexy_ibm(...)` (or reuse an existing `xy` intermediate) inside the `if (lxydump .or. lxytdump)` block in `statsdump` +- [ ] Accumulate it into the time-average with the same `(old*(tstatsdumpp-tsamplep) + new*tsamplep)*tstatsdumppi` running-mean pattern, inside `if (lxytdump)` +- [ ] Bump `nstatxyt` in the module header and `call ncinfo(ncstatxyt(N,:), 'name', 'description', 'units', 'gridtag')` in `initstatsdump` +- [ ] Add `varsxyt(:,N) = myvarxyt(kb:ke)` next to the write call in `statsdump` +- [ ] Document the new output variable in `docs/udales-output-files.md` if that page enumerates `xytdump` fields + +## Adding an example case + +Template: compare `examples/001` (minimal, no buildings, pressure-gradient forcing) with `examples/102` (buildings, warmstart, scalar point source) — together they show the full range of what an example directory can contain. + +Every example under `examples//` needs, at minimum: + +- `namoptions.` — the namelist file; the last three digits of every filename in the case must match `iexpnr` in `&RUN`. +- `config.sh` — sets `DA_EXPDIR`, `DA_TOOLSDIR`, `DA_BUILD`, `DA_WORKDIR`, `NCPU` (as absolute, user-specific paths — copy and edit one from an existing example, e.g. `examples/001/config.sh` or `examples/102/config.sh`, don't reuse it as-is). +- Geometry/IBM inputs produced by pre-processing: `facets.inp.`, `factypes.inp.`, `facetarea.inp.`, `facet_sections_{c,u,v,w}.txt`, `fluid_boundary_{c,u,v,w}.txt`, `solid_{c,u,v,w}.txt`, and either an `.stl` file (e.g. `examples/001/flat_ground.stl`, `examples/102/geom.102.STL`) or a MATLAB/Python-generated set from `stl_file` in `&INPS`. The corresponding facet/point counts (`nfcts`, `nsolpts_*`, `nbndpts_*`, `nfctsecs_*`) go in `&WALLS` in `namoptions.` and are produced by the pre-processing tools, not hand-written. +- `prof.inp.` and `lscale.inp.` — initial/forcing profiles, also pre-processing outputs. +- `info.txt` — a short human-readable description of the case (see `examples/001/info.txt`, `examples/102/info.txt`). +- Optional, case-specific inputs: `scalar.inp.` and `scalarsourcep.inp.1.` for scalar sources (as in `102`), `warmstart_files/` and restart files (`initd...`, matched by `startfile` in `&RUN`) for a warmstart (as in `102`), driver/`*driver*` files for driver-driven cases (as in `949`/`950`). + +Getting the geometry/IBM inputs, `prof.inp`, and `lscale.inp` right normally means running the pre-processing pipeline (see `docs/udales-pre-processing.md`) rather than writing them by hand. + +To exercise a new example locally: + +```sh +./u-dales/tools/local_execute.sh examples/ +``` + +`tools/local_execute.sh` reads `config.sh` from the case directory, checks `DA_WORKDIR`/`DA_BUILD`/`DA_TOOLSDIR`/`NCPU` are set, and runs the case. `tools/examples/run_examples.sh` is the CI/local sweep that runs several examples back to back (currently listing `001 002 101 102 201 501 502`, cross-check against the current contents of `examples/` before relying on this list — it also handles downloading the extra warmstart/driver assets for `102`/`502` via `curl`+`unzip`). Add your new case number to that loop if it should be part of the routine sweep. + +Checklist: + +- [ ] Create `examples//` with `namoptions.` and `config.sh` (edit paths, don't commit real absolute paths pointing outside the repo if avoidable) +- [ ] Generate geometry/IBM inputs via pre-processing (or copy+adapt from a similar example) and make sure `&WALLS` counts match +- [ ] Add `prof.inp.`, `lscale.inp.`, and any case-specific `.inp` files +- [ ] Add `info.txt` describing the case +- [ ] Test with `./u-dales/tools/local_execute.sh examples/` +- [ ] Add the case to `tools/examples/run_examples.sh` if it should run in the routine sweep +- [ ] Document the case in `docs/udales-example-simulations.md` (setup table row plus a walkthrough section, following the `001`/`002`/`101`/`102` pattern) + +## Adding a test + +Template: `tests/integration/ibm_sparse_input/run_test.sh` (an MPI/solver-driven integration test) for a case-based suite, or any file in `tools/python/tests/` for a pure-Python unit test. + +Tests live in layered locations by scope, documented in `tests/README.md`: + +- `tools/python/tests/` — unit tests for Python modules (`udbase`, `udprep`, `udgeom`, ...); picked up automatically by `unittest discover`, no manifest entry needed. +- `tests/unit/` — isolated Fortran/solver routine tests with minimal setup. +- `tests/integration/` — end-to-end, multi-component checks, often against committed fixtures in `tests/cases/` (e.g. `tests/cases/101` is shared by `integration/ibm_sparse_input/`). +- `tests/system/` — heavier whole-code, solver-driven validation. +- `tests/regression/` — branch-to-branch or reference-output comparisons (e.g. `tests/regression/david_tests/`). + +New automated test *runs* are wired up through `tests/test_suites.yml`, a hand-parsed YAML manifest read by `tests/run_tests.py`. Its structure (see the schema comment at the top of the file): + +- `groups..suites` — concrete suite entries belonging to that selection. +- `groups..includes` — other group names folded in (e.g. `supported` includes `python-library`). +- Each suite has `label`, `class` (`supported` = required merge-gate, `experimental` = not yet gating), `kind` (`unit`/`integration`/`reference`/`system`/`regression`), `component`, `platform` (`linux`/`macos`/`hpc`/`any`), `cost` (`fast`/`medium`/`slow`), and `command` — an argv list passed straight to `subprocess.run`, with `{python}`, `{repo_root}`, `{tests_dir}`, `{branch_a}`, `{branch_b}`, `{build_type}`, `{build_type_lower}` available as format placeholders. Optional `env_` keys inject environment variables into the child process (e.g. `env_UDALES_BUILD: "{repo_root}/build/{build_type_lower}/u-dales"` used by the `ibm_sparse_input`, `mpi_operators`, and `processor_boundaries` suites). + +`tests/integration/ibm_sparse_input/run_test.sh` is a good template for a solver-driven suite: it resolves `UDALES_BUILD`/`CASE_SOURCE`/`NAMELIST_SOURCE` from environment variables with sane defaults, copies a fixture case (`tests/cases/101`) plus a test-local namelist into a scratch `mktemp -d` directory, derives `NPROCS` from `nprocx`/`nprocy` in the namelist, runs `mpiexec ... "$UDALES_BUILD" "$NAMELIST"`, and propagates the exit code — the same pattern `tests/run_tests.py` expects from any suite command. + +Checklist: + +- [ ] Decide the scope (`unit`/`integration`/`system`/`regression`) and put the test file(s) under the matching `tests//` subdirectory (or `tools/python/tests/` for a pure-Python unit test) +- [ ] If it needs a case fixture, add it under `tests/cases/` (reuse an existing one, e.g. `tests/cases/100` or `101`, if it fits) rather than duplicating a full case +- [ ] Write the test/script so it exits non-zero on failure (Python: raise/assert; shell: propagate `$?`, as in `run_test.sh`) +- [ ] Add a suite entry to `tests/test_suites.yml` under the appropriate group (usually `supported` if it should merge-gate, `experimental` otherwise), filling in `label`, `class`, `kind`, `component`, `platform`, `cost`, `command`, and any `env_` needed +- [ ] Run it via the dispatcher to confirm wiring: `python tests/run_tests.py ` (e.g. `python tests/run_tests.py supported`) +- [ ] Update `tests/README.md` if you introduced a new fixture under `tests/cases/` or a new suite category worth documenting there diff --git a/docs/udales-facets-tutorial.md b/docs/udales-facets-tutorial.md index 3c95cf52a..563cf4f30 100644 --- a/docs/udales-facets-tutorial.md +++ b/docs/udales-facets-tutorial.md @@ -1,6 +1,9 @@ # Working with uDALES facet data in MATLAB +!!! warning "MATLAB path to be retired in v3.0" + These tutorials currently use the MATLAB toolchain. The MATLAB route is deprecated and will be retired from uDALES v3.0 onwards, in favour of the [uDALES Python package](udales-python-package.md). + In this tutorial we cover how to work with uDALES facet data. This pertains to all surface quanties, e.g. shear stress, surface pressure, surface energy balance terms and the temperature inside each of the facets. The **`udbase`** post-processing class reads in most important input parameters, and contains a number of methods to work with facet data: @@ -20,7 +23,7 @@ The **`udbase`** post-processing class reads in most important input parameters, - [**convert_facvar_to_field**](#convert_facvar_to_field-convert-facet-data-to-grid). This method transfers a facet variable onto the grid, so it can be used for post-processing. - [**convert_facflx_to_field**](#convert_facflx_to_field-convert-facet-data-to-3d-density-field). This method converts a facet variable to a density in a 3D field, so it can be used for post-processing (e.g. calculating distributed drag). -**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_mlx.** +**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_udbase.** ## Initialising udbase diff --git a/docs/udales-fields-tutorial.md b/docs/udales-fields-tutorial.md index 1bed60d48..518d78908 100644 --- a/docs/udales-fields-tutorial.md +++ b/docs/udales-fields-tutorial.md @@ -1,6 +1,9 @@ # Working with uDALES field data in MATLAB +!!! warning "MATLAB path to be retired in v3.0" + These tutorials currently use the MATLAB toolchain. The MATLAB route is deprecated and will be retired from uDALES v3.0 onwards, in favour of the [uDALES Python package](udales-python-package.md). + This tutorial describes how to read and process field data output of the LES code uDALES using MATLAB. In addition, it describes some important concepts, such as the [grid layout](#udales-grid-layout), [variable locations](#averages-used-in-udales-output) and [averaging procedures](#averages-used-in-udales-output). The **`udbase`** post-processing class reads in most important input parameters, and contains a number of methods to load field data: @@ -12,7 +15,7 @@ The **`udbase`** post-processing class reads in most important input parameters, - [**load_slice**](#load_slice-loading-instantaneous-2d-slice-data). This method loads instantaneous 2D slices of instantaneous 3D data from the file `Xslicedump.expnr.nc`. Several output times may be present in the data. - [**plot_trees**](#load_stat_tree-and-plot_tree-loading-and-plotting-tree-data). This method plots tree patches. -**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_mlx.** +**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_udbase.** ## Initialising udbase diff --git a/docs/udales-geometry-tutorial.md b/docs/udales-geometry-tutorial.md index 42d048bf9..159b750a6 100644 --- a/docs/udales-geometry-tutorial.md +++ b/docs/udales-geometry-tutorial.md @@ -1,6 +1,9 @@ # uDALES urban surface creation +!!! warning "MATLAB path to be retired in v3.0" + These tutorials currently use the MATLAB toolchain. The MATLAB route is deprecated and will be retired from uDALES v3.0 onwards, in favour of the [uDALES Python package](udales-python-package.md). + uDALES supports several manners to create urban surfaces. It is possible to run uDALES without specifying an urban surface, in which case a flat terrain is assumed. In most cases however, an urban surface will be specified. uDALES contains a number of functions in the [**udgeom**](#the-udgeom-class) package that aid geometry generation: - [**udgeom.createFlatSurface**](#udgeomcreateflatsurface). This function creates a flat surface. The main reason for using this function is to be able to discretise the surface in as many facets as possible in case heterogeneous boundary conditions are required. @@ -8,7 +11,7 @@ uDALES supports several manners to create urban surfaces. It is possible to run - [**udgeom.createCubes**](#udgeomcreatecubes). This function creates cubes, either a single one or an array of cubes. - [**udgeom.createRealistic**](#udgeomcreaterealistic). This function creates a realistic urban surface based on an stl file that contains the buildings. The function adds the ground surface. -**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_mlx.** +**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_udbase.** ## Initialising udbase diff --git a/docs/udales-how-to-cite.md b/docs/udales-how-to-cite.md index e303a9142..413f130b7 100644 --- a/docs/udales-how-to-cite.md +++ b/docs/udales-how-to-cite.md @@ -2,7 +2,7 @@ When using uDALES or data generated by uDALES please cite: -> Owens *et al.* (2024). A conservative immersed boundary method for the multi-physics urban large-eddy simulation model uDALES v2.0. *Geophys. Mod. Dev.* **16**:6277-6300, https://doi.org/10.5194/gmd-17-6277-2024 +> Owens *et al.* (2024). A conservative immersed boundary method for the multi-physics urban large-eddy simulation model uDALES v2.0. *Geophys. Mod. Dev.* **17**:6277-6300, https://doi.org/10.5194/gmd-17-6277-2024 > Suter *el al.* (2022), uDALES 1.0: a large-eddy-simulation model for urban environments. *Geophys. Mod. Dev.* **15**, 5309–5335, https://doi.org/10.5194/gmd-15-5309-2022 diff --git a/docs/udales-installation.md b/docs/udales-installation.md index b59ff61c8..f02349425 100644 --- a/docs/udales-installation.md +++ b/docs/udales-installation.md @@ -2,8 +2,6 @@ Getting started with uDALES to set up your own experiments is straightforward. This page goes through the steps required to [install](#installation) uDALES and to [set-up the directory structure](#repository-set-up). Results are output in netCDF format, for a quick inspection you can use GUI tools such as [Panoply](https://www.giss.nasa.gov/tools/panoply/) or [ncview](http://meteora.ucsd.edu/~pierce/ncview_home_page.html). To learn more about pre- and post-processing steps see the [what's next section](#whats-next). -If you have [Singularity](https://sylabs.io/) available on your system, you can use the provided scripts under `tools/singularity` to build and run uDALES cases locally or on HPC environments. See [Singularity](#singularity) for instructions; otherwise, see the next section. - ## Prerequisites ### uDALES @@ -71,6 +69,9 @@ cd uDALES git clone --recurse-submodules https://github.com/uDALES/u-dales.git ``` +!!! tip "Which version to use" + For running simulations, we recommend using the latest release — check out its tag after cloning, e.g. `git checkout v2.2.0` (see the [releases page](https://github.com/uDALES/u-dales/releases) for the current version). The `master` branch is the bleeding-edge development version: it is where new features land first, but it may contain unresolved issues. Use `master` if you are [developing uDALES](udales-development.md). + Create directories for experiment set-ups and output data: ```sh @@ -104,7 +105,7 @@ In the next steps we will assume your current working directory is the top-level ## Build on common systems -To compile uDALES (in release mode) on common/local uvuntu or mac systems using helper script, run: +To compile uDALES (in release mode) on common/local Ubuntu or Mac systems using the helper script, run: ```sh # We assume you are running the following commands from the u-dales directory @@ -181,13 +182,19 @@ By default uDALES will compile in `Release` mode. You can change this by specify | Name | Options | Default | Description | | ------------------------------- | ------------------ | --------- | --------------------------------------------- | | `CMAKE_BUILD_TYPE` | `Release`, `Debug` | `Release` | Whether to optimise/build with debug flags | -| `NETCDF4_DIR` | `` | - | Path to netCDF-C installation directory | +| `NETCDF_DIR` | `` | - | Path to netCDF-C installation directory | | `NETCDF_FORTRAN_DIR` | `` | - | Path to netCDF-Fortran installation directory | | `SKIP_UPDATE_EXTERNAL_PROJECTS` | `ON`, `OFF` | `OFF` | Whether to skip updating external projects | +| `DOUBLE_PRECISION` | `ON`, `OFF` | `ON` | Build 2DECOMP&FFT with double precision | +| `SINGLE_PRECISION_OUTPUT` | `ON`, `OFF` | `OFF` | Write output in single precision | +| `FFT_Choice` | `fftw3`, `generic` | `fftw3` | FFT backend for 2DECOMP&FFT | + +## Singularity (not recommended) -## Singularity +!!! warning "Outdated image" + The Singularity image has not been updated in several years and still corresponds to uDALES 1.0. Building natively (see above) is the recommended route; the instructions below are retained for legacy use only. -If you are looking for information on how to install or use Singularity on your system, please refer to the [Singularity documentation](https://sylabs.io/docs). The use of Singularity is undoubtedly the easiest way to build and run cases in uDALES as all dependencies are provided and uDALES will compile out of the box. Furthermore, users wishing to achieve a reasonable level of scientific reproducibility may archive software, tools, and data with their Singularity image containing OS and external libraries to an open access repository (e.g. [Meyer et al., 2020](https://doi.org/10.1029/2019MS001961)). +If you are looking for information on how to install or use Singularity on your system, please refer to the [Singularity documentation](https://sylabs.io/docs). Users wishing to achieve a reasonable level of scientific reproducibility may archive software, tools, and data with their Singularity image containing OS and external libraries to an open access repository (e.g. [Meyer et al., 2020](https://doi.org/10.1029/2019MS001961)). First clone the uDALES repository with: diff --git a/docs/udales-model-overview.md b/docs/udales-model-overview.md new file mode 100644 index 000000000..4e152caa9 --- /dev/null +++ b/docs/udales-model-overview.md @@ -0,0 +1,39 @@ +# Model overview + +uDALES (urban Dutch Atmospheric Large-Eddy Simulation) is an open-source large-eddy simulation model for the urban microclimate. It simulates the flow of air through and above cities at building-resolving resolution, together with the transport of heat, moisture, and pollutants. uDALES originates from the atmospheric LES code DALES and has been extended into a multi-physics urban modelling tool [@Suter2022; @Owens2024]. + +## Dynamical core + +uDALES solves the filtered, incompressible Navier–Stokes equations under the Boussinesq approximation on a staggered Cartesian grid, with grid stretching supported in the vertical. Subgrid-scale turbulence is represented by an eddy-viscosity closure (the Vreman model or a one-equation subgrid TKE model). Several advection schemes are available, including second-order central and flux-limited kappa schemes for scalars; the pressure Poisson equation is solved with an FFT-based method. + +## Urban surfaces + +Buildings and terrain are represented by an immersed boundary method (IBM). Since uDALES 2.0, the geometry is specified as a triangulated surface (STL), and the IBM is discretely mass-, momentum-, and scalar-conservative [@Owens2024]. Surface fluxes of momentum, heat, and moisture at the resolved surfaces (facets) are computed with urban wall functions. + +## Surface energy balance + +Each facet can carry a full surface energy balance: shortwave and longwave radiation (with view factors between facets computed during pre-processing), conduction through multi-layer walls, and turbulent sensible and latent heat fluxes [@Suter2022]. This enables coupled simulations of urban airflow and heat transfer, including the effects of shading and radiative trapping in street canyons. + +## Vegetation + +Trees and other vegetation are represented as porous drag elements that also exchange heat, moisture, and scalars with the air, allowing studies of the effect of urban greening on flow, temperature, and air quality [@Grylls2021a]. + +## Dispersion and air quality + +uDALES transports an arbitrary number of passive or reactive scalars released from point, line, or volume sources. A basic NO~x~–O~3~ chemistry scheme is available for street-scale air quality studies [@Grylls2019], alongside idealised air purifiers and heat-pump sources. + +## Inflow and boundary conditions + +Simulations can be run with periodic lateral boundaries or with inflow–outflow conditions. Turbulent inflow can be supplied by a precursor (driver) simulation — see [precursor simulations](udales-driver-simulations.md) — or generated synthetically. Details of all boundary condition options are given in the [boundary conditions](udales-boundary-conditions.md) reference. + +## Parallelisation + +The code is parallelised with MPI using a 2-D pencil domain decomposition based on the 2DECOMP&FFT library, and scales to large domains and core counts on HPC systems — see the [parallelisation notes](udales-2decomp.md). + +## Where to read more + +The model formulation and validation are described in the uDALES description papers: uDALES 1.0 [@Suter2022] and the conservative IBM of uDALES 2.0 [@Owens2024]. Studies using uDALES are collected in the [publication list](udales-pub-list.md). To get started with the model itself, see the [workflow overview](udales-workflow.md). + +## References + +\bibliography diff --git a/docs/udales-namoptions-overview.md b/docs/udales-namoptions-overview.md index 974dd91a4..dabad12af 100755 --- a/docs/udales-namoptions-overview.md +++ b/docs/udales-namoptions-overview.md @@ -17,7 +17,7 @@ This list refers to the original code-base [DALES](https://github.com/dalesteam/ | trestart | 10000. | 0 < trestart < runtime | Time at which restart files are written, trestart > runtime will prevent restart files being written. | [s] | | dtmax | 20 | > 0 | Maximum allowed numerical integration timestep. | [s] | | ladaptive | .false. | .true. or .false. | Switch for adaptive time-stepping, .true. recommended. | - | -| courant | 1.1 | 1 <= courant <=2 | Courant number, default sets it to 1.5 or 1.1 (if Kappa or upwind scheme is used). | - | +| courant | 1.5 | 1 <= courant <=2 | Courant number, default sets it to 1.5 or 1.1 (if Kappa or upwind scheme is used). | - | | lrandomize | .true. | .true. or .false. | Switch that determines whether initial field is randomised.| - | | irandom | 43 | `INTEGER` > 0 | Seed for random number generation. | - | | randu | 0.01 |`REAL` > 0 | Amplitude of velocity field randomisation. | [m/s] | @@ -55,8 +55,6 @@ This list refers to the original code-base [DALES](https://github.com/dalesteam/ | lvoutflowr | .false. | .true. or .false. | Switch that determines whether u-velocity is corrected to get a fixed outflow rate. *Only functional when y-direction is not parellelised.* | | | luvolflowr | .false. | .true. or .false. | Switch that determines whether u-velocity is corrected to get a fixed volume flow rate. | - | | lvvolflowr | .false. | .true. or .false. | Switch that determines whether u-velocity is corrected to get a fixed volume flow rate. | - | -| luflowr | .false. | .true. or .false. | Switch that determines whether u-velocity is corrected to get a fixed flow velocity. | - | -| lvflowr | .false. | .true. or .false. | Switch that determines whether v-velocity is corrected to get a fixed flow velocity. | - | | uflowrate | 1. | `REAL` | U-velocity flow rate for out- or volume-flow forcing. | [m/s] | | vflowrate | 1. | `REAL` | V-velocity flow rate out- or volume-flow forcing. | [m/s] | | lprofforc | .false. | .true. or .false. | Switch for nudging flow to a profile (forcing). | - | @@ -191,7 +189,6 @@ BCs at the bottom (BCbot; only effective if not covered with ground facets): 1 = | Name | Default | Possible values | Description | Unit | | ---- | ------- | --------------- | ----------- | ---- | -| nblocks | 0 | `INTEGER` | Number of blocks specified in `blocks.inp`. | - | | nfcts | -1 | `INTEGER` | Number of facets specified in `facets.inp`. | - | | iwallmom | 2 | 1, 2, 3 (1 means zero flux) | Building wall momentum flux. | - | | iwalltemp | 1 | 1, 2 | Building wall temperature flux. | - | @@ -234,7 +231,7 @@ BCs at the bottom (BCbot; only effective if not covered with ground facets): 1 = | nnz | 0 | `INTEGER` | Number of non-zero view factors (only used with sparse view factor format. | - | | lperiodicEBcorr | .false. | .true. or .false. | Switch for preventing over-heating and moisture saturation in periodic simualtions. | - | | sinkbase | 0 | `INTEGER` > 0 | k index above which the periodicEBcorr sink is applied (should be above height of tallest building). | - | -| fraction | 0 | `REAL` > 0 | Ratio of domain height to uncapped boundary layer height. | - | +| fraction | 1 | `REAL` > 0 | Ratio of domain height to uncapped boundary layer height. | - | ## Namelist SCALARS @@ -252,11 +249,10 @@ BCs at the bottom (BCbot; only effective if not covered with ground facets): 1 = | ---- | ------- | --------------- | ----------- | ---- | | idriver | 0 | 0, 1, 2 | Options for running precursor driver simulations where \*driver\* files will be written (`= 1`) and reading a completed driver simulation as the inlet to a simulation (`= 2`). Default (`= 0`) will do neither. | - | | tdriverstart | 0. | `REAL` | Time at which \*driver\* files start being written. In use for `idriver = 1`. | s | -| dtdriver | 0. | `REAL` | Timestep at which \*driver\* file planes are written. In use for `idriver = 1`. | s | +| dtdriver | 0.1 | `REAL` | Timestep at which \*driver\* file planes are written. In use for `idriver = 1`. | s | | iplane | - | `INTEGER` | Index of the position on the x-axis of the plane that will be written to \*driver\* files. In use for `idriver = 1`. | | -| driverstore | 0. | `INTEGER` | Number of timesteps (`idriver = 1`) to be written to \*driver\* files or (`idriver = 2`) contained in \*driver\* files to be read. | - | +| driverstore | - | `INTEGER` | Number of timesteps (`idriver = 1`) to be written to \*driver\* files or (`idriver = 2`) contained in \*driver\* files to be read. Must be set when using driver simulations. | - | | driverjobnr | - | - | Job number of the \*driver\* files to be read. These files should be copied into the experiments folder of the driven simulation. In use for `idriver = 2`. | - | -| lsdriver | .false. | .true., .false. | Switch for reading scalar driver files. In use for `idriver = 2`. | - | ## Namelist OUTPUT @@ -273,7 +269,6 @@ BCs at the bottom (BCbot; only effective if not covered with ground facets): 1 = | lytdump | .false. | .true. or .false. | Switch to output y- and time- averaged statistics. *Only functional if x-direction is not parallelised.* | - | | lxydump | .false. | .true. or .false. | Switch to output x- and y- averaged statistics. | - | | lxytdump | .false. | .true. or .false. | Switch to output x-, y- and time-averaged statistics. | - | -| lslicedump | .false. | .true. or .false. | Switch to output slices in the xy-plane. | - | diff --git a/docs/udales-output-files.md b/docs/udales-output-files.md new file mode 100644 index 000000000..28c8f44d0 --- /dev/null +++ b/docs/udales-output-files.md @@ -0,0 +1,37 @@ +# Output files + +uDALES writes its outputs as NetCDF files. Simulations run on more than one CPU write one file per CPU for most output types; files that are already reduced across the whole domain (e.g. the y-, xy- and xy-time-averaged statistics, the TKE budget and the facet energy balance files) are written once, by rank 0. Per-CPU files use the naming pattern `....nc`, where `` and `` are the three-digit x- and y-pencil ranks; reduced files use `..nc`. Use `gather_outputs.sh` (or its HPC/ARCHER2 wrappers) to merge the per-CPU files into single files per experiment — see [Post-processing](udales-post-processing.md). + +All switches and frequency parameters below live in the `&OUTPUT`, `&TREES`, `&WALLS` and `&ENERGYBALANCE` namelists of `namoptions`; see the [Configuration](udales-namoptions-overview.md) page for their defaults and constraints. + +## Output files + +| File | Enabled by | Frequency | Contents | +| ---- | ---------- | --------- | -------- | +| **Statistics (time- and/or space-averaged)** | | | | +| `tdump....nc` | `ltdump` | `tstatsdump` | Time-averaged 3D statistics per CPU: mean velocity, temperature, moisture, pressure and scalar fields; PSS defect; turbulent momentum/heat/scalar fluxes; variances and TKE; SGS scalar fluxes. | +| `mintdump....nc` | `lmintdump` | `tstatsdump` | Reduced time-averaged 3D statistics per CPU: mean u, v, w, temperature and moisture, and pressure only (lighter-weight alternative to `tdump`). | +| `xytdump..nc` | `lxytdump` | `tstatsdump` | x-, y- and time-averaged 1D (height) statistics: mean velocity/temperature/moisture/pressure profiles, turbulent/kinematic/SGS fluxes, temperature and momentum variances, TKE. | +| `xydump..nc` | `lxydump` | `tsample` | x- and y-averaged instantaneous 1D (height) profiles: velocity, temperature, moisture, pressure, turbulent/SGS momentum and heat fluxes, advective fluxes. | +| `ytdump..nc` | `lytdump` | `tstatsdump` | y- and time-averaged 2D (x, height) statistics: mean velocity/temperature/moisture/scalar profiles, turbulent/kinematic/SGS fluxes, variances. Only functional if the x-direction is not parallelised. | +| `ydump..nc` | `lydump` | `tsample` | y-averaged instantaneous 2D (x, height) profiles: velocity, temperature, moisture, scalars, turbulent/SGS momentum and heat fluxes, advective fluxes. Only functional if the x-direction is not parallelised. | +| `tkedump..nc` | `ltkedump` | `tstatsdump` | xy-averaged TKE budget terms vs. height: buoyancy production, total transport, advection, turbulent transport, SGS transport, shear production, viscous and SGS dissipation. Currently marked unsupported in the namoptions documentation — treat as experimental. | +| `treedump....nc` | `ltreedump` (`&TREES`) | `tstatsdump` | Time-averaged 3D vegetation/tree terms per CPU: drag in x, y, z; temperature and moisture source/sink terms; scalar source/sink terms; decoupling factor. | +| **Instantaneous fields** | | | | +| `fielddump....nc` | `lfielddump` | `tfielddump` | Instantaneous 3D snapshots of the variables listed in `fieldvars` (e.g. velocity components, temperature, moisture, pressure, scalars, wall stresses, heat flux, IBM masks, divergence), per CPU. | +| **Slices** | | | | +| `kslicedump....nc` | `lkslicedump` | `tsample` | Instantaneous horizontal (x, y) slice at `k = kslice` of u, v, w, temperature and moisture, per CPU. | +| `islicedump....nc` | `lislicedump` | `tsample` | Instantaneous vertical (y, z) slice at `i = islice` of u, v, w, temperature and moisture; written only by the CPU(s) owning that i-index. | +| `jslicedump....nc` | `ljslicedump` | `tsample` | Instantaneous vertical (x, z) slice at `j = jslice` of u, v, w, temperature and moisture; written only by the CPU(s) owning that j-index. | +| **Facets** | | | | +| `fac..nc` | `lwritefac` (`&WALLS`) | `dtfac` | Facet momentum and heat-transfer data per facet, averaged over the `dtfac` window: surface shear stresses `tau_x`, `tau_y`, `tau_z`, pressure and pressure fluctuation, heat transfer coefficients. | +| `facT..nc` | `lwriteEBfiles` (requires `lEB`) | `dtEB` | Facet layer temperature (`T`) and temperature gradient (`dTdz`) per facet and facet layer. | +| `facEB..nc` | `lwriteEBfiles` (requires `lEB`) | `dtEB` | Facet surface energy-balance terms per facet: net shortwave, incoming longwave, outgoing longwave, sensible heat flux, latent heat flux, soil water content. | + +Statistics that are sampled every `tsample` and only dumped every `tstatsdump` (`tdump`, `mintdump`, `xytdump`, `ytdump`, `treedump`, `tkedump`) report the average over the preceding `tstatsdump` window; the switches whose frequency is `tsample` (`xydump`, `ydump`, the slice dumps) instead write an instantaneous sample every `tsample`. Sampling/averaging only starts once the simulation time passes `tstatstart`. + +## Restart files + +If `trestart` is smaller than `runtime`, uDALES periodically writes unformatted (non-NetCDF) restart files per CPU: `initd__.` holds the flow fields (`mindist`, `wall`, `u0`, `v0`, `w0`, `pres0`, `thl0`, `e120`, `ekm`, `qt0`, `ql0`, `ql0h`, plus the current time and timestep), and, if `nsv > 0`, a companion `inits__.` holds the scalar fields `sv0`. `ntrun` is an 8-digit run counter embedded in the filename. These are written by `writerestartfiles` in `modsave.f90`, at the interval set by `trestart`, and additionally whenever an `exit_now.` file is detected or (for driver simulations) when the inlet file store is exhausted. + +To resume from a restart file, set `lwarmstart = .true.` (or `lstratstart = .true.`) and point `startfile` at the `initd...` filename of the desired restart point (the trailing digits of `startfile` must match the experiment's `iexpnr`); the matching `inits...` file, if present, is read automatically. See the `RUN` and `SCALARS` namelists on the [Configuration](udales-namoptions-overview.md) page for the exact parameter definitions. diff --git a/docs/udales-post-processing.md b/docs/udales-post-processing.md index f2bb5d01d..43db3d8f1 100644 --- a/docs/udales-post-processing.md +++ b/docs/udales-post-processing.md @@ -54,7 +54,7 @@ Replace 009 and 010 with the numbers of your simulations. ## Different output files explained -The output files generated depend on the parameters specified under `&OUPUT` in the `namoptions` file of your simulation (see [Configuration](udales-namoptions-overview.md) for details), and the name of the output file(s) matches the name of that switch, e.g. if `lxytdump` is selected for experiment `009` then there will be an output file called `xytdump.009.nc`. If `lfielddump` is selected, note that there will be a `fielddump.xxx.009.nc` file for each cpu. +The output files generated depend on the parameters specified under `&OUTPUT` in the `namoptions` file of your simulation (see [Configuration](udales-namoptions-overview.md) for details), and the name of the output file(s) matches the name of that switch, e.g. if `lxytdump` is selected for experiment `009` then there will be an output file called `xytdump.009.nc`. If `lfielddump` is selected, note that there will be a `fielddump.xxx.009.nc` file for each cpu. ## Reading output files diff --git a/docs/udales-pre-processing.md b/docs/udales-pre-processing.md index 928028e16..9b520a415 100644 --- a/docs/udales-pre-processing.md +++ b/docs/udales-pre-processing.md @@ -24,7 +24,7 @@ export PREPROC_MEM="128gb" # Defaults to 128gb When running preprocessing on an Imperial HPC compute node with `write_inputs.sh ... c`, the wrapper uses `PREPROC_WALLTIME="24:00:00"` and `PREPROC_MEM="128gb"` unless these are set in `config.sh` or the calling environment. These size the preprocessing PBS job only; they are separate from the solver job variables `WALLTIME` and `MEM` used by `hpc_execute.sh`. `PREPROC_MEM` must be written as a number followed by lowercase `gb`, such as `128gb`; a unitless value such as `128` is rejected before submitting the PBS job. Unless `VIEW3D_MAX_DENSE_MATRIX_GIB` is explicitly set, the default View3D configuration derives the dense-matrix guard from `PREPROC_MEM`: requests above `16gb` leave 16 GiB for overhead, while smaller requests use the requested GiB value. For example, `PREPROC_MEM="128gb"` gives `VIEW3D_MAX_DENSE_MATRIX_GIB=112`. -Before running the Python preprocessing route, build the Python virtual environment. The setup script creates the environment, installs all dependencies, and builds the preprocessing tools (View3D and f2py extension modules). For the MATLAB route, the same setup is still a convenient way to build View3D and the bundled Python helpers used by some preprocessing paths, such as vegetation conversion. For more details on virtual environment setup see [here](./../tools/python/README_VENV.md): +Before running the preprocessing, one must build virtual python environment as uDALES preprocessing setup is gradually moving towards Python; the MATLAB route is deprecated and will be retired from uDALES v3.0 onwards. Run the virtual environment setup script from the repository root as given below. It creates the virtual environment, installs all dependencies, and builds the preprocessing tools (View3D and f2py extension modules). For the MATLAB route, the same setup is still a convenient way to build View3D and the bundled Python helpers used by some preprocessing paths, such as vegetation conversion. For more details on virtual environment set up see the [virtual environment README](https://github.com/uDALES/u-dales/blob/master/tools/python/README_VENV.md): ```bash # For a local machine @@ -70,10 +70,10 @@ For ICL HPC # General syntax: write_inputs.sh <-m|-p> experiments/exp_id run_node_type -# To run preprocessing on HPC log in node (not recomended) +# To run preprocessing on HPC login node (not recommended) ./u-dales/tools/write_inputs.sh -m experiments/001 l -# To run preprocessing on HPC compute node (recomended) +# To run preprocessing on HPC compute node (recommended) # For MATLAB ./u-dales/tools/write_inputs.sh -m experiments/001 c # For Python diff --git a/docs/udales-pub-list.md b/docs/udales-pub-list.md index 67fa99205..37ce06a19 100644 --- a/docs/udales-pub-list.md +++ b/docs/udales-pub-list.md @@ -1,5 +1,5 @@ # Publication list -uDALES was used in the following publications [@Owens2025; @Majumdar2025; @Owens2024; @Grylls2022; @Suter2022; @Lim2022; @Sutzl2021; @Grylls2021; @Grylls2021a; @Sutzl2020; @Grylls2020; @Grylls2019]. +uDALES was used in the following publications [@Wilson2026; @Fellini2026; @Owens2025; @Wilson2025; @Reeuwijk2025; @Majumdar2025; @Owens2024; @Grylls2022; @Suter2022; @Lim2022; @Sutzl2021; @Grylls2021; @Grylls2021a; @Sutzl2020; @Grylls2020; @Grylls2019]. \bibliography diff --git a/docs/udales-python-package.md b/docs/udales-python-package.md new file mode 100644 index 000000000..b40cdfc9b --- /dev/null +++ b/docs/udales-python-package.md @@ -0,0 +1,82 @@ +# The uDALES Python package + +!!! note "Status: under active testing" + The uDALES Python package is under active testing and its interfaces may + change. It is intended to replace the MATLAB toolchain, which is + deprecated and will be retired from uDALES v3.0 onwards. + +The package lives in `tools/python` and provides a set of modules for +pre-processing uDALES cases and for loading, analysing, and visualising +simulation output, as a Python-native alternative to the MATLAB tools. + +## What it provides + +| Module | Purpose | +| --- | --- | +| `udprep` | Pre-processing: builds a case's input files (grid, IBM, vegetation, radiation, SEB) from `namoptions` and STL geometry via the `UDPrep` class, driving the compiled f2py/Fortran extensions and View3D. | +| `udbase` | Loading and analysing simulation output: the `UDBase` class reads a case's NetCDF fields, statistics, and facet data, mirroring the MATLAB `udbase` class. | +| `udgeom` | Geometry handling: the `UDGeom` package for STL mesh checking/repair, building separation, outline calculation, and View3D view-factor computation. | +| `udvis` | 3-D visualisation: `UDVis` gives a stable plotting entry point (`sim.vis`) on top of `UDBase`, keeping the rendering backend an internal detail. | +| `udstats` | Statistics: stateless array-in/array-out helpers (e.g. time averaging, variance/covariance) used internally by `UDBase`. | +| Supporting modules | `udconfig`, `udgrid`, `udnetcdf`, `udfacet`, and `exceptions` provide namelist parsing, grid-coordinate maths, NetCDF loading, and facet-data helpers used internally by `udbase` and `udprep`. | + +## Installation + +Set up the Python virtual environment from the repository root using +`tools/python/setup_venv.sh`, passing `common` for a local machine or `icl` +for the Imperial College HPC cluster. This creates the virtual environment +at `tools/python/.venv`, installs the runtime and build dependencies, and +builds the compiled preprocessing extensions (View3D and the +`directshortwave`/`ibm_preproc` f2py modules): + +```bash +# For a local machine +bash tools/python/setup_venv.sh common + +# For the Imperial HPC machine +bash tools/python/setup_venv.sh icl +``` + +See [README_VENV.md](https://github.com/uDALES/u-dales/blob/master/tools/python/README_VENV.md) for the full reference, +including the Windows PowerShell setup script, manual rebuilds, and +registering the environment as a Jupyter kernel. + +## Using it for pre-processing + +The `write_inputs.sh` wrapper script can drive either the MATLAB or the +Python preprocessing route; pass `-p` to use Python (`-m` selects MATLAB). +The Python route requires the virtual environment above to be set up first: + +```sh +# We assume you are running the following commands from your +# top-level project directory. + +./u-dales/tools/write_inputs.sh -p experiments/001 +``` + +This invokes `tools/write_inputs.py`, which drives the `UDPrep` stack to +generate the same set of input files as the MATLAB route (grid, IBM, +vegetation, and, if the energy balance is enabled, radiation/SEB files). See +[Pre-processing](udales-pre-processing.md) for the full set of input files +and namelist options. + +## Tutorials + +The documented tutorials on this site remain MATLAB-based for now. Python +tutorial notebooks and example scripts live in `tools/python/examples/` in +the repository: + +- `udbase_tutorial.ipynb` — loading and analysing simulation output with `UDBase`. +- `fields_tutorial.ipynb` — working with flow field output. +- `facets_tutorial.ipynb` — working with facet data. +- `geometry_tutorial.ipynb` — building and manipulating geometry with `udgeom`. +- `geometry_QA_tutorial.ipynb` — geometry quality-assurance checks. +- `plot_prep_tutorial.ipynb` — visualising preprocessing output. +- `udprep_tutorial.py` — driving the `UDPrep` pre-processing workflow. +- `udprep_radiation_tutorial.py` — radiation/shortwave pre-processing. +- `udprep_vegetation_tutorial.py` — vegetation pre-processing. + +## Feedback + +The Python package is new and still evolving — please report issues or +unexpected behaviour on the [uDALES GitHub issue tracker](https://github.com/uDALES/u-dales/issues). diff --git a/docs/udales-simulation-setup.md b/docs/udales-simulation-setup.md index 6ddded762..c2c03e57a 100644 --- a/docs/udales-simulation-setup.md +++ b/docs/udales-simulation-setup.md @@ -1,4 +1,4 @@ -## Running uDALES +# Running uDALES The scripts `local_execute.sh` (for local machines), `hpc_execute.sh` (for ICL cluster) and `archer_execute.sh` (for ARCHER2) in `u-dales/tools` are used as wrappers to run simulations. These scripts contain several helpers to run the simulations and merge outputs (see [Post-processing](./udales-post-processing.md) for more info) from several CPUs into a single file. @@ -13,7 +13,7 @@ Example cases shipped with uDALES are located under `u-dales/examples/` and are Note that you need to choose the number of CPUs you are using to run the simulation such that the product of `nprocx` and `nprocy` (in the `namoptions` input file) is equal to the total number of CPU asked, i.e., `nprocx * nprocy = NCPU` for local machines, and `nprocx * nprocy = NNODE * NCPU` for ICL HPC or ARCHER2 clusters. -### Run on common systems +## Run on common systems ``` sh # Contents of the config.sh file @@ -41,7 +41,7 @@ Then, to start the simulation, run: ./u-dales/tools/local_execute.sh experiments/009 ``` -### Run on ICL cluster +## Run on ICL cluster ``` sh # Contents of the config.sh file @@ -73,7 +73,7 @@ Then, to start the simulation, run: ./u-dales/tools/hpc_execute.sh experiments/009 ``` -### Run on ARCHER2 +## Run on ARCHER2 ``` sh # Contents of the config.sh file diff --git a/docs/udales-software-docs.md b/docs/udales-software-docs.md new file mode 100644 index 000000000..b5fd5b518 --- /dev/null +++ b/docs/udales-software-docs.md @@ -0,0 +1,10 @@ +# Fortran API reference + +The API documentation of the uDALES Fortran source — modules, procedures, variables, and their call graphs — is generated automatically from comments in the source code using [FORD](https://github.com/Fortran-FOSS-Programmers/ford). + +[:octicons-arrow-right-24: Browse the Fortran API reference](https://udales.github.io/u-dales/udales-docs-software/index.html) + +This reference is intended for developers working on the solver; for using the model, start from the [workflow overview](udales-workflow.md) instead. + +!!! note + The API reference is built separately from this site (in CI, or locally with `ford ford.md` from the repository root), so the link above points to the published documentation and will not work in a local `mkdocs serve` preview. diff --git a/docs/udales-testing.md b/docs/udales-testing.md new file mode 100644 index 000000000..0a9729b35 --- /dev/null +++ b/docs/udales-testing.md @@ -0,0 +1,139 @@ +# Testing and CI + +This page is a developer-facing guide to how uDALES is tested, how to run +those tests locally, and what continuous integration (CI) checks on a pull +request. For the full test philosophy, layer definitions, and manifest +schema, see [`tests/README.md`](https://github.com/uDALES/u-dales/blob/master/tests/README.md); +this page summarises the parts a contributor needs day to day. + +## Test layout + +Tests live in two places, split by what they validate rather than by +language: + +- **`tests/`** — repo-level tests: solver/MPI-driven checks, case-fixture + integration tests, and branch-to-branch regressions. Organised into + `unit/`, `integration/`, `system/`, and `regression/` subdirectories, plus + `cases/` for shared committed case fixtures (e.g. cases `100`, `101`, + `525`, `526`). +- **`tools/python/tests/`** — unit tests for the Python package (`udbase`, + `udgeom`, `udprep`, `udvis`, and related utilities), discovered by + `unittest discover` on the `test_*.py` naming pattern. + +Exploratory or plotting-heavy scripts that aren't stable automated tests +belong in `tools/python/examples/`, not either test tree. Directory location +alone doesn't decide what runs where or under what policy — that's +controlled by the manifest described below. + +## Running tests locally + +All curated test selections are dispatched through +[`tests/run_tests.py`](https://github.com/uDALES/u-dales/blob/master/tests/run_tests.py), +which reads groups from [`tests/test_suites.yml`](https://github.com/uDALES/u-dales/blob/master/tests/test_suites.yml). +Each suite in the manifest is tagged with a `class` (`supported` or +`experimental`), a `kind` (scope), a `component`, a `platform`, and a rough +`cost` tier; these are printed as the suite runs. + +List the available selections by passing an invalid one (or reading the +manifest directly — there's no separate `--list` flag): + +```bash +python tests/run_tests.py --help +``` + +Run a named selection: + +```bash +python tests/run_tests.py python-library +python tests/run_tests.py supported --branch-a master --branch-b HEAD --build-type Release +python tests/run_tests.py experimental +``` + +- `python-library` — `tools/python` unit tests plus the Python-driven + integration/reference suites (directshortwave, udprep integration, + udbase-vs-MATLAB parity, Python-vs-MATLAB preprocessing parity). Needs the + conda `udales` environment (or equivalent) and `tools/python` installed + (`pip install -e tools/python --no-deps`); no compiled solver, no MPI. +- `supported` — the curated merge-gating selection: `python-library` plus + the branch-comparison regression harness and the solver/MPI-driven + integration suites (IBM sparse input, MPI operators, processor + boundaries). These need a compiled uDALES build; the MPI suites read the + build path from `UDALES_BUILD` (set per suite by the manifest as + `build//u-dales`) and expect `mpiexec` on `PATH`. The + regression suite additionally needs `--branch-a`/`--branch-b` (default + `master`/`HEAD`) and builds both branches itself. +- `supported-macos` — a temporary macOS compatibility selection: + `python-library` plus only the IBM sparse-input suite, excluding the + branch-comparison regression until an unrelated macOS/Homebrew CMake + incompatibility on `master` is fixed. +- `experimental` — coverage not yet in the merge gate: slower + directshortwave unit/periodic checks, the vegetation-module-vs-`v2.2.0` + regression, and the MPI averaging regression; also solver/branch-build + dependent. +- `all` — `supported` + `experimental`. + +`--python` (or `UDALES_TEST_PYTHON`) selects the interpreter used to launch +child suites; by default it's whatever interpreter you invoked +`run_tests.py` with, so activate your conda environment or +`tools/python/.venv` first. + +To run only the Python package tests directly, without the dispatcher, use +`python -m unittest discover -s tools/python/tests -p "test_*.py"`. This +needs `tools/python` on the Python path (`pip install -e tools/python +--no-deps`, as CI does) and `environment.yml`'s dependencies. The +visualisation tests (`test_udbase_vis.py`) additionally need +`PYVISTA_OFF_SCREEN=true` and a headless GL/Xvfb setup, and can pull in the +optional Plotly backend (`tools/python/requirements-plotly.txt`). + +For suites that need a compiled solver (the solver/MPI-driven suites under +`supported`/`experimental`), build first — see the [development +notes](udales-development-notes.md) for a `Debug` build, or the +[installation guide](udales-installation.md) for a `Release` build. + +## Continuous integration + +`.github/workflows/ci.yml` runs on every push and pull request (a +`[skip ci]` marker in the head commit message skips it on `push`). It has +four jobs: + +- **`build-and-supported-tests`** — matrix over `os: [ubuntu-latest, + macos-latest]` × `build-type: [Debug, Release]`. Installs system + dependencies (`gfortran`, MPI, NetCDF, Graphviz, FFTW) and the conda + `udales` environment, builds the preprocessing tools and installs + `tools/python` editable, builds uDALES with CMake, summarises compiler + warnings (see below), then runs `tests/run_tests.py supported` — except on + `macos-latest`, which currently runs `supported-macos` instead. +- **`python-viz`** — Linux-only; installs headless OpenGL/Xvfb and the + optional Plotly backend, then runs `test_udbase_vis.py` off-screen with + both the PyVista and Plotly backends via `xvfb-run`. +- **`docs`** — builds the MkDocs site (`mkdocs build --site-dir build/html`) + and the FORD Fortran API docs (`ford ford.md`), uploads the combined HTML + as an artifact. +- **`publish-docs`** — only on `push` to `master` (and not `[skip ci]`); + downloads the `docs` artifact and publishes it to GitHub Pages. + +Compiler warnings from each build are summarised by +`.github/scripts/summarise_warnings.sh` into the GitHub Actions step +summary: it counts warnings by `-W` class, flags known-benign classes +(`-Wcompare-reals`, `-Wunused-value`, reviewed in #334) versus everything +else as "review", and lists the actionable warning sites. This step is +reporting-only and never fails the build — CI doesn't pin compiler versions, +so warning sets legitimately vary across runner images. + +`.github/workflows/ci-rerun-on-cancel.yml` watches for CI runs that GitHub +itself cancelled (infra/runner loss, not a genuine test failure) and +automatically re-triggers them, capped at 3 attempts, so contributors don't +need to manually re-run infra-flaky jobs. + +## Expectations for contributions + +Before requesting review, a pull request should have a green +`build-and-supported-tests` matrix (all four OS/build-type combinations) and +a green `python-viz` job — together these run the `supported` (or +`supported-macos`) test selection plus both visualisation backends. The +`docs` job should also build cleanly if you touched documentation. Compiler +warnings are reported but do not gate merging, so a new warning won't fail +CI, but check the step summary and avoid introducing avoidable ones. +`experimental` and `heavy` suites are not part of the required gate, but are +worth running locally if your change touches the areas they cover (see +`tests/test_suites.yml` for what each suite exercises). diff --git a/docs/udales-troubleshooting.md b/docs/udales-troubleshooting.md new file mode 100644 index 000000000..58edc5eea --- /dev/null +++ b/docs/udales-troubleshooting.md @@ -0,0 +1,43 @@ +# Troubleshooting + +This page collects common problems and their fixes, grouped by the stage of the [workflow](udales-workflow.md) where they tend to show up. If your problem isn't listed here, please open a [GitHub issue](https://github.com/uDALES/u-dales/issues). + +## Build problems + +**`Could NOT find NetCDF (missing: NETCDF_LIBRARIES NETCDF_INCLUDE_DIRS NETCDF_HAS_INTERFACES)`** — CMake could not locate the NetCDF-C and/or NetCDF-Fortran libraries and headers. On HPC systems in particular, CMake often cannot find NetCDF automatically; hint the install locations explicitly with `-DNETCDF_DIR= -DNETCDF_FORTRAN_DIR=` as shown in the [HPC build instructions](udales-installation.md#build-on-hpcs). On local systems, install the `libnetcdf-dev`/`libnetcdff-dev` (Linux) or `netcdf`/`netcdf-fortran` (macOS Homebrew) packages listed in [Installation](udales-installation.md#prerequisites). + +**`Only GNU, Intel, and Cray Fortran compilers are supported`** — CMake found a Fortran compiler that isn't GNU (`gfortran`), Intel, or Cray, so it doesn't know which compile flags to apply (see the `elseif` chain in `CMakeLists.txt`). Set `FC`/`CC` (or `-DCMAKE_Fortran_COMPILER=...`) to point at a supported compiler before configuring, e.g. `FC=mpiifort cmake ...` as in the [HPC build example](udales-installation.md#build-on-hpcs). + +**`Build type 'X' not supported.`** — `CMAKE_BUILD_TYPE` was set to something other than `Debug` or `Release`. Reconfigure with `-DCMAKE_BUILD_TYPE=Release` or `Debug` (see the [build options table](udales-installation.md#build-defaultsoptions)). + +**CMake reports NMake generator errors, or `numpy.f2py`/build tools fail near a truncated OneDrive-style path** — both are native-Windows preprocessing build issues (wrong default generator, and paths containing spaces breaking f2py). These are already documented, with fixes, in [Windows setup for preprocessing libraries](udales-preprocessing-windows.md#troubleshooting) — use `-G Ninja` explicitly and build from a no-space `subst` alias path. + +## Pre-processing problems + +**`The preprocessing route and path to case/experiment folder must be set.`** / **`config.sh must be set inside `** — `write_inputs.sh` requires both a route flag (`-m` for MATLAB or `-p` for Python) and a case directory, and the case directory must contain a `config.sh` defining `DA_TOOLSDIR` and `DA_EXPDIR`. See the usage message and checks in `tools/write_inputs.sh` and the [pre-processing guide](udales-pre-processing.md) for a template `config.sh`. + +**`Python virtual environment not found or not executable: /.venv/bin/python`** — the Python pre-processing route (`write_inputs.sh -p`) expects a virtual environment at `$DA_TOOLSDIR/python/.venv`. Create it first with `bash $DA_TOOLSDIR/python/setup_venv.sh ` as the script itself suggests, then rerun `write_inputs.sh -p`. + +**`Unrecognised preprocessing route: X`** — `write_inputs.sh` only accepts `-m` (MATLAB) or `-p` (Python) as its first argument; any other value is rejected. See [Pre-processing](udales-pre-processing.md) for the two supported routes. + +## Runtime problems + +**`ERROR: Namoptions does not exist`** — the executable was started without a valid namoptions path, or the file isn't in the working directory the run script `cd`'d into. `local_execute.sh`/`hpc_execute.sh` invoke the executable as `$DA_BUILD namoptions.$exp` from inside the copied output directory, so the file must be named `namoptions.` and match the three-digit experiment number in the case path. + +**`ERROR: Problem in namoptions ` with `iostat error: `** — a namelist block (`RUN`, `DOMAIN`, `PHYSICS`, `BC`, `WALLS`, `TREES`, ...) failed to parse, usually because of a typo'd key, a missing value, or a stray/misplaced entry. Each namelist has its own check in `src/modstartup.f90` (and similar per-module checks, e.g. `NAMSUBGRID` in `src/modsubgrid.f90`, `NAMCHECKSIM` in `src/modchecksim.f90`); compare your `namoptions` block against the [input parameters overview](udales-namoptions-overview.md). + +**`STOP ERROR IN NUMBER OF PROCESSORS` / `nprocx must divide itot!!!` / `nprocy must divide jtot!!!`** — `nprocx` and `nprocy` in `namoptions` must exactly divide `itot`/`jtot` (and `ktot` for `nprocy`) so the domain decomposes into equal pencils; see the [`nprocx`/`nprocy` constraints](udales-namoptions-overview.md#namelist-run) and the [parallelisation overview](udales-2decomp.md). This is distinct from the related 2DECOMP&FFT startup error **`Invalid 2D processor grid - nproc /= p_row*p_col`**, which fires when the number of MPI ranks you launched with doesn't equal `nprocx * nprocy`; the run scripts require `nprocx * nprocy = NCPU` (or `NNODE * NCPU` on HPC), see [Running uDALES](udales-simulation-setup.md). + +**`Number of CPU cores NCPU must be set inside /config.sh` / `NCPU must be equal to the product of nprocx and nprocy set in /namoptions.`** — `local_execute.sh`/`hpc_execute.sh` check for `NCPU` (and, on HPC, `NNODE`) before launching `mpiexec`/`mpirun`; set these in `config.sh` so their product matches `nprocx * nprocy`, per the note above. + +**`ERROR: no restartfile set`** — `lwarmstart` or `lstratstart` was set to `.true.` in the `RUN` namelist without a corresponding `startfile`. Set `startfile` to the name of an existing restart file (`initdNNNNNNNN_xxx_xxx.000`), see the [`RUN` namelist](udales-namoptions-overview.md#namelist-run). + +**`ERROR: invalid itree_mode. Supported values are 1 (drag only), 2 (sveg), 99 (legacy SEB).`** / **`ERROR: legacy tree SEB (itree_mode=99) cannot be combined with lEB=.true.`** — `ltrees=.true.` requires `itree_mode` to be one of the supported drag/SEB modes, and the legacy tree SEB mode (`99`) cannot be combined with the building energy balance (`lEB=.true.`). Check the `TREES` namelist against `src/modstartup.f90`. + +**`ERROR: zgrid.inf does not exist`** — a driven simulation (`idriver = 2`) couldn't find the `zgrid.inf`/`zgrid.inl` inflow files written by the corresponding precursor run. Make sure the precursor (`idriver = 1`) has completed and `driverjobnr` points at the right experiment number, per [Driver simulations](udales-driver-simulations.md). + +**Simulation stops partway through a driven run** — for `idriver = 2`, `runtime` must be `(driverstore-1)*dtdriver` seconds or less, where `driverstore`/`dtdriver` come from the precursor; exceeding this runs out of saved inflow data, per the note in [Driver simulations](udales-driver-simulations.md#running-driven-simulations). + +**Simulation crashes with a floating-point exception (SIGFPE), or diverges/produces `NaN` output** — GNU builds are compiled with `-ffpe-trap=invalid,zero,overflow` (`CMakeLists.txt`), so an invalid operation, division by zero, or overflow anywhere aborts the run immediately rather than silently propagating `NaN`s. This is usually a sign the timestep is too large or the flow has become numerically unstable. Watch the Courant number, diffusion number, and divergence printed periodically by `modchecksim` (controlled by `tcheck` in `NAMCHECKSIM`); if the Courant number is climbing, reduce `dtmax` or enable `ladaptive` so the solver keeps `courant` within its recommended `1 <= courant <= 2` range — see the [`dtmax`/`ladaptive`/`courant` entries](udales-namoptions-overview.md#namelist-run) and the [`tcheck` entry](udales-namoptions-overview.md#namelist-namchecksim). + +**`Error: y-averaged statistics not currently implemented for nprocx > 1.`** / **`Error: constant x outflow only possible for nprocx = 1.`** / **`Error: constant y outflow only possible for nprocy = 1.`** — `lydump`/`lytdump`, `luoutflowr`, and `lvoutflowr` each have a decomposition restriction: run with a single pencil in the relevant direction (`nprocx = 1` or `nprocy = 1`), or disable the option. See `src/modstartup.f90`. diff --git a/docs/udales-udbase-tutorial.md b/docs/udales-udbase-tutorial.md index 67d73c9fc..d7c9ea2d3 100644 --- a/docs/udales-udbase-tutorial.md +++ b/docs/udales-udbase-tutorial.md @@ -1,6 +1,9 @@ # Introduction to uDALES post-processing with MATLAB +!!! warning "MATLAB path to be retired in v3.0" + These tutorials currently use the MATLAB toolchain. The MATLAB route is deprecated and will be retired from uDALES v3.0 onwards, in favour of the [uDALES Python package](udales-python-package.md). + This tutorial describes how to read and process facet data of the LES code uDALES using MATLAB. This tutorial introduces the `udbase` post-processing class. There are separate tutorials for processing field files and facet files. The **`udbase`** post-processing class reads in most important input parameters, and contains a number of methods to load field and facet data. @@ -29,7 +32,7 @@ The **`udbase`** post-processing class reads in most important input parameters, - **convert_facflx_to_field**. This method converts a facet variable to a density in a 3D field, so it can be used for post-processing (e.g. calculating distributed drag). - **convert_facvar_to_field**. This method assigns facet variables to the 3D grid, so it can be used for post-processing (e.g. averaging over y). -**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_mlx.** +**The live matlab file of this tutorial can be found in the repository in the folder /docs/tutorial_udbase.** ## Initialising udbase diff --git a/docs/udales-utility-tutorial.md b/docs/udales-utility-tutorial.md index ab22ee205..902198bf2 100644 --- a/docs/udales-utility-tutorial.md +++ b/docs/udales-utility-tutorial.md @@ -1,6 +1,9 @@ # Using uDALES utilities in MATLAB +!!! warning "MATLAB path to be retired in v3.0" + These tutorials currently use the MATLAB toolchain. The MATLAB route is deprecated and will be retired from uDALES v3.0 onwards, in favour of the [uDALES Python package](udales-python-package.md). + This tutorial demonstrates how to use the uDALES MATLAB utilities for post-processing simulation data. It covers 1) time-averaging; 2) merging short-term time-averaged data to long-term averages, and 3) coarse-graining fields for Spatial filtering (coarse-graining method). The utility functions include: - [**time_average**](#time_average-time-averaged-mean-and-variance-of-instantaneous-data). This routine calculates the time-average of a quantity (assumes time is in the last index of the variable). diff --git a/docs/udales-workflow.md b/docs/udales-workflow.md index 1c7ba2782..d4bae8571 100644 --- a/docs/udales-workflow.md +++ b/docs/udales-workflow.md @@ -1,3 +1,54 @@ -# Workflow +# Workflow overview -Page under development ... +A uDALES study follows the same sequence of steps regardless of whether you run on a laptop or an HPC cluster. This page gives the end-to-end picture; each step links to the detailed guide. + +```mermaid +flowchart LR + A["Install & build"] --> B["Set up experiment"] + B --> C["Pre-process
write_inputs.sh"] + C --> D["Run
local/hpc/archer_execute.sh"] + D --> E["Merge outputs
gather_outputs.sh"] + E --> F["Analyse
udbase"] +``` + +## 1. Install and build + +Install the prerequisites and build the uDALES executable, either manually with CMake or with the helper script `tools/build_executable.sh`. See [Installation](udales-installation.md). + +## 2. Set up an experiment + +Each simulation lives in its own experiment directory (e.g. `experiments/001`), identified by a three-digit experiment number. It contains: + +- `namoptions.001` — the simulation configuration; see the [input parameters overview](udales-namoptions-overview.md) and [boundary conditions](udales-boundary-conditions.md), +- an STL file describing the building geometry (except for special cases); see [geometry generation](udales-geometry-tutorial.md), +- `config.sh` — paths and settings used by the helper scripts. + +The quickest way to start is to adapt one of the [example simulations](udales-example-simulations.md) or [copy an existing setup](udales-copy-inputs.md). Simulations with inflow-outflow boundary conditions may need a [precursor (driver) simulation](udales-driver-simulations.md) to generate inflow data. + +## 3. Pre-process + +The pre-processing step turns the namoptions file and geometry into the input files the model reads at startup (facets, grids, initial profiles, etc.): + +```sh +./u-dales/tools/write_inputs.sh -m experiments/001 +``` + +See [Pre-processing](udales-pre-processing.md) for setup and options, including the Python route (`-p`) and running on clusters. + +## 4. Run the simulation + +Launch the solver through the wrapper script for your platform — `local_execute.sh` (desktop), `hpc_execute.sh` (ICL cluster), or `archer_execute.sh` (ARCHER2): + +```sh +./u-dales/tools/local_execute.sh experiments/001 +``` + +See [Running uDALES](udales-simulation-setup.md), and [cluster workflows](cluster_workflows.md) for cluster-specific notes. + +## 5. Merge outputs + +uDALES writes one NetCDF file per CPU. After the run, merge them into single output files with `gather_outputs.sh` (done automatically by `local_execute.sh`). See [Post-processing](udales-post-processing.md). + +## 6. Analyse + +Load and analyse the merged NetCDF output with the `udbase` MATLAB class — see the [udbase](udales-udbase-tutorial.md), [fields](udales-fields-tutorial.md), and [facets](udales-facets-tutorial.md) tutorials. A [Python package](udales-python-package.md) providing the same functionality is under active testing and will replace the MATLAB toolchain from uDALES v3.0 onwards. diff --git a/docs/udales-docs-software.md b/ford.md similarity index 65% rename from docs/udales-docs-software.md rename to ford.md index 58147e749..db8d0018e 100644 --- a/docs/udales-docs-software.md +++ b/ford.md @@ -2,11 +2,11 @@ project: uDALES author: The uDALES Team project_github: https://github.com/uDALES/u-dales -summary: TODO: +summary: API documentation of the Fortran source code of uDALES, the open-source multi-physics large-eddy simulation model for urban environments. github: https://github.com/uDALES -src_dir: ../src -output_dir: ../build/html/udales-docs-software -media_dir: assets +src_dir: ./src +output_dir: ./build/html/udales-docs-software +media_dir: ./docs/assets docmark: \! predocmark: \!! docmark_alt: $$$ @@ -22,6 +22,3 @@ creation_date: %Y-%m-%d %H:%M %z md_extensions: markdown.extensions.toc markdown.extensions.smarty --- - - - diff --git a/mkdocs.yml b/mkdocs.yml index cf1bd64bf..8467bc140 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -5,9 +5,9 @@ site_author: 'The uDALES Team' site_url: 'https://uDALES.github.io/u-dales' # Repository -repo_name: 'u-dales' +repo_name: 'uDALES/u-dales' repo_url: 'https://github.com/uDALES/u-dales' -edit_uri: '' # disable 'Edit this page' option (https://www.mkdocs.org/user-guide/configuration/#edit_uri) +edit_uri: '' # disable 'Edit this page' option (https://www.mkdocs.org/user-guide/configuration/#edit_uri) # Copyright copyright: 'The uDALES Team' @@ -16,74 +16,121 @@ copyright: 'The uDALES Team' theme: name: 'material' language: 'en' + icon: + logo: material/city + repo: fontawesome/brands/github palette: - primary: 'light-blue' - accent: 'pink' + # Palette toggle for light mode + - media: '(prefers-color-scheme: light)' + scheme: default + primary: 'light blue' + accent: 'deep orange' + toggle: + icon: material/brightness-7 + name: Switch to dark mode + # Palette toggle for dark mode + - media: '(prefers-color-scheme: dark)' + scheme: slate + primary: 'light blue' + accent: 'deep orange' + toggle: + icon: material/brightness-4 + name: Switch to light mode font: text: 'Roboto' code: 'Roboto Mono' - + features: + - navigation.tabs + - navigation.tabs.sticky + - navigation.top + - navigation.footer + - search.suggest + - search.highlight + - content.code.copy + - toc.follow # Extensions markdown_extensions: - - pymdownx.arithmatex + - admonition + - attr_list + - md_in_html + - footnotes + - toc: + permalink: true + - pymdownx.arithmatex: + generic: true - pymdownx.betterem: smart_enable: all - pymdownx.caret - pymdownx.critic - pymdownx.details - pymdownx.emoji: - emoji_generator: !!python/name:pymdownx.emoji.to_svg + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg + - pymdownx.highlight: + guess_lang: false - pymdownx.inlinehilite - pymdownx.magiclink - pymdownx.mark - pymdownx.smartsymbols - - pymdownx.superfences + - pymdownx.snippets: + check_paths: true + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.tasklist: custom_checkbox: true - pymdownx.tilde - - codehilite: - guess_lang: false - - toc: - permalink: true - - footnotes - - pymdownx.mark - - pymdownx.details - - pymdownx.superfences - - pymdownx.tasklist extra_javascript: - - 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-MML-AM_CHTML' + - 'javascripts/mathjax.js' + - 'https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js' # Page tree nav: - Home: index.md - - Installation: udales-installation.md - - "Simulation inputs": + - Getting started: + - Model overview: udales-model-overview.md + - Installation: udales-installation.md + - Workflow overview: udales-workflow.md + - Example simulations: udales-example-simulations.md + - Troubleshooting: udales-troubleshooting.md + - Simulation inputs: - List of input parameters: udales-namoptions-overview.md - Boundary conditions: udales-boundary-conditions.md - Geometry generation: udales-geometry-tutorial.md - Copying from other simulations: udales-copy-inputs.md - Precursor simulations: udales-driver-simulations.md - - "Running the simulation": + - Running simulations: - Pre-processing: udales-pre-processing.md - Windows preprocessing setup: udales-preprocessing-windows.md - Running uDALES: udales-simulation-setup.md - Post-processing: udales-post-processing.md - - "Data post-processing": + - Output files: udales-output-files.md + - Cluster workflows: cluster_workflows.md + - Tutorials: - udbase: udales-udbase-tutorial.md - Flow fields and statistics: udales-fields-tutorial.md - Facets: udales-facets-tutorial.md - Utility routines: udales-utility-tutorial.md - - Example simulations: udales-example-simulations.md - - How to cite: udales-how-to-cite.md - - Publication list: udales-pub-list.md - - Development notes: DEVELOP.md - - How to contribute: CONTRIBUTING.md - - Software docs: udales-docs-software/index.html + - Python package: udales-python-package.md + - Development: + - Overview: udales-development.md + - Code architecture: udales-architecture.md + - Extending uDALES: udales-extending.md + - Testing and CI: udales-testing.md + - Development notes: udales-development-notes.md + - How to contribute: udales-contributing.md + - Parallelisation (2DECOMP&FFT): udales-2decomp.md + - Using agents: udales-agents.md + - Fortran API reference: udales-software-docs.md + - About: + - How to cite: udales-how-to-cite.md + - Publication list: udales-pub-list.md plugins: - search - bibtex: bib_file: "docs/references.bib" - cite_style: "pandoc" \ No newline at end of file