Getting started

Getting started

Add AMR/LB to your OpenFOAM case

Pre-requisites

1. An OpenFOAM installation (Up to dynamicMesh libs, with a few mesh manipulation utilities)
2. The blastAMR libraries (simply run the Allwmake script)
3. A solver that
   • supports OpenFOAM's dynamic meshes
   • including correct flux interpolation/reconstruction on mesh changes
4. Ability to load shared libraries at case level (set controlDict.libs to load blastAMR libraries)

Important

The first mesh.update() call in each time step will trigger AMR and LB functionality. Subsequent calls in the same time step will not.

What case files need to be added/changes?

Here is a quick git workflow to figure out the steps needed to add AMR/LB support for some default OpenFOAM tutorials:

# 0.0 Grab the first commit in which I touched the case
myfirstcommit=$(git log --format='%H' --author=Elwardi --reverse -- tutorials/damBreak2D | head -1)
# 0.1 Look for the original folder name for the target tutorial
orig_path=$(git log --diff-filter=R --summary | grep -o 'tutorials/{.* => damBreak2D}' | uniq | sed -e 's/[{}]//g' -e 's/\(.*\)=>.*/\1/')

# 1.0 Explore the diffs
# You want the diffs between the last commit from the original subtree and the most recent commit
# However, files will most likely be moved around and renamed, hence the use of $orig_path
git diff $myfirstcommit~..HEAD -M -- $orig_path tutorials/damBreak2D

The diffs will always yield up-to-date steps to add AMR/LB support for your OpenFOMA case; however, there are some common trends for the simple cases (using the adaptiveFvMesh dynamic mesh type):

1. Library loading in system/controlDict
2. Addition of constant/dynamicMeshDict
3. Adaptation of run scripts (Allrun), mostly to run updateMesh utility before the solver a few times.

The same git workflow applies to solvers and library parts.

A complete constant/dyanmicMeshDict reference

Note

As it is an issue of all documentation of this kind that has ever existed, this list may not be complete as some defaulted parameters may unfortunately be sneaked-in by the code. Open issues if you are unsure of a particular keyword or feature.

// The dynamic mesh class to use.
// adaptiveFvMesh is available only if you load libamrDynamicFvMesh.so in your controlDict
// Hint: you can replace with staticFvMesh to test things out without the AMR/LB being active
// however, make sure your mesh is then fine enough to make fair comparisons 
dynamicFvMesh              adaptiveFvMesh;

// The refinement engine to use.
// Options:
// - polyRefiner: This the preferred one as it handles any cell shapes
// - hexRefiner: Use this if your base mesh is purely-hex, especially if you have
//               refinement histories from other utilities (eg. SHM)
refiner polyRefiner;

// Boolean to activate load balancing
// Pre-requisites:
// - Your BCs must allow for zero-sized boundary patches
// - Your solver must handle changing patches
balance yes;

// Fraction of load-imbalance between MPI ranks that necessitates
// executing the load-balancing logic
// eg. A value of 0.2 (default) waits for the imbalance to exceed 20% of the average load
// Currently, the "load" is the number of cells each processor is responsible for
allowableImbalance 0.2;

// Load policy for calculating processor imbalance
// Options: cellCount (default), cpuLoad
loadPolicy cellCount;

// Coefficient for particle contribution to load
// load = cellCount + particleCoeff * particleCount
// For now, only used with cellCount policy when Lagrangian clouds are present
particleCoeff 1.0;

// Minimum cells per processor to avoid degenerate distributions
// Load balancing is skipped if any processor would end up with fewer cells
// This is important to set in lagrangian-centered simulations where the background
// mesh is coarse
minCellsPerProc 5;


// Interval to run the refinement logic periodically (in terms of timeSteps)
refineInterval             100;
// Interval to run the coarsening logic periodically (in terms of timeSteps)
// recommended to be less often, as the coarsening logic processes mesh vertices instead of cells
// which is more costly and scales worse.
unrefineInterval           500;

// How far to refine, choose of one maxRefinement or minDx

// split 0-level cells' edges maxRefinement times
// this is preferred as it's clear how many times to run updateMesh in initialization phases
maxRefinement 2;
// refine until this cell size is reached
minDx 1e-3;

// AMR cells selector, to control which cells to refine/unrefine/leave-alone
// one of https://github.com/STFS-TUDa/blastAMR/tree/v2212/src/errorEstimators if
// the libamrIndicators.so lib is loaded
// Although only the following ones are reliable enough:
// - delta: tracks the interface on a field
// - coded: UDF to supply your own code to select the cells
// Each estimator/indicator will then have its own settings added
// straight to the top level of the dictionary
errorEstimator             <estimatorTypeName>;

// common estimator setting: lower and upper bounds for refinement
// refine cells with: lowerRefineLevel < indicator < upperRefineLevel
lowerRefineLevel 0.01;
upperRefineLevel 0.99;

// common estimator setting: lower and upper bounds for unrefinement
// unrefine around points if: indicator < unrefineLevel || indicator > upperUnrefineLevel
unrefineLevel   0.01;
upperUnrefineLevel 0.99;


// number of cells between unrefinement levels
// it is important to keep this > 1 as to mitigate errors of size-based interpolation
// between adjacent cells with high aspect ratio
nBufferLayers  3;
// You can also control buffer layers for refinement/unrefinement individually
// these will be prioritized over nBufferLayers if supplied
nRefinementBufferLayers    3;
nUnrefinementBufferLayers  6;

// List of patches to protect from refinement, useful for example if you
// don't want the AMR engine to mess with your boundary layers, or if you have
// coupled BCs that don't support non-conforming patches
protectedPatches();

// number of cells protected from refinement next to protected patches
// coordinate this with nBufferLayers and maxRefinement to make sure your boundary layers
// are safe
nPatchesBuffers 3;

// Do not interpolate the flux on newly created faces
// this setting is solver-dependent. If you have flux issues, try to instruct the flux correction
// to construct any phi-like field from the velocity, or try preventing interpolation on new faces
correctFluxes
(
    ( phi none )
);

// Propagate refinement to probes function objects
refineProbes               no;

// Interval to check load balancing (in terms of timeSteps)
// defaults to refineInterval if not specified
balanceInterval            100;

// Time window control for refinement/unrefinement/balancing
// Operations are only performed when simulation time (seconds) is within [begin, end]
beginRefine                0;
endRefine                  GREAT;
beginUnrefine              0;
endUnrefine                GREAT;
beginBalance               0;
endBalance                 GREAT;

// Maximum number of cells allowed in the mesh
// Refinement is disabled when this limit is reached
maxCells                   labelMax;

// Advanced: expire sampled surfaces after load balancing
// Workaround for surfaceFieldValue function objects with stale face indices
// Set to true if you use surfaceFieldValue and encounter post-LB crashes
expireSampledSurfacesOnLB  false;

// debug options
dumpLevel                  no;

As for the specific options for each error estimator/indicator, check this wiki page.

For a complete tabular reference of all options (including defaults and types), see the Options Reference page

Alternative: Function Object Route

Instead of using adaptiveFvMesh, you can use the loadBalancedAMR function object. This is useful when you need to combine AMR/LB with other dynamicFvMesh types, such as dynamicMotionSolverFvMesh for mesh motion.

Important

Usage of of the provided function object does not lift the dynamic mesh support requirement. This is provided stricktly as a way to use LB routines in mesh-motion cases.

See the Function Object Route page for complete documentation