Runtime and solvers
Final steps before launching a simulation
Runtime setup
At this stage in the setup workflow, the user defines the runtime information to configure the runtime behaviour of the flow solver, including the time step to use (only meaningful for transient solutions), as well as information about how often to write results to disk. XCALibre.jl provides a the Runtime to perform this operation.
XCALibre.Solve.Runtime — Type
Runtime(;
# keyword arguments
iterations::I,
write_interval::I,
time_step::N,
adaptive::A
) where {I<:Integer,N<:Number} = begin
# returned Runtime struct
Runtime{I<:Integer,F<:AbstractFloat}
(
iterations=iterations,
dt=time_step,
write_interval=write_interval,
adaptive=adaptive
)
endThis is a convenience function to set the top-level runtime information. The inputs are all keyword arguments and provide basic information to flow solvers just before running a simulation.
Input arguments
iterations::Integer: specifies the number of iterations in a simulation run.write_interval::Integer: defines how often simulation results are written to file (on the current working directory). The interval is currently based on number of iterations. Set to-1to run without writing results to file.time_step::AbstractFloat: the time step to use in the simulation. Notice that for steady solvers this is simply a counter and it is recommended to simply use1.adaptive::Union{Nothing, AdaptiveTimeStepping}: optionally enables adaptive time stepping. Pass anAdaptiveTimeSteppingobject to automatically adjustdtbased on the Courant number during transient simulations. Defaults tonothing, meaning a fixed time step is used.
Example
runtime = Runtime(iterations=2000, time_step=1, write_interval=2000)Configuration object
Once all the simulation configuration information has been defined, from discretisation scheme to runtime information, all settings must be wrapped in a Configuration object. The definition, including expected input arguments, for the Configuration object are detailed below.
XCALibre.Simulate.Configuration — Type
struct Configuration{SC,SL,RT,HW,PP}
schemes::SC
solvers::SL
runtime::RT
hardware::HW
postprocess::PP
endThe Configuration type is passed to all flow solvers and provides all the relevant information to run a simulation.
Inputs
schemes::NamedTuplethis keyword argument is used to pass distretisation scheme information to flow solvers. See Numerical setup for details.solvers::NamedTuplethis keyword argument is used to pass the configurations for the linear solvers for each field information to flow solvers. See Runtime and solvers for details.runtime::NamedTuplethis keyword argument is used to pass runtime information to the flow solvers. See Runtime and solvers for details.hardware::NamedTuplethis keyword argument is used to pass the hardware configuration and backend settings to the flow solvers. See Pre-processing for details.
Optional keywords
postprocessthis keyword argument is used to pass any fields that need to be post-processed. See Post-processing for details.
Example
config = Configuration(
solvers=solvers, schemes=schemes, runtime=runtime, hardware=hardware, boundaries=BCs)Initialising fields
The last (optional) step before running the simulation is to provide an initial guess for all the fields being solved. Although this step is optional, in most cases the flow solvers will perform better when initialised. To set an initial value for a field, the initialise! function is provided, which assigns a starting value to a given field.
XCALibre.Fields.initialise! — Function
function initialise!(field, value) # dummy function for documentation
# Assign `value` to field in-place
nothing
endThis function will set the given field to the value provided in-place. Useful for initialising fields prior to running a simulation.
Input arguments
fieldspecifies the field to be initialised. The field must be either aAbractScalarFieldorAbstractVectorFieldvaluedefines the value to be set. This should be a scalar or vector (3 components) depending on the field to be modified e.g. for anAbstractVectorFieldwe can specify asvalue=[10,0,0]
Note: in most cases the fields to be modified are stored within a physics model i.e. a Physics object. Thus, the argument value must fully qualify the model. For example, if we have created a Physics model named mymodel to set the velocity field, U, we would set the argument field to mymodel.momentum.U. See the example below.
Example
initialise!(mymodel.momentum.U, [2.5, 0, 0])
initialise!(mymodel.momentum.p, 1.25)initialise! also accepts a functions arguments and initialises a field by evaluating the function at each cell centre. The function should have the signature f(x, y, z) and it must return either a scalar or an SVector type (requirement to work on GPUs)
A uniform initial guess leaves a velocity field that does not satisfy continuity, which the first few solver iterations must work off before making physical progress. For external and internal flows around bodies, a potential-flow field is a much better starting point: it already satisfies continuity and carries the shape of the geometry.
potential_flow! solves a velocity-potential equation on the current mesh and projects the velocity field onto the resulting divergence-free field. Boundary conditions for the potential are inferred from those already assigned to pressure, so no extra setup is needed.
XCALibre.Solvers.potential_flow! — Function
potential_flow!(model, config; ncorrectors=0, pref=nothing, time=0)Project the current velocity field onto a divergence-free potential-flow field. Velocity boundary conditions supply the initial face flux. Velocity-potential boundary conditions are inferred from pressure: fixed pressure becomes fixed zero potential, periodic patches remain periodic, and other patches use zero normal gradient.
The corrected face-volume flux is returned with the linear-solver residual.
Call it after initialise! and before run!:
initialise!(model.momentum.U, velocity)
initialise!(model.momentum.p, 0.0)
potential_flow!(model, config; ncorrectors=5)
residuals = run!(model, config)On meshes with appreciable non-orthogonality, pass ncorrectors to run non-orthogonal correctors on the potential equation. Set it to zero for orthogonal meshes.
AMG solver
The AMG linear solver can be selected directly in SolverSetup. It supports mode=AMGSolver() for a standalone multigrid solve and mode=Cg() for AMG-preconditioned conjugate gradient on symmetric systems such as pressure equations. mode takes an instance, not a symbol: mode=:cg throws an ArgumentError.
SolverSetup(
solver = AMG(
mode = Cg(),
coarsening = SmoothAggregation(),
smoother = AMGJacobi()
),
preconditioner = Jacobi(),
convergence = 1e-7,
relax = 1.0,
rtol = 0.0,
atol = 1e-5
)For non-symmetric systems use mode=Bicgstab(), which gives AMG-preconditioned stabilised biconjugate gradient. A typical case is the pressure equation in compressible flow, where the implicit pressure convection is upwinded and therefore non-symmetric. Cg() rejects a non-symmetric matrix. examples/2D_cylinder_transonic_RANS_AMG_BICGStab.jl uses it for the pressure equation of a transonic RANS case.
Choice of coarsening for non-symmetric operators: Bicgstab() has been tested with the default SmoothAggregation(). RugeStuben() and Geometric() are accepted but have not been validated on non-symmetric operators - RugeStuben() builds its strength of connection from each row's entries only, not from the transpose - so prefer the default unless you have checked the alternative on your case.
If Bicgstab() breaks down - the shadow residual going orthogonal, or the stabilising step collapsing - the half-step iterate is kept and the solve restarts with a fresh shadow residual, at most twice.
scale_correction (on by default) is supported. It makes the V-cycle preconditioner depend on its input, but the solver is right-preconditioned, so the solution and residual it carries stay consistent, and convergence is confirmed against the true residual b - A*x.
SolverSetup(
solver = AMG(
mode = Bicgstab(),
smoother = AMGGaussSeidel(sweep = AMGForwardSweep())
),
preconditioner=DILU(),
convergence = 1e-7,
relax = 1.0,
rtol = 0.0,
atol = 1e-5
)Solving the coarsest level
The coarsest level of the hierarchy is small but is visited on every V-cycle, so on a GPU the default host round trip can dominate. coarse_solve selects how it is solved:
OnDevice(; max_rows = 512)(default) — device-resident factorisation, no host round trip. A coarsest level with more rows thanmax_rowsfalls back to the host, since the factor is rebuilt on every coarse refresh.OnHost()— always copy the coarsest right-hand side to the host and solve there. Also writtencoarse_solve = CPU().OnDeviceJacobi(; omega, iterations)andOnDeviceChebyshev(; degree, eig_ratio, lambda_scale)— apply a fixed number of smoother sweeps on the device instead of solving. Both are constant linear operators, so they are valid withmode = Cg()as well asmode = AMGSolver().OnDeviceKrylov(; solver, rtol, atol, itmax)— run an inner Krylov solve on the device. An inner Krylov solve is nonlinear in its right-hand side, which breaks the fixed-operator assumption of preconditioned CG, so this one requiresmode = AMGSolver()and is rejected withmode = Cg(). Passing a GPU backend, as incoarse_solve = CUDABackend(), selects it.
AMG(mode = AMGSolver(), coarsening = SmoothAggregation(), smoother = AMGJacobi(),
coarse_solve = OnDeviceChebyshev(degree = 8))On a CPU backend every OnDevice* option behaves as OnHost().
Launching flow solvers
In XCALibre.jl the run! function is used to start a simulation, which will dispatch to the appropriate flow solver for execution. Once the simulation is complete a NamedTuple containing residual information is returned for users to explore the convergence history of the simulation.
XCALibre.Solvers.run! — Method
function run!(
model::Physics, config;
output=VTK(), pref=nothing, ncorrectors=0, inner_loops=0
)
# here an internal function is used for solver dispatch
return residuals
endThis is the top level API function to initiate a simulation. It uses the user-provided model defined as a Physics object to dispatch to the appropriate solver.
Dispatched flow solvers
- Steady incompressible (SIMPLE algorithm for coupling)
- Transient incompressible (PISO algorithm for coupling)
- Steady weakly compressible (SIMPLE algorithm for coupling)
- Transient weakly compressible (PISO algorithm for coupling)
Input arguments
modelreference to aPhysicsmodel defined by the user.configConfiguration structure defined by the user with solvers, schemes, runtime and hardware structures configuration details.outputselect the format used for simulation results fromVTK()orOpenFOAM()(default =VTK())prefReference pressure value for cases that do not have a pressure defining BC. Incompressible solvers only (default =nothing)ncorrectorsnumber of non-orthogonality correction loops (default =0)inner_loopsnumber to inner loops used in transient solver based on PISO algorithm (default =0)
Output
This function returns a NamedTuple for accessing the residuals (e.g. residuals.Ux). The fields available within the returned residuals tuple depend on the solver used. For example, for an incompressible solver, a x-momentum equation residual can be retrieved accessing the Ux field i.e. residuals.Ux. Look at reference guide for each dispatch method to find out which fields are available.
Example
residuals = run!(model, config)
# to access the pressure residual
residuals.p Restarting simulations
It should be noted that when running a simulation with run!, the solution fields in the Physics model are mutated. Thus, running the simulation from the previous solution is simply a matter of reissuing the run! function. At present, this has the side effect of overwriting any existing solution files (.vtk or .vtu). Users must be aware of this behaviour.
In some cases, it may be desirable to solve a problem with a steady solver and use the solution to run transient simulations. This is possible using the change function.
XCALibre.ModelPhysics.change — Function
change(model::Physics, property, value) => updatedModel::PhysicsA convenience function to change properties of an exisitng Physics model.
Input arguments
model::PhysicsaPhysicsmodel to modifypropertyis a symbol specifying the property to changevalueis the new setting for the specifiedproperty
Output
This function return a new Physics object
Example
To change a model to run a transient simulation e.g. after converging in steady state
modelTransient = change(model, :time, Transient())