Pre-processing
First steps required to set up a simulation
Mesh generation and requirements
As with all CFD solvers, defining a suitable mesh is one of the most important steps. XCALibre.jl does not provides mesh generation utilities (yet). Thus, the user will have to generate the grids using external mesh generation tools. However, XCALibre.jl does provide a number of mesh conversion tools to allow user to import their grids.
XCALibre.jl is an unstructured Finite Volume Method (FVM) library, therefore, we are able to support grids of arbitrary polyhedral cells. In XCALibre.jl a cell-centred FVM approach has been implemented, which is popular since it allows the representation of complex geometries and it is used by most commercial and many open source CFD solvers.
XCALIbre.jl does not enforce units strictly. Units are implicitly provided by the mesh. You can use the scale
keyword when importing a mesh to control the units used e.g. if the mesh file was created in millimetres, set scale=0.001
to work in metres. Thus, it is important to ensure that consistent units are used through all configuration entries. It is highly recommended that SI units are used.
Mesh conversion
XCALibre.jl at present supports .unv
mesh formats (which can be generated using SALOME) for simulations in 2D and 3D domains. XCALibre.jl also supports the OpenFOAM mesh format for simulations in 3D only (for now).
Currently, XCALibre.jl only supports loading mesh files stored in ASCII format. Please ensure that when saving grid files they are not saved in binary format. Most mesh generation programmes offer the option to export in ASCII (text-based) formats.
The following functions are provided for importing mesh files:
XCALibre.UNV2.UNV2D_mesh
— FunctionUNV2D_mesh(meshFile; scale=1, integer_type=Int64, float_type=Float64)
Read and convert 2D UNV mesh file into XCALibre.jl
Input
meshFile
– path to mesh file.
Optional arguments
scale
– used to scale mesh file e.g. scale=0.001 will convert mesh from mm to metres defaults to 1 i.e. no scalinginteger_type
- select interger type to use in the mesh (Int32 may be useful on GPU runs)float_type
- select interger type to use in the mesh (Float32 may be useful on GPU runs)
XCALibre.UNV3.UNV3D_mesh
— FunctionUNV3D_mesh(unv_mesh; scale=1, integer_type=Int64, float_type=Float64)
Read and convert 3D UNV mesh file into XCALibre.jl. Note that a limitation of the .unv mesh format is that it only supports the following 3D cells:
- Tetahedrals
- Prisms
- Hexahedrals
Input
unv_mesh
– path to mesh file.
Optional arguments
scale
– used to scale mesh file e.g. scale=0.001 will convert mesh from mm to metres defaults to 1 i.e. no scalinginteger_type
- select interger type to use in the mesh (Int32 may be useful on GPU runs)float_type
- select interger type to use in the mesh (Float32 may be useful on GPU runs)
XCALibre.FoamMesh.FOAM3D_mesh
— FunctionFOAM3D_mesh(mesh_file; scale=1, integer_type=Int64, float_type=Float64)
Read and convert 3D OpenFOAM mesh file into XCALibre.jl. Note that, at present, it is not recommended to run 2D cases using meshes imported using this function.
Input
mesh_file
– path to mesh file.
Optional arguments
scale
– used to scale mesh file e.g. scale=0.001 will convert mesh from mm to metres defaults to 1 i.e. no scalinginteger_type
- select interger type to use in the mesh (Int32 may be useful on GPU runs)float_type
- select interger type to use in the mesh (Float32 may be useful on GPU runs)
These conversion functions will read mesh information and generate a mesh object (Mesh2
or Mesh3
depending on whether the mesh is 2D or 3D, respectively) with the following properties:
XCALibre.Mesh.Mesh3
— Typestruct Mesh3{VC, VI, VF<:AbstractArray{<:Face3D}, VB, VN, SV3, UR} <: AbstractMesh
cells::VC # vector of cells
cell_nodes::VI # vector of indices to access cell nodes
cell_faces::VI # vector of indices to access cell faces
cell_neighbours::VI # vector of indices to access cell neighbours
cell_nsign::VI # vector of indices to with face normal correction (1 or -1 )
faces::VF # vector of faces
face_nodes::VI # vector of indices to access face nodes
boundaries::VB # vector of boundaries
nodes::VN # vector of nodes
node_cells::VI # vector of indices to access node cells
get_float::SV3 # store mesh float type
get_int::UR # store mesh integer type
boundary_cellsID::VI # vector of indices of boundary cell IDs
end
Mesh limitations and requirements
In this section we summarise the key limitations of the mesh loaders presented above, and we also highlight specific requirements.
UNV mesh files
- Only ASCII files are supported
- For 2D simulations the mesh must be contained in the X-Y plane only
- In 3D only hex, tet and prism elements are supported
OpenFOAM mesh files
- Only ASCII files are supported
- Boundary groups are not supported (must be deleted manually or the conversion may fail)
- Boundary information is not preserved (walls, symmetry, etc)
- 2D setups are not currently supported (but will be)
Backend selection
In XCALibre.jl the mesh object is very important, as it will not only provide geometry information about the simulation/s, but it is also used to automatically dispatch methods to run on the appropriate backend. Therefore, users must first select the backend they wish to use for the simulations, and then "adapt" the mesh to use the correct backend.
XCALIbre.jl
aims to work with all the backends supported by KernelAbstractions.jl. However, since internally XCALibre.jl
uses sparse arrays to reduce its memory footprint, some GPU backends are not currently supported since this functionality is not yet available. Thus, currently only a subset of backends are supported:
- CPU (multithreaded and tested)
- NVidia GPUs (tested)
- AMD GPUs (not tested - feedback welcome)
Selecting a given backend is straight-forward. The examples below show how to assign a backend (CPU or GPU) to the symbol backend
and converting the mesh object to run a simulation on the corresponding backend. The converted mesh is assigned to the symbol mesh_dev
for clarity.
CPU backend
Selecting the CPU backend is straight-forward. See the example below. Notice that CPU()
is a backend type provided by KernelAbstractions.jl which we re-export for convenience.
CPU Example
mesh = # call function to load mesh e.g. UNV2_mesh, UNV3_mesh or FOAM3D_mesh
backend = CPU()
mesh_dev = mesh # dummy reference to emphasise the mesh in on our chosen dev (or backend)
GPU backends
To execute the code on GPUS, the process is also quite simple, but does require a few additional steps.
- Install the corresponding Julia library that supports your hardware. For NVidia GPUs, the CUDA.jl package is required. For AMD GPUs, the AMDGPU.jl package is needed.
- Move the mesh object to the backend device using the
adapt
method, which for convenience we re-export from Adapt.jl
Example for Nvidia GPU
mesh = # call function to load mesh e.g. UNV2_mesh, UNV3_mesh or FOAM3D_mesh
backend = CUDABackend()
mesh_dev = adapt(backend, mesh) # make mesh object backend compatible and move to GPU
Example for AMD GPU
mesh = # call function to load mesh e.g. UNV2_mesh, UNV3_mesh or FOAM3D_mesh
backend = ROCBackend()
mesh_dev = adapt(backend, mesh) # make mesh object backend compatible and move to GPU
Hardware configuration
In order to configure the backend the Hardware
function can be used.
XCALibre.Simulate.Hardware
— Typehardware = Hardware(backend, workgroup)
Struct used to configure the backend.
Inputs
backend
: used to specify the backend e.g.CPU()
,CUDABackend()
or other backends supported byKernelAbstraction.jl
workgroup::Int
this is an integer specifying the number of workers that cooperate in a parallel run. For GPUs this could be set to the size of the device's warp e.g.workgroup = 32
. On CPUs, the default value inKernelAbstractions.jl
is currentlyworkgroup = 1024
.
Output
This function returns a Hardware
object with the fields backend
and workgroup
which are accessed by internally in XCALibre.jl
to execute a given kernel in the target backend
.