ENVIRON

Main menu:

CAMx Scalability with Parallelization

We present an example of CAMx runtimes on the US EPA's High Performance Computing (HPC) system (Atmos).  The CAMx v6.40 configuration includes:

  • Single domain over the eastern US, 225x225x25 grid at 12 km resolution
  • CB6r2 gas-phase chemistry + CF aerosol chemistry
  • PiG invoked for major point sources
  • Source Apportionment: 9 regions x 1 sector, OSAT + PSAT (sulfer and nitrogen families), 220 total tracers

The plot below shows model speed for 1 simulation day using combinations of OMP and MPI parallelization and combinations of standard disk and solid state (RAM) I/O.  Speed improves up to 512 cores:  

  • Multiple points shown for each number of total cores results from different OMP/MPI combinations
  • At 128 total cores, 128 MPI x 1 OMP is slowest, 32 MPI x 4 OMP is fastest
  • Fast I/O (such as solid state drives) become important at large numbers of cores
  • We recommend using OMP and MPI in combination
  • Conduct tests to determine which OMP/MPI combinations work best for your model application and computer system

Acknowledgment: We thank US EPA Region 7 for providing these results.