(Multi-scale, Multi-physics Modeling and Petascale
Scientific Computing)
We have recently been actively collaborating with researchers
from the Laboratory for
Computational Astrophysics at
the University of California at
San Diego, addressing fundamental questions regarding
reionization of the early universe following the Big Bang.
Modern observational astronomy has offered some glimpses into
the earliest precursors of galactic formation, giving rise to
debate as to the underlying physical processes that formed our
universe. Through this collaboration, we aim to address the
core components of this debate, through developing new models
for cosmic reionization in an attempt to match computational
simulation to telescope observation.
For these studies, we are developing coupled
radiation-hydrodynamic-chemical kinetics simulations that will
attempt to test some of these modern theories against
experimental observation. As with our research efforts in
fusion energy and core-collapse
supernovae, these models involve the coupling of a
large number of physical processes, including the compressible
Euler equations for gas motion, nonlinear radiation diffusion
equations for multi-frequency radiation transport, chemical
kinetics models for tracking the ionization state of primordial
elemental species, as well as a model for gravitational
acceleration due to star clustering and galactic formation. This
problem therefore combines all challenges facing modern
scientific computation: consistent modeling of the relevant
physical processes, analysis of the well-posedness of the
resulting PDE modeling system, the derivation and development of
high-fidelity numerical methods for accurately approximating
processes at large (cosmic) and ``small'' (solar) scales, and
the invention of solution strategies for efficiently solving the
resulting PDE model system on some of
the largest NSF
supercomputers available.
Figure 1:
Numerical Simulations of Cosmic Reionization
(left) and Cluster Formation (right). Images courtesy of
collaborators at the LCA.
In this effort we have been working on nearly all of these
fronts: deriving a coupled PDE modeling system that will both
capture the dominant physical processes while lending itself
to efficient numerical solution; deriving an accurate
time-evolution technique for the coupled radiation transport and
chemical kinetics processes occurring on multiple time and space
scales, and implementing computational algorithms based on
these approaches designed to utilize next-generation petascale
computing hardware [1, 2].
Figure 2:
Density field (left) and Neutral
Fraction (right) in cosmological reionization simulations
using new scalable algorithms (256^3 grid).
We show results from simulations of the early universe. The results
in Figure 1 are from simulations by collaborators prior to
the development of our new self-consistent radiation transport
module. The results in Figures 2-4 are from simulations using
the new fully implicit solver for radiation transport, chemical
ionization and gas energy feedback.
Figure 3:
Density field (top-left), star sources (top right),
ionized fraction (bottom left) at redshift z=6, for an early universe
simulation at 512^3 spatial resolution. Bottom right: weak scaling of
the radiation solver on a test cosmology run, showing ideal
O(n log(n)) growth in wall-clock time as the problem size is increased.
Figure 4:
Density, Radiation Energy, and Temperature
fields (left to right) at redshifts 20, 15, 12, 11, 10 and 9
(top to bottom), for an early universe simulation at 1024^3
spatial resolution (using 4096 cores on Kraken).
M.L. Norman, G.L.Bryan, R. Harkness, J. Bordner,
D. Reynolds, B. O'Shea, and R. Wagner. Petascale Computing:
Algorithms and Applications, chapter "Simulating cosmological
evolution with Enzo." CRC Press, 2007.
[2]
D.R. Reynolds, J.C. Hayes, P. Paschos, and
M.L. Norman. "Self-Consistent Solution of Cosmological
Radiation-Hydrodynamics and Chemical
Ionization." Journal of Computational Physics,
228:6833-6854, 2009.
[3]
M.L. Norman, D.R. Reynolds, and G.C. So. "Cosmological
Radiation Hydrodynamics with Enzo," Recent Directions in
Astrophysical Quantitative Spectroscopy and Radiation
Hydrodynamics, AIP, 2009.
[4]
I.T. Iliev, D. Whalen, K. Ahn, S. Baek, N.Y. Gnedin,
A.V. Kravtsov, G. Mellema, M. Norman, M. Raicevic,
D.R. Reynolds, D. Sato, P.R. Shapiro, B. Semelin,
J. Smidt, H. Susa, T. Theuns, and M. Umemura.
"Cosmological Radiative Transfer Codes Comparison Project
II: The Radiation-Hydrodynamic Tests," MNRAS,
400:1283-1316, 2009.
[5]
Michael L. Norman, Daniel R. Reynolds, Geoffrey C. So
and Robert P. Harkness,
"Direct Numerical Simulation of Reionization in Large
Cosmological Volumes I: Numerical Methods and Tests,"
submitted, 2013.
Funding Support
NSF AST Grant 1109008 (co-PI; with M. Norman),
2011-2014.
DOE INCITE Awards "How High Redshift Galaxies
Reionized the Universe" (co-PI; with M. Norman & R. Harkness),
2011-2012, 2012-2013.
NSF AAG Grant 0808184 (co-PI; with M. Norman),
2008-2011.
NSF OCI Grant 0832662 (supporting; with B. O'Shea),
2009-2011.
