Terascale application for cosmological hydrodynamics

An application in the NCSA Alliance Performance Expedition.

We are interested in porting to and optimizing for the NSF TeraGrid the parallel structured adaptive mesh refinement (AMR) cosmology software package "Enzo", written by Greg Bryan with Michael Norman. This document serves to organize information that is potentially useful to this project, to help facilitate collaborations with software tool providers, and to document our progress.

PI: Michael L. Norman (mnorman@cosmos.ucsd.edu)
Technical contact: James Bordner (jbordner@cosmos.ucsd.edu)

Affiliation: Laboratory for Computational Astrophysics (LCA), a division of the Center for Astrophysics & Space Sciences (CASS) at the University of California, San Diego (UCSD)

Enzo-related web pages

• Obtaining the AMR code (Greg Bryan)
• Enzo User Guide (Greg Bryan)
• Enzo Primer (Alexei Kritsuk)
• Enzo: Demonstration of the Computational Grid (Zhiling Lan)
• Enzo parameters (grouped by function) (copy of Greg Bryan's page with minor cosmetic changes)
• Enzo parameters (sorted alphabetically) (sorted version of the above)

Technical papers

General

• Achieving Extreme Resolution in Numerical Cosmology Using Adaptive Mesh Refinement: Resolving Primordial Star Formation, Greg L. Bryan, Tom Abel, and Michael L. Norman, Proceedings, SC2001.
• A Hybrid AMR Application for Cosmology and Astrophysics, Greg L. Bryan and Michael L. Norman, arXiv:astro-ph/9710187, 16 October 1997.
• Cosmological Adaptive Mesh Refinement, Greg L. Bryan and Michael L. Norman, arXiv:astro-ph/9807121, 13 July 1998.

Load balancing

• Dynamic Load Balancing for Adaptive Mesh Refinement Applications: Improvements and Sensitivity Analysis, Z. Lan, V. Taylor, and G. Bryan, Proc. of 13th IASTED International Conference PDCS'2001,Anaheim, CA, August 2001.
• Dynamic Load Balancing for Structured Adaptive Mesh Refinement Applications, Z. Lan, V. Taylor, and G. Bryan, Proc. of 30th International Conference on Parallel Processing'2001, Valencia, Spain.
• Dynamic Load Balancing of SAMR applications on Distributed Systems, Z. Lan, V. Taylor, and G. Bryan, Proc. of SC'2001 (formerly known as Supercomputing), Denver, CO.

Software tools

Message passing tools

• MPI "MPI is a library specification for message-passing, proposed as a standard by a broadly based committee of vendors, implementors, and users."
• MPICH: MPICH is a freely available, portable implementation of MPI, the Standard for message-passing libraries.

Performance analysis tools

• PAPI: " PAPI aims to provide the tool designer and application engineer with a consistent interface and methodology for use of the performance counter hardware found in most major microprocessors."
• Pervis: Performance Visualization for Parallel Programs
• Pablo Toolkit: "...a portable performance data analysis environment that can be used with a variety of massively parallel systems.
• fpmpi: "...this tool reports a summary of all MPI calls, providing information about message lengths, [and] communication patterns."

AMR tools

• KeLP: "a framework for implementing portable scientific applications on distributed memory parallel computers. It is intended for applications with special needs, in particular, that adapt to data-dependent or hardware dependent conditions at run time."

Data formats

• HDF: "The HDF project involves the development and support of software and file formats for scientific data management."

Storage systems

• High Performance Storage System (HPSS): "...provides a scalable parallel storage system for highly parallel computers as well as traditional supercomputers and workstation clusters."
• Parallel Virtual File System (PVFS): "...an effort to provide a high-performance and scalable parallel file system for PC clusters."

Data access

• Storage Resource Broker (SRB): "The SDSC Storage Resource Broker (SRB) is client-server middleware that provides a uniform interface for connecting to heterogeneous data resources over a network and accessing replicated data sets."

Visualization

• Advanced Visual Systems (AVS): "3D graphics system for scientific visualization."
• Interactive Data Language (IDL): ...software for data analysis, visualization, and cross-platform application development.
• LCA Vision: "your does-what-you-need-it-to-and-more-with-less-pain vis tool."

Hardware platforms

The NCSA IA32 cluster "Platinum"

• Technical summary
• User documentation
• Cluster monitor

The NCSA IA64 cluster "Titan"

• Technical summary
• User documentation
• Cluster monitor

Other related pages

• Performance Engineering and Computational Methods Group (PECM): "...a part of the [NCSA] Scientific Computing Division, provides support to high-end users to help them achieve high performance on NCSA supercomputers.