Presentation of the G8 Projects
A short slide presentation of all projects is available here:
http://www.exascale.org/mediawiki/images/a/a5/Talk-14-Tang.pdf

ECS (Enabling Climat Simulations at Exascale) (F. Cappello, PI)
Policy decisions for both mitigating and adapting to climate change are subjects of great discussion in the G8 countries and throughout the world. It is essential to reduce, as soon as possible, the current uncertainties about future climate change. Exascale simulations are fundamental toward this objective. As climate codes are ported to Exascale platforms, changes will be needed in current (10 km resolution) codes, running on O(100,000) cores, to handle a less friendly execution environment; major innovation will be required for ultra-resolution (1 km) on (10-100M) cores. The ECS project objectives are: 1) Understand what will be the major obstacles that the climate community will face when retooling for the Exascale regime and 2) Propose and evaluate possible ways to overcome these obstacles. The project explores three main topics scalability, node performance and resilience taking as reference codes the latest version of CESM and NICAM codes. In particular, we concentrate our efforts on modeling and auto-tuning/scheduling for intra-node heterogeneity with massive number of cores (e.g., CPUs + GPUs), exploiting extensive latency hiding to effectively hide traffic between core-memory, task decomposition to improve land-balancing, building approximate performance models of CESM modules, identifying likely scalability bottlenecks & define possible solutions (load balancing, overlap comm./comp., jitter tolerance), new naturally fault-tolerant computational algorithms, new fault-tolerance programming constructs and new hybrid fault-tolerant protocols. Experiments are run on the largest publically available systems, BlueGene/P, Tsubame2, the K computer and BlueWaters. After one year of collaborative research, several important limiting factors have been observed. ECS project members have started exploring solutions to improve scalability, node level performance and resilience.

ICOMEX (G. Zaengl, PI):
The primary goal of ICOMEX is to prepare four state-of-the-art Earth system models (ESMs) based on icosahedral grids - one being operational since a few years, the others being at advanced stages of development - for future application on exascale computing platforms. Such computing capacity will, for example, be needed for global convection-resolving climate simulations. These will allow circumventing the long-standing problem of convection parameterization and its related limitations to the reliability of predicted climate trends, and will provide a much better basis for human adaptation measures than presently available. The sub-projects focus on the development of key infrastructure components for exascale computing (e.g. parallel I/O, parallel internal post-processing, GPU usage, appropriate implicit solvers, and abstraction schemes for platform-specific optimization of memory bandwidth usage), plus an overarching iterative model inter-comparison effort. As most of the project staff started only in 2012, results are only starting to emerge now, but an I/O performance analysis tool being advanced in one of the sub-projects already proved useful for identifying specific bottlenecks in one of the participating models.

ExArch (Martin Juckes, PI) (http://proj.badc.rl.ac.uk/exarch)
Climate science demands on data management are growing rapidly as climate models grow in the precision with which they depict spatial structures and in the completeness with which they describe a vast range of physical processes. For the Climate Model Inter-comparison Project 5 (CMIP5), a distributed archive is being constructed to provide access to what is expected to be in excess of 10 Peta-bytes of global climate change projections. The data will be held at 30 or more computing centres and data archives around the world, but for users it will appear as a single archive described by one catalogue. In addition, the usability of the data will be enhanced by a three-step validation process and the publication of Digital Object Identifiers (doi) for all the data. For many users the spatial resolution provided by the global climate models (around 150km) is inadequate: the CORDEX project will provide data scaled down to around 10km. Evaluation of climate impacts often revolves around extremes and complex impact factors, requiring high volumes of data to be stored. At the same time, uncertainty about the optimal configuration of the models imposes the requirement that each scenario be explored with multiple models.
ExArch will explore the challenges of developing a software management infrastructure which will scale to the multi-exabyte archives of climate data which are likely to be crucial to major policy decisions in by the end of the decade. Support for automated processing of the archived data and metadata will be essential. In the short term goal, strategies will be evaluated by applying them to the CORDEX project data.

SEISMIC IMAGING (J. Tromp, PI):
The goal of our G8 project is the development of sophisticated 3D seismic imaging tools for the characterization of earthquakes, Earth noise, and mapping of Earth’s interior on all scales. Thus far, we have further developed and enhanced our open-source spectral-element applications for GPU systems, achieving excellent weak scaling up to 900 CPUs on ORNL's Cray XK6 "Titandev". Our first application of adjoint methods has been to image the European upper mantle. Using three-component traveltime and dispersion measurements from 190 earthquakes recorded by 745 seismographic stations, we performed 30 tomographic iterations, requiring a total of 17,100 wavefield simulations and 2.3 million CPU core hours. During the inversion, smaller scale structures —such as slabs, upwellings, and delaminations— naturally emerge from the smooth background of the 3D starting model, thereby bridging the gap between high-resolution traveltime tomography and lower resolution inversions based on long-period waves and free oscillations.

NuFuSE (G. Ackland, PI):
The goal of the G8-NuFuSE project is to support efforts to produce Nuclear Fusion power as a clean abundant energy source by delivering high fidelity predictive capability for simulating multi-physics, multi-scale burning plasmas in ITER – a multi-billion dollar burning plasma experiment involving 7 governments representing over half of the world’s population. This requires development of the needed software to utilize computing at the extreme scale. In addressing the key issue of exploiting local concurrency in Fusion Energy Sciences (FES) codes, NuFuSE will carry out computational science research in 4 areas: (i) Plasma Physics by developing GPU versions of CPU codes and by improving data locality in particle-in-cell (PIC) codes through sorting particles according to positions on the grid; (ii) Edge Physics via development of tree codes (mesh-free simulations) to deal with the geometric complexity; (iii) Materials Physics by developing techniques for rare events such as temperature accelerated dynamics, parallel replica dynamics, and hyper-dynamics using dynamic time and spatial time stepping methods; and (iv) Computer Science & Applied Math for developing the needed algorithm changes to respond to extreme scale computing challenges such as data locality and low memory per core issues. Examples of progress include: (i) improved physics insights from PIC code investigations of the reasons behind improvement of confinement with increasing plasma size predicted for ITER; (ii) advances in software tools such as XcalableMP for testing, developing, and implementing new capabilities; (iii) demonstrated scalability of kinetic plasma codes to 200K processors on Cray XT5 and Blue-Gene-P platforms in the US and Germany – with plans in place for deployment on the K-machine in Japan; and (iv) development & improvement of proto-type GPU version of a global PIC code that represents progress in better use of Flops (cheap) to improve use of memory (limited and expensive to access).

INGENIOUS (M. Taiji, PI):
The kinetics of biomolecular processes such as protein-ligand binding or membrane ion channel transport define many biomedical technologies and for this reason represent the fundamental basis for a large segment of pharmaceutical industries. The main challenge for numerical simulations is modeling atomistic water that takes up to 90% of computing resources and makes the calculations prohibitively expensive. Although the atomistic details are unnecessary in the areas distant from the bimolecular, in the vicinity of it the explicit representation of water molecules is decisively important. When studying bimolecular systems of experimentally relevant sizes (thousands of atoms) even the most advanced computer simulations are limited to deal with very short times. The problem of extending the simulations to much longer times calls for a global solution. One of the approaches combines the atomistic simulation of all chemically important molecules with the continuum hydrodynamic simulation of the bulk water. The combination of new hardware architectures with novel mathematical techniques generates fundamentally new concepts that become actively developed recently. The objectives of INGENIOUS are 1) to develop a new efficient theoretical and computational framework for hybrid molecular dynamics - hydrodynamics simulation of bio-chemically important processes at realistic time and space scales; 2) to implement and test this framework in the world fastest supercomputing facility; 3) to conduct large scale simulations of trimethoprim (TMP) binding to dihydrofolate reductase (DHFR) and compare the predicted kinetic properties, the binding rate, with measured experimental values. The drastic reduction of the cost of the water model will transform the technologies by allowing routing investigations of systems at the next level of time and space scales.