NASA Advanced Supercomputing Division
The NASA Advanced Supercomputing Division is located at NASA Ames Research Center, Moffett Field in the heart of Silicon Valley in Mountain View, California. It has been the major supercomputing and modeling and simulation resource for NASA missions in aerodynamics, space exploration, studies in weather patterns and ocean currents, and space shuttle and aircraft design and development for over thirty years.
The facility currently houses the petascale Pleiades, Aitken, and Electra supercomputers, as well as the terascale Endeavour supercomputer. The systems are based on SGI and HPE architecture with Intel processors. The main building also houses disk and archival tape storage systems with a capacity of over an exabyte of data, the hyperwall visualization system, and one of the largest InfiniBand network fabrics in the world. The NAS Division is part of NASA's Exploration Technology Directorate and operates NASA's High-End Computing Capability Project.
History
Founding
In the mid-1970s, a group of aerospace engineers at Ames Research Center began to look into transferring aerospace research and development from costly and time-consuming wind tunnel testing to simulation-based design and engineering using computational fluid dynamics models on supercomputers more powerful than those commercially available at the time. This endeavor was later named the Numerical Aerodynamic Simulator Project and the first computer was installed at the Central Computing Facility at Ames Research Center in 1984.Groundbreaking on a state-of-the-art supercomputing facility took place on March 14, 1985 in order to construct a building where CFD experts, computer scientists, visualization specialists, and network and storage engineers could be under one roof in a collaborative environment. In 1986, NAS transitioned into a full-fledged NASA division and in 1987, NAS staff and equipment, including a second supercomputer, a Cray-2 named Navier, were relocated to the new facility, which was dedicated on March 9, 1987.
In 1995, NAS changed its name to the Numerical Aerospace Simulation Division, and in 2001 to the name it has today.
Industry Leading Innovations
NAS has been one of the leading innovators in the supercomputing world, developing many tools and processes that became widely used in commercial supercomputing. Some of these firsts include:- Installed Cray's first UNIX-based supercomputer
- Implemented a client/server model linking the supercomputers and workstations together to distribute computation and visualization
- Developed and implemented a high-speed wide-area network connecting supercomputing resources to remote users
- Co-developed NASA's first method for dynamic distribution of production loads across supercomputing resources in geographically distant locations
- Implemented TCP/IP networking in a supercomputing environment
- Developed a batch-queuing system for supercomputers
- Developed a UNIX-based hierarchical mass storage system
- Co-developed the first IRIX single system image 256-, 512-, and 1,024-processor supercomputers
- Co-developed the first Linux-based single-system image 512- and 1,024-processor supercomputers
- A 2,048-processor shared memory environment
Software Development
NAS develops and adapts software in order to "complement and enhance the work performed on its supercomputers, including software for systems support, monitoring systems, security, and scientific visualization," and often provides this software to its users through the NASA Open Source Agreement.A few of the important software developments from NAS include:
- NAS Parallel Benchmarks were developed to evaluate highly parallel supercomputers and mimic the characteristics of large-scale CFD applications.
- Portable Batch System was the first batch queuing software for parallel and distributed systems. It was released commercially in 1998 and is still widely used in the industry.
- PLOT3D was created in 1982 and is a computer graphics program still used today to visualize the grids and solutions of structured CFD datasets. The PLOT3D team was awarded the fourth largest prize ever given by the NASA Space Act Program for the development of their software, which revolutionized scientific visualization and analysis of 3D CFD solutions.
- FAST is a software environment based on PLOT3D and used to analyze data from numerical simulations which, though tailored to CFD visualization, can be used to visualize almost any scalar and vector data. It was awarded the NASA Software of the Year Award in 1995.
- INS2D and INS3D are codes developed by NAS engineers to solve incompressible Navier-Stokes equations in two- and three-dimensional generalized coordinates, respectively, for steady-state and time varying flow. In 1994, INS3D won the NASA Software of the Year Award.
- Cart3D is a high-fidelity analysis package for aerodynamic design which allows users to perform automated CFD simulations on complex forms. It is still used at NASA and other government agencies to test conceptual and preliminary air- and spacecraft designs. The Cart3D team won the NASA Software of the Year award in 2002.
- OVERFLOW is a software package developed to simulate fluid flow around solid bodies using Reynolds-averaged, Navier-Stokes CFD equations. It was the first general-purpose NASA CFD code for overset grid systems and was released outside of NASA in 1992.
- Chimera Grid Tools is a software package containing a variety of tools for the Chimera overset grid approach for solving CFD problems of surface and volume grid generation; as well as grid manipulation, smoothing, and projection.
Supercomputing History
| Computer Name | Architecture | Peak Performance | Number of CPUs | Installation Date |
| Cray XMP-12 | 210.53 megaflops | 1 | 1984 | |
| Navier | Cray 2 | 1.95 gigaflops | 4 | 1985 |
| Chuck | Convex 3820 | 1.9 gigaflops | 8 | 1987 |
| Pierre | Thinking Machines CM2 | 14.34 gigaflops | 16,000 | 1987 |
| Pierre | Thinking Machines CM2 | 43 gigaflops | 48,000 | 1991 |
| Stokes | Cray 2 | 1.95 gigaflops | 4 | 1988 |
| Piper | CDC/ETA-10Q | 840 megaflops | 4 | 1988 |
| Reynolds | Cray Y-MP | 2.54 gigaflops | 8 | 1988 |
| Reynolds | Cray Y-MP | 2.67 gigaflops | 88 | 1988 |
| Lagrange | Intel iPSC/860 | 7.88 gigaflops | 128 | 1990 |
| Gamma | Intel iPSC/860 | 7.68 gigaflops | 128 | 1990 |
| von Karman | Convex 3240 | 200 megaflops | 4 | 1991 |
| Boltzmann | Thinking Machines CM5 | 16.38 gigaflops | 128 | 1993 |
| Sigma | Intel Paragon | 15.60 gigaflops | 208 | 1993 |
| von Neumann | Cray C90 | 15.36 gigaflops | 16 | 1993 |
| Eagle | Cray C90 | 7.68 gigaflops | 8 | 1993 |
| Grace | Intel Paragon | 15.6 gigaflops | 209 | 1993 |
| Babbage | IBM SP-2 | 34.05 gigaflops | 128 | 1994 |
| Babbage | IBM SP-2 | 42.56 gigaflops | 160 | 1994 |
| da Vinci | SGI Power Challenge | 16 | 1994 | |
| da Vinci | SGI Power Challenge XL | 11.52 gigaflops | 32 | 1995 |
| Newton | Cray J90 | 7.2 gigaflops | 36 | 1996 |
| Piglet | SGI Origin 2000/250 MHz | 4 gigaflops | 8 | 1997 |
| Turing | SGI Origin 2000/195 MHz | 9.36 gigaflops | 24 | 1997 |
| Turing | SGI Origin 2000/195 MHz | 25 gigaflops | 64 | 1997 |
| Fermi | SGI Origin 2000/195 MHz | 3.12 gigaflops | 8 | 1997 |
| Hopper | SGI Origin 2000/250 MHz | 32 gigaflops | 64 | 1997 |
| Evelyn | SGI Origin 2000/250 MHz | 4 gigaflops | 8 | 1997 |
| Steger | SGI Origin 2000/250 MHz | 64 gigaflops | 128 | 1997 |
| Steger | SGI Origin 2000/250 MHz | 128 gigaflops | 256 | 1998 |
| Lomax | SGI Origin 2800/300 MHz | 307.2 gigaflops | 512 | 1999 |
| Lomax | SGI Origin 2800/300 MHz | 409.6 gigaflops | 512 | 2000 |
| Lou | SGI Origin 2000/250 MHz | 4.68 gigaflops | 12 | 1999 |
| Ariel | SGI Origin 2000/250 MHz | 4 gigaflops | 8 | 2000 |
| Sebastian | SGI Origin 2000/250 MHz | 4 gigaflops | 8 | 2000 |
| SN1-512 | SGI Origin 3000/400 MHz | 409.6 gigaflops | 512 | 2001 |
| Bright | Cray SVe1/500 MHz | 64 gigaflops | 32 | 2001 |
| Chapman | SGI Origin 3800/400 MHz | 819.2 gigaflops | 1,024 | 2001 |
| Chapman | SGI Origin 3800/400 MHz | 1.23 teraflops | 1,024 | 2002 |
| Lomax II | SGI Origin 3800/400 MHz | 409.6 gigaflops | 512 | 2002 |
| Kalpana | SGI Altix 3000 | 2.66 teraflops | 512 | 2003 |
| Cray X1 | 204.8 gigaflops | 2004 | ||
| Columbia | SGI Altix 3000 | 63 teraflops | 10,240 | 2004 |
| Columbia | SGI Altix 4700 | 10,296 | 2006 | |
| Columbia | SGI Altix 4700 | 85.8 teraflops | 13,824 | 2007 |
| Schirra | IBM POWER5+ | 4.8 teraflops | 640 | 2007 |
| RT Jones | SGI ICE 8200, Intel Xeon "Harpertown" Processors | 43.5 teraflops | 4,096 | 2007 |
| Pleiades | SGI ICE 8200, Intel Xeon "Harpertown" Processors | 487 teraflops | 51,200 | 2008 |
| Pleiades | SGI ICE 8200, Intel Xeon "Harpertown" Processors | 544 teraflops | 56,320 | 2009 |
| Pleiades | SGI ICE 8200, Intel Xeon "Harpertown"/"Nehalem" Processors | 773 teraflops | 81,920 | 2010 |
| Pleiades | SGI ICE 8200/8400, Intel Xeon "Harpertown"/"Nehalem"/"Westmere" Processors | 1.09 petaflops | 111,104 | 2011 |
| Pleiades | SGI ICE 8200/8400/X, Intel Xeon "Harpertown"/"Nehalem"/"Westmere"/"Sandy Bridge" Processors | 1.24 petaflops | 125,980 | 2012 |
| Pleiades | SGI ICE 8200/8400/X, Intel Xeon "Nehalem"/"Westmere"/"Sandy Bridge"/"Ivy Bridge" Processors | 2.87 petaflops | 162,496 | 2013 |
| Pleiades | SGI ICE 8200/8400/X, Intel Xeon "Nehalem"/"Westmere"/"Sandy Bridge"/"Ivy Bridge" Processors | 3.59 petaflops | 184,800 | 2014 |
| Pleiades | SGI ICE 8400/X, Intel Xeon "Westmere"/"Sandy Bridge"/"Ivy Bridge"/"Haswell" Processors | 4.49 petaflops | 198,432 | 2014 |
| Pleiades | SGI ICE 8400/X, Intel Xeon "Westmere"/"Sandy Bridge"/"Ivy Bridge"/"Haswell" Processors | 5.35 petaflops | 210,336 | 2015 |
| Pleiades | SGI ICE X, Intel Xeon "Sandy Bridge"/"Ivy Bridge"/"Haswell"/"Broadwell" Processors | 7.25 petaflops | 246,048 | 2016 |
| Endeavour | SGI UV 2000, Intel Xeon "Sandy Bridge" Processors | 32 teraflops | 1,536 | 2013 |
| Merope | SGI ICE 8200, Intel Xeon "Harpertown" Processors | 61 teraflops | 5,120 | 2013 |
| Merope | SGI ICE 8400, Intel Xeon "Nehalem"/"Westmere" Processors | 141 teraflops | 1,152 | 2014 |
| Electra | SGI ICE X, Intel Xeon "Broadwell" Processors | 1.9 petaflops | 1,152 | 2016 |
| Electra | SGI ICE X/HPE SGI 8600 E-Cell, Intel Xeon "Broadwell"/"Skylake" Processors | 4.79 petaflops | 2,304 | 2017 |
| Electra | SGI ICE X/HPE SGI 8600 E-Cell, Intel Xeon "Broadwell"/"Skylake" Processors | 8.32 petaflops | 3,456 | 2018 |
| Aitken | HPE SGI 8600 E-Cell, Intel Xeon "Cascade Lake" Processors | 3.69 petaflops | 1,150 | 2019 |
| Computer Name | Architecture | Peak Performance | Number of CPUs | Installation Date |
Storage Resources
Disk Storage
In 1987, NAS partnered with the Defense Advanced Research Projects Agency and the University of California, Berkeley in the Redundant Array of Inexpensive Disks project, which sought to create a storage technology that combined multiple disk drive components into one logical unit. Completed in 1992, the RAID project lead to the distributed data storage technology used today.The NAS facility currently houses disk mass storage on an SGI parallel DMF cluster with high-availability software consisting of four 32-processor front-end systems, which are connected to the supercomputers and the archival tape storage system. The system has 192 GB of memory per front-end and 7.6 petabytes of disk cache. Data stored on disk is regularly migrated to the tape archival storage systems at the facility to free up space for other user projects being run on the supercomputers.
Archive and Storage Systems
In 1987, NAS developed the first UNIX-based hierarchical mass storage system, named NAStore. It contained two StorageTek 4400 cartridge tape robots, each with a storage capacity of approximately 1.1 terabytes, cutting tape retrieval time from 4 minutes to 15 seconds.With the installation of the Pleiades supercomputer in 2008, the StorageTek systems that NAS had been using for 20 years were unable to meet the needs of the greater number of users and increasing file sizes of each project's datasets. In 2009, NAS brought in Spectra Logic T950 robotic tape systems which increased the maximum capacity at the facility to 16 petabytes of space available for users to archive their data from the supercomputers. As of March 2019, the NAS facility increased the total archival storage capacity of the Spectra Logic tape libraries to 1,048 petabytes with 35% compression. SGI's Data Migration Facility and OpenVault manage disk-to-tape data migration and tape-to-disk de-migration for the NAS facility.
As of March 2019, there is over 110 petabytes of unique data stored in the NAS archival storage system.
Data Visualization Systems
In 1984, NAS purchased 25 SGI IRIS 1000 graphics terminals, the beginning of their long partnership with the Silicon Valley-based company, which made a significant impact on post-processing and visualization of CFD results run on the supercomputers at the facility. Visualization became a key process in the analysis of simulation data run on the supercomputers, allowing engineers and scientists to view their results spatially and in ways that allowed for a greater understanding of the CFD forces at work in their designs.The hyperwall
In 2002, NAS visualization experts developed a visualization system called the "hyperwall" which included 49 linked LCD panels that allowed scientists to view complex datasets on a large, dynamic seven-by-seven screen array. Each screen had its own processing power, allowing each one to display, process, and share datasets so that a single image could be displayed across all screens or configured so that data could be displayed in "cells" like a giant visual spreadsheet.The second generation "hyperwall-2" was developed in 2008 by NAS in partnership with Colfax International and is made up of 128 LCD screens arranged in an 8x16 grid 23 feet wide by 10 feet tall. It is capable of rendering one quarter billion pixels, making it the highest resolution scientific visualization system in the world. It contains 128 nodes, each with two quad-core AMD Opteron processors and a Nvidia GeForce 480 GTX graphics processing unit for a dedicated peak processing power of 128 teraflops across the entire system—100 times more powerful than the original hyperwall. The hyperwall-2 is directly connected to the Pleiades supercomputer's filesystem over an InfiniBand network, which allows the system to read data directly from the filesystem without needing to copy files onto the hyperwall-2's memory.
In 2014, the hyperwall was upgraded with new hardware: 128 Intel Xeon "Ivy Bridge" processors and NVIDIA Geforce 780 Ti GPUs. The upgrade increased the system's peak processing power from 9 teraflops to 57 teraflops, and now has nearly 400 gigabytes of graphics memory.
Concurrent Visualization
An important feature of the hyperwall technology developed at NAS is that it allows for "concurrent visualization" of data, which enables scientists and engineers to analyze and interpret data while the calculations are running on the supercomputers. Not only does this show the current state of the calculation for runtime monitoring, steering, and termination, but it also "allows higher temporal resolution visualization compared to post-processing because I/O and storage space requirements are largely obviated... may show features in a simulation that would otherwise not be visible."The NAS visualization team developed a configurable concurrent pipeline for use with a massively parallel forecast model run on the Columbia supercomputer in 2005 to help predict the Atlantic hurricane season for the National Hurricane Center. Because of the deadlines to submit each of the forecasts, it was important that the visualization process would not significantly impede the simulation or cause it to fail.
NASA Advanced Supercomputing Resources
-
Other Online Resources