The self-hosted design of next-generation Xeon Phi processors eliminates wait for data arriving via PCI-E bus.
Intel’s new, but still-unreleased, supercomputer microprocessor has found its first home – in a Cray supercomputer scheduled to join a host of other supercomputers already running at the National Energy Research Scientific Computing Center (NERSC).
NERSC – which is based at the UC Oakland Scientific Facility in Oakland, California and managed by the Dept. of Energy’s Lawrence Berkeley National Laboratory – has signed an order for a Cray XC30 supercomputer built on the next-generation Knights Landing edition of Intel’s Xeon Phi processor, which is spec’d with a peak performance of more than 3 teraFLOPS per single-socket node.
NERSC is the main computing center for the U.S. Department of Energy’s (DOE) Office of Science, which is responsible for enough of the really impressive scientific research paid for by the U.S. government that the Office of Science had to create a (fairly long) table to list all the Nobel Prizes awarded to scientists working under the DOE’s umbrella during the past 89 years.
NERSC is home base for more than 5,000 scientists working on an average of 700 active projects, and is also one of the few sites in the country to house not just one computer super enough to have a proper name, but six.
Hopper – a Cray XE6 built from 153,216 Opteron cores, 211 terabytes of memory and a top speed of 8.4 gigaFLOPS per core is the one singled out for unflattering comparison to the Knight’s Landing-based system, which is expected to deliver more than 10 times the performance of NERSC’s XE6. Hopper is no slug, but the leader of NERSC’s current pack is the 2.6 teraFLOPS Intel Ivy Bridge-based Cray XC30 named Edison.
Rather than crowding the other supers, the new Knights Bridge-based XC30 – which will be named Cori in honor of Gerty Cori, the Prague-born American biochemist who was the first woman to win a Nobel Prize for science – will be installed in the $145 million Computational Research and Theory facility under construction at UC Berkeley. When the CRT building is finished and occupied in January, 2015, it will include a datacenter for Cori that is supported by an earthquake-resistant floor and cooling systems that use outside air and water rather than refrigerators to wick heat away from Cori’s processors and recycle it to heat the rest of the building, according to NERSC briefing documents. When it’s finally finished, Cori’s datacenter will use half the electricity it would have using traditional power-management techniques, and should rate a nearly-perfect 1.1 on the Green Grid’s Power Usage Effectiveness(PUE) scale.
The $70 million Cori system is due for delivery in mid-2016 and will be one of the first supercomputers to be based on a self-hosted processor, meaning it will house its own memory chips within the same package as the processor rather than sticking the memory outside on the motherboard and connect to it via PCI-Express bus, as does the current version of Xeon Phi.
The result is a processor that doesn’t require complex programming to help it manage external memory and the latency that comes with even tiny bits of extra distance and which also creates a higher-bandwidth connection that allows the processor to pull in more data with each cycle as well.
Cori will contain more than 9,300 Knights Landing compute nodes, which will be fed data by a monstrous Cray Lustre File System storage system with 28 petabytes of disk space and 400 Gbit/sec of I/O bandwidth. Cray’s Aries high-speed interconnect, Dragonfly network topology and Cray HPC-optimized programming environment.
The goal, according to NERSC’s procurement-requirement sheet, is a system that will be able to deliver enough compute power to meet a level of demand expected to be 47 times as high in 2016 as in 2011, equaling a total of 75 petaFLOPS from all the center’s supercomputers combined.
The benefit of the self-hosted fast-memory access architecture of the Knight’s Landing processors isn’t the number of cores or even the horsepower, however. It’s the higher performance possible from workloads whose performance is held back more by the need to move data on and off a co-processor than by the raw performance of the processor itself, according to Katie Antypas, head of NERSC’s Services Department told HPCWire.
“This agreement is a significant step in advancing supercomputing design toward the kinds of computing systems we expect to see in the next decade as we advance to exascale,” according to a NERSC statement quoting Steve Binkley, Associate Director of the Office of Advanced Scientific Computing Research. “U.S. leadership in HPC, both in the technology and in the scientific research that can be accomplished with such powerful systems, is essential to maintaining economic and intellectual leadership. This project was strengthened by a great partnership with DOE’s National Nuclear Security Administration.”