or we’ll need 1000x more compute (Exascale) than we have today, and we can do that via a proper exascale architecture for general purpose computing (i.e. without the special purpose computing approaches proposed by Intel competitors) – this is the latest message from Intel.
Just two recent headlines from the media:
Intel unveils 72-core x86 Knights Landing CPU for exascale supercomputing [ExtremeTech, Nov 26, 2013]
Intel rushes to exascale with redesigned Knights Landing chip [Computerworld, Nov 19, 2013]
Then other two headlines which are reflecting another aspect of Intel’s move:
Intel claims Knight’s Landing will slay HUMA [*] and bare all CUDA[**]‘s flaws [PC Perspective, Nov 20, 2013]
* Heterogeneous Uniform Memory Access from AMD
** Compute Unified Device Architecture from NVIDIA
Nvidia to Intel: ‘Which HPC chip brain will win? Let the people decide’ [The Register, Nov 20, 2013]
Referring to: Chip Shot: Intel Reveals More Details of Its Next Generation Intel® Xeon Phi™ Processor at SC’13 [Intel Newsroom, Nov 19, 2013]
Today at the Supercomputing Conference in Denver, Intel discussed form factors and memory configuration details of the next generation Intel® Xeon Phi™ processor (code named “Knights Landing”). The new revolutionary design will be based on the leading edge 14nm manufacturing technology and will be available as a host CPU with high-bandwidth memory on a processor package. This first-of-a-kind, highly integrated, many-core CPU will be more easily programmable for developers and improve performance by removing “off-loading” to PCIe devices, and increase cost effectiveness by reducing the number of components compared to current solutions. The company has also announced collaboration with the HPC community designed to deliver customized products to meet the diverse needs of customers, and introduced new Intel® HPC Distribution for Apache Hadoop* and Intel® Cloud Edition for Lustre* software tools to bring the benefits of Big Data analytics and HPC together. View the tech briefing.
High-bandwidth In-Package Memory:
Performance for memory-bound workloads
Flexible memory usage models
From: Intel Brings Supercomputing Horsepower to Big Data Analytics [press release, Nov 19, 2013]
Intel discloses form factors and memory configuration details of the CPU version of the next generation Intel® Xeon Phi™ processor (code named “Knights Landing”), to ease programmability for developers while improving performance.
During the Supercomputing Conference (SC’13), Intel unveiled how the next generation Intel Xeon Phi product (codenamed “Knights Landing“), available as a host processor, will fit into standard rack architectures and run applications entirely natively instead of requiring data to be offloaded to the coprocessor. This will significantly reduce programming complexity and eliminate “offloading” of the data, thus improving performance and decreasing latencies caused by memory, PCIe and networking.
Knights Landing will also offer developers three memory options to optimize performance. Unlike other Exascale concepts requiring programmers to develop code specific to one machine, new Intel Xeon Phi processors will provide the simplicity and elegance of standard memory programming models.
In addition, Intel and Fujitsu recently announced an initiative that could potentially replace a computer’s electrical wiring with fiber optic links to carry Ethernet or PCI Express traffic over an Intel® Silicon Photonics link. This enables Intel Xeon Phi coprocessors to be installed in an expansion box, separated from host Intel Xeon processors, but function as if they were still located on the motherboard. This allows for much higher density of installed coprocessors and scaling the computer capacity without affecting host server operations.
Several companies are already adopting Intel’s technology. For example, Fovia Medical*, a world leader in volume rendering technology, created high-definition, 3D models to help medical professionals better visualize a patient’s body without invasive surgery. A demonstration from the University of Oklahoma’s Center for Analysis and Prediction of Storms (CAPS) showed a 2D simulation of an F4 tornado, and addressed how a forecaster will be able to experience an immersive 3D simulation and “walk around a storm” to better pinpoint its path. Both applications use Intel® Xeon® technology.
Intel @ SC13 [HPCwire YouTube channel, Nov 22, 2013]
Intel presents technical computing solutions from SC13 in Denver, CO. [The CAPS demo is from [4:00] on]
From: Exascale Challenges and General Purpose Processors [Intel presentation, Oct 24, 2013]
CERN Talk 2013 presentation by Avinash Sodani, Chief Architect, Knights Landing Processor, Intel Corporation
The demand for high performance computing will continue to grow exponentially, driving to Exascale in 2018/19. Among the many challenges that Exascale computing poses, power and memory are two important ones. There is a commonly held belief that we need special purpose computing to meet these challenges. This talk will dispel this myth and show how general purpose computing can reach the Exascale efficiencies without sacrificing the benefits of general purpose programming. It will talk about future architectural trends in Xeon-Phi and what it means for the programmers.
About the speaker
Avinash Sodani is the chief architect of the future Xeon-Phi processor from Intel called Knights Landing. Previously, Avinash was one of the primary architects of the first Core i7/i5 processor (called Nehalem). He also worked as a server architect for Xeon line of products. Avinash has a PhD in Computer Architecture from University of Wisconsin-Madison and a MS in Computer Science from the same university. He has a B.Tech in Computer Science from Indian Institute of Technology, Kharagpur in India.
Summary
Many challenges to reach Exascale – Power is one of them
General purpose processors will achieve Exascale power efficiencies
– Energy/op trend show bridgeable gap of ~2x to Exascale (not 50x)
General purpose programming allows use of existing tools and programming methods.
Effort needed to prepare SW to utilize Xeon-Phi’s full compute capability. But optimized code remains portable for general purpose processors.
More integration over time to reduce power and increase reliability
From: Intel Formally Introduces Next-Generation Xeon Phi “Knights Landing” [X-bit labs, Nov 19, 2013]
According to a slide from an Intel presentation that leaked to the web earlier this year, Intel Xeon Phi code-named Knights Landing will be released sometimes in late 2014 or in 2015.
The most important aspect about the Xeon Phi “Knights Landing” product is its performance, which is expected to be around or over double precision 3TFLOPS, or 14 – 16GFLOPS/w; up significantly from ~1TFLOPS per current Knights Corner chip (4 – 6GFLOPS/w). Keeping in mind that Knights Landing is 1.5 – 2 years away, three times performance increase seem significant and enough to compete against its rivals. For example, Nvidia Corp.’s Kepler has 5.7GFLOPS/w DP performance, whereas its next-gen Maxwell (competitor for KNL) will offer something between 8GFLOPS/w and 16GFLOPS/w.
More from: Intel Brings Supercomputing Horsepower to Big Data Analytics [press release, Nov 19, 2013]
New Intel® HPC Distribution for Apache Hadoop* and Intel® Cloud Edition for Lustre* software tools bring the benefits of Big Data analytics and HPC together.
Collaboration with HPC community designed to deliver customized products to meet the diverse needs of customers.
High Performance Computing for Data-Driven Discovery
Data intensive applications including weather forecasting and seismic analysis have been part of the HPC industry from its earliest days, and the performance of today’s systems and parallel software tools have made it possible to create larger and more complex simulations. However, with unstructured data accounting for 80 percent of all data, and growing 15 times faster than other data1, the industry is looking to tap into all of this information to uncover valuable insight.
Intel is addressing this need with the announcement of the Intel® HPC Distribution for Apache Hadoop* software (Intel® HPC Distribution) that combines the Intel® Distribution for Apache Hadoop software with Intel® Enterprise Edition of Lustre* software to deliver an enterprise-grade solution for storing and processing large data sets. This powerful combination allows users to run their MapReduce applications, without change, directly on shared, fast Lustre-powered storage, making it fast, scalable and easy to manage.
The Intel® Cloud Edition for Lustre* software is a scalable, parallel file system that is available through the Amazon Web Services Marketplace* and allows users to pay-as-you go to maximize storage performance and cost effectiveness. The software is ideally suited for dynamic applications, including rapid simulation and prototyping. In the case of urgent or unplanned work that exceeds a user’s on-premise compute or storage performance, the software can be used for cloud bursting HPC workloads to quickly provision the infrastructure needed before moving the work into the cloud.
With numerous vendors announcing pre-configured and validated hardware and software solutions featuring the Intel Enterprise Edition for Lustre, at SC’13, Intel and its ecosystem partners are bringing turnkey solutions to market to make big data processing and storage more broadly available, cost effective and easier to deploy. Partners announcing these appliances include Advanced HPC*, Aeon Computing*, ATIPA*, Boston Ltd.*, Colfax International*, E4 Computer Engineering*, NOVATTE* and System Fabric Works*.
* Other names and brands may be claimed as the property of others.
1 From IDC Digital Universe 2020 (2013)
Mark Seager: Approaching Big Data as a Technical Computing Usage Model [ieeeComputerSociety YouTube channel, recorded on October 29, published on November 12, 2013]
Mark Seager, CTO for technical computing at Intel, discusses the amazing new capabilities that are spreading across industries and reshaping the world. Watch him describe the hardware and software underlying much of the parallel processing that drives the big data revolution in his talk at the IEEE Computer Society’s “Rock Stars of Big Data” event, which was held 29 October 2013 at the Computer History Museum in Santa Clara, CA. Mark leads the HPC strategy for Intel’s High Performance Computing division. He is working on an ecosystem approach to develop and build HPC systems for Exascale and new storage paradigms Big Data systems. Mark managed the Platforms portion of the Advanced Simulation and Computing (ASC) program at Lawrence Livermore National Laboratory (LLNL) and successfully developed with industry partners and deployed the five generations of TOP1 systems. In addition, Mark developed the LLNL Linux strategy and award winning industry partnerships in storage and Linux systems developments. He has won numerous awards including the prestigious Edward Teller Award for “Major Contributions to the State-of-the-Art in High Performance Computing.”
From: Discover Your Parallel Universe [The Data Stack blog from Intel, Nov 18, 2013]
That’s Intel’s theme at SC’13 this week at the 25th anniversary of the Supercomputing Conference. We’re using it to emphasize the importance of modernizing codes and algorithms to take advantage of modern processors (think lots of cores and threads and wide vector units found in Intel Xeon processors and Intel Xeon Phi coprocessors). Or simply put, “going parallel” as we like to call it. We have a fantastic publication called Parallel Universe Magazine for more on the software and hardware side of going parallel.
But we’re also using it as inspiration for the researchers, scientists, and engineers that are changing the world every day. We’re asking them to envision the universe we’ll live in if the supercomputing community goes parallel. A few examples:
In a parallel universe there is a cure
In a parallel universe natural disasters are predicted
In a parallel universe ideas become reality
Pretty lofty huh? But also inevitable. We will find a 100% cure to all forms of cancer according to the National Cancer Institute. We will be able to predict the weather 28-days in advance according to National Oceanic and Atmospheric Association. And everyone will eventually use computing to turn their ideas into products.
The only problem is it’ll be the year 2190 before we have a cure to pancreatic cancer, we’ll need 1000x more compute (Exascale) than we have today to predict the weather 28-days in advance, and the cost and learning curve of technical computing will need to continue to drop before everyone has access.
That’s our work here at Intel. We solve these problems. We drive more performance at lower cost which gives people more compute. The more compute, the better cancer researchers will understand the disease. We’ll shift that 2190 timeline left. We’ll also solve the challenges to reaching Exascale levels of compute which will make weather forecast more accurate. And we’ll continue to drive open standards. This will create a broad ecosystem of hardware and software partners which drives access on a broad scale.
From: Criteria for a Scalable Architecture 2013 OFA Developer Workshop, Monterey, CA [keynote on 2013 OpenFabrics International Developer Workshop, April 21-24, 2013]
By Mark Seager, CTO for the HPC Ecosystem, Intel Technical Computing Group
In this video from the 2013 Open Fabrics Developer Workshop, Mark Seager from Intel presents: Criteria for a Scalable Achitecture. Learn more at: https://www.openfabrics.org/press-room/2013-intl-developer-workshop.html
………………………………………………………..
Exascale Systems Challenges are both Interconnect and SAN
• Design with system focus that enables end-user applications
• Scalable hardware
– Simple, Hierarchal
– New storage hierarchy with NVRAM
• Scalable Software
– Factor and solve
– Hierarchal with function shipping
• Scalable Apps
– Asynchronous coms and IO
– In-situ, in-transit and post processing/visualization
Summary
• Integration of memory and network into processor will help keep us on the path to Exascale
• Energy is the overwhelming challenge. We need a balanced attack that optimizes energy under real user conditions
• B:F and memory/core while they have their place, they can also result in impediments to progress
• Commodity interconnect can deliver scalability through improvements in Bandwidth, Latency and message rates
………………………………………………………..
SAN: Storage Area Network Ci: Compute nodes NVRAM: Non-Volatile RAM
OSNj: ?Operating System and Network? SNk: ?Storage Node?
Lustre: the dominant parallel file system for HPC and ‘Big Data’
Moving Lustre Forward: Status & Roadmap [RichReport YouTube channel, Dec 2, 2013]
In this video from the DDN User Meeting at SC13, Brent Gorda from the Intel High Performance Data Division presents: “Moving Lustre Forward: Status & Roadmap.” Learn more: http://www.whamcloud.com/about/ and http://ddn.com
Intel Expands Software Portfolio for Big Data Solutions [press release, June 12, 2013]
New Intel® Enterprise Edition for Lustre* Software Designed to Simplify Big Data Management, Storage
NEWS HIGHLIGHTS
Intel® Enterprise Edition for Lustre* software helps simplify configuration, monitoring, management and storage of high volumes of data.
With Intel® Manager for Lustre* software, Intel is able to extend the reach of Lustre into new markets such as financial services, data analytics, pharmaceuticals, and oil and gas.
When combined with the Intel® Distribution for Apache Hadoop* software, Hadoop users can access Lustre data files directly, saving time and resources.
New software offering furthers Intel’s commitment to drive new levels of performance and features through continuing contributions the open source community.
SANTA CLARA, Calif., June. 12, 2013 – The amount of available data is growing at exponential rates and there is an ever-increasing need to move, process and store it to help solve the world’s most important and demanding problems. Accelerating the implementation of big data solutions, Intel Corporation announced the Intel® Enterprise Edition for Lustre* software to make performance-based storage solutions easier to deploy and manage.
Businesses and organizations of all sizes are increasingly turning to high-performance computing (HPC) technologies to store and process big data workloads due to its performance and scalability advantages. Lustre is an open source parallel distributed file system and key storage technology that ties together data and enables extremely fast access. Lustre has become the popular choice for storage in HPC environments for its ability to support tens of thousands of client systems and tens of petabytes of storage with access speeds well over 1 terabyte per second. That is the equivalent to downloading all “Star Wars”* and all “Star Trek”* movies and television shows in Blu-Ray* format in one-quarter of a second.
“Enterprise users are looking for cost-effective and scalable tools to efficiently manage and quickly access large volumes of data to turn valuable information into actionable insight,” said Boyd Davis, vice president and general manager of Intel’s Datacenter Software Division. “The addition of the Intel Enterprise Edition for Lustre to our big data software portfolio will help make it easier and more affordable for businesses to move, store and process data quickly and efficiently.”
The Intel Enterprise Edition for Lustre software is a validated and supported distribution of Lustre featuring management tools as well as a new adaptor for the Intel® Distribution for Apache Hadoop*. This new offering provides enterprise-class reliability and performance to take full advantage of storage environments with worldwide service, support, training and development provided experienced Lustre engineers at Intel.
The Intel® Manager for Lustre provides a consistent view of what is happening inside the storage system regardless of where the data is stored or what type of hardware is used. This tool enables IT administrators to easily manage tasks and reporting, provides real-time system monitoring as well as the ability to quickly troubleshoot. IT departments are also able to streamline management, shorten the learning curve and lower operational expenses resulting in time and resource savings, better risk mitigation and improved business decision-making.
When paired with the Intel® Distribution for Apache Hadoop, the Intel Enterprise Edition for Lustre software allows Hadoop to be run on top of Lustre, significantly improving speed in which data can be accessed and analyzed. This allows users to access data files directly from the global file system at faster rates and speeds up analytics time, providing more productive use of storage assets as well as simpler storage management.
As part of the company’s commitment to drive innovation and enable the open source community, Intel will contribute development and support as well as community releases to the development of Lustre. With veteran Lustre engineers and developers working at Intel contributing to the code, Lustre will continue its growth in both high-performance computing and commercial environments and is poised to enter new enterprise markets including financial services, data analytics, pharmaceuticals, and oil and gas.
The Intel Enterprise Edition for Lustre will be available in early in the third quarter of this year.
Filed under: Cloud Computing strategy, Enterprise computing, servers Tagged: 14nm, Big Data, exascale, exascale architecture, exascale computing, exascale efficiencies, exascale supercomputing, exascale systems, general purpose computing, general purpose processors, HPC, In-Package Memory, Intel, Intel Cloud Edition for Lustre, Intel HPC Distribution for Apache Hadoop, Intel Xeon Phi, Knights Landing, Lustre, many-core computing, Mark Seager, next generation Intel Xeon Phi, parallel file system, special purpose computing, supercomputing