Recent academic research has been awakening to the disaggregated datacenter phenomenon already happening as the “next big thing” in the industry, as evidenced by the following excerpts from the Network Support for Resource Disaggregation in Next-Generation Datacenters [research paper* on HotNets-XII**, Nov 21-22, 2013]
Datacenters have traditionally been architected as a collection of servers wherein each server aggregates a fixed amount of computing, memory, storage, and communication resources. In this paper, we advocate an alternative construction in which the resources within a server are disaggregated and the datacenter is instead architected as a collection of standalone resources.
Disaggregation brings greater modularity to datacenter infrastructure, allowing operators to optimize their deployments for improved efficiency and performance. However, the key enabling or blocking factor for disaggregation will be the network since communication that was previously contained within a single server now traverses the datacenter fabric. This paper thus explores the question of whether we can build networks that enable disaggregation at datacenter scales.
…
Figure 2: Architectural differences
between server-centric and resource-centric datacenters***
As illustrated in Figure 2, the high-level idea behind diaggregation is to develop standalone hardware “blades”for each resource type including CPUs, memory, storage, and network interfaces as well as specialized components (GPUs, various ASIC accelerators, etc.). Those resource
blades are interconnected by a datacenter-wide network fabric. Understanding the specifications and nature of this network fabric is our focus in this paper.
* Sangjin Han (U.C.Berkeley), Norbert Egi (Huawei Corp.), Aurojit Panda, Sylvia Ratnasamy (U.C.Berkeley), Guangyu Shi (Huawei Corp.), Scott Shenker (U.C.Berkeley and ICSI)
** Twelfth ACM Workshop on Hot Topics in Networks
*** I should emphasize here that a disaggregated datacenter with shared disaggregated memory (as on the (b) part of the Figure 2. above) is NOT a kind of academic exageration but a relatively “near term reality” of the future.
Abbreviations used above (in addition to “C” for CPU and “M” for Memory):
QPI: QuickPath Interconnect (Intel)
IOH: I/O Hub (an Intel chipset architecture)
PCIe: PCI Express (Peripheral Component Interconnect Express)
SATA: Serial ATA (Advanced Technology Attachment)
NIC: Network Interface Controller
SAN: Storage Area Network
NAS: Network-Attached Storage
GPU: Graphics Processing Unit
FPGA: Field-Programmable Gate Array
ASIC: Application-Specific Integrated Circuit
From Network Support for Resource Disaggregation in Next-Generation Datacenters [presentation slides on HotNets-XII*, Nov 21-22, 2013]
The Trends: Disaggregation
HP MoonShot
– Shared cooling/casing/power/mgmt for server blades
AMD SeaMicro
– Virtualized I/O
[from the research paper:]
SeaMicro’s server architecture [6] uses a looser coupling of components within a single server … the network in SeaMicro’s architecture implements a 3D torus interconnect, which only disaggregate I/O and does not scale beyond the rack … [6] SeaMicro Technology Overview.
Intel Rack Scale Architecture
[from the research paper: SeaMicro’s server architecture [6] uses a looser coupling of components within a single server,] while Intel’s Rack Scale
Architecture (RSA) [15] extends this approach to rack scales. …
[15] Intel Newsroom. Intel, Facebook Collaborate on Future Data Center Rack Technologies.
Open Compute Project
Closing Remarks
Disaggregated datacenter will be “the next big thing”
– Already happening. We [i.e. the academic research] need to catch up!
From “Sharks” in the press at HP Discover, Barcelona – Day One coverage [HP Converged Infrastructure blog, Dec 10, 2013]
… we were hosting a large press announcement that went out over the wire on Monday at 3 pm local time (CET).
Here’s a brief summary of the announcement that was presented by Tom Joyce, Senior Vice President and General Manager, HP Converged Systems. The HP ConvergedSystem is a new product line completely reengineered up based on 21st-century assets and architectures for the New Style of IT. This is an important point as Tom emphasized – this is not a collection of piece parts, this is a completely new engineered solution, built on core building that are workload-optimized systems which are easy to buy, manage, and support – order to operations in as few as 20 days, with ONE tool to manage and most importantly having ONE point of accountability.
Built using HP Converged Infrastructure’s best-in-class servers, storage, networking, software and services, the new HP ConvergedSystem family of products deliver a total systems experience “out of the box.”
HP ConvergedSystem for Virtualization helps clients easily scale computing resources to meet business needs with preconfigured, modular virtualization systems supporting 50 to 1,000 virtual machines at twice the performance, and at an entry price 25 percent lower than competitive offerings.
HP ConvergedSystem 300 for Vertica speeds big data analytics, helping organizations turn data into actionable insights at 50 to 1,000 times faster performance and 70 percent lower cost per terabyte than legacy data warehouses.
HP ConvergedSystem 100 for Hosted Desktops, based on the award-winning HP Moonshot server, delivers a superior desktop experience compared to traditional virtual desktop infrastructure. This first PC on a chip for the data center delivers six times faster graphics performance and 44 percent lower total cost of ownership
The physical press release in my opinion was pretty cool, and one of the better ones I have attended. The new HP ConvergedSystem for Virtualization 300 and 700 debuted on stage with the theme from Jaws, with much snapping of camera flashes. Tom explained why the sharks theme was so integral to this particular system with core attributes of most “efficient”, ”best in class”, extremely “fast”, very “agile” and that it “never sleeps”!!
The best one liner from Tom Joyce during the session was “If I were VCE [VMware/Cisco/EMC combination] I would be getting out of the water!!” which was capture on the HP live streaming video s found here. Check it out as it is worth watching. I have also included the full “HP Shark” press release HP Introduces Innovations Built for the Data Center of the Future.
Here is a detailed press report on that: HP Targets VCE With Converged System Lineup [Dec 10, 2013].
The HP “Sharks” are in the Water [HP Converged Infrastructure blog, Dec 9, 2013]
Written by guest blogger Tom Joyce, Senior Vice President and General Manager, HP Converged Systems
Seven months ago HP announced the formation of our new Converged Systems business unit. I was excited to be asked to lead this new team because so many of our customers had told us they needed truly converged platforms for their datacenters. Over the last five years HP had developed Converged Infrastructure technologies for storage, networking and servers that enabled better and more cost effective solutions, but it was time to take it to the next level. We needed to bring all those technologies together in a way that collapsed the cost of IT infrastructure and made everything faster and easier.
Starting last summer, we built our team. We hired the best of the best from within HP and from elsewhere. We put in place an operating model and set of processes that allow us to do agile product development and deliver products to market rapidly and with high quality. And we got really creative in our thinking. We were also fortunate to get a lot of time with Meg [Ryan, HP CEO] and other top people throughout HP. This was critical because to deliver a game changing set of new products, we had to break down or change a lot of established processes in development, manufacturing, support and go-to-market. We had to break some glass, and Meg helped us do that by making this a high priority.
Based on the customer input, there were some critical things I knew we needed to do.
Move fast. The IT market is changing quickly, and I wanted to get our first set of products out by the end of the calendar year.
Do more than just combine existing server, storage, networking and software components. We needed to engineer these new products to deliver more with less infrastructure, and to handle the most important customer workloads exceptionally well.
Everything had to be simple – the ordering process, the system design, management, support, easy upgrades – everything.
Think about the “whole offer” and experience for the customer, not just the product itself. This meant providing a better process from end to end.
Deliver exceptional economics. The new product had to be priced to market with a clear return on investment for the customer.
Most importantly, we needed to make sure that our channel partners could make money selling this product, and could provide specialize services around it.
After developing our plan, we started “Project Sharks”. We called it this because if you think about it, a shark is perfectly engineered to accomplish its mission – it is the ideal hunting machine. When I was a kid I was fascinated by sharks. People tend to think of sharks as primitive creatures, but they are actually extremely sophisticated. Everything is designed with a purpose, and there is no waste. Sharks have a unique hydroskeleton, musculature, and skin. All these parts are connected to maximize thrust so that the animal can move fast, like a torpedo. Sharks are noted for being able to sense blood in the water, but beyond that they have an amazingly complete set of sensors – perhaps the most sophisticated set of “sensors in the sea.”
Our goal with “project sharks” was to build a perfectly designed virtual infrastructure machine. This week at HP Discover, Barcelona, we announced the new HP ConvergedSystem for Virtualization. Click here to find out more information. The two models are designed to be core building blocks for constructing a converged data center. They are very fast and efficient, delivering better raw IOPS for virtualization at a great cost point. They can handle a lot more virtual machines than a traditional configuration. They can also deliver about a 58% lower cost per VM over a 3 year period, as compared to our closest competitor.
Perhaps more important, we redesigned our whole delivery process as part of “project sharks”. The result is that HP or a channel partner can actually produce a configuration and quote for an HP ConvergedSystem in about 20 minutes, and the whole thing will be on one sheet of paper. HP ConvergedSystem 300 and the 700 installed and in production in a customer data center in as few as 20 days. We have also fully integrated the management, to make it simple, and the support. If support is needed, only one call to HP is required; you don’t need to deal with a server vendor, a storage vendor, etc. When it is time for firmware upgrades, the process for the whole system is integrated. And when you need additional capacity, we can ship a module out from our factory in one day, and it will be up and running in about five days.
These new “sharks” are not just for virtualization. We also announced that the HP ConvergedSystem 300 for Vertica as a new platform for big data analytics. The HP ConvergedSystem 100 is based on HP Moonshot servers, and ships as a Citrix XenDesktop system.
In the future the HP ConvergedSystem products will support additional workloads and ISV applications, and will be used as building blocks for HP CloudSystem private clouds, so stay tuned for more.
Our new Converged Systems business unit team is very excited about the opportunity to unleash these new “sharks”, and put them in the water. We are looking forward to hearing from our customers and partners about what they want us to do next, because the spirit of innovation is alive and well at HP.
On the Dec 10 HP Discover 2013 keynote HP’s hybrid cloud strategy was presented with the following slides, with comments made by the presenter added only for the HP CloudSystem private clouds part:
Bill Hilf Vice President, Converged Cloud Products and Services, is driving HP’s entire cloud roadmap (who came to HP 6 months ago from Microsoft where he was GM of Windows Azure Product Management): “HP Next Gen CloudSystem … to be released in the 1st half of 2014” with the following major characteristics:
“We created a killer interface. An easy to use, consumer inspired interface that is consistent across multiple types of experiences (from classic PC, administration, to mobile experiences). We also designed and optimized the interface for the different types of roles in the organization (from architect who might be designing a service, to end user or consumer of that service, as well as for IT operator and adminstrator).”
“We spent considerable effort and energy an choice and ability to really give customers the heterogeneous workload suport they need. And now we are taking openess to an entirely new level. And so for the first time with CloudSystem we are shipping HP Cloud OS which is our enterprise class, OpenStack**** platform which gives customers the great innovation from OpenStack to build modern cloud workloads. But we are also supporting the power of matrix, so that you can bridge today’s and
tomorrow’s workloads on the same system.”
*** OpenStack APIs are compatible with Amazon EC2 (see Nova/APIFeatureComparison)and Amazon S3 (see Swift/APIFeatureComparison) and thus client applications written for Amazon Web Services can be used with OpenStack with minimal porting effort
“And finally we’re giving customers and partners more confidence
than they’ve ever had before in this type of solution. … And that will be available in both a quick-ship, channel-ready fixed configuration as well as in a highly customizable solution. In addition CloudSystem will ship with cloud service automation (CSA), the industry-leading orchestration and hybrid cloud management software [read NEW! HP's solution for managing private and hybrid clouds] that gives an easy experience and easy management of next hybrid cloud environment. That could be clouds delivered in any physical infrastructure: public, managed or private. And lastly, when customers use clouds as to build private cloud there is boundless growth, because you can extend CloudSystem with public cloud resources: from the HP public cloud, or Amazon, or Savis. And this week we are also announcing support for Windows Azure, as well as two very important European partners: SFR and arsys, a service provider right here in Spain.
Underlying core technologies:
HP Converged Cloud brings OpenStack to the Enterprise [HewlettPackardVideos YouTube channel, Nov 6, 2013]
HP Converged Cloud delivers choice, confidence, and consistency. Learn how HP Cloud OS as part of the HP Converged Cloud portfolio leverages OpenStack to enable workload portability, simplified installation, and enhanced service lifecycle management.http://hp.com/cloud
Open source clouds and the enterprise [The HP Blog Hub, Nov 24, 2013]
Open source has long been linked to innovation. With a history tracing back to the origins of the public web, the concept of open source relies on the assumption that shared knowledge produces more and better innovation, which is better for everyone—as well as the business world.
Some pundits believe that it is the combination of cloud and the power of the open source community that has enabled such rapid cloud development, adoption, and innovation.
OpenStack: cloud source code at the ready
OpenStack® provides the building blocks for developing private and public cloud infrastructures. OpenStack comprises a series of interrelated projects, characterized by their powerful capabilities and massive scalability.
Like all open source projects, OpenStack is a group collaboration, consisting of a global community of developers and cloud computing technologists. HP is a top contributor and driving force behind OpenStack, helping it to become a leading software for open cloud platforms.
In other words, there’s a bright future for OpenStack, which is why HP chose it as the foundation for its hybrid cloud solutions.
HP Cloud OS
HP Cloud OS is the world’s first OpenStack-based cloud technology platform for hybrid delivery. HP Cloud OS enables our existing cloud solutions portfolio and new innovative offerings by providing a common architecture that is flexible, scalable, and easy to build on.
“We are in a new phase of cloud computing. Enterprises, government agencies, and industry are all placing demands on cloud computing technologies that exceed a singular, one-size-fits all delivery model,” says Bill Hilf, vice president of product management for HP Cloud. “HP Cloud OS, built on the power of OpenStack, is the foundation for the HP Cloud portfolio and a key part of the HP solutions that enable real customer choice and consistency.”
Watch the HP Cloud OS story at HP Discover
Attendees at HP Discover 2013 in Barcelona, don’t miss this opportunity to hear the inside story of HP’s development of HP Cloud OS. Join the Innovation Theater session:
IT3261 – The rise of open source clouds
In this session, Bill Hilf will walk you through his experiences working with large public cloud systems, the rise of open source clouds in the enterprise, and HP’s strategy and innovation with OpenStack, including a discussion of HP Cloud OS (Wednesday, 12/11/13, 4:30 pm).
Highlights from the presentation include:
How open source has affected the development of the cloud
The requirements of enterprises related to cloud computing
How OpenStack enables HP’s cloud platform
Top ten lessons learned when building HP’s public cloud
HP’s overall cloud strategy
William Franklin on HP Cloud and OpenStack Strategy for HP [hpcloud YouTube channel, Nov 5, 2013]
Discussion with William Franklin, VP OpenStack & Technology Enablement talks about HP Cloud and our open source strategy with OpenStack at the OpenStack Summit Hong Kong 2013.
OpenStack Technology [HewlettPackardVideos YouTube channel, Oct 29, 2013]
Monty Taylor, Distinguished Technologist and OpenStack Guru, talks about the OpenStack community, HP’s contributions to Havana and OpenStack projects, and the future of OpenStack.
Gartner’s Allessandro Perilli’s latest observations about the OpenStack (he is focusing on private cloud computing in the Gartner for Technical Professionals (GTP) division):
- What I saw at the OpenStack Summit [Nov 12, 2013] in which he is particularly describing how OpenStack vendors are divided into two camps that I called “purists” and “pragmatists”. He notes that purists tend to ignore the fact that many large enterprises are interested in OpenStack for the reason of reducing their dependency from VMware and frightened by rewriting their traditional multi-tier LoB applications into new cloud-aware applications advocated by purists.
- Why vendors can’t sell OpenStack to enterprises [Nov 19, 2013] where he notes that: “In fact, for the largest part, vendors don’t know how to articulate the OpenStack story to win enterprises. They simply don’t know how to sell it.” Then he gives at least four reasons for why vendors can’t tell a resonating story about OpenStack to enterprise prospects:
1. “Lack of clarity about what OpenStack does and does not.”
2. “Lack of transparency about the business model around OpenStack.”
3. “Lack of vision and long term differentiation.”
4. “Lack of pragmatism”, i.e. “purist” approach described in his previous post.
HP Cloud OS [Technology Preview] Technical Overview [hpcloud YouTube channel, Nov 5, 2013]
J.R. Horton, HP CloudOS Sr. Product Manager details the HP Cloud OS technology preview allowing developers access to a complete enterprise-grade OpenStack package for fast installation and deployment.
Converged Cloud: HP Cloud OS Whiteboard Demo [hpcloud YouTube channel, June 12, 2013]
[Mark Perreira, Chief Architect of HP Cloud OS:] This video demonstrates how HP Cloud OS can help simplify delivery, enhance lifecycle management and optimize workloads for your cloud environment. It includes information on Cloud OS architecture, kernel and base services, and administrative tools.
HP Cloud OS Whiteboard Demo – Hybrid Cloud [hpcloud YouTube channel, Oct 29, 2013]
Mark Perreira, Chief Architect of HP Cloud OS, whiteboards the hybrid provisioning capabilities in HP Cloud OS.
An Open Architecture for Hybrid Cloud Delivery [hpcloud YouTube channel, Dec 10, 2013]
J.R. Horton, HP Cloud OS Sr. Product Manager presents the HP Cloud architecture at HP Discover in Barcelona 2013. [Note that in addition to HP other OpenStack Foundation Platinum Members (providing a significant portion of the funding) are: AT&T, Canonical, IBM, Nebula, Rackspace, Red Hat, Inc., SUSE. Just today the news came as well that Oracle raised its membership to Platinum level.]
Moonshot: one of the “INFRA” (see above in the “HP Cloud OS Whiteboard Demo” video) building blocks for the HP CloudOS, actually the most future-oriented one
My Software defined server without Microsoft: HP Moonshot [‘Experiencing the Cloud’, April 10, 2013 – updated Dec 6, 2013] post introduced the HP Moonshot System as follows:
On the right is the Moonshot System with the very first Moonshot servers (“microservers/server appliances” as called by the industry) based on Intel® Atom S1200 processors and for supporting web-hosting workloads (see also on right part of the image below). Currently there is also a storage cartridge (on the left of the below image) and a multinode for highly dense computing solutions (see in the hands of presenter on the image below). Many more are to come later on.
The initial Moonshot System launched in April’13 had support just for light workloads, such as such as website front ends and simple content delivery. This meant, nevertheless, a lot in the hosting space as evidenced by serverCONDO Builds its Business on Moonshot [Janet Bartleson YouTube channel, Dec 9, 2013] video:
serverCONDO President John Brown wanted to expand to offer dedicated hosting, and traditional 1U servers looked pretty good, until the team discovered HP Moonshot. Hear more about what he was looking for and the results serverCONDO achieved. http://www.servercondo.com http://www.hp.com/go/moonshot
More information from the same source:
- Why serverCONDO is in the Dedicated Hosting Business
- Old School and New School Cloud Servers (serverCONDO)
OR taking a true large-scale example watch this HP.com Takes 3M Hits on Moonshot [Janet Bartleson YouTube channel, Nov 26, 2013] video:
Volker Otto talks about the results of using Moonshot for HP.com’s web site, caching, and ftp downloads. http://www.hp.com/go/moonshot
According to Meg Ryan’s keynote at Discover 2013 on Dec 10 they would be able to go from 6 datacenters to 4 thanks to Moonshot, even considering the future needs and workloads. Something as dramatic as when HP moved previously (3 years ago) from 86 datacenters to 6 datacenters.
So, to appreciate the full potential of Moonshot one should, on the other hand, understand the following system architecture information provided in the HP Moonshot System, the world’s first software defined servers [April 10, 2013] technical whitepaper:
HP Moonshot System
HP Moonshot System is the world’s first software defined server accelerating innovation while delivering breakthrough efficiency and scale with a unique federated environment, and processor-neutral architecture. Traditional servers rely on dedicated components, including management, networking, storage, power cords and cooling fans in a single enclosure. In contrast, the HP Moonshot System shares these enclosure components. The HP Moonshot 1500 Chassis has a maximum capacity of 1800 servers per 47U rack with quad server cartridges. This gives you more compute power in a smaller footprint, while significantly driving down complexity, energy use and costs.
The first server available on HP Moonshot System is HP ProLiant Moonshot Server based on Intel® Atom™ processor S1260, and it provides an ideal solution for web serving, offline analytics and hosting.
HP Moonshot 1500 Chassis design
The HP Moonshot 1500 Chassis incorporates independent component design and hosts 45 cartridges, two network switches, and the infrastructure components within the chassis. The Moonshot 1500 Chassis’ electrically passive design makes this completely hot pluggable design possible. The Moonshot 1500 Chassis uses no active electrical components, other than EEPROMs required for manufacturing and configuration control purposes.
Figure 1 shows the elements of the Moonshot 1500 Chassis. HP controls the design on all elements of the chassis except for the server (initial server contain a single server) and the network switch module which may be designed by the Moonshot server or network switch partners.
Figure 1.
The HP Moonshot 1500 Chassis accommodates up to 45 individually serviceable hot plug cartridges. Two high-density, low-power HP Moonshot 45G Switch Modules, each with a 10g x6 HP Moonshot 6SFP Uplink Module, handle network communication for all cartridges in the chassis. These switches use Layer 2/Layer 3 routing, QoS management (CLI, SFLOW), and require no license keys. The dual network switches and I/O modules provide traffic isolation, or stacking capability for resiliency. Rack level stacking simplifies the management domain.
The Moonshot System uses the HP Moonshot 1500 Chassis Management module (CM) module for complete chassis management, including power management with shared cooling. The server platform is powered by four 1200W Common Slot Power Supplies in an N+1 configuration and cooled by five hot pluggable fans also in an N+1 configuration. The CM uses component-based satellite controllers to communicate with and manage chassis elements. The modular faceplate design allows for future feature development.
…
HP ProLiant Moonshot Server
Each software defined server contains its own dedicated memory, storage, storage controller, and two NICs [Network Interface Controllers] (1Gb). For monitoring and management, each server contains management logic in the form of a Satellite Controller with a dedicated internal network connection (100 Mb). Figure 5 shows HP ProLiant Moonshot Server with a single Intel® Atom™ processor S1260and a single SFF drive.
Figure 5. HP ProLiant Moonshot Server and functional block diagram
These servers provide the base hardware functionality of the system. Future software defined servers can take the following forms:
One or more discrete server with separate compute, storage, memory and I/O
One or more complete cartridge designs with integrated compute, storage, memory, and I/O
One or more forms of storage accessible to adjacent cartridges
Future servers will incorporate these descriptions to provide a wide degree of flexibility for customizing and tuning based on the desired performance, cost, density, and power constraints.
The available ProLiant Moonshot server design includes one processor and a single HDD or SDD. This server is ideal for application workloads such as website front ends and simple content delivery. Table 1 gives you the current server component descriptions.
The Intel Atom is the world’s first 6-watt server-class processor. In addition to lower power requirements, it includes data-center-class features such as 64-bit support, error correcting code (ECC) memory, increased performance, and broad software ecosystem. These features, coupled with the revolutionary HP Moonshot System design are ideal for workloads using many extreme low-energy servers densely packed into a small footprint can be much more efficient than fewer standalone servers.
Intel® Atom™ processor S1260 integrates two CPU cores, single-channel memory controller, and PCI Express 2.0 interface. Each CPU core will has its own dedicated 32KB instruction and 24 KB data L1 caches, and 512 KB L2 cache. The processors incorporate Hyper-Threading, which allows them to run up to 4 threads simultaneously. Additionally, the chips have VT-x virtualization enabled.
Each Moonshot server boots from a local hard drive, or the network using PXE [Preboot eXecution Environment]. The Moonshot System use HP BIOS and “headless” operation (no video or USB). No additional HP software is required to run the cartridge. NIC, storage, and other drivers are included in the compatible Linux distributions (described later in the OS management section).
…
Fabrics and topology
We designed the HP Moonshot System to provide application-specific processing for targeted workloads. Creating a fabric infrastructure capable of accommodating a wide range of application-specific workloads requires highly flexible fabric connectivity. This flexibility allows the Moonshot System fabric architecture to adapt to changing requirements of hyperscale workload interconnectivity.
The Moonshot System design includes three physical production fabrics, the Radial Fabric, the Storage Fabric, and the 2D Torus Mesh Fabric. The fabrics are connected to 45 cartridges slots, two slots for the network switches, and two corresponding I/O modules.
Figure 9 shows the eight 10Gb lanes routed from each of the cartridge slots to the pair of core network fabric slots in the center of the Moonshot 1500 chassis. Four lanes from each cartridge go to one core network fabric slot and four to the other (A and B). From each core fabric slot there are 16 10Gb lanes routed to the back of the chassis to attach to an I/O module.
Figure 9.
Radial Fabric
The Radial Fabric provides a high-speed interface between each cartridge and the two core fabric slots.
The Radial fabric includes these links:
• 2x GbE channels
• One port to each network switch
Figure 10 illustrates a torus topology interlinking cartridge to cartridge in combination with the radial topology linking to the network switches.
Figure 10.
The Radial fabric handles all Ethernet-based traffic between the cartridge and external targets. The exception is iLO* management network traffic using the dedicated iLO port.
*[iLO: Integrated Lights-Out]
Storage fabric
A Moonshot System Storage Fabric will use existing Moonshot 1500 Chassis connections to span each 3×3 cartridge slot subsection within the chassis baseboard (Figure 11). The Storage Fabric will be part of future HP Moonshot System releases. This fabric implementation will use the Storage Fabric as a connection between servers and local storage devices.
Figure 11.
In this implementation, SAS/SATA is sent over lanes between each adjacent cartridge for primary storage along with additional lanes to other cartridges in the subsection for redundancy or other storage requirements. Although the figure shows a specific configuration of compute and storage nodes, there is flexibility to configure the subsections in different ways as long it does not violate the rules of the interface or storage technology. While the example in Figure 11 shows the proximal fabric being used for SAS/SATA, any type of communication is possible due to the dynamic nature of the fabric.
2D Torus Mesh Fabric
Like the Storage Fabric, future releases of the HP Moonshot System will use existing Moonshot 1500 Chassis connections to implement the 2D Torus Mesh Fabric, providing a high speed general purpose interface among the cartridges for those applications that benefit from high bandwidth node-to-node communication. The 2D Torus Mesh fabric can be used as Ethernet, PCIe, or any other interface protocol. At chassis power on, the CM [Chassis Management] ensures the compatibility on all interfaces before allowing the cartridges to power on.
The 2D Torus Mesh fabric is routed as torus ring configuration capable of providing four 10Gb bandwidths in each direction to its north, south, east and west neighbors. This allows the HP Moonshot System to meet many unique HPC [High-Performance Computing] applications where efficient localized traffic is needed.
16 lanes from each cartridge
Four up, four down, four left, and four right
Can support speeds up to 10Gb
Topologies
Topologies utilize the physical fabric infrastructure to achieve a desired configuration. In this case, Radial and 2D Torus Mesh fabrics are the desired Moonshot topologies. The Radial Fabric pathways are optimized for a network topology utilizing two Ethernet switches. The 2DTorus Mesh fabric pathways are passive copper connections negotiated with neighbors and optimized for topology protocols that change over time to accommodate future Moonshot System releases.
Moonshot System network configurations
Moonshot System network switches and uplink modules provide resiliency and efficiency when configured as stand-alone or stackable networks. This feature allows you to connect up to nine Moonshot 1500 Chassis and then to your core network, eliminating the need for a top of rack (TOR) switch.
Dual switches provide traffic isolation or can be stacked
Rack level stacking simplifies management domain
Redundant switch configurations provide a more resilient infrastructure
Layer 2, Layer 3 Routing & QoS, Management (CLI, SNMP, SFLOW). No license keys
Moonshot 1500 Chassis stacking
Stacking allows you to select a tradeoff between overall performance and cost of TOR switches. Stacking can eliminate the cost of TOR switches for wor