Upping the stakes in the emerging and highly competitive market for container orchestration tools, Mesosphere on Tuesday took the much-anticipated, though vitally necessary, step of releasing the Apache Mesos-based Data Center Operating System (DC/OS) into open source, under an Apache v2 license.
“Mesosphere views DC/OS as a critical open source project and is committed to having a lively and engaged community,” stated Matt Trifiro, Mesosphere’s chief marketing officer, in a note to The New Stack. “We have a long history of running successful open source projects (e.g., Apache Mesos), and we will use that expertise to enrich this community as well.”
Being open source, DC/OS is now on par with one of the chief rivals for placement in the data center, Google’s Kubernetes, also being developed as open source.
Perhaps you’ve noticed, Mesosphere has also endowed the abbreviation of its platform with a slash mark – or, as a former editor of mine would have corrected, a virgule. Henceforth, the Data Center Operating System will be known as DC/OS, rather than DCOS. It’s an artifact that harks back to the heady days of PL/I, CP/M, and OS/2. It’s also a way to get people to quit saying “dee · kos.”
The Logical Complement
In a blog post Tuesday, Mesosphere founder Ben Hindman reminded customers that it was his intent, in founding the company three short years ago, to build a core data center OS around Apache Mesos. But in pursuing that vision, he realized that Mesos might end up being just one part.
“DC/OS is the inevitable next step for a project like Mesos,” Hindman wrote. “Mesos is necessary but insufficient, and DC/OS is the best way to take advantage of Mesos while providing all of the other capabilities that organizations need to run their software. It is not DC/OS versus Mesos; DC/OS is the logical complement for Mesos… While Mesos will continue to add and improve upon its OS-like primitives (for example, upcoming support for external storage volumes and a universal containerizer), DC/OS will continue to focus on complementary features such as service discovery, load balancing, privacy and ease of installation.”
Trifiro told us more than 60 partners will contribute to the project in some substantive fashion, noting practitioners such as Autodesk and Yelp; systems consultants such as Accenture (featured in a recent New Stack @Scale podcast) and Container Solutions; independent software vendors such as MemSQL and Crate; network integrators such as Canonical and Cisco. Also throwing in its support is Microsoft; Microsoft Azure, whose newly-launched Container Service offers DC/OS as one of two options alongside Docker Swarm for at-scale deployment.
A Microsoft spokesperson confirmed to The New Stack Tuesday that it intends to continue contributing to both DC/OS and Apache Mesos, with an eye toward completing its port of Mesos to Windows Server. “This will ensure that customers can run mixed Linux and Windows container workloads,” the spokesperson said.
“Our view is that container orchestration is just table stakes, and only solves a small part of the challenge faced by modern enterprise applications.” — Matt Trifiro
Virtualization provider Citrix added its name to that list on Tuesday, saying it plans to integrate DC/OS into its NetScaler CPX — a version of its application delivery controller deployed via containers, currently available as a tech preview.
“NetScaler CPX obviously does not run in isolation, but as an integral part of the fabric of running and scaling a container-based deployment,” wrote Nand Mulchandani, Citrix’ vice president for market development, in a Tuesday blog post. “One of our largest customers just helped pull together an integrated stack featuring Mesosphere Marathon, Nuage Networks, Infoblox, and NetScaler CPX. This was possible through the integration that Citrix pulled together with Mesos Marathon.”
And Cisco chipped in, with an announcement that it has been working with Mesosphere — and will continue to do so — in conjunction with its rapid distributed service platform, called Mantl.
“Mantl is a batteries-included, end-to-end solution for your microservices infrastructure,” wrote Cisco CTO for cloud infrastructure services Ken Owens Tuesday. “Mantl’s ethos is to focus a community around an end-to-end solution, not around an individual component; designed to solve business and team challenges, one glue, improved by the community, tested everywhere. We have worked very closely with Mesosphere because Apache Mesos is a critical component of the modern application development model with the large ecosystem of frameworks that have been developed in the community.”
Too Early to Declare Victory
As the container workload deployment and orchestration space matures, different options for enterprises are taking shape.
On the public cloud side, there is the AWS EC2 Container Service, Joyent Triton, and the recently launched Microsoft Azure service. For in-house Docker environments, Docker Swarm and Docker Datacenter pools together resources into clusters for the deployment of workloads natively created within the Docker environment, as Docker images.
But Docker’s partners, including Microsoft, are demonstrating that the integration of container workloads with existing workloads may require an orchestration and deployment system that is not centered around containers. Mesos was engineered for co-existence. And Kubernetes is successfully being deployed in environments where co-existence is necessary, as CoreOS’ partnership with Intel demonstrates.
Responding to some recent comments, including from CoreOS CEO Alex Polvi — who declared that the orchestration market needs a “winner,” Mesosphere’s Trifiro told The New Stack, “Frankly, the Kubernetes hype surprises me. It’s way too early to proclaim winners in this space.
“Let me be frank: Kubernetes is a container orchestrator, akin to our DC/OS Marathon,” Trifiro continued. “DC/OS is a full data center operating system, which is inclusive of container orchestration — but container orchestration is just a tiny piece of the puzzle.
“Our view is that container orchestration is just table stakes, and only solves a small part of the challenge faced by modern enterprise applications. We also believe we have the most proven and production-ready container orchestrator, Marathon,” Trifiro said.
Trifiro went on to express his view that modern applications are far greater than mere containerized microservices, requiring complex resources like Cassandra distributed databases. Only DC/OS, he asserted, can run all the necessary components on the same cluster in a multi-tenant setup. And by Mesos’ two-level scheduling, DC/OS can run Kubernetes and other orchestrators simultaneously… if that were necessary.
And some say it is, arguing that Kubernetes is the choice of developers when staging works in progress while Marathon may be preferable for production environments.
Intel is also one of Mesosphere’s many partners, which means it has an iron in both fires. Indeed, Intel Capital was an early investor in Mesosphere, just as it was with commercial OpenStack provider Mirantis. And when the Linux Foundation announced its FD.io effort last February, to work toward an open I/O services framework, Intel and Mesosphere were both equally spotlighted as partners in that effort.
In a Mesosphere blog post published just after Intel launched its Cloud for All initiative, Intel principal engineer, and SDI architect Das Kamhout told the company, “We partner with Mesosphere and we also work directly on Apache Mesos. We strongly believe in a few types of solutions that are necessary to hit the broad market for people running data centers at scale. Where we like Mesosphere is its dual-level scheduling, which allows for a lot of things to run on top of a massive resource pool.”
In a discussion for a New Stack Makers podcast, Kamhout touted recent observations of Kubernetes’ success with Intel’s customers as the reason for choosing CoreOS for Universal Resource Scheduler, announced at Intel Cloud Day.
“Kubernetes is getting a ton of traction as an app orchestration layer for containers,” said Kamhout. “We go out and talk to people, and it’s pretty common to see, at least at the app layer, that it’s getting a lot of traction. Anybody building apps now — maybe they’re using Docker for their container runtime. But optimally, when you look at how they’re orchestrating, it’s pretty common to find the Kubernetes API in place, and they’re learning the ‘pod’ concepts and basically building on top of Kubernetes.”
When Kamhout says, “at least at the app layer,” he’s being sure to distinguish resource orchestration — which is where Kubernetes and Mesosphere both excel — from service orchestration, such as the types of services delivered by telcos and other upstream providers over next-generation, IP-based networks.
Where does Docker Swarm fit in this picture? (You remember Docker, the company that first popularized the container format so many months ago?) Swarm remains a reality in the developer space. But last March 30 at Microsoft’s Build 2016 conference, during a demo of its Azure Container Service (which was released to GA Tuesday), Ross Gardler, a senior program manager for Container Service, clearly framed its choice of orchestration platforms — Swarm and Mesosphere — as playing to developers and to production managers separately.
“What we’re working with Docker on is making sure that Azure Container Service, with the Docker Swarm flavor, is a great place to be able to run their tools,” said Gardler, listing Docker Inc.’s Trusted Registry and Universal Control Plane components, “which is useful when you’re working with the Docker ecosystem at scale.
“Mesosphere’s products with DC/OS,” he continued, “have some additional advantages over the Docker-native environment for certain kinds of workloads.” He had explained earlier how Mesos was developed at a time and place before containers came to fruition. “So you can run hybrid container workloads. If you want to run Spark or HDFS (Hadoop) or Cassandra, or any of these other technologies, [they’ve] been proven in those environments as well.”
While Docker Inc. continues to make strides toward grafting these other classes of workloads into Docker-native technologies, Mesosphere clearly presents itself — and is being presented — as a realm where all that grafting doesn’t need to happen. Containers, VMs, “big data,” and NoSQL workloads can simply co-exist.
Table Stakes
Anyone leaving the Open Networking Summit in San Francisco in mid-March would have come away believing Mesosphere, not Kubernetes, was making the deepest inroads in the resource orchestration space, at least with communications engineers. While these engineers noted that workload scaling principles do not translate to service scaling, at least for now, many of the floor demos we saw paid tribute to Mesosphere as one very sensible approach to scaling and distributing workloads.
And if those demos didn’t make the point technically enough, Andreessen Horowitz General Partner Peter Levine made the case regarding business sense. While praising Red Hat as having achieved the business scale necessary to make open source software succeed in enterprise, Levine said, “I believe that the future of open source is also going to require business innovation, as opposed to just thinking that, we’re going to set up another open source company that’s going to give away the software for free and set up a support and service organization.” That comment came in the context of a discussion with ONS founder Guru Parulkar, where Mesosphere was mentioned by name as one of Andreessen Horowitz’ key investments
Next week’s OpenStack Summit will put the allegiances of telcos like AT&T, builders like Intel, and networkers like Cisco, Juniper, and Brocade to the test. When orchestrating workloads at huge scale, there are clear benefits to both Kubernetes and Mesos, and their commercial options. While Mesosphere will be part of the “table stakes,” to borrow Matt Trifiro’s phrase, for several panel discussions next week about how to orchestrate containers at huge scale, it’s interesting how he casts OpenStack as perhaps not the biggest player at the table.
“We see a lot of OpenStack deployments, particularly in smaller cloud providers (those outside of the Big Four) and in a lot of enterprises, but OpenStack is primarily about provisioning VMs,” wrote Trifiro. “DC/OS is a great addition to an OpenStack deployment because you can use OpenStack to provision the underlying machines, and then layer the DC/OS on top of it create a container-driven fabric for deploying modern applications.”
Cisco, CoreOS, Docker, Intel, and Mesosphere are sponsors of The New Stack.
Feature image via Pixabay.
The post The Inevitable: Mesosphere Open Sources the Mesos-based Data Center Operating System appeared first on The New Stack.