Magento 2 has lifted the minimum supported versions of PHP, MySQL and other supporting software to higher levels – the level anticipated to be the norm when Magento 2 is released at GA. However a lot of people have reported current issues with running Magento 2 in environments such as MAMP (and for myself XAMP on Windows) which come with MySQL 5.5 by default (Magento 2 now requires MySQL 5.6). It is likely to be an ongoing challenge for developers who need to switch between Magento 1 and 2 projects. This post explores some virtualization technologies (in particular Vagrant and Docker) showing how they can be used to solve the ‘version hell’ problem (not DLLs this time, but rather all the software packages required to build real sites).
A Brief Overview of Virtualization
I am not going to attempt to describe virtualization in depth in this post. There are lots of other sources that do that. But I did want to briefly introduce two popular virtualization related technologies and the potential they offer.
The first more traditional form of virtualization is where you run software such as VMware or VirtualBox on a host Operating System (OS). Inside the virtualization software you can run a Virtual Machine (VM) with a different OS to the host OS. There are several use cases where this is useful:
A hosting provider can buy racks of standard hardware and then carve out subsets of resources to lease to different customers. For example an 8 CPU machine might be carved into 3 virtual machines allocated as 4 CPUs to customer 1, 2 CPUs to customer 2, and 2 CPUs to customer 3. Each VM is isolated so customers in one VM have no access into other VMs for security.
A VM allows the application being deployed to control the exact version of the OS and software packages it has installed. One VM can install PHP 5.3, another PHP 5.4, and another HHVM. So a VM can built for a specific full stack of software packages (and their versions) without concern for the needs of other applications.
The other form of virtualization that is has gained particular attention this year (2014) is Linux containers, as used by Docker. Containers are built into the Linux Kernel. Instead of having a full new OS running within virtualization software, containers use Linux OS features like chroot to guarantee a set of processes in one “container” do not have access to the contents (process, files on disk, network ports, memory, etc) of other containers, unless explicitly granted. This approach is more efficient as it is not an OS running within another OS – instead it is a single OS that uses security mechanisms to isolate processes.
With Linux containers, you share the Linux kernel between containers, but within each container there is a copy of the software that the container needs. This is how different containers can have different versions of software installed even though they share the same underlying kernel.
So how do Vagrant and Docker potentially fit into a Magento 2 project’s development lifecycle? Should you use one, the other, or both on your projects? The following is my current thoughts on this, influenced from many sources including recent community feedback.
Development using Vagrant
Vagrant (see http://vagrantup.com/) allows you to “Create and configure lightweight, reproducible, and portable development environments”. There are different ways in which Vagrant can be utilized. The following example is based on a Windows laptop as the host.
The Magento 2 code base is checked out onto the Windows file system and so can be edited using Windows text editors (such as PHP Storm).
The web browser is run directly on Windows.
A Vagrant ‘box’ is created that mounts the directory the code resides in. Within the box (VM) runs PHP 5.5, Apache, and MySQL 5.6. The Apache web server port is made accessible from the host OS.
In this case the name ‘box’ is an apt description. You really can consider it to be a black box that exposes a web server port. The contents of the box does not matter (much) to a developer, as long as the site works. They can edit the application on the Windows laptop (editing, viewing the results on the web site). (Yes, I am skipping over the important topic of debugging here for the sake of simplicity.)
Another key advantage of Vagrant is it is straightforward to share an environment definition with other users (in a GIT repository or in a public Vagrant specific repository). This can make support easier as there will be fewer environmental differences between users. For example, I have defined a Vagrant “box” suitable for Magento 2 development at https://github.com/alankent/vagrant-magento2-apache-base. This is relatively new and still under development (subject to change with little notice). The README file lists the commands to use the box. Hopefully sharing this box definition will reduce the friction for new developers exploring Magento 2.
For example, to switch to work on a Magento 1 project, a different box can be defined with the correct versions of the software that Magento 1 requires. Software installed within different boxes then has no risk of conflict.
Magento Development versus Project Development
Before moving on to talk about Docker, it is worth clarifying there are two different ways that developers are likely to interact with the Magento 2 code base now with the public GitHub repository now accepting pull requests.
Developers wishing to contribute to the Magento 2 code base (e.g. to submit a pull request with a bug fix) will clone the Magento 2 repository.
Developers building a customer production site should not clone the Magento 2 repository – they should instead use the officially released Composer packages (with version numbers), downloaded via Composer. They are also likely to make a number of local customizations that they would manage for that specific site.
The Vagrant example I have published in GitHub above is based on the first scenario (which will be the most common during the developer beta period). Only the second scenario would likely be pushed to production. While the way in which the site is assembled is different between the two cases, the use of Vagrant and Docker to support them is not significantly affected.
Production with Docker
I have written a number of posts recently about Docker. The use of Vagrant during the development phase above does not diminish the potential of Docker in production. The reality is development and production configurations of Magento will frequently be different. For example, developing on a laptop will normally be based on a single web server and MySQL instance. In production, there may be multiple instances of both, plus Varnish (or similar) caching.
That is, there is nothing stopping use of Vagrant during development and Docker in production. When designing a scalable production topology, the modular nature of Docker is appealing. Containers can be defined for MySQL, Apache (built to include the Magento 2 code), Varnish, Redis, and so forth. These containers can then be assembled in different topologies as required per site. Ideally, only the Apache (web server) container is the one that needs to change per deployment. The other container defintions can be reused and shared across projects.
More important to me than picking exclusively between Vagrant and Docker is making the underlying platform flexible enough to be used in a range of deployment scenarios with minimal overheads. If the Vagrant and Docker definitions are relatively short and easy to understand, then the overhead of duplicating the definitions is pretty low. This is one of the reasons the installation process has a command line based version, not only web based as in Magento 1.
Problems
It is worth noting that at the time of writing this post there are still problems to be resolved with the Vagrant box above. The Vagrant box works, but on my laptop is extremely slow. I believe this is most likely due to misconfiguration on my behalf which I will continue to investigate. I did however want to get this post out quickly as possible assistance to anyone having installation troubles due to software version incompatibilities, so I have decided to release this post and come back and adjust when the performance issue is resolved.
Conclusions
This post describes a way in which Vagrant can be used to reduce configuration pain during development with Docker being used in production. Some other projects already do the same, and I can well believe this may become the norm. However there are no plans to make Vagrant or Docker a requirement for using Magento 2 – I believe Vagrant will be one option available during development and Docker will be one option available in production. This blog post does not in any way invalidate the current ways in which Magento 1 is deployed today, particularly for the more advanced sites.
But I do believe there is benefit, particularly in the lower end of the market if a Merchant does not go with a solution partner, in providing more guidance on development and production practices to reduce the number of problems they encounter. But I would always recommend considering the option of developing a relationship with a suitably skilled Magento partner. As scenarios get more complicated, proforma solutions are less likely to be satisfactory.
So what is next? I would like to flesh out the second use case in greater detail – the scenario of using released composer packages, not a clone of the Magento 2 GitHub repository. How much can Docker simplify the roll out of code to production? What is one set of development practices for seamless development and then roll out to production? How might this work with Magento Connect? If this can be made Docker based, this may simplify the deployment strategies (for low end sites) since Docker is gaining adoption across a range of hosting providers. The ultimate goal is to reduce friction for Merchants who decide not to go with an appropriately skilled partner.
Acknowledgements: This post contains thoughts from many sources including several community suggestions.
