This week Microsoft launched the release of Windows Server 2016 along with its ultra light headless deployment option - Nano Server. The Nano server images are many times smaller than what we have come to expect from a Windows server image. A Nano Vagrant box is just a few hundred megabytes. These machines also boot up VERY quickly and require fewer updates and reboots.
Earlier this year, I blogged about how to run a Chef client on Windows Nano Server. Things have come a long way since then and this post serves as an update. Now that the RTM Nano bits are out, we will look at:
How to get and run a Nano server
How to install the chef client on Windows Nano
How to use Test-Kitchen and Inspec to test your Windows Nano Server cookbooks.
The sample cookbook I'll be demonstrating here will highlight some of the new Windows container features in Nano server. It will install docker and allow you to use your Nano server as a container host where you can run, manipulate and inspect Windows containers from any Windows client.
How to get Windows Nano Server
You have a few options here. One thing to understand about Windows Nano is that there is no separate Windows Nano ISO. Deploying a Nano server involves extracting a WIM and some powershell scripts from a Windows 2016 Server ISO. You can then use those scripts to generate a .VHD file from the WIM or you can use the WIM to deploy Nano to a bare metal server. There are some shortcuts available if you don't want to mess with the scripts and prefer a more instantly gratifying experience. Lets explore these scenarios.
Using New-NanoServerImage to create your Nano image
If you mount the server 2016 ISO (free evaluation versions available here), you will find a "NanoServer\NanoServerImageGenerator" folder containing a NanoServerImageGenerator powershell module. This module's core function is New-NanoServerImage. Here is an example of using to to produce a Nano Server VHD:
This will generate a Nano Hyper-V capable image file of a Container/DSC/IIS ready Nano server. You can read more about the details and other options of this function in this Technet article.
Direct EXE/VHD download
As I briefly noted above, you can download evaluation copies of Windows Server 2016. Instead of downloading a full multi gigabyte Windows ISO, you could choose the exe/vhd download option. This will download an exe file that will extract a pre-made vhd. You can then create a new Hyper-V VM from the vhd. With that vm, just login to the Nano console to set the administrative password and you are good to go.
This is my installation method of choice. I use a packer template to automate the download of the 2016 server ISO, the generation of the image file and finally package the image both for Hyper-V and VirtualBox Vagrant providers. I keep the image publicly available on Atlas via mwrock/WindowsNano. The advantage of these images is that they are fully patched (key for docker to work with Windows containers), work with VirtualBox and enable file sharing ports so you can map a drive to Nano.
Vagrant Nano bug
One challenge working with Nano Server and cross platform automation tools such as vagrant is that Nano exposes a Powershell.exe with no -EncryptedCommand argument which many cross platform WinRM libraries leverage to invoke remote Powershell on a Windows box.
Shawn Neal and I rewrote the WinRM ruby gem to use PSRP (powershell remoting protocol) to talk powershell and allow it to interact with Nano server. This has been integrated with all the Chef based tools and I will be porting it to Vagrant soon. In the meantime, a "vagrant up" will hang after creating the VM. Know that the VM is in fact fully functional and connectable. I'll mention a hack you can apply to get Test-Kitchen's vagrant driver working later in this post.
Connecting to Windows Nano Server
Once you have a Nano server VM up and running. You will probably want to actually use it. Note: There is no RDP available here. You can connect to Nano and run commands either using native Powershell Remoting from a Windows box (powershell on Linux does not yet support remoting) or use knife-windows' "knife winrm" from Windows, Mac or Linux.
Note that knife winrm expects "cmd.exe" style commands by default. Use "--winrm-shell powershell" to send powershell commands.
Installing Chef on Windows Nano Server
Quick tip: Do not try to install a chef client MSI. That will not work.
Windows Nano server jettisons many of the APIs and subsystems we have grown accustomed to in order to achieve a much more compact and cloud friendly footprint. This includes the removal of the MSI subsystem. Nano server does support the newer appx packaging system currently best known as the format for packaging Windows Store Apps. With Nano Server, new extensions have been added to the appx model to support what is now known as "Windows Server Applications" (aka WSAs).
At Chef, we have added the creation of appx packages into our build pipelines but these are not yet exposed by our Artifactory and Bintray fed Omnitruck delivery mechanism. That will happen but in the mean time, I have uploaded one to a public AWS S3 bucket. You can grab the current client (as of this post) here. To install this .appx file (note: if using Test-Kitchen, this is all done automatically for you):
Either copy the .appx file via a mapped drive or just download it from the Nano server using this powershell function.
Run "Add-AppxPackage -Path <path to .appx file>"
Copy the appx install to c:\opscode\chef:
The last item is a bit unfortunate but temporary. Microsoft has confirmed this to be an issue with running simple zipped appx applications. The ACLs on the appx install root are seriously restricted and you cannot invoke the chef client from that location. Until this is fixed, you need to copy the files from the appx location to somewhere else. We'll just copy to the well known Chef default location on Windows c:\opscode\chef.
With the chef client installed, its easiest to work with chef when its on your path. To add it run:
Now you can run the chef client just as you would anywhere else. Here I'll check the version using knife:
Not all resources may work
I have to include this disclaimer. Nano is a very different animal than our familiar 2012 R2. I am confident that the newly launched Windows Server 2016 should work just as 2012 R2 does today, but nano has APIs that have been stripped away that we have previously leveraged heavily in Chef and Inspec. One example is Get-WmiObject. This cmdlet is not available on Nano Server so any usage that depends on it will fail.
Most of the crucial areas surrounding installing and invoking chef are patched and tested. However, there may be resources that either have not yet been patched or will simply never work. The windows_package resource is a good example. Its used to install MSIs and EXE installers not supported on Nano.
Test-Kitchen and Inspec on Nano
The WinRM rewrite to leverage PSRP allows our remote execution ecosystem tools to access Windows Nano Server. We have also overhauled our mixlib-install gem to use .Net core APIs (the .Net runtime supported on Nano) for the chef provisioners. With those changes in place, Test-Kitchen can install and run Chef, and Inspec can test resources on your Nano instances.
There are a few things to consider when using Test-Kitchen on Windows Nano:
Specifying the Chef appx installer
As I mentioned above, the "OmniTruck" system is not yet serving appx packages to Nano. However, you can tell Test-Kitchen in your .kitchen.yml to use a specific .msi or .appx installer. Here is some example yaml for running Test-Kitchen with Nano:
Inspec requires no configuration changes.
Working around Vagrant hangs
Until I refactor Vagrant's winrm communicator, it cannot talk powershell with Windows Nano. Because Test-Kitchen and Inspec talks to Nano directly via the newly PSRP supporting WinRM ruby gem, they make Vagrant's limitation nearly unnoticeable. However the RTM Nano bits exacerbated the Vagrant bug causing it to hang when it does its initial winrm auth check. This can unfortunately hang your kitchen create. You can work around this by applying a simple "hack" to your vagrant install:
Update C:\HashiCorp\Vagrant\embedded\gems\gems\vagrant-1.8.5\plugins\communicators\winrm\communicator.rb (adjusting the vagrant gem version number as necessary) and change:
This should get your test-kitchen runs unblocked.
Running on Azure hosted Nano images
If you prefer to run Test-Kitchen and Inspec against an Azure hosted VM instead of vagrant, use Stuart Preston's excellent kitchen-azurerm driver:
See the kitchen-azurerm readme for details regarding azure authentication configuration. As of the date of this post, RTM images are not yet available but thats probably going to change very soon. In the meantime, use TP5.
Using Chef to Configure a Docker host
One of the exciting new features of Windows Server 2016 and Nano Server is their ability to host Windows containers. They can do this using the same Docker API we are familiar with with linux containers. You could walk through the official instructions for setting this up or you could just have Chef do this for you.
Updating the Nano server
Note that in order for this to work on RTM Nano images, you must install the latest Windows updates. My vagrant boxes come fully patched and ready but if you are wondering how do you install updates on a Nano server, here is how:
Then just reboot and you are good.
A sample cookbook to install and configure the Docker service
I converted the above mentioned instructions for installing Doker and configuring the service into a Chef cookbook recipie. Its fairly straightforward:
This downloads the appropriate docker binaries, installs the docker service and configures it to listen on port 2375.
To validate that all actually worked we have these Inspec tests:
If this all passes, we know our server is listening on the expected port and that docker commands work.
Converge and Verify
So lets run these with kitchen verify:
Ok our docker host is ready.
Creating and running a Windows container
First if you are running Nano on VirtualBox, you need to add a port forwarding rule for port 2375. Also note that you will need the docker client installed on the machine where you intend to run docker commands. I'm running them from my Windows 10 laptop. To install docker on Windows 10:
No matter what platform you are running on, once you have the docker client, you need to tell it to use your Nano server as the docker host. Simply set the DOCKER_HOST environment variable to "tcp://<ipaddress of server>:2375".
So now lets download a nanoserver container image from the docker hub repository:
Now lets run a command...heck lets just launch an interactive powershell session inside the container with:
Here is what we get:
So we have made alot of progress over the last few months but the story is not entirely complete. We still need to finish knife bootstrap windows winrm and plug in our azure extension.
Please let us know what works and what does not work. I personally want to see Nano server succeed and of course we intend for Chef to provide a positive Windows Nano Server configuration story.