In WSUS (Windows Server Update Services) and Windows Update, Microsoft has one of the most reliable patch management engines in existence. According to Microsoft, Windows Update currently updates more than 800 million PCs worldwide. But over the past two years Microsoft patches themselves have been causing more and more problems, such as those users had installing Windows 7 Service Pack 1 last April and the recent Skype (now owned my Microsoft) patch - a package that was supposed to be update-only, but actually initiated full deployments to all targeted systems.
In other cases, the patch packages fail to find machines that need updates, or attempt to install patches where they’re not needed. Over the last year, I’ve heard two to three cases where Microsoft patches reported a failed installation when the patch has actually installed. Sometimes, a report of a failed update is actually triggered by an older update, not the most recent one.
This leads to a lot of confusion and wasted effort for the patch administrator. Usually, something happens unexpectedly and you don’t know why – like a report of an unsuccessful patch but you can’t identify where it failed. The other side of that, which is even more dangerous, is when a patch needs to be applied, but it can’t identify all the systems that need it and thus never gets deployed. What you don’t know can hurt you.
In only a very few cases is the problem a question of interaction between the patch and WSUS. Most often, it’s flagrant errors in the way the patch logic itself has been written. After close to 14 years of solid performance one can ask why has the quality of Microsoft patches fallen so quickly?
Can Product Groups Do Patches, Too?
I don’t have a window into Microsoft, but my sense is there has been reorganization in the WSUS and the Windows Update hierarchy recently, and perhaps resources once devoted to WSUS, a mature offering, have been diverted to other projects such as System Center Configuration Manager. It’s almost as if WSUS has gone into maintenance mode, with Microsoft treating it as a product that doesn’t need continued enhancement.
Another factor, I suspect, is that the updates for each product are developed by the individual product groups, each of whom is supposed to use Microsoft’s standard methodology for developing update packs. But expecting each product group to have the same skill sets, level of experience and, frankly, level of commitment to patching is asking for trouble. For a process as complex as patching, and a product line as complex as Microsoft’s, a more centralized management structure is the only way to deliver the quality customers expect.
What can Microsoft Do Better?
What steps would I like to see Microsoft take? First, publish (to both Microsoft product groups and externally) more details about how the WSUS infrastructure works, and the best practices for working with it. This should include information about how to use Microsoft’s tools for building packages and injecting them into the WSUS database. Internally, this documentation will help keep the product groups on track with best practices for creating patches. For those of us on the outside, this detailed documentation will help us understand what has gone wrong when a Microsoft patch blows up.
Second, the folks in Redmond need to do whatever it takes – whether that means organizational or personnel moves, or both – to follow those best practices themselves. Millions of customers worldwide rely on Microsoft to lead the way in patch management, at least with their own systems. If we can’t trust Microsoft to patch their own products correctly, what else can we trust them with?
And finally, it would be helpful if Microsoft would expose in the WSUS console the actual logic rules defined in a Microsoft patch package. The SolarWinds Patch Manager console exposes the logic rules that we use for third party packages – and that readily accessible information has helped immensely with customer troubleshooting of unexpected behaviors.
Being able to see, for example, that IE9 on Vista Service Pack 2 had an unexpected dependency on a certain video driver would have helped hundreds of patch administrators last year when nobody could figure out why IE9 wouldn’t install on Vista SP2 systems.