“The Update” For Windows Server 2012 R2, Windows 8.1, and Windows RT 8.1 Is GA … But Not For Everyone

The much talked about “Update 1” which is actually called “Update” (and can be thought of as Service Pack 2, after the big GA update) is out via Windows Update. And that’s it. You won’t get this update via WSUS, and downstream products.

And that’s because there’s a problem.

I applaud Microsoft for taking the following action to avoid breaking security patching:

Microsoft plans to issue an update as soon as possible that will correct the issue and restore the proper behaviour for Windows 8.1 Update scanning against all supported WSUS configurations. Until that time, we are temporarily suspending the distribution of the Windows 8.1 Update to WSUS servers.

So if you’re desperate for The Update then you can manually download it via Windows Update but forget about deploying it in business for the time being until Microsoft resolves the WSUS SSL issue.

BTW, if you are using Veeam to backup WS2012 R2 hyper-V then MVP Hans Vredevoort has important news of a hotfix from Veeam that you should read before approving The Update.

KB2919394 – Update Rollup For WS2012 R2

Microsoft has released a February update rollup for Windows Server 2012 R2, as well as Windows 8.1 and Windows 8.1 RT. This is a much bigger UR than the one that was just released for Windows Server 2012.

I’ve noticed two updates that Hyper-V and/or clustering folks should be aware of:

KB2920469 deals with a situation where you cannot change the schedule for CAU self-updating mode in Windows Server 2012 or Windows Server 2012 R2 by using CAU GUI.

Assume that you have a cluster that runs Windows Server 2012 or Windows Server 2012 R2. When you try to change the Cluster-Aware Updating (CAU) schedule by using the GUI, the time for the automatic update schedule is not changed, and the old time remains.

KB2920196 is for when you cannot create or change the name of a debugger named pipe for a VM when it is running in Windows 8.1 or Windows Server 2012 R2.

When you have a virtual machine that is running in Windows 8.1 or Windows Server 2012 R2, you cannot create or change a debugger named pipe’s name for the virtual machine while it is running.

This issue occurs because of a regression that is introduced in Windows 8.1.

As I said in the WS2012 post, don’t be an IT haemophiliac; delay approval of this update for a month and let the rest of the world be Microsoft’s test lab. If you don’t see any update to this update in one month, then approve and deploy it if you’re happy.

KB2919393 – Update Rollup For WS2012

Microsoft has released a February update rollup for Windows Server 2012, as well as Windows 8 and Windows RT (8). There’s one included update that Hyper-V or clustering folks should be aware of:

KB2920469 deals with a situation where you cannot change the schedule for CAU self-updating mode in Windows Server 2012 or Windows Server 2012 R2 by using CAU GUI.

Assume that you have a cluster that runs Windows Server 2012 or Windows Server 2012 R2. When you try to change the Cluster-Aware Updating (CAU) schedule by using the GUI, the time for the automatic update schedule is not changed, and the old time remains.

As usual with update rollups, don’t be an IT haemophiliac; delay approval of this update for a month and let the rest of the world be Microsoft’s test lab. If you don’t see any update to this update in one month, then approve and deploy it if you’re happy.

Complaints About Bad Microsoft Patches Goes Mainstream

It was inevitable.  MVPs have privately voiced concerns to Microsoft about the quality of patches coming out of Sustained Engineering.  That feedback went somewhere up the chain and out the back door.

Then after many months and the July 2013 disasters, some of us decided to talk about it publicly on social media.  Some Microsoft people in Redmond agree with our concerns, expressing embarrassment that their hard work is being diminished by a laughable resource planning policy decision.  Once again, no notable changes to the CA-style “testing”.

Now I’ve just read on Silicon Republic that MVP Susan Bradley (AKA the SBS Diva) has written to Microsoft CEO Steve Ballmer to address her concerns.

“On behalf of everyone in this community, may I respectfully request that you assign someone in a management position to investigate what is going on with quality control with patch testing lately?” Bradley asked Ballmer.

A certain negative response from a few Microsoft people to Susan’s letter is reported in the article.  I do recognise that experience.

I’m glad this has gone “main stream” and been picked up by the media.  To be honest, I think we have to embarrass whatever executive is responsible for this mess into making a much-needed change.

EDIT:

Mary Jo Foley just pinged me on Twitter to let me know that Larry Seltzer had previously posted a similar story on ZDnet.  And don’t forget that myself and Hans Vredevoort also raised issues in Windows Server and System Center in July.  To be honest, I think there’s a mindset with the power-that-is that will only increase the cost of testing if sales are hit.  The power to make a change is in your hands.

EDIT:

And Rod Trent also posted a story on this problem on Windows IT Pro.

EDIT:

Add Redmond Magazine to the list.

Technorati Tags: ,

KB2734608: Enable WSUS 3.0 SP2 To Support Windows Server 2012 And Windows 8

Microsoft has released an update for WSUS 3.0 SP2 that enables Windows Server Update Services to provide updates for Windows Server 2012 and Windows 8.  It is available as an x86 and x64 download.

According to the Microsoft SUS blog, this update will fix:

This update lets servers that are running Windows Server Update Services (WSUS) 3.0 SP2 provide updates to computers that are running Windows 8 or Windows Server 2012.

This update fixes the following issues:

  • Installation of update 2720211 may fail if Service Pack 2 was previously uninstalled and then reinstalled.
  • After you install update 2720221, health monitoring may fail if the WSUS server is configured to use SSL.

Additionally, this update includes the following fixes:

  • 2530678 System Center Update Publisher does not publish customized updates to a computer if WSUS 3.0 SP2 and the .NET Framework 4 are installed
  • 2530709 "Metadata only" updates cannot be expired or revised in WSUS 3.0 SP2
  • 2720211 An update for Windows Server Update Services 3.0 Service Pack 2 is available

Virtual Machine Servicing Tool 2012 Beta

Microsoft has launched the beta for VMST 2012.  This tool is used in conjunction with System Center 2012 Virtual Machine Manager (VMM) to update offline virtual machine resources from ConfigMgr/WSUS.

VMST 2012 helps you more effectively—and safely—manage the workflow of updating you offline virtualization environment. Using VMST 2012, you can now service:

– Offline virtual machines in a SCVMM library.

– Stopped and saved state virtual machines on a host.

– Virtual machine templates.

– Offline virtual hard disks in a SCVMM library by injecting update packages.

Patching A Windows Server 2012 Failover Cluster, Including Hyper-V

Cluster Aware Updating (CAU) is a new feature that makes running Windows or Automatic Updates on a Hyper-V cluster easier than ever, as well as any other WS2012 cluster.

If you currently have a Windows Server 2008/R2 Hyper-V cluster, then you have a few options for patching it with no VM downtime:

  • Manually Live Migrate VM workloads (Maintenance Mode in VMM 2008 R2makes this easier), patch, and reboot each host in turn, which is a time consuming manual task.
  • Use System Center Opalis/Orchestrator to perform a runbook against each cluster node in turn that drains the cluster node of it’s roles (VMs), patches it and reboots it.
  • Use the patching feature of System Center 2012 Virtual Machine Manager – which is limited to Hyper-V clusters and adds more management to your patching process.

CAU is actually pretty simple:

  1. Have some patching mechanism configured: e.g. enable Automatic Updates on the cluster nodes (e.g. Hyper-V hosts), approve updates in WSUS/ConfigMgr/etc.  Make sure that you exempt your cluster nodes from automatic installation/rebooting in your patching policy; CAU will do this work.
  2. Log into Failover Clustering from a machine that is not a cluster node (Hyper-V host) member.  Run the CAU wizard.
  3. Here, you can either manually kick off a patching job for the cluster nodes or schedule it to run automatically.  The scheduled automatic option requires that you have deployed a CAU role on the cluster in question to orchestrate the patching.

When a patching job runs the following will happen:

  1. Determine the patches to install per node.
  2. Put node 1 in a paused state (maintenance mode).  This drains it of clustered roles – in other words your Hyper-V VMs will Live Migrate to the “best possible” hosts.  Failover Clustering uses amount of RAM to determine the best possible host.  VMM’s advantage is that it uses more information to perform Intelligent Placement.
  3. Node 1 is removed from a paused state, enabling it to host roles (VMs) once again.
  4. CAU will wait then patch and reboot Node 1.
  5. When Node 1 is safely back online, CAU will move onto Node 2 to repeat the operation.

VMs are Live Migrated throughout the cluster as the CAU job runs and each host is put into a paused state (automatically Live Migrating VMs off), patching, rebooting, and un-pausing.  It’s a nice simple operation.

The process is actually quite configurable, enabling you to definite variables for decisions, execute scripts at different points, and define a reboot timeout (for those monster hosts).

Something to think of is how long it will take to drain a host of VMs.  A 1 GbE Live Migration network will take an eternity to LM (or vMotion for that matter) 192 GB RAM of VMs, even with concurrent LMs (as we have in Windows Server 2012).

Sounds nice, eh?  How about you see it in action:

 

 

 

I have edited the video to clip out lots of waiting:

  • These were physical nodes (Hyper-V hosts) and a server’s POST takes forever
  • CAU is pretty careful, and seems to deliberately wait for a while when a server changes state before CAU continues with the task sequence.

 

 

Interesting Analysis on Patching and Attacks

Microsoft produces a document called the Security Intelligence Report on a regular basis.  Some of it hit the news today so I decided to take a peek.  The report doesn’t restrict itself to exploits of Microsoft products and is based on data that they gather from a number of sources.

“In this supplemental analysis, zero-day exploitation accounted for about 0.12 percent of all exploit activity in 1H11, reaching a peak of 0.37 percent in June”.

OK, so that tells us that the vast majority of exploits take advantage of old vulnerabilities that have had patches available previously.

“Of the attacks attributed to exploits in the 1H11 MSRT data, less than half of them targeted vulnerabilities disclosed within the previous year, and none targeted vulnerabilities that were zero-day during the first half of 2011”.

People aren’t patching like they should be. That explains this:

Conficker is still (STILL!!!!) the leading infection on domain joined computers. Seriously!?!?!? It is not in the top 10 of non-domain joined PCs.

And it explains this:

“Exploits that target CVE-2010-2568, a vulnerability in Windows Shell, increased significantly in 2Q11, and were responsible for the entire 2Q11 increase in operating system exploits. The vulnerability was first discovered being used by the family Win32/Stuxnet in mid-2010”.

This report covers up to H2 2011 and MS10-046 is still being exploited because people haven’t deployed the patch.

“Detections of exploits targeting Adobe Flash, although uncommon in comparison to some other types of exploits, increased in 2Q11 to more than 40 times the volume seen in 1Q11 … Two vulnerabilities accounted for the bulk of zero-day exploit activity … Both vulnerabilities affect Adobe Flash Player”.

Adobe Flash is one of those products that is constantly badgering me to get updated at home.  I leave this turned on because Flash is a real target for attackers. 

“The most commonly observed types of exploits in 1H11 were those targeting vulnerabilities in the Oracle (formerly Sun) Java Runtime Environment (JRE), Java Virtual Machine (JVM), and Java SE in the Java Development Kit (JDK). Java exploits were responsible for between one-third and one-half of all exploits observed in each of the four most recent quarters”.

Other products like Java and Adobe Reader are nice targets too because they have vulnerabilities and are rarely patched.  At work, we patch the Adobe products via System Center Essentials.  You can also use ConfigMgr 2007 to do this.

“As in previous periods, infection rates for more recently released operating systems and service packs are consistently lower than earlier ones, for both client and server platforms. Windows 7 and Windows Server 2008 R2, the most recently released Windows client and server versions, respectively, have the lowest infection rates”.

A) Newer products always do more under the hood to protect themselves.  B) Newer home PCs will have current AV.  C) Newer business deployments will have had a fresh installation of patching/security systems that some more mature environments have lacked, e.g. lack of WSUS, etc.

Interestingly, in the regional analysis, Italy appears to lead the pack at minimizing most malware infections (congrats!) but is second worst when it comes to adware infections (boo!). 

Don’t be so quick to blame Microsoft: 44.8% of exploits are because of the weakness that is found between the keyboard and the chair, where the user is handing over some piece of information or OK-ing something bad. 

Drive by attack download sites (innocent web sites that have been compromised, e.g. adspace that was sold and contains a Flash exploit) are on the rise.

There’s a lot of good info in the Security Intelligence Report.  You should give it a read if considering the security of your business.

VMM 2012 System Requirements

The official TechNet content is a bit scattered about so I through I’d reorganise it and consolidate to make stuff easier to find.  The software requirements of Virtual Machine Manager (VMM/SCVMM) 2012 are easy:

  • Windows Server 2008 R2 Standard, Enterprise or Datacenter with SP1
  • Windows Remote Management (WinRM) 2.0 – a part of W2008 R2
  • .NET 3.5 with SP1 (a feature in W2008 R2)
  • WAIK  for Windows 7

There’s a significant change for the database.  SQL Express is no longer supported.  You will need to migrate the VMM database to one of the supported versions/editions:

  • SQL Server 2008 R2 Enterprise/Standard x86/x64 (no news of support for the recent SP1 yet)
  • SQL Server 2008 Enterprise/Standard x86/x64 with Service Pack 2

Here’s the system requirements for VMM 2012:

Manage Up To 150 Hosts

Let’s be honest; how many of us really have anything close to 150 hosts to manage with VMM?  Hell; how many of us have 15 hosts to manage?  Anyway, here’s the system requirements and basic architecture for this scale of deployment.

image

You can run all of the VMM roles on a single server with the following hardware configuration:

Component Minimum Recommended
CPU Pentium 4, 2 GHz (x64)

Dual-Processor, Dual-Core, 2.8 GHz (x64) or greater

Memory

2 GB

4 GB
Disk space (no local DB)

2 GB

40 GB
Disk Space (local DB) 80 GB 150 GB

Although you can run all the components on a single server, you may want to split them out onto different servers if you need VMM role fault tolerance.  You’re looking at something like this if that’s what you want to do:

image

A dedicated SQL server will require:

Component Minimum Recommended
CPU Pentium 4, 2.8 GHz (x64)

Dual-Processor, Dual-Core, 2 GHz (x64) or greater

Memory

2 GB

4 GB
Disk space (no local DB)

80 GB

150 GB

A dedicated library server will require:

Component Minimum Recommended
CPU Pentium 4, 2.8 GHz (x64)

Dual-Processor, Dual-Core, 3.2 GHz (x64) or greater

Memory

2 GB

2 GB
Disk space (no local DB)

Depends on what you store in it

Depends on what you store in it

A dedicated Self-Service Portal server will require:

Component Minimum Recommended
CPU Pentium 4, 2.8 GHz (x64)

Dual-Processor, Dual-Core, 2.8 GHz (x64) or greater

Memory

2 GB

2 GB
Disk space (no local DB)

512 MB

20 GB

If all you want is hardware fault tolerance for VMM then the simple solution is to run VMM in a highly available virtual machine.  I don’t like System Center being a part of a general production Hyper-V cluster.  That’s because you create a chicken/egg situation with fault monitoring/responding.  If you want to virtualise System Center then consider setting up a dedicated host or cluster for the VMM, OpsMgr, ConfigMgr VMs.  DPM is realistically going to remain physical because of disk requirements.

Manage More Than 150 Hosts

It is recommended that you:

  • Not use VMM server to host your library.  Set the library up on a dedicated server/cluster.
  • Install SQL Server on a dedicated server/cluster.

The VMM server requirements are:

Component Minimum Recommended
CPU Pentium 4, 2.8 GHz (x64)

Dual-Processor, Dual-Core, 3.6 GHz (x64) or greater

Memory

4 GB

8 GB
Disk space (no local DB)

10 GB

50 GB

The database server requirements are:

Component Minimum Recommended
CPU Pentium 4, 2 GHz (x64)

Dual-Processor, Dual-Core, 2.8 GHz (x64) or greater

Memory

4 GB

8 GB
Disk space (no local DB)

150 GB

200 GB

A dedicated library server will require:

Component Minimum Recommended
CPU Pentium 4, 2.8 GHz (x64)

Dual-Processor, Dual-Core, 3.2 GHz (x64) or greater

Memory

2 GB

2 GB
Disk space (no local DB)

Depends on what you store in it

Depends on what you store in it

A dedicated Self-Service Portal server will require:

Component Minimum Recommended
CPU Pentium 4, 2.8 GHz (x64)

Dual-Processor, Dual-Core, 3.2 GHz (x64) or greater

Memory

2 GB

8 GB
Disk space (no local DB)

10 GB

40 GB

VMM Console

The software requirements are:

  • Either Windows 7 with SP1 or Windows Server 2008 R2 with SP1
  • PowerShell 2.0 (included in the OS)
  • .NET 3.5 SP1 (installed by default in Windows 7 and a feature in W2008 R2 – VMM setup will enable it for you)

Managing up to 150 hosts will require:

Component Minimum Recommended
CPU Pentium 4, 550 MHz

Pentium 4, 1 GHz or more

Memory

512 MB

1 GB
Disk space (no local DB)

512 MB

2 GB

Managing over 150 hosts will require:

Component Minimum Recommended
CPU

Pentium 4, 1 GHz

Pentium 4, 2 GHz or more

Memory

1 GB

2 GB
Disk space (no local DB)

512 MB

4 GB

Managed Hosts

Supported Hyper-V hosts are below. 

Parent OS Edition Service Pack
Windows Server 2008 R2 (Full or Server Core)

Enterprise or Datacenter

Service Pack 1 or earlier

Hyper-V Server 2008 R2  
Windows Server 2008 (Full or Server Core)

Enterprise or Datacenter

Service Pack 1 or earlier

Please note that the following are not listed as supported:

  • Hyper-V Server 2008
  • Windows Server 2008 R2 Standard edition
  • Windows Server 2008 Standard edition

In the beta, Windows Server 2008 is not supported.

Supported VMware hosts are listed below.  They must be managed by vCenter Server 4.1.

  • ESXi 4.1
  • ESX 4.1
  • ESXi 3.5
  • ESX 3.5

There is no mention of vSphere/ESXi 5 at the moment.  That’s understandable – both VMM and the VMware v5 product set were being developed at the same time.  Maybe support for v5 will appear later.

Citrix XenServer 5.6 FP1 can also be managed as standalone hosts or as Resource Pools if you deploy the Microsoft SCVMM XenServer Integration Suite to your hosts.

Bare Metal Host Deployment

The requirements for being able to use VMM 2012 to deploy Hyper-V hosts to bare metal machines are:

Item Notes
Windows Server 2008 R2 Windows Deployment Services (WDS) PXE Server to boot the bare metal up on the network.  No other PXE service is supported.
Boot Management Controller (BMC)

This is a server management card:

  • Intelligent Platform Management Interface (IPMI) versions 1.5 or 2.0
  • Data Center Management Interface (DCMI) version 1.0
  • Hewlett-Packard Integrated Lights-Out (iLO) 2
  • System Management Architecture for Server Hardware (SMASH) version 1.0 over WS-Management (WS-Man)
VHD image A Windows Server 2008 R2 host OS captured as a generalized VHD image.  Have a look into WIM2VHD or maybe using a VM to create this.
Host Hardware Drivers NIC, Storage, etc.

Update Management

A dedicated WSUS root server, running WSUS 3.0 SP2.  It cannot be a downstream server because that is not supported.  There will be a lot of processed updates so this may require a dedicated server (possible a VM).  If you install WSUS on a VMM server cluster then you must install the WSUS Administrator Console on each node in that cluster.

Deploying New Hyper-V Integration Components

Imagine this: you are running a pretty big Hyper-V environment, Microsoft releases a service pack that adds a great new feature like Dynamic Memory (DM), legacy OS’s will require the new ICs, and you really want to get DM up and running.  Just how will you get those ICs installed in all those VMs?

First you need to check your requirements for Dynamic Memory.  The good news is that any Windows Server 2008 R2 with SP1 VM will have the ICs.  But odds are that if you have a large farm then things aren’t all that simple for you.  Check out the Dynamic Memory Configuration Guide to see the guest requirements for each supported OS version and edition. 

OK, let’s have a look at a few options:

By Hand

Log into each VM, install the ICs, and reboot.  Yuk!  That’s only good in the smallest of environments or if you’re just testing out DM on one or two VMs.

VMM

VMM has the ability to install integration components into VMs.  The process goes like this:

  1. Shut down a number of VMs
  2. Select the now shut down VMs (CTRL + select)
  3. Right-click and select the option to install new integration components
  4. Power up the VMs

You’ll see the VM’s power up and power down during the installation process.  Now you’re done.

WSUS

Here’s an unsupported option that will be fine in a large lab.  You can use the System Center Updates Publisher to inject updates into a WSUS server.  Grab the updates from a W2008 R2 SP1 Hyper-V server and inject them into the WSUS server.  Now you let Windows Update take care of your IC upgrade.

Configuration Manager

This is the one I like the most.  ConfigMgr is the IT megalomaniac’s dream come true.  It is a lot of things but at it’s heart is the ability to discover what machines are and distribute software to collections of machines that meet some criteria.  So for example, you can discover if a Windows machine is a Hyper-V VM and put it in a collection.  You can even categorise them.

You may notice that Windows Server 2008 with SP2 Web and Standard editions require a prerequisite update to get DM working

So, you can advertise the ICs to a collection of W2008 with SP2 standard and web editions, making that update a requirement.  The update gets installed, and then the ICs get installed.  All other OS’s: it’s just an update.  And of course, you just need to install SP1 on your W2008 R2 VMs.  As you may have noticed, I’[m not promoting the use of the updates function of ConfigMgr; I’m talking about the ability to distribute software.

I’ll be honest – I don’t know if the ConfigMgr method is supported or not (like the WSUS option) but it’s pretty tidy, and surely must be the most attractive of all in a large managed environment.  And because it’s a simple software distribution, I can’t see what the problem might be.