KB2878635 – A December 2013 Update To Improve CSV Backup Resiliency On WS2012 Hyper-V

Microsoft released an update in December 2013 that that improves the resiliency of the cloud service provider in Windows Server 2012. That’s a little “marketing speak”. In truth, this update is designed to resolve issues with CSV backup on Windows Server 2012 Hyper-V. This update two fixes. Please note the post-installation instructions!!!!

Symptoms

This article introduces an update that improves the resiliency of the cloud service provider in Windows Server 2012. This update is dated December 2013.
This update replaces update 2870270, which is used to improve resiliency. Also, this update includes update 2869923 and update 2908415. Additionally, the update resolves several issues that occur in the following scenario: 

• You have a Hyper-V failover cluster.
• The Hyper-V resources are saved in .vhd files on Cluster Shared Volumes File System (CSVFS) volumes.
• You use a backup solution. For example, you use System Center Data Protection Manager (DPM) in the Hyper-V environment.
• You try to perform a backup, and a snapshot is taken of the CSVFS volume.
• The current active node encounters an error, and the cluster fails over to another node.
• DPM may start a consistency check on the volume unexpectedly.

Issue 1

Snapshots that are no longer being used are not cleaned up. Therefore, Volume Shadow Copy Service (VSS) snapshots may accumulate on Cluster Shared Volumes (CSV) and guest virtual machines. This causes a deadlock in the Resource Hosting Subsystem (RHS) process, and causes  CSV failures. Additionally, all Hyper-V instances that uses the VHD files go down.
Additionally, the following events are logged separately in the Cluster log and in the System log:

Software snapshot creation on Cluster Shared Volume(s) (‘volume location‘) with snapshot set id ‘snapshot id‘ failed with error ‘HrError(0x80042308)(2147754760)’. Please check the state of the CSV resources and the system events of the resource owner nodes.
Log Name: System
Source: Microsoft-Windows-FailoverClustering
Date: Date and time
Event ID: 5120
Task Category: Cluster Shared Volume
Level: Error
Keywords:
User: SYSTEM
Computer: Computer name
Description: Cluster Shared Volume ‘Volume1‘ (‘name’) is no longer available on this node because of ‘STATUS_IO_TIMEOUT(c00000b5)’. All I/O will temporarily be queued until a path to the volume is reestablished.

Log Name: System
Source: Microsoft-Windows-FailoverClustering
Date: Date and time
Event ID: 5142
Task Category: Cluster Shared Volume
Level: Error
Keywords:
User: SYSTEM
Computer: Computer name
Description: Cluster Shared Volume ‘Volume3‘ (‘Cluster Disk 4‘) is no longer accessible from this cluster node because of error ‘ERROR_TIMEOUT(1460)’. Please troubleshoot this node’s connectivity to the storage device and network connectivity.

Issue 2

If a CSVFS volume is repeatedly added and removed from a cluster, or if CSVFS snapshots are repeatedly created, the Plug and Play (PnP) hive in the following registry path may grow with many additional registry keys: HKEY_LOCAL_MACHINESYSTEMCurrentControlSetEnumSTORAGEVolumeSnapshot.
Therefore, installation of PnP volumes (which occurs usually during a resource move or failure) may become slow.  This update prevents future unnecessary growth in the volume snapshot registry key, but does not clean up existing registry entries.

 

Resolution

A supported hotfix is available from Microsoft.

Post-Installation Instructions

After you install this hotfix on a Hyper-V server, you must update the integration components in the virtual machines that are running Windows Server 2012. To do this, use Hyper-V Manager to connect to the virtual machine. This starts the Virtual Machine Connection tool. Then, on the Action menu, click Insert Integration Services Setup Disk. Run the Setup.exe file on the Integration Services Setup disk to update the integration component.