2013
09.04

Microsoft added a new feature in Windows Server 2012 R2 Hyper-V called Virtual RSS or vRSS.  Receive Side Scaling (RSS) is a feature used in physical NICs to allow a server’s networking capacity to scale out.  Microsoft describes RSS as:

… a network driver technology that enables the efficient distribution of network receive processing across multiple CPUs in multiprocessor systems.

Long story, short, without RSS inbound networking scalability in a server is bottlenecked by the processing power of Core 0 of CPU 0, no matter how many cores or processors that you have in that server; all the networking interrupts go to that single core.  With RSS in a NIC and enabled in the advanced settings (changes depending on your NIC) then Windows Server 2012 and later can spread the processing load across the cores/processors in the server (including Hyper-V hosts).

As I said, RSS is a function of physical NICs, and therefore a function of physical servers/hosts.  Since WS2012 Hyper-V we have the ability to create massive VMs with 64 virtual processors, 1 TB RAM, and oodles of VHDX storage.  I bet some of those workloads would like to do lots of inbound networking.  Unfortunately, there was no RSS that you could use in the Hyper-V Virtual Ethernet Adapter.

That changes with WS2012 R2 Hyper-V because it allows us to turn on vRSS.  The concept is simple enough.  In the host you have one or more 10 Gbps or faster physical NICs (pNICs) that are connected to the virtual switch, probably via a NIC team (if there is more than one pNIC).  Virtual Machine Queue (VMQ) is enabled in this NICs.  VMQ is a cousin of RSS; it uses the same circuitry to increase the scalability of inbound VM networking on the host.

You can check the status of VMQ using PowerShell if you want using Get-NetAdapterVMQ.

image

You then enable vRSS in the properties of the virtual NIC in the guest OS of the VM.   I went one step further by enabling Jumbo Frames.

image

Alternatively you can enable vRSS in the VM by using PowerShell in the guest OS if you want:

Enable-NetAdapterRSS –Name <AdapterName>

Voila; you have increased the scalability of the VM’s networking … assuming that the VM has lots of virtual prcessors.  Now you can push some data to your VM(s).  Open up Task Manager and the logical processors view and you can see the workload being scaled out across the virtual processors of the VM.

image

My reason for enabling vRSS was that I was building a second Scale-Out File Server, which needed to be virtual due to equipment shortages. I built up 2 VMs with Shared VHDX.  Each VM in the virtual SOFS has 3 virtual NICs:

  1. Management: connected to a 1 Gbps virtual switch
  2. Storage1: connected to a 10 Gbps virtual witch, with dVMQ enabled on the pNIC
  3. Storage2: connected to another 10 Gbps virtual witch, with dVMQ enabled on the pNIC

All my cluster and SMB 3.0 traffic goes over Storage1 and Storage2 via SMB Multichannel.  I enabled vRSS in the guest OS of the VMs.  The above CPU performance was snapped from when I had a job running to create 60 WS2012 R2 VMs on the virtual SOFS.  You can see that the processor load is better balanced … better for the VM’s scalability and probably better for the VM’s neighbours on the host.

Note that the supported guest OS’s are Windows 8.1 and Windows Server 2012 R2.

Note: NVGRE is compatible with vRSS.  SR-IOV is not compatible with vRSS

9 comments so far

Add Your Comment
  1. Hi Aidan,

    You wrote: “Voila; you have increased the scalability of the VM’s networking … assuming that the VM has lots of virtual NICs.”

    Did you actually mean “… assuming that the VM has lots of virtual CPUs.”

    • Yes.

  2. Hi Aidan,
    do you advise Broadcom or Intel NICs? In combination with Dell servers & HP Procurve switches.
    thanks,
    Haroun.

    • Short answer: yes.

  3. great article yaar, thanks a lot

  4. Does vRSS also work on the host vNIC? e.g live migration or Management vNIC?

    • Unfortunately, no.

  5. Great. What about the vSwitch performance with RSS enabled on the host?
    Does vSwitch run on the different CPU of the host?

    • You’ve got the concept wrong. It’s the queues inside the guest OS of the VMs that are optimized, not the virtual switch. There’s no point in optimizing the switch.

Get Adobe Flash player