TechEd NA 2014–SCVMM Bare Metal Host Deployment

Speakers: Damian Flynn, MVP and Marc Van Eijk, MVP.

Confusing topic for many. The guys are very experienced in the real world so a good session to take notes from and share.

Environment Preparation

• Rack the servers and configure the BMC card.
• Build an OS image and add it to the library.
• Configure DNS aliases for BMCs, set up certs (if required), and set up VMM Run As profiles, e.g. join a domain, log into BMC, etc.

Infrastructure Services

• Set up WDS for SCVMM
• You need a DHCP scope for your bare metal hosts for the deployment phase. The hosts will get static IPs after deployment.
• Prep SCVMM – Import WDS, add OS image to the library (and refresh), add a Run As account for the domain join, and add a Run As account for the BMC.

The deployment

• Configure the networking stack in SCVMM
• Do a deep discovery to get hardware of the host
• Deploy the VHD as boot-from-VHD on the host, install drivers, join domain, configure networking, enable Hyper-V,  etc.

YOU NEED TO UNDERSTAND YOUR TARGET

Concepts of the network in SCVMM

• Logical network: A role
• Network site: Part of logical network
• Subnet/VLAN ID: A subnet that’s a part of a site
• IP Pool: A pool of IPs for a subnet

A VM Network is an abstraction of a logical network. Required to connect NICs to a logical network.

Demo of Logical Network

Marc asks who has used VMM. Almost everyone. Who has done bare metal deployment: Very few. Who was successful first time: one brave person puts his hand up (I mock him – seeing as he is a friend).

Marc does:

1. Create a host group.
2. Creates a logical network called management. He sets VLAN-basd independent networks. There will be converged networks that are split up based on VLANs.
3. Creates a network site called host that is set to be available on the host group. He sets a VLAN on 0 for PXE boot, and sets the IP subnet.
4. Additional network site for Live Migration with a different VLAN
5. Then he adds a third site for cluster communications with a VLAN. So one logical network with 3 network sites.
6. Creates IP pools for each network site. Use to assign static IPs during deployment. Configures gateway and DNS settings for the management network.

Note that there is no need to do anything special to enable NVGRE. No subnets, logical networks, or anything else. A check box was left checked to allow NVGRE to be used when creating the logical network.

1. Creates a new logical network called Cloud Network. This is what would appear in WAP when a customer creates a virtual network – so choose a suitable name.
2. Checks “allow new VM networks ….” to use NVGRE.
3. Creates a site with a VLAN and associates with the host group.
4. Now he creates an IP pool for that site/logical network. The number of IPs in the pool will limit the number of VMs. No DNS or gateway settings.

So now we have two logical networks: Management and Cloud Network. The Cloud Network appears to be used for the PA Space.

1. A third logical network called tenant VLANs
2. Network site: Names the site after the VLAN ID.
3. Adds more network sites, named based on the VLAN IDs.
4. Adds IP pools.

These VLANs appear to be used for tenants.

1. Creates VM network for host/management.
2. Creates VM network for cluster.
3. Creates VM network for live migration.
4. Creates a VM network for tenant A and another for tenant B

Back to presentation.

Network Design

Note that for VMM to create a team, you need to create a logical switch. BAD! Needless creation of virtual switches and limits things like RDMA. Complet convergence also not good for some storage, e.g. RDMA or iSCSI storage. Might do some convergence and non-converge your storage networks.

Benefit of logical switch

Repeatable consistency.

Note: also required for NVGRE, unless you want to go to PowerShell hell.

The design they are deploying:

Demo

1. Create an uplink port profile to define a NIC team. This one is created for HNV/Tenants. Selects the Cloud Network and the tenant VLAN network sites. Also makes sure the enable NVGRE check box is left enabled.
2. Creates an uplink port profile for the mangagement network. Now adds the cluster, host, and live migration network sites.

What he’s done: configure the above two network teams from the diagram and defined which networks will pass through the respective teams.

1. Creates a logical switch for management. Selects the management uplink port profile and selects the teaming option. Even if you have just one NIC, you can add a NIC later and join it to the team. Now to define the convergence by adding virtual ports. A step in this is to define port classification – this does QoS. Select Host Management and match with management network – repeat for the reset of management networks.
2. Creates a logical switch for tenants. And also teams with the tenant HNV uplink port profile. Adds three adapters (port profile classifications) for QoS – low, medium, and high (out of the box – weights of 1, 3, and 5).

1. Next up: create a physical computer profile. Choose a host profile. Select the virtual hard disk from the library that will be the host OS. Now the fun bit – network configuration in Hardware Configuration.
2. Tip: Expand this dialog using the control in the bottom right corner.
3. It starts with a single physical NIC with the management role. Add 4 more physical NICs.
4. First and second added to the logical switch of management.
5. Configure 3rd and 4th to the tenant logical switch.
6. Edit the original physical NIC and select “Create a Virtual Network Adapter as the management NIC”. Set the transient physical newtork adapter as NIC 1. Apply a classification – host management. Set the IP Pool as Host.
7. Add 2 virtual NICs. Connect 2st to logical switch management. Set to Live Migration. Connect the 3rd to the mangement logical switch and configure for cluster.
8. Can also do some other stuff like filtering drivers from the library for precise PNP.
9. Continue the wizard – set domain join and runas account. Set the local admin password, the company info and product key. An answer file can be added to customize the OS more, and you can run tasks with GUIRUNONCE.
10. You can skip the default VM storage path for clustered hosts – VMM will control this in other ways later.

Deployment Demo

1. Kicks off the wizard from the host group. Provision the new machine.
2. Select a host group that has a valid host profile. Select the host profile.
3. Kick off the deep discovery. The host reboots into WinPE to allow VMM to audit the host h/w. With CDN enabled, you can pre-bind NICs to logical switches/teams. Without it, you’ll need to know which NIC is plugged into which switch port, then you can bind NICs to right logical switches. The server schedules a shutdown after the audit.
4. In VMM you can finish the host configuration: naming of the host. Binding of NICs to logical switches if you don’t have CDN in the host. If you’re quick, the server will not shutdown and the setup will kick off.

Notes

Converging things like SMB 3.0 or Live Migration through a logical/virtual switch disables RSS so you limit 10 GbE bandwidth to 3.5 Gbps or thereabouts. Can create multiple management OS vNICs for SMB multichannel, where VMQ dedicates a queue/core to each vNIC.

My approach: I do not converge my SMB/cluster/storage rNICs. They are not teamed, so they are basic logical networks. No need then for logical switch.