Waiver: What you do following reading this post is up to you.
After my earlier post on “Top Hyper-V Implementation Issues” I had some feedback on my preference to keep antivirus (AV) off of the Hyper-V hosts.
The configuration that you should have is in KB961804. That article also says what can happen if you do install AV on your hosts, not follow that guidance, and scan everything. One day you’ll end up with nasty errors such as 0x800704C8, 0x80070037 or 0x800703E3 and find lots of VMs (with their business apps and data) have:
- Disappeared from your Hyper-V console
- Disappeared from your VMM console
- Are not running
The files are still there but, damn, the VMs will not start up or appear in a management tool. That’s because AV has gotten in the way and screwed up with things. I blogged about this back during the W2008 Hyper-V beta (can’t find the post now) in early 2008. It happened to me. I was unlucky; I set the required exclusions and restarted the host in question (a lab machine). My VM configuration files were corrupted. The solution was the recreate the VM’s and point them at the existing VHD’s containing the safe OS, programs, and data. Time consuming – and how many people document/remember their VM configurations? And come to think of it, how many businesses would be OK with their LOB applications being offline for half a day or more while admins do this?
I learned something in 2004. There is a balancing act between security and business. Sometimes it has to swing one way, sometimes another. This is one of those cases.
I do not trust any antivirus product completely. They are stupid assassins. They are given rules of engagement, get a target list, and they attack. But all too often, program updates, definition file updates, or dumb human operator error make mistakes. It is not unknown for one of these to reset the exception list. Yes; it has happened – and even happened recently. Do you really want one of these things to undo the necessary configurations of your Hyper-V cluster – a thing that is effectively a mainframe running many/most/all of your LOB applications, and putting them at risk?
So I say: do not install AV on the parent partition or host OS. Sure, go ahead and install it in the VMs. If you can, choose an AV product that is aware of things like virtualisation and minimises redundant scanning. On the host, make sure you apply security fixes. Keep the service pack up to date. And keep the Windows Firewall running. Finally, restrict who has logon rights to the hosts. If you can, prevent the hosts from having proxy/web access. People should never browse from a server but I just don’t trust human nature. All that should secure the parent pretty well.
Now let’s get back to why you’re installing AV on the parent partition. Odds are there is a security officer who has a list of things that [booming voice] “must be done to all Windows computers” [/booming voice]. And if you do not do these things you will be fired! One of them is: “you must install anti virus and scan everything because Windows is a threat to life itself”. Hmm, someone’s been reading the SANS website again! I hate checklist security experts.
Here’s my response to that person:
- I’d point them to KB961804. In fact, you might even want to show them the Microsoft required exceptions list. It says “recommended” in the title but try having that argument with a MSFT support engineer when your SYSVOL is corrupted!
- If they insist, then say you’ll comply but you have one requirement. Never say “no” because that’s career suicide. Give them a waiver form. This form will clearly state that you the operator/administrator/engineer/consultant will not be held responsible for any corruption or loss of virtual machines because of the mandate to scan all things on the Hyper-V hosts. All responsibility will lie with the undersigned security officer – and demand their signature. Then keep a copy for yourself, give one to your boss, and one to the CIO. At least then you know who will get fired when incorrectly configured AV causes your VMs to disappear.
It’s funny; security officers are usually career politicians. And politicians do not like being nailed down to a something like that. Taking responsibility is not in a politician’s nature. I bet you get your way after that.
Maybe as a compromise, you might offer to take a host offline once in a while to perform a complete system scan of the C: drive.
Anyway, that’s my opinion on the matter.
You’re making a judgment call, weighing all the options with their pros and cons. And in my humble opinion you’re right.Compare this to vaccins/inoculations to protect the majority of your population. You don’t have to get a 100% complete coverage to be succesfull in containing an outbreak. Just a sufficiently large enough part including your most vulnerable and most at risk population. This fits that bill. Check list “security officers” do more harm than good.
Thanks Didier.
Some more to think about: implement correct cloud email scanning, on-server email scanning, proxy scanning, and server/desktop scanning, and you have protected all avenues. If you really need to, you could go extreme, and set up a AD forest/firewalled network just for your parent partitions – but that’ll complicated management stuff like you won’t believe!
Aha, yes, one of my designs is a AD domain just to be used for management/clustering and is extremly isolated. The actual production domain(s) run as guests on the Hyper-V hosts of a cluster in that management domain. Minimal to no contact between partent partitions guest and in between seperate guest environments depending on your needs 🙂 I love that concept.
Its not supported by Microsoft, but in a pinch I have used the following article to re-add VMs to the host if the config is still intact but it’s just not showing up in the Management console (note it does work in 2008 R2):
http://blogs.msdn.com/b/robertvi/archive/2008/12/19/howto-manually-add-a-vm-configuration-to-hyper-v.aspx
Also, once in a while I have seen the config file just have an extra ‘>’ on the end when the host doesn’t shut down cleanly, but that was on a 2008 host.
Considering that all antivirus on-access scanning can be a hog if a file is changing size all the time, and as an ex-security person, I would always ask, “do we have a file that is constantly being changed? If I knew it was being changed all the time, I would do an exclude of scanning for that file. Was I correct in doing this? I did not care, as long as the file was not .dll, .exe and etc. With a host server, the on access scanner would have nothing to do once the host is up and running. Would you scan the VM and the files that make each VM work and any of its files? IF you have a thug requiring you to use antivirus, limit what is being scanned and avoid scanning of the VM folders.
Just avoiding those folders is not enough. You have to comply with the entire set of exceptions. Fail to do so and *bang* you’re done. And I did hear of one AV product that did reset its exceptions after an upgrade recently. The risk is just too high IMO.
I agree, I am with you. Do not install AV and save your weekend for golf or boating. I do not see this one as a lesser of 2 evils, as I have seen a DAT update break thousands of clients and they all needed a manual fix or a push of code to get them all happy.