VMware Inc.’s ESX hypervisor could let IT staff steal sensitive data by abusing administrative access, particularly if customers fail to implement role-based access controls, the security vendor BeyondTrust Software Inc. argued at the recent VMworld conference.
IT staff with root access to VMware ESX can steal virtual machine disk files and then erase log files and other traces of the illicit activity by manipulating the service console, a Linux-based instance that manages the VMware hypervisor, BeyondTrust says. This could make it easy to steal medical records, financial data, or any other files tied to virtual machines, says Jordan Bean, principal systems engineer for Carlsbad, Calif.-based BeyondTrust. Bean provided a demonstration of this type of attack on the VMworld conference exposition floor.
But in response, VMware noted that root access to any sort of IT product could let users do malicious things. VMware doesn’t have built-in access controls for the service console, but does offer a recommended set of best practices to enable role-based access controls and has partnered with third parties – including BeyondTrust – to track and manage access into virtualized environments.
Moreover, VMware is eliminating the service console in future versions of its core hypervisor platform. For the past several versions of vSphere – formerly known as VMware Infrastructure – the vendor has offered both the ESX and ESXi architectures in parallel. But in the next release, the date of which has not yet been announced, ESX will be eliminated leaving only ESXi, which lacks a service console.
ESXi has a much smaller attack surface, roughly 100 MB instead of 2 GB, largely because the Linux-based service console has been replaced by APIs and modules that let administrators create and manage virtual machines.
Problems related to root access are possible in any IT product, says Venu Aravamudan, a senior director for product marketing at VMware.
“It’s not as much a vulnerability in that clearly if you got the root password to anything” – like a SQL Server or a router – “you can do whatever you want. You’re never going to stop that from occurring for any product in the marketplace today,” he says.
But the specific scenario of stealing virtual machine disk files demonstrated by BeyondTrust is much harder to achieve with ESXi than it is with ESX.
“You’re still going to have one root password,” says Charu Chaubal, senior technical marketing manager for VMware. “But this phenomenon of one user that can do everything is highly mitigated [in ESXi]. By going to the ESXi architecture, it’s almost like you’re closing the garage door, and now you can only go through the windows, and every window can be locked individually.”
Under the ESX architecture with the service console, Bean says by logging in with the same username and password used to create an ESX host, a user can essentially operate invisibly to VMware’s security.
“At this level, logged in with this account, I don’t do anything through VMware,” he says. “I don’t care about their processes. I’m going underneath VMware’s own security.”
After accessing and copying file systems to a personal drive, the user attempting to steal data can prevent detection by deleting the history and log files.
“There’s very little activity logging at this level,” Bean says. “There are records that I’ve logged in, a history of activity. But before I log out all I have to do is blow it away, overwrite it, and there’s nothing left for them to see.”
Bean acknowledges he’s not aware of any actual attacks “in the wild,” but says “I’m sure it has [happened]. We don’t hear about it, but I’m sure people are doing it.”
Bean also says root access can be manipulated in similar ways in any Xen-based hypervisor.
In response, Simon Crosby, chief technology officer of Citrix Systems Inc., said his firm’s XenServer is like ESXi in that it lacks a service console.
“XenServer does not have the concept of a service console as ESX does,” Crosby writes in an e-mail. “XenServer, like ESXi, has a tiny embedded runtime that is entirely locked down with no user access. It provides driver support and runs the embedded management stack. There is no notion of an administrator logging on to this embedded VMM runtime. It supports no access other than SSH.”
Crosby downplayed the issue raised by BeyondTrust, saying that if any hypervisor is compromised, whether it be VMware’s, Citrix’s or Microsoft’s, then all virtual machines associated with the hypervisor would be at risk.
“The ‘service console’ challenge occurs in any ‘OS + Hypervisor’ approach,” he writes. That includes “Linux with Xen or KVM” and “WS08 [Windows Server 2008] parent partition with Hyper-V. In this case you have the entire attack surface of a traditional OS.”