The architecture of ESXi
Chapter 1, An Introduction to Server Virtualization using VMware, introduced you to the architecture of a type-1 hypervisor and the differences between ESX Classic (no longer developed) and ESXi, in terms of their architecture.
The ESXi hypervisor is made up of the VMkernel, the User World layer and the Virtual Machine Monitors. What these components provide is the foundation for other processes to enable core functionalities.
The VMkernel layer
VMkernel is a Posix-like operating system which is primarily responsible for resource scheduling. In an effort to allocate a fair share of the hardware resources between all the hosted virtual machines, it creates a level of abstraction so that the guest operating systems on the virtual machines can consume resources like they would do on a physical machine, without interfering in the operations of the guest operating systems running on other virtual machines. VMkernel includes a storage stack which supports the use of a proprietary distributed file system called the Virtual Machine File System (VMFS), and a network stack that enables the creation of virtual switches.
User World APIs
The User World provides a set of APIs that help other processes to natively interact with VMkernel and those include:
- Management agents: hostd and vpxa
- The HA agent: FDM
- CIM Broker
- DCUI
- Syslog daemon
- Software iSCSI adapter
- NTP client
- SNMP
- VMX User World helper processes for every running virtual machine. The helper process will handle the device I/O, virtual machine snapshots, and so on.
The management agents, hostd and vpxa, help send commands to the hypervisor through a management interface such as the vSphere Client or the vCenter Server.
The hostd is commonly referred to as the host management agent and is involved in every user interaction with the hypervisor. When the vSphere Client is connected directly to ESXi, commands are sent to the hostd agent, which uses the vSphere API (User World API) to complete the requested actions. hostd also has an embedded SNMP agent than can be used to send traps or receive polling requests from an SNMP server.
The vpxa is an agent used for communication between the vCenter and the ESXi hosts it manages. It proxies the commands received from the vCenter Server to the hostd agent. Every ESXi host will have a vpxa agent when it is connected to a vCenter Server.
The vSphere HA agent or Fault Domain Manager Agent will be installed on an ESXi host when HA is enabled on it. It is the vCenter that will transfer and install the agent on the host.
The CIM Broker is a set of APIs that can be used by an application to fetch the status of the hardware devices on an ESXi host. Hardware vendors are allowed to develop their own CIM plugins to use the broker APIs to monitor and collect health information.
The Direct Console User Interface (DCUI) is a menu-driven console interface which is used to manage an ESXi host. It can be used to perform very basic management and configuration changes to an ESXi host. We will learn more about DCUI later in this chapter.
The syslog daemon is used to redirect the storing of logs onto a remote syslog server.
The Software iSCSI adapter can be used for iSCSI target discovery without the need for a hardware-based solution.
The NTP Client helps synchronize the ESXi host with an NTP server for accurate time keeping.
VMM worlds
The Virtual Machine Monitor worlds provide an execution environment for every running virtual machine. They are responsible for virtualizing CPU instructions issued by the guest operating systems and also manage the virtual machine's memory.
ESXi's in-memory filesystem
ESXi has a very minimal storage footprint when installed. It is so small that it can be loaded directly into the memory of the server without the need for secondary storage. ESXi maintains an in-memory filesystem which is used to store all the configuration files that enable ESXi's core functionalities. The in-memory filesystem forms a work desk for ESXi to operate. The work desk has several drawers for storing essential files of different types and which serve different purposes. So, the work desk here is a disk partition layout and the drawers are different partitions that are formed during the deployment and installation.
Here is a list of five partitions that ESXi uses:
- Bootloader partition
- Bootbank partition
- Alternative bootbank partition
- Store partition
- Core dump partition
The bootloader partition is where the bootloader is stored. This partition is 4 megabytes in size. The bootloader is responsible for booting the system and handing over the control the VMkernel.
The bootbank is a 48-megabyte partition that will hold the ESXi hypervisor code along with any of the vendor specific drivers and so on.
The alternative bootbank is the same size as the bootbank and holds an identical copy of the hypervisor code. This bank is used during hypervisor upgrades as a working partition, and the original bootbank will act as a failsafe during the first boot after the upgrade. We will learn more about what happens during the upgrade later in this book.
The store partition is a 540-megabyte partition that acts as runtime storage for the VMkernel and also holds other essential binaries including VMware Tools packages for all supported guest operating system types and VMware vSphere Client binaries. The vSphere Client binaries are just Windows installation packages that are available for download from the ESXi host's web page.
The core dump partition is 110 megabytes in size and used to store the VMkernel's memory dump in the event of an unrecoverable error affecting the running state of the ESXi hypervisor. The core dump partition can also be a remote partition.