The effect on the power consumed by turning off those unused VM's!

Last year I wrote a little blog about power consumptions and created a little dashboard to see if we can save some power.

Link: Monitor Performance while reducing power usage/ C02 Emission ? Yes we can! (debruinonline.net)

In the past many best practices preached: disable power management features! Policy to High Performance. No delay is acceptable! Electricity is cheap...but the world has changed.

I noticed that when I talk about saving power 90% of the time the discussion immediately turns to DPM (Distributed Power Management). Although true, yes this saves the most power but it is not always possible.. when using VSAN for instance.. or simply not desirable when the boot time of the hosts are extremely long. But the world is not black and white. We don't need to turn off the host to start saving some power.

First a little theory taken from: Host Power Management in VMware vSphere 7.0 Performance Study for Optimal Power Consumption September 28, 2021:

Modern processors all have power-focused enhancements. These are available though the ACPI (Advanced Configuration and Power Interface) an open standard developed by several hardware vendors.

The ACPI standard defines C-States (commonly known as idle states or power states) and P-States (widely known as operational states or performance states). Hardware vendors such as Intel and AMD have introduced hardware support for P-States and C-States, which are accessible through the on-chip power control unit (PCU) [1] [2]

And vSphere/ESXI can take control of these states. You can control which states by settings the power policy's.

High Performance: This policy tries to maximize performance by disabling C-State and P-State management. It always keeps the CPU in the highest possible operating frequency (P0-State) and only uses the top two shallow C-States (C0 when running and C1 when idle).

The High-Performance policy is geared toward latency-sensitive applications and provides predictable and consistent performance.

Balanced: This policy is designed to minimize host power consumption while having little to no impact on performance. The balanced policy is the default power policy in vSphere 7.0. The policy determines the system load and selects the appropriate P State. The Demand Based Switching (DBS) implementation here has little to no impact on application performance. vSphere chooses a suitable deep C-State (like C2) based on its prediction of when the CPU cores need to be active again.

Low Power: This policy is designed to reduce power consumption compared to the other policies substantially. The P-State and C-State selection algorithms are more aggressive toward power saving. This policy, however, can impact the performance of latency-sensitive applications.

Custom: This policy, by default, has the same settings as Balanced. However, it provides the user the ability to tune individual parameters for specific use cases.

The link to the full article is below.

To summarize, modern CPU's can switch to different states to save power and vSphere/ESXI can control these states.

Theory... fun and all but today I did a little test in my homelab on one of my older hosts and happy to share what I've seen.

The infrastructure component are not hosted on this host so Instant Clones only.

the measured Hardware:

one HP DL380 G7 with dual Xeon E5620 (Told you so, older host)

vSphere and ESXi 7.x

Horizon 8 2303 using Instant Clones

Some NVME disks, some spinning disks

start power policy: High Performance

The test and result :

10:15/10:20 get the host out of "Standby" by using the DPM feature. well takes some time.. good old G7. With no VM's turned on the power consumption of the host is around 180 Watts

11:00 Enabled provisioning in horizon, you can see the number of registered and powered on VM's starts rising (blistering fast of course, awesome Instant Clone technology). Notice that in the second graph the power consumption starts to rise in parallel with the CPU demand and usage. No suprise the CPU in this setup demands the most power. Power consumptions is now at 268 Watts

11:55 changed the pool size to 5 Instant Clones. Instant Clones are starting to turn off and be destroyed since it is a non Persistent Floating pool. The power consumption starts dropping to 190 Watts.

12:20 changed the power policy to "low power". As you can see the power consumption starts to drop in a little cascade, vSphere/ESXi starts lowering power consumption to a 167 Watts.

Conclusion so far:

1. Yes, more VM's active means more power is consumed, even in high performance some power savings apply

2. Switching from "High performance" policy to "Low Power" policy squeezes out some more power savings.

3. Even the power saved might not seem much. it's a 100 Watts but on only one host. What if you have something around a 1000 hosts? Think big...

Should you turn all your hosts to Low Power mode? Turn off all your VM's? No of course not. Currently we are working on some cool ideas for this. More on that later.

Little side note: The BIOS of your host(s) needs to be configured correctly for these policies to work

see if the ACPI P-states and ACPI C-states are reported

Thanks for reading! Care to debate on this topic? Questions? Remarks? Please let me know!

Additional Links

1. Host Power Management in VMware vSphere 7.0: Performance Study for Optimal Power Consumption

2. Monitor Performance while reducing power usage/ C02 Emission ? Yes we can! (debruinonline.net)