Troubleshooting Mindset Series #014: HCI Node Memory >95% — Capacity Problem or VM Sizing Problem?

During routine health monitoring of our Sangfor HCI environment, I noticed a Critical Alert indicating that one of the HCI nodes had exceeded the configured memory usage threshold.

The alert showed:

• Allocatable Memory: 259.5 GB
  
  
• Memory Occupied: 260.3 GB
  
  
• Memory Usage: Exceeds 95%
  
  
• Impact: The node will not be able to run additional VMs
  
  

The system recommends reducing the memory configured for VMs or expanding the capacity of the node.

But before immediately adding RAM, I think there is another important question to answer:

Is this really a capacity problem, or is the workload simply not distributed efficiently across the cluster?

Troubleshooting Approach

When a node reaches this level of memory utilization, I would avoid jumping directly to a hardware upgrade.

1. Check VM Memory Allocation

First, review the memory configuration of the VMs running on the affected node.

Look for:

• VMs with unnecessarily high memory allocation
  
  
• Recent increases in VM memory
  
  
• VMs that were provisioned with more resources than their workload actually requires
  
  

Configured memory should always be compared with actual workload requirements.

2. Check Actual Memory Consumption

A VM having a large amount of configured memory does not necessarily mean that it is actively using all of it.

Identify which VMs are consuming the most memory and determine whether the consumption is:

• Normal for the workload
  
  
• Consistently high
  
  
• Suddenly increased
  
  

This helps distinguish between normal resource demand and an abnormal change.

3. Check What Changed Before the Alert

One of the questions I always find useful during troubleshooting is:

“What changed before the problem started?”

For example:

• A new VM was deployed
  
  
• VM memory was increased
  
  
• A workload suddenly became heavier
  
  
• Several VMs were migrated to the same node
  
  

This can often provide more useful information than simply looking at the current utilization percentage.

4. Compare Memory Utilization Across Nodes

Next, check the other HCI nodes.

If the affected node is above 95% while other nodes still have significant available memory, this may indicate a workload distribution issue rather than an overall capacity shortage.

This is where the DRS configuration becomes relevant.

Possible Solution: Smart Resource Scheduling / DRS

During the investigation, I also found that Smart Resource Scheduling was not enabled.

The DRS interface provides two modes:

• Better Performance — focuses on improving overall service performance
  
  
• Less Cost — aims to increase VM density and reduce resource/cost consumption
  
  

For a situation where one node is heavily loaded while other nodes still have available resources, Better Performance would be the more relevant direction to evaluate.

If the cluster has sufficient spare capacity, DRS can help with workload balancing by moving workloads away from heavily utilized nodes.

However, I would still monitor the cluster after enabling the feature rather than assuming that DRS alone has solved the problem.

Common Mistake

A common mistake is seeing:

Memory Usage >95% → Add More RAM

That may solve the immediate capacity limitation, but it doesn't necessarily explain why the node reached that level in the first place.

If the actual problem is VM over-allocation or uneven workload distribution, adding hardware may only treat the symptom.

Likewise, enabling DRS is not a replacement for capacity planning.

If every node in the cluster is already heavily utilized, workload redistribution will not create additional capacity.

My Troubleshooting Sequence

For this type of alert, my preferred sequence would be:

Check Node → Check VM Allocation → Check Actual Usage → Compare Other Nodes → Check DRS → Identify Recent Changes → Decide on Resize / Migration / Capacity Expansion

This gives us a clearer picture before making a major change to the environment.

Key Takeaway

A >95% memory alert does not automatically mean that more RAM is the first answer.

Before expanding hardware, determine whether the issue is caused by:

VM sizing → workload distribution → recent changes → or genuine cluster capacity limitations.

Sometimes the right solution is to resize a VM or redistribute workloads.

Sometimes enabling DRS / Smart Resource Scheduling can help.

And sometimes, after all the checks, the correct answer really is:

The cluster needs more capacity.

The important part is making that decision based on evidence rather than the alert alone.

Thanks to share!