My Understanding about NUMA


NUMA mean non-uniformed memory access. So by default all processor has uniform access to memory. NUMA became more popular with number of cores per socket started rising.Since multi-core system in simple term is nothing but to put more in less. If you have Single Socket quad core machine, it means it has 4 cores. So it is not single processor which need access to memory but 4 processors(cores) will need access to memory. In old day only on processor(Car) use to travel on memory bus now there are 4 cores(cars) travelling on same memory bus. Now imagine 4 Socket Quad core machine i.e 16 cores(car) travelling on same road.So option was either to widen the bus(road) or limit the traffic on the bus(road). Later was easier and so NUMA was born. With NUMA memory controller is attached per socket and processor access to memory is done using high speed bus i.e. limiting the processors(cars) accessing the bus(road). High speed bus AMD calls it as Hypertransport and Intel calls it QuickPath.

Important pre-requisite before vSphere starts utilizing the NUMA.

  1. Node interleaving should be disabled in the bios. By default it is enabled (which means NUMA is disabled)
  2. There should be minimum 2 nodes and each node should have at least 2 cores. In processor terminology you need to have 2 Socket Dual core processor. NUMA will be not enabled on this machines.

Things to remember

  • Virtual machine SMP should be either less or equal to the number of cores present in the node (Node Size) for efficient utilization of NUMA architecture
  • If Virtual machine SMP is more than the Node Size, then NUMA won’t manage this Virtual machine and will not be benefitted.
  • every 2 sec, virtual machines are re-evaluated to see if the node change is going to benefit

If workload is imbalanced, ESX will migrate VM across the node.The algorithm used to migrate Virtual machine is based on amount of memory Virtual machine is accessing in it’s home node and overall priority of VM.Any new memory pages requested from VM are created on new home node and for accessing old pages it must traverse bus to the old node. To reduce the impact, page migration across the nodes takes place at 25pages/sec

For more information & Reference: Network Computing

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s