Installing memory on 2-socket servers – memory mathematics

Background – 2-sockets, memory channels and DPC

On a 2-socket server (2 processors on the motherboard), each Romley processor has 4 memory channels (up from 3 of pre-Romley). On each memory channel if you use 3 DPC (3 DIMMs per channel) you have:

2 sockets x 4 memory channels per socket x 3 DIMMs per channel = 24 DIMM sockets
or
8 x 3 DPC = 24 DIMM sockets.

Using 16GB or 32GB memory modules at 1 DPC, 2 DPC and 3 DPC (i.e. various levels of loading on the memory bus):

8 x 3 DPC = 24 DIMM sockets – 384GB (16GB) – 768GB (32GB)
8 x 2 DPC = 16 DIMM sockets – 256GB (16GB) – 512GB (32GB)
8 x 1 DPC = 8 DIMM sockets – 128GB (16GB) – 256GB (32GB)

As you go from 1 DPC to 2 DPC to 3 DPC, you are increasing the (electrical) load of the memory on the memory channel.

At 3 DPC you have maximum memory load on the memory channel.

As memory load increases the achievable bandwidth goes down – so for example at 3 DPC the standard speed is NOT achievable but is a speed grade lower.

If standard memory speed is 1333MT/s (mega transfers per second – alternatively also sometimes written 1333MHz), then at 3 DPC this drops to 1066MHz or 800MHz.

For example:

– 1 DPC at 1333MHz
– 2 DPC at 1066MHz
– 3 DPC at 800MHz

This is a basic problem with memory for servers i.e. if you add lots of memory, it runs slower.