What are IBM HCDIMMs and HP HDIMMs ?

Comparing RDIMMs, LRDIMMs and HCDIMMs/HDIMMs

UPDATE: 07/06/2012 – VMware certifies Netlist as sole memory vendor

Adding this as an entry point (or crash course) for folks asking about HCDIMMs and HDIMMs on twitter.

IBM HCDIMMs are Netlist HyperCloud.

HP HDIMMs are Netlist HyperCloud (called “HP Smart Memory HyperCloud”).

HP Smart Memory includes some other diagnostic/error features, and so that feature set has been extended to the HyperCloud and it is therefore also called a “HP Smart Memory”.

There may be some repetition, as I have tried to keep each section readable on it’s own.

Usage

Depending on how much memory you need, you may choose to populate the DIMM slots on your motherboard at 1 DPC, 2 DPC or 3 DPC (DPC = DIMMs per channel).

At 16GB memory module sizes:

– 1 DPC and 2 DPC – use 16GB RDIMMs (2-rank)
– 3 DPC – you should use HyperCloud – since 16GB RDIMMs (2-rank) experience speed slowdown at 3 DPC

If you are planning to upgrade to 3 DPC eventually, you may consider getting the HyperCloud from the start.

At 32GB memory module sizes:

– 3 DPC, 2 DPC and possibly 1 DPC also – require HyperCloud – since 32GB RDIMM 4-rank experience abysmal slowdown

32GB RDIMMs will only be available in 4-rank (because 2-rank requires 8Gbit DRAM die which will not be available for a few years) and 4-rank suffers abysmal performance at not just 3 DPC but also 2 DPC and even 1 DPC (see IBM doc below).

Thus the use for HyperCloud will expand from 3 DPC at 16GB – to 3 DPC and 2 DPC and possibly even 1 DPC at 32GB memory module levels.

In the future, HyperCloud (currently being copied in LRDIMMs) will be licensed for DDR4 – as DDR4 copies the load reduction and rank multiplication features that LRDIMM has taken from HyperCloud, but will also incorporate the symmetrical lines and distributed buffer chipset on the HyperCloud.

Therefore HyperCloud is essentially a smooth continuation onto DDR4, while LRDIMMs are “end-of-life” with their asymmetrical lines (cause of the latency issues) and centralized buffer chipset.

Viability of LRDIMMs vs. HyperCloud

LRDIMMs have high latency issues and fail to achieve 1333MHz at 3 DPC – which leads to this summary (see other posts for why):

– 16GB LRDIMMs are not viable vs. 16GB RDIMMs (2-rank)
– 16GB LRDIMMs are not viable vs. 16GB HyperCloud
.
– 32GB LRDIMMs ARE viable vs. 32GB RDIMMs (4-rank) (this is only because these will only be available in 4-rank for a while as an 8Gbit DRAM die will not be available for a few years))
– 32GB LRDIMMs are not viable vs. 32GB HyperCloud

In addition HyperCloud underpins both LRDIMMs and the upcoming DDR4 (which copies NLST’s IP even more closely – with the asymmetrical lines and centralized buffer chipset). See other articles here for that.

HyperCloud vs LRDIMMs vs RDIMMs

– LRDIMMs require a BIOS modification before the motherboard can understand LRDIMMs (Romley motherboards qualified for LRDIMMs should have gotten this BIOS update at the factory)
.
– HyperCloud is plug and play and requires no BIOS updates of the motherboard (demonstrated on pre-Romley systems)
.
.
– LRDIMMs have a “5 ns latency penalty” compared to RDIMMs (from Inphi LRDIMM blog).
.
– NLST HyperCloud have similar latency as RDIMMs (a huge advantage) and have a rather significant “4 clock latency improvement” over the LRDIMM (quote from Netlist Craig-Hallum conference)
.
– Cisco UCS has a “6 ns latency penalty” compared to RDIMMs (just read this figure on a blog – not sure of it’s accuracy) – listing this for comparison as Cisco UCS has now switched away from their ASIC-on-motherboard solution which suggests they are going to be using the LRDIMM/HyperCloud solutions which are ASIC-on-memory-module solutions for “load reduction”
.
.
– LRDIMMs cannot deliver 1333MHz at 3 DPC (peak at 1066MHz at 3 DPC)
.
– NLST HyperCloud delivers 1333MHz at 3 DPC
.
.
– LRDIMMs are not interoperable with standard RDIMMs
.
– NLST HyperCloud are interoperable with standard RDIMMs (even though IBM and HP are marketing the HyperCloud in all-HyperCloud configurations – perhaps because you get maximum load reduction this way)
.
.
– LRDIMMs have legal risk (Inphi currently the only maker of LRDIMM buffer chipsets – others have backed off – lost a challenge of Netlist IP at the USPTO – as a result the Netlist patents have become stronger and are going to come back and bite Inphi in Netlist vs. Inphi which was stayed pending these patent reexaminations – patents which survive reexamination can never again be challenged in court – NLST patents ‘537 and ‘274 survived with ALL claims intact which is a powerful statement on the strength of their IP – Inphi has appealed to the BPAI but the USPTO decision is telling)
.
– NLST HyperCloud invented “load reduction” and “rank multiplication” – as well as VLP memory if my understanding is correct from their earlier days – and hold significant IP in this area (if you recall MetaRAM from some years ago – they conceded to NLST and went out of business because they were found to be copying NLST IP)
.
.
– LDIMMs are “end-of-life” with their asymmetrical lines and centralized buffer chipset
.
– NLST HyperCloud symmetrical lines and distributed buffer chipset is being copied by DDR4 (as that is the only way DDR4 can scale to the higher speeds required)
.
.
– 32GB LRDIMMs are expensive (much more than 2x the 16GB LRDIMMs)
.
– 32GB HyperCloud will be cheaper than 32GB LRDIMMs also – because the 32GB LRDIMMs use 4Gbit x 2 (DDP) while NLST 32GB HyperCloud uses 4Gbit (monolithic) and leverages their Planar-X IP to make 32GB HyperCloud

Viability of HyperCloud vs. RDIMMs

UPDATE: added 05/29/2012

Netlist HyperCloud offers “load reduction” and “rank multiplication” (NLST IP that is being copied in the LRDIMMs and in future DDR4).

NLST HyperCloud have similar latency as RDIMMs (a huge advantage) and have a rather significant “4 clock latency improvement” over the LRDIMM (quote from Netlist Craig-Hallum conference).

However if a simpler 2-rank RDIMM can deliver the same speed, it is better to buy the RDIMM (unless you plan on upgrading to 3 DPC later – see below).

At higher memory loading the 2-rank RDIMMs will experience slowdown (at 3 DPC).

The 4-rank RDIMMs will experience slowdown at not just 3 DPC, but 2 DPC and possibly 1 DPC (see IBM docs below).

– 2-rank RDIMMs ARE preferable over HyperCloud at 1 DPC and 2 DPC (RDIMMs speed is ok)
– 2-rank RDIMMs are not preferable over HyperCloud at 3 DPC (RDIMMs speed is slow)

– 4-rank RDIMMs are not preferable over HyperCloud at 3 DPC, 2 DPC and possibly 1 DPC (4-rank speed is slow)

16GB RDIMMs are available at 2-rank these days (thanks to availability of 4Gbit DRAM die).

32GB RDIMMs are NOT available at 2-rank, but ONLY in 4-rank (and this situation will persist for a few years because 8Gbit DRAM die is required to make 2-rank and this die will not be available for a few years).

This means that:

– 16GB RDIMMs (2-rank) ARE preferable over 16GB HyperCloud at 1 DPC and 2 DPC
– 16GB RDIMMs (2-rank) are not preferable over 16GB HyperCloud at 3 DPC (RDIMM slows down)
.
– 32GB RDIMMs (4-rank) are not preferable over 16GB HyperCloud at 3 DPC, 2 DPC, 1 DPC (4-rank slows down)
.
– none of the LRDIMMs (16GB or 32GB) are viable vs. RDIMMs/HyperCloud

Market ownership for RDIMM, HyperCloud, LRDIMM

Which means that a portion of the 16GB market (1 DPC and 2 DPC) belongs to RDIMMs.

And a portion of the 16GB market (3 DPC) belongs to HyperCloud.

And a more significant part of the 32GB market (3 DPC, 2 DPC and possibly 1 DPC also) belongs to HyperCloud.

At the higher speeds required for DDR4, a “load reduction” solution will be essential.

HyperCloud will be required at DDR4 – as the higher speeds of DDR4 will make the problem worse – load reduction will be required to minimize the load on the memory bus, and the rank multiplication will be required so dense memory modules can be made from less dense memory while exhibiting the same 2-rank (virtual rank) on the bus.

By the logic in this article, none of the market should belong to LRDIMMs (there may be marketing or lack of information which may allow it’s sale).

Bottom Line – more than 256GB requires HyperCloud

On a 2-socket server (2 processors on the motherboard), each Romley processor has 4 memory channels (up from 3 of pre-Romley). On each memory channel if you use 3 DPC (3 DIMMs per channel) you have:

2 sockets x 4 memory channels per socket x 3 DIMMs per channel = 24 DIMM sockets
or
8 x 3 DPC = 24 DIMM sockets.

Using 16GB or 32GB memory modules at 1 DPC, 2 DPC and 3 DPC (i.e. various levels of loading on the memory bus):

8 x 3 DPC = 24 DIMM sockets – 384GB (16GB) – 768GB (32GB)
8 x 2 DPC = 16 DIMM sockets – 256GB (16GB) – 512GB (32GB)
8 x 1 DPC = 8 DIMM sockets – 128GB (16GB) – 256GB (32GB)

Which means that if you need less than 256GB on a 2-socket Romley server, you can buy 16GB RDIMMs (2-rank) – use them at 1 DPC or 2 DPC.

And if you need MORE than 256GB on a 2-socket Romley server, you need 16GB or 32GB HyperCloud – use them at 3 DPC (using 16GB HyperCloud) or 3 DPC, 2 DPC or even 1 DPC (using 32GB HyperCloud).
.

HP “exclusive” deal for HyperCloud

IBM and HP did deals some time back for Netlist (NLST) HyperCloud. The HP deal was an “exclusive” deal of some sort for a certain period of time (undisclosed).

The “exclusive” nature of the HP deal for HyperCloud may have been related to the development of the “HP Smart Memory” version of HyperCloud (i.e. integrate some of the HP functionality into HyperCloud memory modules).

But it could also be related (more likely since a time period fits this more) to HP offering HyperCloud as a exclusive FIO (Factory Installed Option) – it is only offered at 24 DIMMs on the HP DL360p and HP DL380p (high volume virtualization servers) i.e. “fully loaded” with memory.

IBM and HP (exclusive) deal with NLST:

http://www.secinfo.com/dVut2.p1nj.htm
Netlist Inc · 10-K · For 12/31/11
Filed On 2/28/12 5:22pm ET · SEC File 1-33170 · Accession Number 1047469-12-1788

Collaborative Agreements

In 2011, the Company entered into two memory technology Collaboration Agreements. The first agreement is a HyperCloud™ Technology Collaboration Agreement (the “IBM Agreement”) with International Business Machines (“IBM”). Under the IBM Agreement, IBM and the Company have agreed to cooperate with respect to the qualification of HyperCloud™ technology for use with IBM servers and to engage in certain joint marketing efforts if qualification is achieved. IBM and the Company have agreed to commit resources and funds in support of these activities. The IBM Agreement is non-exclusive.

The second agreement is a Collaboration Agreement (the “HP Agreement”) with Hewlett-Packard Company (“HP”). Under the HP Agreement, HP and the Company have agreed to cooperate with respect to the qualification of HyperCloud™ technology for use with HP servers and to engage in certain joint marketing efforts if qualification is achieved. HP and the Company have agreed to commit resources and funds in support of these activities. The HP Agreement is exclusive for a period of time. HP and the Company agree to collaborate on the future use of HyperCloud™ load reduction and rank multiplication technologies for next generation server memory for HP.

In 2011, the Company reimbursed IBM and HP $0.2 million and $0.1 million, respectively, for the cost of certain qualification activities. The payments are included in research and development expense in the accompanying consolidated statement of operations. The Company may be required to make additional payments to both IBM and HP for joint marketing activities if and when the HyperCloud™ technology is qualified. The Company’s net sales will be determined after deduction of such customer allowances, in accordance with ASC 605-50.

HP performance figures for RDIMMs, LRDIMMs, HCDIMMs

HP HDIMMs or “HP Smart Memory HyperCloud” is the only memory delivering 1333MHz at 3 DPC – and is available as a Factory Installed Option (FIO) fully loaded at 3 DPC (i.e. 24 DIMM slots):

http://h18004.www1.hp.com/products/quickspecs/14225_na/14225_na.html
Overview
HP SmartMemory

Load Reduced DIMMs (LRDIMM)
HP 32GB (1x32GB) Quad Rank x4 PC3L-10600L (DDR3-1333) Load Reduced CAS-9 Low Voltage Memory Kit 647903-B21

HyperCloud DIMMs (HDIMM)
HP 16GB (1x16GB) Dual Rank x4 PC3-10600H (DDR3-1333) HyperCloud CAS-9 FIO Memory Kit 678279-B21
NOTE: This is a Factory Installed Option (FIO) only.

Performance
Because HP SmartMemory is certified, performance tested and tuned for HP ProLiant, certain performance features are unique with HP SmartMemory. For example, while the industry supports DDR3-1333 RDIMM at 1.5V, today’s Gen8 servers support DDR3-1333 RDIMM up to 3 DIMMs per channel at 1066MT/s running at 1.35V. This equates to up to 20% less power at the DIMM level with no performance penalty and now with HyperCloud Memory on DL360p Gen8 and the DL380p Gen8 servers will support 3 DIMMs per channel at 1333MT/s running at 1.5 V. In addition, the industry supports UDIMM at 2 DIMMs per channel at 1066MT/s. HP SmartMemory supports 2 DIMMs per channel 1333MT/s, or 25% greater bandwidth.

WARNING – this HP document seems to have plausible speed figures (i.e. matches the IBM speed info and other data). However, some of the other HP docs I have seen seem to have egregious typos – UDIMMs running full speed at 3 DPC for instance.

HP resellers (lower prices)

Since it is a Factory Installed Option (FIO) it may not be buyable as a separate item – though these resellers seem to list it somehow:

http://h71016.www7.hp.com/option.asp?SuperCategoryID=34&SuperCatDesc=Memory&CategoryID=634&ComponentId=258722&ParentId=30&ParentDesc=&oi=E9CED&BEID=19701&SBLID=
HP 16GB (1x16GB) 2Rank x4 PC3-10600H-9 HC Memory
Part number
678279-B21

Price snapshot (05/29/2012): $549.00

http://shop.computer-trade.ch/en/products/1409601-678279-b21-hewlett-packard-hp-16gb-2rx4-pc3-10600h-9-hc-
Hewlett Packard HP 16GB 2Rx4 PC3-10600H-9 HC FIO Kit
HP – Memory – 16 GB – HCDIMM 240-pin – DDR3 – 1333 MHz / PC3-10600 – CL9 – registered – ECC – factory integrated – for ProLiant DL360p Gen8, DL380p Gen8

http://www.darestshop.ch/fr/products/600597-678279-b21-hewlett-packard-hp-16gb-2rx4-pc3-10600h-9-hc-f
Hewlett Packard HP 16GB 2Rx4 PC3-10600H-9 HC FIO Kit
HP 16GB 2Rx4 PC3-10600H-9 HC FIO Kit

http://www.abbo-shop.ch/en/products/4205882-678279-b21-hewlett-packard-hp-16gb-2rx4-pc3-10600h-9-hc-
Hewlett Packard HP 16GB 2Rx4 PC3-10600H-9 HC FIO Kit
HP – Memory – 16 GB – HCDIMM 240-pin – DDR3 – 1333 MHz / PC3-10600 – CL9 – registered – ECC – factory integrated – for ProLiant DL360p Gen8, DL380p Gen8 Hewlett Packard

http://www.ohc24.ch/de/products/1496992-678279-b21-hewlett-packard-hp-16gb-2rx4-pc3-10600h-9-hc-
Hewlett Packard HP 16GB 2Rx4 PC3-10600H-9 HC FIO Kit
HP – Memory – 16 GB – HCDIMM 240-pin – DDR3 – 1333 MHz / PC3-10600 – CL9 – registriert – ECC – werkseitig integriert – für ProLiant DL360p Gen8, DL380p Gen8 Hewlett Packard

IBM price list

http://www-304.ibm.com/shop/americas/webapp/wcs/stores/servlet/default/CategoryDisplay?catalogId=-840&storeId=1&langId=-1&dualCurrId=73&categoryId=4611686018425093824
Memory
Boost your xSeries system performance.

Earlier snapshot (March 2012):

Netlist 16GB HCDIMM:
00D4964 16GB (1x16GB, 1.5V)PC3-10600 CL9 ECC DDR3 1333MHz LP HyperCloud DIMM $779.00
.
16GB LRDIMM:
49Y1567 New! 16GB (1x16GB, 4Rx4, 1.35V) PC3L-10600 CL9 ECC DDR3 1333MHz LP LRDIMM $779.00
.
32GB LRDIMM (other IBM docs suggest “Available later in 2012”):
90Y3105 32GB (1x32GB, 4Rx4, 1.35V) PC3L-10600 CL9 ECC DDR3 1333MHz LP LRDIMM $4,499.00

Snapshot (05/29/2012):

16GB RDIMM – 1600MHz:
00D4968 16GB (1x16GB, 2Rx4, 1.5V) PC3-12800 CL11 ECC DDR3 1600MHz LP RDIMM $499.00
.
16GB RDIMM – 1333MHz:
49Y1563 16GB (1x16GB, 2Rx4, 1.35V) PC3L-10600 CL9 ECC DDR3 1333MHz LP RDIMM $459.00
.
16GB RDIMM – 1066MHz:
49Y1400 16GB (1x16GB, 4Rx4, 1.35V) PC3L-8500 CL7 ECC DDR3 1066MHz LP RDIMM $459.00
.
.
16GB HyperCloud:
00D4964 16GB (1x16GB, 1.5V)PC3-10600 CL9 ECC DDR3 1333MHz LP HyperCloud DIMM $575.00
.
.
16GB LRDIMM:
49Y1567 16GB (1x16GB, 4Rx4, 1.35V) PC3L-10600 CL9 ECC DDR3 1333MHz LP LRDIMM $575.00
.
32GB LRDIMM:
90Y3105 32GB (1x32GB, 4Rx4, 1.35V) PC3L-10600 CL9 ECC DDR3 1333MHz LP LRDIMM $4,399.00

IBM resellers (lower prices)

http://www.costcentral.com/proddetail/IBM_Memory/00D4964/11581855/11581855
IBM – Memory
Mfg Part #: 00D4964

Price snapshot (05/29/2012): $560.63

http://www.vecmar.com/IBM_memory_16_GB_DIMM_240-pin_DDR3/00D4964/p/S10401536#Realtime
IBM memory – 16 GB – HCDIMM 240-pin – DDR3
Mfg Part#: 00D4964

Price snapshot (05/29/2012): $575.00

http://www.glcomp.com/products/servers/ibm-systemx/x3650-m4/memory/ibm-16gb-1x-16-gb-2rx4-1-5-v-pc3-10600-1333-mhz-lp-hcdimm
IBM 16GB (1x 16 GB, 2Rx4, 1.5 V) PC3-10600 1333 MHz LP HCDIMM
Product SKU: 00D4964

Price snapshot (05/29/2012): $575.00

http://www.pcsuperstore.com/products/11581855-IBM-00D4964.html
IBM 16GB ECC LP DDR3 DIMM PC3-10600 1333MHZ HYPERCLOUD CL9 1.5V
IBM 00D4964
Manufacturer IBM
Manufacturer Part # 00D4964

Price snapshot (05/29/2012): $547.28

IBM performance figures for RDIMMs, LRDIMMs, HCDIMMs

IBM HCDIMMs (NLST HyperCloud) are the only memory delivering 1333MHz at 3 DPC (fully loaded memory).

http://www.redbooks.ibm.com/abstracts/tips0850.html
IBM System x3650 M4
IBM Redbooks Product Guide

Table 5. Maximum memory speeds:

RDIMM – dual-rank (2-rank) – at 1.5V
– 1 DPC at 1333MHz
– 2 DPC at 1333MHz
– 3 DPC at 1066MHz

RDIMM – quad-rank (4-rank) – at 1.5V
– 1 DPC at 1066MHz
– 2 DPC at 800MHz
– 3 DPC not supported (because 4 ranks x 3 DPC = 12 ranks which exceeds the 8 ranks per memory rank limit of current systems)

LRDIMM – at 1.5V
– 1 DPC at 1333MHz
– 2 DPC at 1333MHz
– 3 DPC at 1066MHz

HCDIMM – at 1.5V
– 1 DPC at 1333MHz at 1.5V
– 2 DPC at 1333MHz at 1.5V
– 3 DPC at 1333MHz at 1.5V

UPDATE: 07/06/2012 – VMware certifies Netlist as sole memory vendor

VMware certifies Netlist as the sole memory vendor for it’s products. The Netlist 16GB, 32GB HyperCloud (supplied by IBM/HP) and the Netlist 16GB VLP RDIMM (supplied by IBM) are the only memory products certified for use with VMware:

https://ddr3memory.wordpress.com/2012/07/05/memory-for-vmware-virtualization-servers/
Memory for VMware virtualization servers
July 5, 2012

Advertisements

9 Comments

Filed under Uncategorized

9 responses to “What are IBM HCDIMMs and HP HDIMMs ?

  1. Pingback: LRDIMMs future and end-user risk factors | ddr3memory

  2. Pingback: JEDEC fiddles with DDR4 while LRDIMM burns | ddr3memory

  3. Pingback: Why are LRDIMMs single-sourced by Inphi ? | ddr3memory

  4. Pingback: Why are 16GB LRDIMMs non-viable ? | ddr3memory

  5. Pingback: Memory buying guide – when to use RDIMMs ? | ddr3memory

  6. Pingback: Where are the 1600MHz LRDIMMs/HyperCloud for Romley ? | ddr3memory

  7. Pingback: Infographic – memory buying guide for Romley 2-socket servers | ddr3memory

  8. Pingback: Examining LRDIMMs | ddr3memory

  9. Pingback: Examining Netlist | ddr3memory

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s