One of the highlights of Hot Chips 2019 was the startup Cerebras showcasing its product – a large ‘wafer-scale’ AI chip that was literally the size of a wafer. The chip itself was rectangular, but it was cut from a single wafer, and contained 400,000 cores, 1.2 trillion transistors, 46225 mm2 of silicon, and was built on the 16 nm process of TSMC.
The whole thing created a very big buzz, and later that year the company unveiled the first system, CS-1, which is a 15U unit to send one chip. Power consumption was in the 15 kW range, and the unit cost a few million. They are already deployed by research institutions.
Initially, Cerebras was set to announce their 2nd generation product here at Hot Chips this year, however, the timeline has not been precisely tuned, so the company is continuing its software procedures instead. But at the end of the slide deck there is a special slide with some details about the next generation.
Of course, you can not just add more die area when doing waffle scale, so the only way is to optimize star area per core and take advantage of smaller process nodes. That means for TSMC 7nm, there are now 850,000 cores and 2.6 trillion transistors. Cerebras had to develop new technologies to deal with multi-reticle designs, but they succeeded with the first gene, and transferred the teachings to the new chip. We expect more details about this new product later this year.
