last week, I (referred to the author of this article) was fortunate to host a group discussion at the artificial intelligence hardware and marginal artificial intelligence summit, which is the memory challenge of the next generation of artificial intelligence/machine learning.Mlcommons's David KANTER, Microsoft's Brett Dodds and AMD's Nuwan Jayasena have joined my ranks. These three effective views have different views on the importance of AI/ML.Our discussion focuses on some challenges and opportunities facing DRAM and memory systems.As the performance requirements of AI/ML continue to grow rapidly, the importance of memory continues to increase.
    In fact, when talking about artificial intelligence memory, we see all the above needs, especially:
    more capacity -the model size is huge and grows rapidly.David quotes the embedded table used by Baidu in its recommendation system and requires 10 TB.Such a scale of assets require more and more DDR main memory capacity.
    More bandwidth -As the amount of data that needs to be moved is huge, we have witnessed that all DRAM types are constantly pursuing higher data rates to provide more memory bandwidth.
    lower latency -another aspect of this speed demand is lower latency, so that the processor will not wait for data idle.
    lower power consumption -unfortunately, we are facing the limits of physics, and power consumption has become an important limit factor in the artificial intelligence system.The demand for higher data rates is pushing high power consumption.
    In order to alleviate this problem, the IO voltage is decreasing, but this will reduce the chance of voltage and increase errors. This makes us ...
    higher reliability -In order to solve the increasing error rate under higher speed, lower voltage, and smaller process geometric conditions, we see more and more use of ECC and advanced signal technology on the filmCome on compensation.
    another important topic we discuss is the challenges and opportunities of new storage technology in artificial intelligence.New technologies have many potential benefits, including:
    optimize capacity, bandwidth, delay and power consumption for a group of key cases.Artificial intelligence is a huge and important market. There are a lot of funds behind it. This is a great combination that can promote the development of new storage technology.In the past, GDDR (for the development of graphics market), LPDDR (for the development of mobile market) and HBM (for high -bandwidth application development including AI) were created to meet the needs of the unable to meet the needs of the case.
    CXL? - CXL provides the opportunity to greatly expand memory capacity and increase bandwidth, and also abstracts the type of memory from the processor.In this way, CXL provides a good interface for integrating new memory technology.The CXL memory controller provides a conversion layer between the processor and memory, allowing a new memory layer after the memory connected locally.
    Although new memory types for specific cases may be beneficial to many applications, they face additional challenges:
    DRAM, SRAM and flash in the foreseeable future will continue to exist, so do not expect anything to completely replace these technologies.The annual R & D and capital expenditure investment of these technologies, coupled with decades of high -yield manufacturing experience, is basically impossible to replace any technologies in the short term.Any new memory technology must cooperate well with these memory to be adopted.
    The scale of artificial intelligence deployment and risks related to the development of new memory technology make it difficult to use new memory.Memory development is usually 2-3 years, but the development of artificial intelligence is so fast that it is difficult to predict specific functions that may be needed in the future.The gambling injection is high, and the risk of relying on new technologies is also high.
    The performance advantage of any new technology must be high enough to offset any additional costs and risks.Considering the requirements of infrastructure engineering and deployment teams, this means that new memory technology needs to overcome a very high obstacle.
    Memory will continue to become a key promotion factor for future artificial intelligence systems.Our industry must continue to innovate the future system to provide faster and more meaningful artificial intelligence, and the entire industry is responding.