Researchers develop hardware device to make AI more energy efficient

• In the design, data remains within the computational random-access memory, eliminating the need for extensive data movement.
• CRAM-based machine learning inference accelerator potentially achievies energy savings of up to 2,500 times compared to conventional methods.
• Advancement of CRAM architecture marks a critical evolution in computing, resolving the traditional von Neumann architecture’s bottleneck between memory and computation.

Engineering experts at the University of Minnesota Twin Cities have developed a hardware device that can lower he energy usage for AI processes by a factor of at least 1,000.

The findings, published in the journal npj Unconventional Computing, reveal  the breakthrough holds the potential to transform the landscape of energy-efficient computing due to the soaring demand for AI technologies,.

As AI applications proliferate, the quest for methods that enhance energy efficiency while maintaining high performance and low costs has become increasingly urgent.

Traditional AI processes typically involve extensive data transfer between logic units, where processing occurs, and memory, where data is stored. The transfer is not only power-intensive but also a significant contributor to overall energy consumption.

Enhancing energy efficiency

The introduction of a novel model known as computational random-access memory (CRAM) addresses this critical issue. In the design, data remains within the memory array, eliminating the need for extensive data movement.

Yang Lv, a postdoctoral researcher in electrical and computer engineering and the paper’s first author, emphasises the groundbreaking nature of the research.

“This work is the first experimental demonstration of CRAM,” he said, highlighting how processing occurs directly within the memory grid, streamlining operations and significantly enhancing energy efficiency.

The implications of the research are substantial. A report from the International Energy Agency forecasts that global energy consumption for AI could soar from 460 terawatt-hours (TWh) in 2022 to an estimated 1,000 TWh by 2026, mirroring the total electricity consumption of Japan.

Given this trajectory, the CRAM-based machine learning inference accelerator presents a much-needed solution, potentially achieving energy savings of up to 2,500 times compared to conventional methods.

A critical evolution in computing

The development of CRAM has been a long journey, rooted in over two decades of innovative research. Jian-Ping Wang, the senior author and a Distinguished McKnight Professor, recalls how the initial idea of utilising memory cells directly for computing was met with scepticism two decades ago.

He credits the collaboration among a diverse interdisciplinary faculty and a dedicated team of students for the eventual success of this ambitious project. Their combined efforts have led to a practical demonstration of this energy-efficient technology.

Furthermore, this research builds upon Wang’s prior accomplishments in Magnetic Tunnel Junctions (MTJs), which have been instrumental in advancements in microelectronics. The advancement of CRAM architecture marks a critical evolution in computing, resolving the traditional von Neumann architecture’s bottleneck between memory and computation.

 As Ulya Karpuzcu, co-author and Associate Professor, explains, CRAM’s flexibility allows for computation to be performed anywhere in the memory array, optimising performance for various AI algorithms while minimising energy requirements.