A Survey on Hardware Accelerators for Large Language Models

Large Language Models (LLMs) have emerged as powerful tools for naturallanguage processing tasks, revolutionizing the field with their ability tounderstand and generate human-like text. As the demand for more sophisticatedLLMs continues to grow, there is a pressing need to address the computationalchallenges associated with their scale and complexity. This paper presents acomprehensive survey on hardware accelerators designed to enhance theperformance and energy efficiency of Large Language Models. By examining adiverse range of accelerators, including GPUs, FPGAs, and custom-designedarchitectures, we explore the landscape of hardware solutions tailored to meetthe unique computational demands of LLMs. The survey encompasses an in-depthanalysis of architecture, performance metrics, and energy efficiencyconsiderations, providing valuable insights for researchers, engineers, anddecision-makers aiming to optimize the deployment of LLMs in real-worldapplications.

Further reading#

• Access Paper in arXiv.org