Emerging AI accelerators increasingly adopt wafer-scale integration, combining hundreds of thousands of cores with\u00a0massive on-chip memory and ultra-high bandwidth. Yet, existing LLM inference systems\u2014designed primarily for GPUs\u2014cannot fully exploit this architecture. In this talk, I will present WaferLLM, the first LLM inference system designed specifically for wafer-scale accelerators. WaferLLM introduces new approaches for wafer-scale prefill and decode parallelism, KV-cache\u00a0management, and high-performance kernels\u2014MeshGEMM and MeshGEMV\u2014to\u00a0maximise hardware utilisation. On commodity hardware (Cerebras WSE-2), WaferLLM achieves 2,700 tokens per second for a single user, translating to less than one millisecond per token and demonstrating its potential for efficient scaling in test-time compute.<\/p>\n\n\n\n
Luo\u00a0Mai is a Reader (Associate Professor) at the University of Edinburgh, where he leads the Large-Scale\u00a0Machine Learning Systems Group and co-directs the UK EPSRC Centre for Doctoral Training in\u00a0Machine Learning Systems. His research spans systems, AI, and data, and has been recognised with multiple research and rising-star awards from academia and industry. He has co-authored a textbook on ML systems and co-founded several widely used open-source AI system libraries. Previously, he worked at Imperial College London and Microsoft Research and received his PhD from Imperial College London with support from a Google Doctoral Fellowship.<\/p>\n\n\n\n
2 March 2026<\/strong> Abstract Emerging AI accelerators increasingly adopt wafer-scale integration, combining hundreds of thousands of cores with\u00a0massive on-chip memory and ultra-high bandwidth. Yet, existing LLM inference systems\u2014designed primarily for GPUs\u2014cannot fully exploit this architecture. In this talk, I will present WaferLLM, the first LLM inference system designed specifically for wafer-scale accelerators. WaferLLM introduces new approaches for wafer-scale […]<\/p>\n","protected":false},"author":1149,"featured_media":3969,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[154],"tags":[],"class_list":{"0":"post-3967","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-talks","8":"czr-hentry"},"jetpack_featured_media_url":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-content\/uploads\/sites\/14\/2026\/05\/20260302_171231820_iOS-scaled.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/posts\/3967","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/users\/1149"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/comments?post=3967"}],"version-history":[{"count":2,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/posts\/3967\/revisions"}],"predecessor-version":[{"id":3971,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/posts\/3967\/revisions\/3971"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/media\/3969"}],"wp:attachment":[{"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/media?parent=3967"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/categories?post=3967"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.qub.ac.uk\/dipsa\/wp-json\/wp\/v2\/tags?post=3967"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Ashby Building, Queen’s University Belfast<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"