{"id":4013,"date":"2026-06-15T14:32:20","date_gmt":"2026-06-15T13:32:20","guid":{"rendered":"https:\/\/blogs.qub.ac.uk\/dipsa\/?p=4013"},"modified":"2026-06-15T14:32:21","modified_gmt":"2026-06-15T13:32:21","slug":"configspec-configuration-selection-framework-for-edgecloud-llm-inference","status":"publish","type":"post","link":"https:\/\/blogs.qub.ac.uk\/dipsa\/configspec-configuration-selection-framework-for-edgecloud-llm-inference\/","title":{"rendered":"ConfigSpec at TDIS ’26: Configuration Selection Framework for Edge\u2013Cloud LLM Inference"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n
\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n<\/figure>\n\n\n\n
<\/div>\n\n\n\n

At the 4th International Workshop on Testing Distributed Internet of Things Systems (TDIS ’26),<\/strong> Babar Ali presented ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge\u2014Cloud Speculative LLM Serving. TDIS<\/strong> held in Edinburgh, Scotland, as part of the broader EUROSYS<\/strong> conference (April 27\u201330, 2026).<\/p>\n\n\n\n

TDIS focuses on the tools and frameworks for testing and evaluating distributed IoT systems, which is an increasingly critical area as the emerging applications are spreading the computations across the edge–cloud continuum. This year’s edition brought together researchers working on distributed systems, edge computing, and AI inference, making it an ideal venue for our work on collaborative LLM deployment across edge and cloud.<\/p>\n\n\n\n

What is ConfigSpec?<\/h3>\n\n\n\n

To remedy this, speculative decoding emerged as a promising solution, which enables collaborative inferencing. In this approach, a lightweight “draft” model on the edge device quickly proposes a sequence of candidate tokens, which are then sent to a powerful “target” model in the cloud for verification. The accepted tokens become part of the final response, and the rejected are discarded. Speculative decoding aids in distributing the work between edge and cloud while offering the response quality of the \u201ctarget\u201d model.<\/p>\n\n\n\n

If not carefully used, the promising Speculative decoding can yield delayed responses and escalate cost, energy, and memory consumption across the heterogeneous hardware. Because there is an enormous number of configuration choices to make: which draft model to deploy? What should be the quantization level? Which model family to prefer? How many tokens to speculate at a time? And on which hardware? Finding answers to select the best configuration can be time-consuming and costly, which leads us to present this work.<\/p>\n\n\n\n

ConfigSpec<\/strong> is our framework for navigating this space systematically. It profiles each edge device and draft model, measuring drafting speed, power draw, and how well the draft model aligns with the target model. From these measurements, it analytically evaluates three integral deployment objectives:<\/p>\n\n\n\n