VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing

The Unseen Energy Crisis in LLM Inference

The latest preprint from arXiv (2509.04827) introduces VoltanaLLM, a system designed to tackle a growing but often overlooked problem in large language model deployment: the enormous and inefficient energy consumption of inference. While much of the industry focus has been on training costs, inference—the actual act of running models for users—now dominates operational expenses at scale. VoltanaLLM proposes a two-pronged approach: adaptive frequency control for hardware and state-space routing for workload distribution.

The core insight is that LLM serving is not monolithic. The "prefill" phase (processing the input prompt) and the "decode" phase (generating tokens one by one) have fundamentally different computational profiles. Prefill is compute-bound and memory-intensive, while decode is latency-sensitive and memory-bandwidth-bound. Existing systems treat both phases uniformly, wasting energy by running hardware at full throttle even when it is unnecessary. VoltanaLLM dynamically adjusts CPU and memory frequencies to match the actual demands of each phase, and uses a state-space model to route requests to the most energy-appropriate server node without violating service-level objectives (SLOs).

Why This Matters for AI Practitioners

For teams running production LLM services, energy costs are no longer a secondary concern. A single large-scale deployment can consume megawatts of power, translating into millions of dollars annually. More critically, energy inefficiency directly impacts throughput and latency. When servers are running at maximum frequency for decode operations that do not require it, heat builds up, throttling occurs, and overall system performance degrades. VoltanaLLM’s approach offers a path to decouple performance from power consumption.

The state-space routing component is particularly significant. Traditional load balancers treat all requests equally, but VoltanaLLM learns the energy and latency characteristics of different request types and server states. This means practitioners could serve more users with the same hardware budget, or reduce their carbon footprint without sacrificing user experience. For organizations operating under strict SLOs—such as real-time chat applications or API services—this is a practical solution rather than a theoretical optimization.

Implications for System Design

VoltanaLLM signals a broader shift in how we think about LLM infrastructure. The disaggregation of prefill and decode is already gaining traction in the research community, but VoltanaLLM adds an energy-aware dimension that has been missing. Practitioners should consider whether their current serving stacks (vLLM, TensorRT-LLM, etc.) expose the granularity needed for such adaptive control. If not, the next generation of serving frameworks will likely need to incorporate frequency scaling and intelligent routing as first-class features.

The paper also highlights a tension: optimizing for energy can conflict with optimizing for raw throughput. VoltanaLLM’s SLO-aware design is a necessary compromise, but it requires careful tuning. AI engineers should expect to invest in monitoring and profiling to reap the benefits.

Key Takeaways

• Energy efficiency is now a first-order concern for LLM inference, not just training. VoltanaLLM shows that significant savings are possible by treating prefill and decode phases differently.
• Adaptive frequency control and state-space routing are practical, implementable techniques that can reduce power consumption without violating latency guarantees.
• Practitioners should evaluate their current serving infrastructure for support of phase-aware optimization. The disaggregated model is likely to become standard in future systems.
• Energy optimization requires trade-offs with throughput and complexity. Teams must invest in profiling and monitoring to deploy these techniques effectively.