CLOUDADV: Decision-Aligned Instance Sizing with Zero-Shot Foundation Models under Drift

The recent arXiv paper on CLOUDADV tackles a persistent and costly problem in cloud computing: the tendency for virtual machine instances to be overprovisioned. The authors propose an interactive advisory system that leverages zero-shot foundation models to recommend optimal instance sizes, specifically designed to handle the challenge of workload drift—where usage patterns change over time, rendering static sizing decisions obsolete.

What Happened

The CLOUDADV system departs from traditional rightsizing approaches that rely on historical data and manual threshold tuning. Instead, it uses zero-shot foundation models—pre-trained on vast, general datasets—to infer resource requirements from workload characteristics without needing task-specific fine-tuning. The "decision-aligned" aspect means the system prioritizes sizing recommendations that align with the engineer’s operational goals (e.g., cost savings vs. performance guarantees), rather than just predicting resource usage. Crucially, it incorporates drift detection to flag when workload patterns have shifted enough to invalidate previous recommendations, prompting re-evaluation.

Why It Matters

Cloud overspending remains a top concern for enterprises, with many organizations wasting 30-40% of their cloud budget on idle or underutilized resources. Traditional auto-scaling and rightsizing tools often fail under drift because they are reactive or rely on stale baselines. CLOUDADV’s use of foundation models is notable for two reasons. First, it reduces the engineering overhead of building and maintaining bespoke sizing models for every workload type. Second, the zero-shot capability means the system can be deployed immediately on new, unseen workloads without a lengthy training period. This is a practical step toward making AI-driven infrastructure optimization more accessible and adaptive.

For AI practitioners, this work also highlights a shift in how foundation models are applied: not just for generative tasks or NLP, but for operational decision-making in systems engineering. The interactive, engineer-facing design acknowledges that human judgment remains critical—the system advises rather than automates blindly, which is prudent given the high stakes of mis-sizing production instances.

Implications for AI Practitioners

Practitioners building similar advisory systems should note the importance of drift detection as a first-class component. Without it, any static model—even a powerful foundation model—will degrade. The paper also underscores the value of aligning model outputs with user-defined objectives rather than raw prediction accuracy. An engineer may prefer a slightly over-provisioned instance if it guarantees latency SLAs, and the system must respect that trade-off.

However, the reliance on zero-shot models raises questions about edge cases: how well do these models generalize to highly specialized or legacy workloads that are underrepresented in training data? Practitioners will need to validate performance on their specific infrastructure before trusting recommendations in production.

Key Takeaways

• CLOUDADV uses zero-shot foundation models to recommend cloud instance sizes, reducing the need for custom model training per workload.
• The system explicitly handles workload drift, a common failure point for static rightsizing tools.
• Its decision-aligned design lets engineers balance cost and performance based on operational priorities, not just resource predictions.
• AI practitioners should consider drift-aware, interactive advisory systems as a practical alternative to fully automated optimization, especially in high-stakes cloud environments.