SoK: Attack and Defense Landscape of Mobile On-device AI Systems

The Emerging Security Frontier in On-Device AI

The publication of this Systematization of Knowledge (SoK) paper on arXiv marks a critical milestone in understanding the security posture of mobile on-device AI (MoAI) systems. As AI inference shifts from cloud servers to smartphones, tablets, and IoT devices, the attack surface expands in ways that traditional mobile security models were never designed to address. This research systematically catalogs both attack vectors and defensive strategies, providing the first comprehensive taxonomy for this rapidly maturing field.

Why This Matters Now

The timing is significant. Major smartphone manufacturers now ship devices with dedicated neural processing units (NPUs), and frameworks like Apple Core ML, Google ML Kit, and Qualcomm AI Engine are enabling increasingly complex models to run locally. This architectural shift brings undeniable privacy and latency benefits—no data leaves the device—but it also introduces novel vulnerabilities. Unlike cloud AI, where security teams control the infrastructure, on-device models are physically accessible to attackers who can perform side-channel attacks, model extraction, or adversarial input manipulation directly on the hardware.

The paper’s contribution lies in mapping the full attack landscape: from adversarial examples that exploit the model’s decision boundaries, to model inversion attacks that reconstruct training data, to hardware-level exploits targeting NPU memory management. For the first time, practitioners have a structured view of how these threats interact with the unique constraints of mobile environments—limited compute, battery sensitivity, and heterogeneous hardware.

Implications for AI Practitioners

For engineers deploying MoAI systems, this research underscores that security cannot be an afterthought. The paper likely reveals that many current defenses—like input sanitization or model obfuscation—are insufficient against determined adversaries with physical device access. Practitioners should expect to implement layered defenses: runtime integrity checks, differential privacy for training data, and secure enclave processing for sensitive model components.

The findings also have direct implications for model architecture choices. Smaller, quantized models that run efficiently on-device may be more vulnerable to certain attacks than their larger cloud counterparts, simply because they have less representational capacity to absorb perturbations. This creates a tension between performance optimization and security robustness that practitioners must navigate carefully.

Perhaps most importantly, this SoK establishes a baseline for future research. As on-device AI becomes the default for applications ranging from health monitoring to financial services, the security community now has a shared vocabulary and threat model to build upon. The paper likely identifies significant gaps in current defenses—particularly around hardware-software interface attacks—that will drive both academic research and commercial security products in the coming years.

Key Takeaways

• Mobile on-device AI introduces a fundamentally different threat model than cloud AI, with physical device access enabling side-channel and hardware-level attacks that cloud systems do not face
• Current defensive measures are fragmented and often inadequate; practitioners should plan for multi-layered security approaches rather than relying on single countermeasures
• The tension between model efficiency (quantization, pruning) and robustness to adversarial inputs presents a new design trade-off that requires careful evaluation
• This systematization provides the first comprehensive framework for classifying MoAI attacks and defenses, establishing a critical foundation for both researchers and engineers building secure on-device AI systems