Hierarchically Robust Zero-shot Vision-language Models

arXiv:2604.18867v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) can perform zero-shot classification but are susceptible to adversarial attacks. While robust fine-tuning improves their robustness, existing approaches align fixed text embeddings with an image embedding, sacrificing...