Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models

arXiv:2605.06510v1 Announce Type: cross Abstract: Transformer-based tabular foundation models (TFMs) dominate small to medium tabular predictive benchmark tasks, yet their inference mechanisms remain largely unexplored. We present the first large-scale mechanistic study of layerwise dynamics in 6...