Information-Theoretic Measures in AI: A Practical Decision Guide

arXiv:2604.23716v1 Announce Type: new Abstract: Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation learning...