Can In-Context Learning Support Intrinsic Curiosity?

The Missing Piece of the Puzzle: Data Selection as a Learned Behavior

A new arXiv preprint asks a deceptively simple question: can large sequence models learn not just from data, but also how to choose what data to learn from next? The paper frames this as "intrinsic curiosity" — the ability of a model to autonomously select training data that maximizes its own learning progress. While the abstract is brief, the implication is significant: we may have been treating data as a static resource when it could be a dynamic, model-driven process.

What This Research Actually Proposes

The core idea is to extend in-context learning beyond pattern completion into active data acquisition. Current large language models (LLMs) excel at processing whatever sequence they are given, but they have no agency over what they see next. This paper explores whether a model can use its own internal state — its uncertainty, its prediction errors, or its knowledge gaps — to decide which data points to request next during training or inference. This mirrors biological curiosity, where an organism seeks out stimuli that reduce uncertainty most efficiently.

The technical mechanism likely involves training the model to predict not just the next token, but also the expected learning gain from different possible inputs. The model would then select the input that maximizes this gain, creating a feedback loop between data selection and model improvement.

Why This Matters for AI Practitioners

If successful, this approach could fundamentally change how we think about data pipelines. Currently, data selection is a human-driven preprocessing step — we curate datasets, filter for quality, and balance distributions. This research suggests that the model itself could learn to perform these tasks dynamically, potentially discovering data selection strategies that humans would never design.

For practitioners, the most immediate implication is in active learning and data efficiency. In domains where data collection is expensive — medical imaging, scientific experiments, or proprietary business data — a model that can ask for the most informative next sample could dramatically reduce annotation costs. It also raises the possibility of self-improving systems that continuously seek out their own knowledge gaps during deployment.

However, there are clear limitations. The computational overhead of evaluating multiple candidate inputs during training or inference could be substantial. Moreover, "curiosity" without proper constraints could lead models to chase noise or adversarial examples. The paper likely addresses these concerns through regularization or by grounding curiosity in task-specific objectives.

Key Takeaways

• Data selection can be learned, not just engineered. This research explores whether in-context learning can extend to choosing what data to learn from, moving beyond passive pattern matching.
• Potential for dramatic efficiency gains. If models can autonomously identify high-value training data, practitioners could reduce annotation costs and improve sample efficiency in data-scarce domains.
• Computational cost is a major hurdle. Evaluating multiple data candidates during training or inference adds overhead that must be carefully managed for practical deployment.
• Curiosity requires guardrails. Without proper constraints, autonomous data selection could lead models to exploit noise or adversarial inputs rather than genuinely informative samples.