Temporal Feature Extractors in EEG Foundation Models: A Controlled Comparison Including a Pretrained Time-Series Model

The Great Feature Extractor Bake-Off in EEG AI

A new preprint from arXiv (2606.30104) has quietly dropped one of the more methodologically rigorous comparisons in the EEG foundation model space. The study systematically evaluates how different temporal feature extractors perform when plugged into EEG foundation models, with a specific focus on whether pretrained time-series foundation models (TSFMs) can serve as effective backbones for brain-signal processing.

The core experiment is deceptively simple: hold everything constant except the temporal encoder. The authors compare standard convolutional architectures, transformer-based extractors, and crucially, a pretrained TSFM that was originally developed for general time-series forecasting (not EEG). The results are revealing—and somewhat deflating for the hype around universal time-series models.

Why This Matters Beyond the Bench

The EEG foundation model field has been plagued by a "kitchen sink" problem. Teams throw transformers, convolutions, and attention mechanisms at brain data without isolating which component actually drives performance. This study provides a much-needed control: it tells us whether the temporal backbone matters more than the pretraining data or the model scale.

The finding that general TSFMs underperform specialized EEG temporal extractors is significant but not surprising. Brain signals have unique statistical properties—non-stationarity, rhythmic oscillations, and event-related potentials—that generic time-series models weren't designed to capture. A model pretrained on stock prices or weather data simply doesn't "see" the same temporal structure as one trained on neural recordings.

More interesting is the implication that EEG-specific temporal inductive biases (like those in convolutional architectures designed for spectral decomposition) still outperform attention-heavy alternatives at equivalent parameter counts. This suggests that the field hasn't yet hit the scaling regime where transformers automatically win.

Implications for AI Practitioners

For teams building EEG applications, this paper offers three actionable insights:

First, don't assume that larger, more general pretrained models will transfer well to neural data. The "foundation model" label doesn't guarantee superiority for brain signals. Second, the temporal feature extractor choice is a first-order design decision—not a detail to be optimized later. Third, the results suggest that hybrid architectures (convolutional front-ends with attention back-ends) may currently represent the optimal trade-off between inductive bias and flexibility.

The study also implicitly validates the continued investment in EEG-specific pretraining datasets. If general TSFMs don't transfer well, then domain-specific foundation models remain necessary—and the bottleneck shifts from architecture design to data collection and curation.

Key Takeaways

• General TSFMs underperform specialized EEG temporal extractors in brain-signal tasks, suggesting that domain-specific inductive biases remain critical for neural data processing.
• Architecture choices matter more than model scale at current parameter counts—convolutional extractors with spectral inductive biases still beat pure attention alternatives.
• Hybrid architectures (convolutional + attention) appear to offer the best current trade-off between capturing EEG-specific temporal structure and maintaining representational flexibility.
• The field still needs more EEG-specific pretraining data, not just bigger general time-series models, to advance foundation model performance for brain-computer interfaces and clinical applications.