Title:Adapting Neural Audio Codecs to EEG
Abstract:EEG and audio are inherently distinct modalities, differing in sampling rate, channel structure, and scale. Yet, we show that pretrained neural audio codecs can serve as effective starting points for EEG compression, provided that the data are preprocessed to be suitable to the codec’s input constraints. Using DAC, a state-of-the-art neural audio codec as our base, we demonstrate that raw EEG can be mapped into the codec’s stride-based framing, enabling direct reuse of the audio-pretrained encoder-decoder. Even without modification, this setup yields stable EEG reconstructions, and fine-tuning on EEG data further improves fid…
Title:Adapting Neural Audio Codecs to EEG
Abstract:EEG and audio are inherently distinct modalities, differing in sampling rate, channel structure, and scale. Yet, we show that pretrained neural audio codecs can serve as effective starting points for EEG compression, provided that the data are preprocessed to be suitable to the codec’s input constraints. Using DAC, a state-of-the-art neural audio codec as our base, we demonstrate that raw EEG can be mapped into the codec’s stride-based framing, enabling direct reuse of the audio-pretrained encoder-decoder. Even without modification, this setup yields stable EEG reconstructions, and fine-tuning on EEG data further improves fidelity and generalization compared to training from scratch. We systematically explore compression-quality trade-offs by varying residual codebook depth, codebook (vocabulary) size, and input sampling rate. To capture spatial dependencies across electrodes, we propose DAC-MC, a multi-channel extension with attention-based cross-channel aggregation and channel-specific decoding, while retaining the audio-pretrained initialization. Evaluations on the TUH Abnormal and Epilepsy datasets show that the adapted codecs preserve clinically relevant information, as reflected in spectrogram-based reconstruction loss and downstream classification accuracy.
| Comments: | Foundation Models for the Brain and Body (BrainBodyFM@NeurIPS) |
| Subjects: | Machine Learning (cs.LG); Sound (cs.SD) |
| Cite as: | arXiv:2511.23142 [cs.LG] |
| (or arXiv:2511.23142v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2511.23142 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ard Kastrati [view email] [v1] Fri, 28 Nov 2025 12:47:05 UTC (6,800 KB)