Abstract:We consider the reconstruction of a codeword from multiple noisy copies that are independently corrupted by insertions, deletions, and substitutions. This problem arises, for example, in DNA data storage. A common code construction uses a concatenated coding scheme that combines an outer linear block code with an inner code, which can be either a nonlinear marker code or a convolutional code. Outer decoding is done with Belief Propagation, and inner decoding is done with the Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm. However, the BCJR algorithm scales exponentially with the number of noisy copies, which makes it infeasible to reconstruct a codeword from more than about four copies. In this work, we introduce BCJRFormer, a tran…
Abstract:We consider the reconstruction of a codeword from multiple noisy copies that are independently corrupted by insertions, deletions, and substitutions. This problem arises, for example, in DNA data storage. A common code construction uses a concatenated coding scheme that combines an outer linear block code with an inner code, which can be either a nonlinear marker code or a convolutional code. Outer decoding is done with Belief Propagation, and inner decoding is done with the Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm. However, the BCJR algorithm scales exponentially with the number of noisy copies, which makes it infeasible to reconstruct a codeword from more than about four copies. In this work, we introduce BCJRFormer, a transformer-based neural inner decoder. BCJRFormer achieves error rates comparable to the BCJR algorithm for binary and quaternary single-message transmissions of marker codes. Importantly, BCJRFormer scales quadratically with the number of noisy copies. This property makes BCJRFormer well-suited for DNA data storage, where multiple reads of the same DNA strand occur. To lower error rates, we replace the Belief Propagation outer decoder with a transformer-based decoder. Together, these modifications yield an efficient and performant end-to-end transformer-based pipeline for decoding multiple noisy copies affected by insertion, deletion, and substitution errors. Additionally, we propose a novel cross-attending transformer architecture called ConvBCJRFormer. This architecture extends BCJRFormer to decode transmissions of convolutional codewords, serving as an initial step toward joint inner and outer decoding for more general linear code classes.
| Comments: | 16 pages, 19 figures, a shortened version was published in the ISIT 2025 conference |
| Subjects: | Information Theory (cs.IT); Machine Learning (cs.LG) |
| Cite as: | arXiv:2511.00999 [cs.IT] |
| (or arXiv:2511.00999v1 [cs.IT] for this version) | |
| https://doi.org/10.48550/arXiv.2511.00999 arXiv-issued DOI via DataCite (pending registration) | |
| Journal reference: | J. Streit, F. Weindel and R. Heckel, “Transformer-Based Decoding in Concatenated Coding Schemes Under Synchronization Errors,” 2025 IEEE International Symposium on Information Theory (ISIT), Ann Arbor, MI, USA, 2025, pp. 1-6 |
| Related DOI: | https://doi.org/10.1109/ISIT63088.2025.11195598 DOI(s) linking to related resources |
Submission history
From: Julian Streit [view email] [v1] Sun, 2 Nov 2025 16:22:08 UTC (323 KB)