Abstract:Recent successes in image analysis with deep neural networks are achieved almost exclusively with Convolutional Neural Networks (CNNs), typically trained using the backpropagation (BP) algorithm. In a 2022 preprint, Geoffrey Hinton proposed the Forward-Forward (FF) algorithm as a biologically inspired alternative, where positive and negative examples are jointly presented to the network and training is guided by a locally defined goodness function. Here, we extend the FF paradigm to CNNs. We introduce two spatially extended labeling strategies, based on Fourier patterns and morphological transformations, that enable convolutional layers to access label information across all spatial positions. On CIFAR10, we show that deepe…
Abstract:Recent successes in image analysis with deep neural networks are achieved almost exclusively with Convolutional Neural Networks (CNNs), typically trained using the backpropagation (BP) algorithm. In a 2022 preprint, Geoffrey Hinton proposed the Forward-Forward (FF) algorithm as a biologically inspired alternative, where positive and negative examples are jointly presented to the network and training is guided by a locally defined goodness function. Here, we extend the FF paradigm to CNNs. We introduce two spatially extended labeling strategies, based on Fourier patterns and morphological transformations, that enable convolutional layers to access label information across all spatial positions. On CIFAR10, we show that deeper FF-trained CNNs can be optimized successfully and that morphology-based labels prevent shortcut solutions on dataset with more complex and fine features. On CIFAR100, carefully designed label sets scale effectively to 100 classes. Class Activation Maps reveal that FF-trained CNNs learn meaningful and complementary features across layers. Together, these results demonstrate that FF training is feasible beyond fully connected networks, provide new insights into its learning dynamics and stability, and highlight its potential for neuromorphic computing and biologically inspired learning.
| Comments: | PEER-REVIEWED VERSION PUBLISHED ON “SCIENTIFIC REPORTS” (2025) DOI: https://doi.org/10.1038/s41598-025-26235-2 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2312.14924 [cs.CV] |
| (or arXiv:2312.14924v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2312.14924 arXiv-issued DOI via DataCite | |
| Journal reference: | Sci Rep 15, 38461 (2025) |
| Related DOI: | https://doi.org/10.1038/s41598-025-26235-2 DOI(s) linking to related resources |
Submission history
From: Riccardo Scodellaro [view email] [v1] Fri, 22 Dec 2023 18:56:35 UTC (3,616 KB) [v2] Mon, 25 Dec 2023 05:47:20 UTC (3,626 KB) [v3] Sun, 7 Jan 2024 10:18:53 UTC (3,613 KB) [v4] Tue, 4 Nov 2025 13:24:09 UTC (3,017 KB)