Under a Creative Commons license

Open access

Abstract

AI models have been expanding dramatically in size and the number of trainable parameters. This rapid growth has introduced many challenges, including increased computational costs and inefficiencies. Dynamic sparse training has emerged as a novel approach to address overparameterization and achieve energy-efficient artificial neural network (ANN) architectures. The highly efficient neuro-inspired sparse design remains underexplored compared to the significant focus on random topology searches. We propose the Topographical Sparse Mapping (TSM) method, inspired by the vertebrate visual system and convergent units. TSM introduces a sparse input layer for MLPs, significantly reducing the number of parameters. Unlike conventional approaches that focus on optimising sparse connectivity patterns through complex computations, our work introduces a biologically inspired sparse connectivity scheme that naturally enhances performance without the need for intricate optimisation. Notably, the number of connections is determined solely by the number of input features, independent of the number of neurons in the receiving layer. An enhanced version of TSM (ETSM) incorporates additional pruning during training to achieve a desired reduction in parameters. This streamlined framework surpasses several state-of-the-art sparse training methods, offering superior accuracy, generalization, and training efficiency. Remarkably, ETSM overcomes the conventional trade-off between simplicity and accuracy, achieving improvements in both simultaneously. Additional experiments further demonstrate that topographically structured input mapping accelerates convergence and enhances final accuracy compared to unstructured pruning. ETSM introduces a novel perspective on computationally efficient ANN design, underscoring the value of topographically sparse connectivity. These findings emphasize the potential of leveraging neurobiological principles to effectively address overparameterization challenges in ANN development.

Keywords

Sparsification

Topographical mapping

Artificial neural networks

MLP

Biologically-inspired

Data availability

No data was used for the research described in the article.

Mohsen Kamelian Rad is a Ph.D. candidate at the Computer Science Research Centre, University of Surrey, UK, where he began his doctoral studies in 2024. His research lies at the intersection of computational neuroscience and artificial intelligence, with a particular focus on applying neuroscientific principles to develop sustainable and efficient AI systems. During his master’s studies, he published four research papers in leading journals, including Journal of Mathematical Biology and Journal of Computational Neuroscience. These publications primarily investigated pain-relieving mechanisms through selective external electrical nerve stimulations designed to target specific neural fibers, as well as map-based modeling of the electrical activities of neuronal cells in response to electromagnetic radiation. Building upon this foundation, his current doctoral work explores how insights from neural dynamics and computational models can inform the next generation of adaptive and biologically inspired AI architectures.

Prof. Ferrante Neri earned his Laurea and Ph.D. degrees in Electrical Engineering from Politecnico di Bari, Italy, in 2002 and 2007, respectively. He also obtained a second Ph.D. in Scientific Computing and Optimisation and a Docent qualification in Computational Intelligence from the University of Jyväskylä, Finland, in 2007 and 2010. From 2009 to 2014, he was an Academy Research Fellow with the Academy of Finland, leading a significant project on Algorithmic Design Issues in Memetic Computing. His academic journey continued at De Montfort University, Leicester, UK (2012–2019), and the University of Nottingham, UK (2019-2022). Since 2022, he has held the role of Full Professor of Machine Learning and Artificial Intelligence at the University of Surrey, Guildford, where he also serves as Associate Dean (International) for the Faculty of Engineering and Physical Sciences. Additionally, he is a Jiangsu Distinguished Professor at Nanjing University of Information Science and Technology, China. With more than two decades in academia, Professor Neri has contributed to undergraduate, postgraduate, and doctoral education across four countries. He is an HEA Senior Fellow, and his educational impact includes a popular textbook on Linear Algebra for Computational Sciences and Engineering, translated into Chinese. Professor Neri is internationally recognized in his field and consistently ranks among the top 2% of scientists worldwide (Stanford World Ranking of Scientists) in Artificial Intelligence and Image Processing.

Dr. Sotiris Moschoyiannis earned his Ph.D. in Theoretical Computer Science at the University of Surrey. He is currently a Reader in Complex Systems and the Postgraduate Research Director (PGRD) in the School of Computer Science and Electronic Engineering at the University of Surrey. Dr. Moschoyiannis’s research is deeply interdisciplinary, blending mathematical methods with computational techniques to understand and control complex networks and dynamical systems. He has been involved in several EU and UK funded research projects, and his work has been supported by prestigious bodies like the Engineering and Physical Sciences Research Council (EPSRC), the European Commission, UK Research and Innovation (UKRI), the UK National Cyber Security Centre (NCSC), and the British Council. Dr Moschoyiannis served on the editorial board of BCS’s The Computer Journal and Elsevier’s Simulation Modelling Practice and Theory. He was Programme Chair of the DeclarativeAI Int’l Conference in 2021. He is in the organisation committee of the Complex Networks series of Int’l conferences for the last 4 years. He also serves in the Executive Committee of IEEE Cloud Computing and IEEE Industrial Informatics, and an Associate Editor of Cloud Computing (Springer).

Dr. Bauer is a Senior Lecturer and the Head of the Nature Inspired Computing and Engineering (NICE) research group at the University of Surrey, UK. He received his Bachelor’s and master’s degree in computational science and engineering from ETH Zuerich, Switzerland. Afterwards, he did his doctoral studies at the Institute for Neuroinformatics (ETH Zürich/Uni Zürich). After postdoctoral fellowships at Newcastle University (UK), he moved to the University of Surrey in 2020, where he leads the COMBYNE research lab. He currently serves on the editorial boards of PLoS Computational Biology and Frontiers in Computational Neuroscience. With a background in Computational Science and Engineering and a Ph.D. in Neuroinformatics from ETH Zurich (Switzerland), Dr Bauer’s research is highly interdisciplinary, at the intersection of computational modeling, neuroscience, and artificial intelligence, with a focus on the brain. His research focuses on developing computational models to better understand complex biological systems such as neural networks of the brain. He integrates multi-scale biological data with computational simulations to capture complex system dynamics. His interdisciplinary approach bridges gaps between experimental biology and computational science, fostering new avenues for collaboration across disciplines. Notably, he is co-founder and spokesperson of the international BioDynaMo collaboration, which has created the open-source and high-performance agent-based modelling software BioDynaMo (www.biodynamo.org).

© 2025 The Authors. Published by Elsevier B.V.