12k Accesses

210 Citations

57 Altmetric

6 Mentions

Explore all metrics

Abstract

Current advances in Artificial Intelligence (AI) and Machine Learning have achieved unprecedented impact across research communities and industry. Nevertheless, concerns around trust, safety, interpretability and accountability of AI were raised by influential thinkers. Many identified the need for well-founded knowledge representation and reasoning to be integrated with deep learning and for sound explainability. Neurosymbolic computing has been an active area of research for many years seeking to bring together robust learning in neural networks with reasoning and explainability by offering symbolic representations for neural models. In this paper, we relate recent and early research in neurosymbolic AI with the objective of identifying the most important ingredients of neurosymbolic AI systems. We focus on research that integrates in a principled way neural network-based learning with symbolic knowledge representation and logical reasoning. Finally, this review identifies promising directions and challenges for the next decade of AI research from the perspective of neurosymbolic computing, commonsense reasoning and causal explanation.

Access this article

Subscribe and save

Springer+

from $39.99 /Month

• Starting from 10 chapters or articles per month
• Access and download chapters and articles from more than 300k books and 2,500 journals
• Cancel anytime

View plans

Buy Now

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

The use of the word representation here is intended to be broader than its use in representation learning in that the choice of vector space precedes the learning of an embedding. 1.

Several reports on the future of AI and its consequences for society have been published in the mainstream media over the past decade. These include the BBC, NY Times, The Observer and others, see e.g., https://www.bbc.com/news/technology-37713629. 1.

Both types of knowledge can exhibit imprecision and may require the modelling of uncertainty, e.g., through the use of probability theory. However, commonsense knowledge is implicit and the conclusions derived from it may need to be retracted frequently in the face of new evidence, whereas expert knowledge is expected to produce over time conclusions with high confidence. 1.

Although knowledge extraction from large networks may not be complete, it should be sound and measurable (d’Avila Garcez et al. 2001; White and d’Avila Garcez 2019). One of the most interesting areas of recent progress addresses the emergence of symbols and their meaning from trained neural networks (Ngan et al. 2022). 1.

The recent developments in Large Language Models, including OpenAI’s ChatGPT, among others is related to our point. Humans in the loop, conversing and symbolically interacting with AI systems is an avenue that received attention recently under the umbrella of reinforcement learning with human feedback (RLHF) (Zhu et al. 2023). One can classify RLHF under the taxonomy of Neurosymbolic systems, as described by Kautz (2022).

References

Arabshahi F, Lu Z, Singh S, Anandkumar A (2019) Memory augmented recursive neural networks. https://arxiv.org/abs/1911.01545

Bader S, Hitzler P, Hölldobler S, Witzel A (2007) The core method: Connectionist model generation for first-order logic programs. In: Hammer B, Hitzler P (eds) Perspectives of neural-symbolic integration. Springer, Berlin, pp 205–232. https://doi.org/10.1007/978-3-540-73954-8_9

Bengio Y, Deleu T, Rahaman N, Ke NR, Lachapelle S, Bilaniuk O, Goyal A, Pal CJ (2020) A meta-transfer objective for learning to disentangle causal mechanisms. In: ICLR

Cameron C, Chen R, Hartford J, Leyton-Brown K (2020) Predicting propositional satisfiability via end-to-end learning. In: AAAI

Carvalho BW, Garcez AD, Lamb LC (2022) Graph-based neural modules to inspect attention-based architectures: a position paper. In AAAI fall symposium 2022. Arlington, Virginia. https://doi.org/10.48550/ARXIV.2210.07117. https://arxiv.org/abs/2210.07117

Chaudhuri S, Ellis K, Polozov O, Singh R, Solar-Lezama A, Yue Y (2021) Neurosymbolic programming. Found Trends Program Lang 7(3):158–243

Article Google Scholar

Chen J, Batmanghelich K (2019) Weakly supervised disentanglement by pairwise similarities. https://arxiv.org/abs/1906.01044

Chomsky N (1956) Three models for the description of language. IRE Trans Inf Theory 2(3):113–124. https://doi.org/10.1109/TIT.1956.1056813

Article MATH Google Scholar

d’Avila Garcez AS, Besold T, de Raedt L, Földiák P, Hitzler P, Icard T, Kühnberger K, Lamb LC, Miikkulainen R, Silver D (2015) Neural-symbolic learning and reasoning: Contributions and challenges. In: AAAI Spring Symposia. http://www.aaai.org/ocs/index.php/SSS/SSS15/paper/view/10281

d’Avila Garcez AS, Jiménez-Ruiz E (eds) (2022) Proceedings of the 16th international workshop on neural-symbolic learning and reasoning. Cumberland Lodge, Windsor Great Park, Sept 28–30. CEUR workshop proceedings, vol 3212. CEUR-WS.org, Aachen. http://ceur-ws.org/Vol-3212

d’Avila Garcez AS, Lamb LC (2003) Reasoning about time and knowledge in neural symbolic learning systems. In: NIPS, pp 921–928

d’Avila Garcez AS, Zaverucha G (1999) The connectionist inductive learning and logic programming system. Appl Intell 11(1):59–77. https://doi.org/10.1023/A:1008328630915

Article Google Scholar

d’Avila Garcez AS, Broda K, Gabbay DM (2001) Symbolic knowledge extraction from trained neural networks: a sound approach. Artif Intell 125(1–2):155–207. https://doi.org/10.1016/S0004-3702(00)00077-1

Article MathSciNet MATH Google Scholar

d’Avila Garcez A, Broda K, Gabbay DM (2002) Neural-symbolic learning systems: foundations and applications. Springer, Berlin

Book MATH Google Scholar

d’Avila Garcez AS, Lamb LC, Gabbay DM (2009) Neural-symbolic cognitive reasoning. Springer, Berlin-Heidelberg

MATH Google Scholar

d’Avila Garcez AS, Gori M, Lamb LC, Serafini L, Spranger M, Tran S (2019) Neural-symbolic computing: an effective methodology for principled integration of machine learning and reasoning. FLAP 6(4):611–632

MathSciNet Google Scholar

De Raedt L, Kimmig A, Toivonen H (2007) Problog: a probabilistic prolog and its application in link discovery. In: Proceedings of the 20th international joint conference on artifical intelligence. IJCAI’07, pp 2468–2473. Morgan Kaufmann Publishers Inc., San Francisco. http://dl.acm.org/citation.cfm?id=1625275.1625673

Evans R, Grefenstette E (2018) Learning explanatory rules from noisy data. JAIR 61:1–64

Article MathSciNet MATH Google Scholar

Fagin R, Halpern JY, Moses Y, Vardi MY (2003) Reasoning about knowledge. MIT Press, Cambridge

MATH Google Scholar

Giunchiglia E, Stoian MC, Lukasiewicz T (2022) Deep learning with logical constraints. In: IJCAI-ECAI 2022. IJCAI/AAAI Press, Vienna

Gori M (2018) Machine learning: a constraint-based approach. Morgan Kaufmann, Burlington

MATH Google Scholar

Hammer B, Hitzler P (eds) (2007) Perspectives of neural-symbolic integration. Springer, Berlin

MATH Google Scholar

Hinton GE, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527

Article MathSciNet MATH Google Scholar

Hochreiter S (2022) Toward a broad AI. Commun ACM 65(4):56–57

Article Google Scholar

Huang Q, Smolensky P, He X, Deng L, Wu D (2017) A neural-symbolic approach to natural language tasks. https://arxiv.org/abs/1710.11475

Hu Z, Ma X, Liu Z, Hovy E, Xing E (2016) Harnessing deep neural networks with logic rules. In: ACL

Jumper J, Evans R, et al (2020) High accuracy protein structure prediction using deep learning. In: 14th critical assessment of techniques for protein structure prediction, CASP-14

Kahneman D (2011) Thinking. Fast and slow. Farrar, Straus and Giroux, New York

Google Scholar

Kautz HA (2022) The third AI summer: AAAI Robert S. Engelmore memorial lecture. AI Mag 43(1):105–125. https://doi.org/10.1002/aaai.12036

Article Google Scholar

Kleinberg JM, Ludwig J, Mullainathan S, Sunstein CR (2020) Algorithms as discrimination detectors. Proc Natl Acad Sci USA 117(48):30096–30100. https://doi.org/10.1073/pnas.1912790117

Article MathSciNet MATH Google Scholar

Kosiorek AR, Sabour S, Teh YW, Hinton GE (2019) Stacked capsule autoencoders. In: NeurIPS 2019, pp 15486–15496. http://papers.nips.cc/paper/9684-stacked-capsule-autoencoders

Lamb LC, d’Avila Garcez AS, Gori M, Prates M, Avelar P, Vardi MY (2020) Graph, neural networks meet neural-symbolic computing: a survey and perspective. IJCAI 2020:4877–4884

Google Scholar

Lample G, Charton F (2020) Deep learning for symbolic mathematics. In: ICLR. https://openreview.net/forum?id=S1eZYeHFDS

LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

Article Google Scholar

Lévy P (2013) The semantic sphere 1: computation, cognition and information economy. ISTE. Wiley, Hoboken. https://books.google.co.uk/books?id=EIhS9DqwLkgC

Lloyd JW (2003) Logic for learning - learning comprehensible theories from structured data. Cognitive Technologies. Springer, Berlin

MATH Google Scholar

Manhaeve R, Dumancic S, Kimmig A, Demeester T, De Raedt L (2018) Deepproblog: neural probabilistic logic programming. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) NIPS 31, Montreal, pp 3749–3759

Mao J, Gan C, Kohli P, Tenenbaum J, Wu J (2019) The neuro-symbolic concept learner: interpreting scenes, words, and sentences from natural supervision. In: ICLR

Marcus G (2020) The next decade in AI: four steps towards robust artificial intelligence. https://arxiv.org/abs/1801.00631

Marra G, Diligenti M, Giannini F, Gori M, Maggini M (2020) Relational neural machines. https://arxiv.org/abs/2002.02193

Marra G, Giannini F, Diligenti M, Gori M (2019) Lyrics: a general interface layer to integrate logic inference and deep learning. In: Joint European conference on machine learning and knowledge discovery in databases, Springer, pp 283–298

McCarthy J (1988) Epistemological challenges for connectionism. Behav Brain Sci 11(1):44–44. https://doi.org/10.1017/S0140525X0005264X

Article Google Scholar

Minervini P, Bosnjak M, Rocktäschel T, Riedel S, Grefenstette E (2020) Differentiable reasoning on large knowledge bases and natural language. In: AAAI, pp 5182–5190

Ngan KH, Garcez AD, Townsend J (2022) Extracting meaningful high-fidelity knowledge from convolutional neural networks. In: 2022 international joint conference on neural networks (IJCNN), pp 1–17 . https://doi.org/10.1109/IJCNN55064.2022.9892194

Page M (2000) Connectionist modelling in psychology: a localist manifesto. Behav Brain Sci 23(4):443–467. https://doi.org/10.1017/S0140525X00003356

Article Google Scholar

Pearl J (2019) The seven tools of causal inference, with reflections on machine learning. Commun ACM 62(3):54–60. https://doi.org/10.1145/3241036

Article Google Scholar

Prates MOR, Avelar PHC, Lemos H, Lamb LC, Vardi MY (2019) Learning to solve NP-complete problems: a graph neural network for decision TSP. In: AAAI

Raedt LD, Kersting K, Natarajan S, Poole D (2016) Statistical relational artificial intelligence: logic, probability, and computation. Synthesis lectures on artificial intelligence and machine learning. Morgan & Claypool, San Rafael

Book MATH Google Scholar

Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1–2):107–136

Article MATH Google Scholar

Riegel R, Gray A, Luus F, Khan N, Makondo N, Akhalwaya I, Qian H, Fagin R, Barahona F, Sharma U, Ikbal S, Karanam H, Neelam S, Likhyani A, Srivastava S (2020) Logical neural networks. https://arxiv.org/abs/2006.13155

Rocktäschel T, Riedel S (2017) End-to-end differentiable proving. https://arxiv.org/abs/1705.11040

Roemmele M, Bejan CA, Gordon AS (2011) Choice of plausible alternatives: an evaluation of commonsense causal reasoning. In: AAAI spring symposia. Stanford

Schlag I, Schmidhuber J (2018) Learning to reason with third-order tensor products. In: NeurIPS

Serafini L, d’Avila Garcez AS (2016) Logic tensor networks: deep learning and logical reasoning from data and knowledge. https://arxiv.org/abs/1606.04422

Socher R, Chen D, Manning C, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: NIPS, pp 926–934

Stehr M-O, Kim M, Talcott CL (2022) A probabilistic approximate logic for neuro-symbolic learning and reasoning. J Log Algebr Methods Program 124:100719. https://doi.org/10.1016/j.jlamp.2021.100719

Article MathSciNet MATH Google Scholar

Tavares AR, Avelar PHC, Flach JM, Nicolau M, Lamb LC, Vardi MY (2020) Understanding boolean function learnability on deep neural networks. https://arxiv.org/abs/2009.05908

Tran S, d’Avila Garcez A (2018) Deep logic networks: inserting and extracting knowledge from deep belief networks. IEEE TNNLS 29:246–258. https://doi.org/10.1109/TNNLS.2016.2603784

Article MathSciNet Google Scholar

Valiant LG (2003) Three problems in computer science. J ACM 50(1):96–99

Article Google Scholar

White A, d’Avila Garcez A (2019) Measurable counterfactual local explanations for any classifier. https://arxiv.org/abs/1908.03020

Xu J, Zhang Z, Friedman T, Liang Y, den Broeck GV (2018) A semantic loss function for deep learning with symbolic knowledge. In: Dy JG, Krause A (eds) ICML. Proceedings of machine learning research, vol 80, Stockholm

Yang F, Yang Z, Cohen WW (2017) Differentiable learning of logical rules for knowledge base reasoning. In: NIPS, pp 2319–2328. http://papers.nips.cc/paper/6826-differentiable-learning-of-logical-rules-for-knowledge-base-reasoning.pdf

Zhu B, Jiao J, Jordan MI (2023) Principled reinforcement learning with human feedback from pairwise or (K)-wise comparisons. https://doi.org/10.48550/ARXIV.2301.11270. https://arxiv.org/abs/2301.11270

Download references

Acknowledgements

We would like to thank Gary Marcus, Francesca Rossi, Pascal Hitzler, Marco Gori, Kristian Kersting, Pierre Lévy, Luc de Raedt, Alessandra Russo, Kristian Kersting, Murray Shanahan, Alexander Gray, Moshe Vardi and the anonymous reviewers who helped us improve this manuscript. Luis Lamb is supported in part by CNPq and CAPES - Finance Code 001.

Author information

Authors and Affiliations

Data Science Institute, City, University of London, Northampton Square, London, EC1V 0HB, UK

Artur d’Avila Garcez 1.

Institute of Informatics, Federal University of Rio Grande do Sul, Av. Bento Gonçalves 9500, Porto Alegre, RS, 91501-970, Brazil

Luís C. Lamb

Authors

1. Artur d’Avila Garcez
2. Luís C. Lamb

Corresponding author

Correspondence to Luís C. Lamb.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article

Garcez, A.d., Lamb, L.C. Neurosymbolic AI: the 3rd wave. Artif Intell Rev 56, 12387–12406 (2023). https://doi.org/10.1007/s10462-023-10448-w

Download citation

Published: 15 March 2023

Version of record: 15 March 2023

Issue date: November 2023

DOI: https://doi.org/10.1007/s10462-023-10448-w

Keywords