RobotAnalyst

Introduction

RobotAnalyst was developed as part of the Supporting Evidence-based Public Health Interventions using Text Mining project to support the literature screening phase of systematic reviews. RobotAnalyst is designed for searching and screening reference collections obtained from literature database queries. It combines search engine functionality with machine learning and text mining technology, including topic modelling and relevancy feedback-based text classification models, to minimise the human workload involved in identifying relevant references.

Features

RobotAnalyst offers the following functionality:

• Create a new collection: Upload a collection of references in RIS format into the system.
• Update an existing collection: Upload further references or import from PubMed.
• Faceted Search: Search a collection for relevant studies by applying various filters based on keywords, multiword terms, authors, publication year or journal name.
• Cluster-based search: Navigate large collections by browsing automatically generated clusters each labelled by descriptive keywords. Choose between different clustering algorithms and the number of clusters.
• Topic-based search: Identify relevant topics by a keyword search or browse a network graph of the topics. Each topic is associated with a set of references pertaining to the topic.
• Similarity-based search: Find the references that are the most similar to a particular reference of interest, based on the topics it contains.
• Mark, annotate and view screening decisions: Mark references for inclusion or exclusion, add notes to references and retrieve results based on screening decisions using the notes to filter the results.
• Screening with relevancy feedback: Prioritise references by relevancy predictions (inclusion confidence) from RobotAnalyst’s text classification functionality and update the predictive model after making new screening decisions.
• Cluster-based screening: Combine relevancy predictions with clustering to screen only the clusters with the highest proportion of references predicted to be relevant. Alternatively, screen clusters that are predicted to be irrelevant and exclude the irrelevant references.
• Quality assurance: Re-examine screening decisions, especially those whose predictions deviate from the system’s predictions, using RobotAnalyst’s filtering and sorting functionality.
• Exporting results: Export the set of references for any search or screening filters (RIS format).
• Finishing screening: Estimate the recall rate for relevant references and automatically finalise screening by applying the predictive model’s suggestions to any references without manual screening.

Short Demo

Duration: 2 mins 21 secs

Long Demo

Duration: 18 mins 42 secs (with voice-over)

Statistics

Current Users

Many of the world’s most dynamic healthcare institutions, academic labs, and scientists use RobotAnalyst for searching and screening reference collections for systematic reviews.

Example organizations

• University of Manchester
• The National Institute for Health and Care Excellence
• University of Oxford
• NHS Wales
• UC Davis School of Medicine
• Ted Rogers Center for Heart Research

User Testimonials

15th July 2025

"RobotAnalyst is an immensely helpful and user-friendly tool for reviews. I was able to use RobotAnalyst to scan thousands of hits from my database search so that I could identify the most relevant articles in just a few minutes. I’ve used RobotAnalyst for literature reviews, scoping reviews, and even systematic reviews, and it’s saved me hours upon hours of work."

— Dr. Gloria Sclar, Implementation Scientist at the MaineHealth Institute for Research

News

12-13th September 2018

NaCTeM will be exhibiting RobotAnalyst at the G-I-N 2018 conference at the Principal Hotel in Manchester on 12th and 13th September, 2018. The G-I-N conference gathers together people who work with guidelines in health and care.

29th March 2018

RobotAnalyst has been selected amongst 23 applicants as one of three winners of UMIP’s Next Big Thing competition, receiving £10,000 to help with commercialisation of the tool.

Evaluation

RobotAnalyst has been evaluated by systematic reviewers to measure its performance and gather feedback. NaCTeM presented the initial evaluation results at the Global Evidence Summit (GES) Cape Town, South Africa, Sept. 13–16, 2017. The workshop Screening evidence for systematic reviews using a text mining system: the RobotAnalyst was organised in collaboration with the National Institute for Health and Care Excellence (NICE) and Cochrane Switzerland. NaCTeM and NICE oraganised another workshop at the GES, RobotAnalyst: an online system to support citation screening in evidence reviewing, focussed on an interactive demonstration of RobotAnalyst’s functionality and use cases. For further details aboutr the evaluation of RobotAnalyst, plese see the following article:

Przybyła, P., Brockmeier, A. J., Kontonatsios, G., Le Pogam, M. -A., McNaught, J., von Elm, E., Nolan, K. and Ananiadou, S. (2018). Prioritising references for systematic reviews with RobotAnalyst: a user study. In: Research Synthesis Methods

Availability

If you wish to use RobotAnalyst, please contact us to request an account.

References

Przybyła, P., Brockmeier, A. J., Kontonatsios, G., Le Pogam, M. -A., McNaught, J., von Elm, E., Nolan, K. and Ananiadou, S. (2018). Prioritising references for systematic reviews with RobotAnalyst: a user study. In: Research Synthesis Methods

Brockmeier, A. J., Mu, T., Ananiadou, S., and Goulermas, J. Y. (2018) Self-tuned descriptive document clustering using a predictive network. IEEE Transactions on Knowledge and Data Engineering

Kontonatsios, G., Brockmeier, A. J., Przybyła, P., McNaught, J., Mu, T., Goulermas, J. Y., and Ananiadou, S. (2017). A semi-supervised approach using label propagation to support citation screening. Journal of Biomedical Informatics

Sato, M., Brockmeier, A. J., Kontonatsios, G., Mu, T., Goulermas, J. Y, Tsujii, J. and Ananiadou, S. (2017).Distributed Document and Phrase Co-embeddings for Descriptive Clustering. In: Proceedings of EACL

Haynes, C., Kay, N., Harrison, K., McLeod, C., Shaw, B., Leng, G., Kontonatsios, G. and Ananiadou, S.. (2016). Using text mining to facilitate study identification in public health systematic reviews. * In: Guidelines International Network (G-I-N) conference *

Hashimoto, K., Kontonatsios, G., Miwa, M. and Ananiadou, S. (2016). Topic Detection Using Paragraph Vectors to Support Active Learning in Systematic Reviews. * In: Journal of Biomedical Informatics*, 62, 5965

Mu, T., Goulermas, J. Y., Korkontzelos, I. and Ananiadou, S. (2016). Descriptive document clustering via discriminant learning in a co-embedded space of multilevel similarities. Journal of the Association for Information Science and Technology, 67, 1

Mo, Y., Kontonatsios, G. and Ananiadou, S. (2015). Supporting Systematic Reviews Using LDA-based Document Representations. Systematic Reviews , 4, 172

O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M. and Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: A systematic review of current approaches. Systematic Reviews **4:**5 (Highly Accessed)

Miwa, M., Thomas, J., O’Mara-Eves, A. and Ananiadou, S. (2014). Reducing systematic review workload through certainty-based screening. Journal of Biomedical Informatics

Contact

To obtain further information about RobotAnalyst, please contact Prof. Sophia Ananiadou.