Abstract

Background

Preprints are becoming more common in the health sciences and allow for instant dissemination of research findings; however, with the risk of compromising quality and transparency. Peer review potentially improves reporting and reduces errors, although its actual impact is not known. The objective of this scoping review was to synthesize evidence comparing preprints in the health areas to their peer-reviewed versions and assess preprint publication rates.

Methods

We searched Embase, Medline OVID, Scopus, and Web of Science from inception to July 2024 for studies comparing preprints with their peer-reviewed versions and/or investigating preprint publication rates. Two reviewers independently conducted screening and extracted data on study characteristics, parameters compared, and preprint publication rates. We conducted a narrative synthesis.

Results

We included 40 studies (published 2019–2024; 92% peer-reviewed). The median number of studies analyzed per article was 356 (range: 19–73,256). 42% of preprints were eventually published among 33 studies that reported publication rates (IQR: 22%–67%). Preprint searches routinely started on January 1, 2020, with a median of 24.3 months and a median difference of 11.5 months between preprint and peer-reviewed search end dates. Commonly compared parameters were primary outcomes/endpoints (37%) and sample size (30%), with peer-reviewed articles showing improved reporting for funding (13%), conflicts of interest (13%).

Conclusion

While peer review enhances transparency and methodological reporting (e.g., funding, conflicts of interest), the content, outcomes, and conclusions of health-related preprints remain largely consistent with their peer-reviewed versions. Preprints facilitate rapid knowledge dissemination but may benefit from stricter reporting standards to improve credibility. Future efforts should focus on standardizing preprint policies to bridge quality gaps without delaying access.

Similar content being viewed by others

Introduction

A preprint is a complete scientific manuscript that is uploaded by authors to a public server without formal review [1]. Although preprints have existed for over 30 years, they have only recently gained widespread acceptance in medical and life sciences research [2]. The COVID-19 pandemic further accelerated preprint use, driven by the urgency to disseminate research findings [3]. Indeed, during the COVID-19 pandemic, preprint servers like medRxiv experienced at least a fivefold increase in new paper uploads over a five-month period, and nearly 9,000 COVID-related preprints posted on medRxiv by December 2020—representing 62% of all preprints on that server during that period [4, 5]. While this allowed timely dissemination of findings, it raised significant concerns about whether preprints’ benefits outweigh the challenges, including risks to data reliability, patient outcomes, and public trust in science [6].

Preprints allow a rapid dissemination of scientific knowledge and the opportunity to receive feedback from peers before publication [7]. However, the absence of a formal peer-review process raises concerns about the quality and reliability of the disseminated research. This has prompted studies comparing preprints with their corresponding peer-reviewed publications, assessing whether certain parameters change with the transition from preprint to peer reviewed publication. These include interpretation of findings, results, data analysis, and disclosure of funding and conflict of interests [8, 9].

The objective of this scoping review was to synthesize evidence comparing preprints in the health areas to their peer-reviewed versions and assess preprint publication rates.

Methods

Design overview

We followed the JBI guidance for the conduct of scoping reviews and the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) for reporting (Appendix 1) [10, 11]. We registered our protocol with the Open Science Framework (OSF) before the beginning of this study [12].

Eligibility criteria

We included studies in the health field that compared preprints to their corresponding peer-reviewed publication, and/or examined the proportion of preprints that were later published as peer reviewed articles. We excluded reviews, commentaries, editorials, opinion pieces, and conference abstracts. We did not restrict eligibility by date, language, or type of publication (whether preprint or peer reviewed).

Literature search

We developed the search strategy with feedback from an experienced information specialist. We searched Embase (via Elsevier) on June 27, 2024, Medline (Ovid) on July 01, 2024, Scopus on July 05, 2024, and Web of Science on July 10, 2024. We used keywords relevant to the concepts of preprints and peer-reviewed publications, and where applicable (i.e., in the Medline and Embase searches), we also used Medical Subject Headings (MeSH) and Emtrees. We did not apply any language or time restrictions in our search. Search strategies are presented in Appendix 2. In addition, we performed forward and backward citation tracking for the included papers and screened the included studies of relevant reviews. We exported the search results to Endnote for de-duplication.

Article selection

After conducting calibration exercises, pairs of reviewers screened the titles and abstracts of articles on Rayyan (www.rayyan.ai) in duplicate and independently. We obtained full texts of articles considered as potentially eligible by at least one of two reviewers. Pairs of reviewers then screened in duplicate and independently the full texts, documenting reasons for exclusion. Any disagreements were resolved through discussion or using the help of a third reviewer. We summarized the literature search and selection processes using a PRISMA flow diagram [13].

Data charting

We developed a data charting form that was uploaded to the Research Electronic Data Capture (REDCap) tool [14, 15]. We pilot tested the form and conducted a calibration exercise whereby reviewers independently abstracted data from five studies, to ensure consistency across reviewers. Reviewers resolved any disagreements by discussion or with the help of a third reviewer.

Two authors collected the following information from selected studies in duplicate and independently:

Study characteristics, including first author name, year, journal or preprint server, type of publication venue (preprint server or peer reviewed journal), funding of the study, conflicts of interest of study authors, and study objectives;

Methods, including inclusion and exclusion criteria, timeframe for preprint and peer-reviewed publication, linkage method between preprint and peer reviewed publication used (if any), parameters compared and how they were assessed;

Results, including number of preprints included, number of peer-reviewed articles included, and findings on each compared parameter, and proportion of preprints that were published.

Data analysis

We exported the data to Excel where we conducted data cleaning and validity checks. We used descriptive statistics to present our findings: frequencies and percentages for categorical variables and medians and interquartile ranges for continuous variables. For each parameter compared between preprint and peer-reviewed publications, we conducted a narrative synthesis of the findings of the included papers.

Results

Search results

Our search identified a total of 7,935 nonduplicate items. We assessed 482 full texts for eligibility and included 39 studies. Figure 1 presents our PRISMA flow diagram which summarizes the search and the study selection processes.

Fig. 1

PRISMA flowchart for articles selection process

Characteristics of included studies

Among the 40 included articles, 30 compared preprints with peer-reviewed publications (30/40; 75%), 30 examined the publication rates of preprints (30/40; 75%), and 21(21/40; 53%) did both. Studies were published between 2019 and 2024. The majority (38/40; 95%) were peer-reviewed articles (with or without a preprint version), and 2 articles (2/40; 5%) were exclusively published as preprints. Of the included studies, 18 (45%) reported receiving funding, 14 (35%) reported no funding, and 8 (20%) did not report on their funding status. Twenty studies (50%) reported no conflicts of interest for any author, 14 reported conflicts of interest for at least one author, and 6 did not report conflicts of interest. Appendix 3 presents the detailed characteristics of included studies.

Table 1 presents the methodological aspects of included studies. Various eligibility criteria were used, with the most common being restricting to COVID-19-related research (54%). The median start date for the search for preprints was 1 January 2020, while the median timeframe between the end date for the search for preprints and the end date for the search for peer reviewed articles was 11.5 months. The median number of studies included per article was 356 (range:19–73,256). The median percentage of preprints that were eventually published was 42% (IQR: 22%−67%) across the 30 studies that examined publication rates (30/39).

Parameters assessed

Figure 2 presents the parameters compared between preprint and peer-reviewed publications in the included papers that compared preprints and their peer-reviewed versions (N = 30). The most assessed parameters were changes in primary outcomes (11/30; 37%), number of participants (9/29; 30%), authorship/contribution (7/30; 23%), title (7/30; 23%), and Altimetric Scores/Citation Counts (9/29; 30%). Parameters less frequently assessed included methods of Outcome Assessment, changes in units of measurements, discussion, interpretations, and limitations (each with 2/30; 7%), and reporting assessed using a reporting checklist (1/30; 3%). Appendix 4 summarizes the findings of the included studies for each parameter.

Fig. 2

Parameters compared between preprint and peer-reviewed publications (%) (N = 30)

Title

Seven papers assessed this parameter. Two found no significant changes [16, 17]. Peng et al. examined the lexical (i.e., character sequence of words) and semantic (i.e., shared meanings) differences in titles, which range from 0 (indicating no change) to 1 (indicating complete change), with a value of 0.3 indicating a difference that “should not be overlooked.” The authors found an average lexical difference of 0.2 and an average semantic difference of 0.12 for titles, indicating minor revisions [18]. The remaining papers found changes in the wording of titles (6/21; 29%) [19], shorter titles in peer-reviewed versions (10%) [20], changes in the reporting of randomization (112/161; 67% in unpublished preprint abstracts versus 116/121; 72% in published abstracts) [21], and changes in the reporting of study design, study population, and location descriptions (20/67; 30%) [22].

Abstract

Five papers assessed this parameter. Two papers respectively found that 40% [20] and 80% [23] of studies had abstract similarity. Peng et al. reported for abstracts an average lexical difference of 0.37 and an average semantic difference of 0.16 [18]. One paper found changes in abstract conclusions (33/139; 24% of studies) [24] and another paper found substantial changes (reflecting increases in subjects included or analyses) in 4/14; (29) % of studies [25].

Authorship and contributions

Seven papers assessed this parameter. Two papers concluded there were no significant differences [17] or extremely high similarity in first and last authors (46/47; 98%) [26]. Several papers concluded more authors in either one of the versions, with authors added in the peer reviewed version (9/67; 13%) [22], (8/47; 17%) [26], (47/200; 24%) [27] or authors added in the preprint version (6/200; 3%, 3/67; 5%, 3/47; 6%) [22, 26, 27]. Three studies respectively found changes in authorship position in (1/47; 2%, 6/67; 9%, 31/200; 16%) of studies [22, 26, 27]. The remaining changes were spelling or wording changes (2/67; 3%) [22] and major authorship changes in 1/14 (7%) of studies [25].

Funding

Four papers assessed funding related aspects. Four papers respectively found that 0% [26], 2/200; 1% [27], 1/19; 5% [16], and 8/67; 12% [22] of studies reported on funding exclusively in their preprint versions and 1/47; 2% [26], 2/67; 3% [22], 18/200; 9% [27], and 2/19; 11% [16] of studies reported on funding exclusively in their peer-reviewed versions. Itani et al. found a change in the funding status in 7/200; 4% of studies reporting on funding status in both versions [27]. The percentage of studies with more funders reported in peer-reviewed versions ranged between 5.4% and 38% [16, 22, 26,27,28,29]. Three studies reported more funders reported in peer reviewed versions (4/67; 6%, 3/47; 6%, 38/200; 25%) [22, 26, 27]. Two papers both found more funders in preprint version (1/47; 2%, 10/200; 5%) [26, 27]. Moreover, there was difference in the level of detail in the funding statement between the two versions in 1.5–3% of studies [22].

Conflict of interest (COI)

Four papers assessed this parameter. Carneiro et al. found a higher reporting in peer reviewed (both independent {34/76; 45% vs 50/76;66%} and paired {26/56; 46% vs 47/56; 84%} samples) [16]. Three papers reported COI statement exclusively in the preprint version (0%, 6/200; 3%, 5/67; 8%) [26]. Similarly, three studies reported COI statement exclusively in the peer reviewed version (1/67; 1%, 8/47; 17%, 45/200; 23%) [22, 27, 28]. Three papers found more COI reported in the preprint versions (1/67; 1%, 2/138; 1%, 2/47; 4%), and peer-reviewed versions (1/138; 1%, 8/67; 12%, 5/47; 11%) [22, 26, 27]. Bero et. al also found that the level of detail of COI disclosures was higher in the peer-reviewed versions of 2/67; 3% of studies [22].

Ethics approval

Three papers assessed ethics approval. Two found that ethical approvals were reported exclusively in preprints (1/67; 1%, 1/47; 2%) [22, 26], and four found reporting exclusively in peer-reviewed (2/67; 3%, 5/119; 4%, 1/10; 10%, 5/47; 11%) [22, 25, 26, 30]. Shi et al. also found more IRB statements in peer reviewed version (1/47; 2%) [26]. Carneiro et al. also found changes, with ethical approval added in two cases (2/19; 10.5%) but removed in one (1/19; 5.3%) [16].

Sample size

Nine papers assessed sample size calculation. Three papers found preprint larger sample sizes (1/67; 1.5%, 1/47; 2%, 2/119; 2%) [22, 26, 30], whereas four found peer-reviewed version larger sizes (2/67; 3%, 5/119; 4%, 1/10; 10%, 5/47; 11%) [22]. Between-version changes varied from 1 to 33% [24, 29, 31, 32]. Bero et al. found other content discrepancies (e.g., different number of patients recruited but same number randomized, non-matching numbers for sampling or analysis, typographical errors) (3/67; 5%) [22]. Spungen et al. reported differences in indication of randomized participants (51/161; 32% in preprint abstracts vs 62/161; 39% in peer-reviewed abstracts) and analyzed participants (61/161; 38% vs 73/161; 45%) [21].

Altimetric scores and citation counts

Seven papers assessed this parameter. Some reported higher altimetric scores in peer-reviewed versions, with higher citation counts also following publication [33,34,35]. Significant difference in citation counts existed between preprint and peer reviewed [36]. Preprints were more likely to have zero citations (15% vs 5%) [24], and peer-reviewed versions had faster citation growth for longer periods [37]. Nishioka et al. used the Gini coefficient (or Gini index) as a statistical measure of citation inequality between preprints and their peer reviewed versions, with 0 being perfect equality and 1 being perfect inequality. The coefficients of preprints were almost twice those of the peer reviewed version (0.23 vs 0.12), indicating a larger citation inequality in preprints compared to their peer reviewed version [37].

Methods of outcome assessment

Two papers assessed the methods of outcome assessment. Spungen et al. reported greater definition of primary outcomes in peer-reviewed versions (113/161; 76% vs 122/161; 70%) [21]. Bero et al. reported extra or more detailed outcome measurement in peer-reviewed articles (3/67 and 4/67; 4.5 and 6%) [22].

Analytical aspects

Five papers assessed this parameter. Some reported changes in statistical methods (17/119; 14%, 4/21; 24%, 17/67; 25%) [19, 22, 30]. Others reported addition of analyses in peer-reviewed versions (7/67; 10.5%) [22], publication of protocols or statistical plans as supplementary documents (3/75; 4%) [29], and additional post-hoc analyses (3/21; 14%) [19]. Carneiro et al. reported increased reporting of clearly defined variation or precision measures in peer reviewed version [16].

Units of measurement

Two papers assessed units of measurement. Zeratkaar et al. identified greater use of standard deviations and interquartile ranges in peer-reviewed publications (6/75; 8%) [29], and Bero et al. identified differences in reporting median (IQR) vs mean between versions (15/67; 22%) [22].

Primary outcomes/endpoints

Eleven papers assessed this parameter. Five identified discordant or inconsistent reporting (13/547; 2.4%, 1/21; 4.8%, 3/47; 6%, 33/119; 28%, 35/75; 47%) [[19](https://link.springer.com/article/10.1186/s41073-026-00189-z#ref-CR19 “Morin C, Padki A, Wong A, Miano T, Kane-Gill SL, Cozzi G, et al. Comparison of COVID-19 preprint and peer-reviewed versions of studies on therapies for cr