Photo by Kanhaiya Sharma on Pexels
The Public Library of Science (PLOS) recently announced the first results of their new Open Science Indicators (OSIs) initiative, which is designed to measure and improve adherence to Open Science practices. Developed in partnership with DataSeer, the initiative uses artificial intelligence and natural language processing to extract numerical indicators from thousands of published research articles. The PLOS Open Science Indicators focus on code sharing, data sharing, and posting of preprints.
Why the need for Open Science Indicators?
Open Science refers to a wide range of practices and principles aimed at making scientific research more transparent, accessible, and reproducible. Since its inception, PLOS has been a champion for the adoption of best practices in Open Science practices. But first there was the need to understand how journals support Open Science. This is where the OSI initiative, built with an AI-driven approach, comes in to help PLOS fulfil its objective.
While the mission is to improve the adoption of Open Science practices, the world lacked a reliable tool to understand these Open Science practices and behaviors in research until the OSI initiative. PLOS is sharing the OSI framework and dataset with researchers and institutions to encourage the wider scientific community to quickly embrace Open Science practices, which in turn is expected to accelerate and increase research impact. The dataset will serve as a baseline for the current level of Open Science practices and will enable PLOS to track progress over the years.
The OSI framework can be used by organizations seeking to understand Open Science practices and assess the impact of policy changes related to Open Science. PLOS plans to use feedback from the research community to build additional indicators this year. Iain Hrynaszkiewicz, Director of Open Research Solutions at PLOS, said, “Open Science Indicators will enable us to better understand and support the researchers we serve, and by sharing OSIs openly we can help others support the adoption of Open Science practices.”1
PLOS Open Science study focused on three key parameters
The PLOS Open Science study focused on data sharing, code sharing, and preprint posting behavior by examining articles published in PLOS and other open-access journals. Data sharing is the practice of making data sets that support the results of published research available to the public, while code sharing is the practice of making the code used to generate the data available to the public. This enables other researchers to easily access, reproduce, and build on existing research. Publishing preprints is the practice of posting scientific manuscripts online, even before it is peer-reviewed. Preprints allow authors to quickly share their work and get valuable feedback from peers, which not only establishes primacy of the work done but also helps them refine their paper before submitting it to a reputed peer-reviewed journal. The initial results of the OSI project provides a promising outlook for the future of Open Science practices, with all three indicators on the rise.
Data shows improvement in good Open Science practices1
The initial OSI dataset covered about 61,000 PLOS research articles published between January 2019 and June 2022, with 10% of these made up of a comparator set of 6,000 publicly available articles from PubMed Central. The PLOS Open Science initiative evaluated the number of researchers who follow these open science practices in a specific scientific field, how often they use them, and the impact their research has had.
Data sharing: The percentage of PLOS articles that had shared the research data in a repository, in keeping with good Open Science practices, increased to 28% in 2022, up from 23% in 2019. Data-sharing was also higher among the set of comparator articles, with 15% of articles using detectable data repositories in 2022, up from 9% in 2019.
Code sharing: There was a similar trend seen between PLOS and comparator articles when it came to code-sharing rates, which increased to 16% of articles in 2022 from just 6% in 2019. The dataset also revealed, for the first time, insights into which research projects generated code and software and those that do not.
Preprint posting: The PLOS Open Science study also indicated an increase in the use of preprints with PLOS articles more likely to have an associated preprint than comparable articles published elsewhere. In 2022 alone, 21% of PLOS articles had associated preprints compared to 19% among the set of comparator articles.
The results of the OSI also underscore the need for greater support for open science practices. Currently, many researchers lack the resources, training, and infrastructure to properly implement open science practices. Additionally, many researchers face disincentives to sharing data and code, such as time-consuming data management and code review processes. To address these challenges, PLOS and other organizations are working to promote and support Open Science practices that include the development of tools and resources to help researchers adopt open data and open-source software practices, and advocating for open access publishing and the use of preprints.
While the first results of the PLOS Open Science Indicators project provide an encouraging snapshot of the current state of Open Science, it is only a fraction of the many ways in which the data can be examined to better understand Open Science practices. In the future, PLOS plans to expand the dataset with new data points, additional publication years, and new indicators related to other aspects of Open Science practice.2 This information, along with community feedback, will allow for a deeper understanding of the challenges and opportunities in making Open Science practices the norm in research.
R Discovery is the largest open-access research repository in the world with more than 40 million open-access articles, including 2 million preprints. It consists of research topics across all major subject areas, including medicine, biology, philosophy, political science, environmental science, social sciences, and psychology.
References:
- PLOS Unveils New Way To Measure Open Science Practices. NISO Member News, Dec 2022. Available at https://www.niso.org/niso-io/2022/12/plos-unveils-new-way-measure-open-science-practices
- Hrynaszkiewicz, I. PLOS partners with DataSeer to develop Open Science Indicators. The Official PLOS Blog, September 2022. Available at https://theplosblog.plos.org/2022/09/plos-partners-with-dataseer-to-develop-open-science-indicators/
