The Public Library of Science (PLOS) has announced the release of the first results of a new initiative to measure researchers’ Open Science practices across the published literature, ‘Open Science Indicators’ (OSI). These initial results show increasing prevalence of good practices in research data and code sharing, and increasing use of preprints by researchers. PLOS is releasing the OSI dataset, and the associated framework used to develop it, to support Open Science initiatives in the wider community.
Earlier this year PLOS partnered with DataSeer to develop and measure, using artificial intelligence and natural language processing, a set of numerical ’indicators’ linked to specific Open Science characteristics and behaviors observable in thousands of published research articles. Now, PLOS is making available the data on three Open Science Indicators: data sharing, code sharing, and preprint posting, and will be developing additional indicators in 2023 guided by feedback from the community.
OSIs will help PLOS better understand researchers by establishing a baseline for the current state of adoption for Open Science practices and allow PLOS to track progress over time. As a framework for measuring and monitoring Open Science, OSIs have broad potential applications, from complementing the aims of the UNESCO Open Science Monitoring Framework Working Group to meeting the needs of organizations that wish to better understand Open Science practices, and assessing the impact of policy changes, like those set forth in the recent OSTP memo.
PLOS’ initial dataset examines data-sharing and code-sharing behaviors, and preprint posting in both PLOS articles, and the same Open Science practices in the wider scientific literature. The data cover approximately 61,000 PLOS research articles published between January 2019 and June 2022, as well as a comparator set of 6,000 publicly available research articles from PubMed Central (10% of the PLOS sample). The OSI method will be further developed and improved over time.
Data highlights include: all Open Science Indicators are increasing in PLOS articles and in comparator articles from 2019 to 2022; 23% of PLOS articles had associated research data that have been shared in a repository, according to good practices in 2019, rising to 28% in 2022. For comparators the rates are less than half those observed in PLOS articles, with 9% of articles using detectable data repositories in 2019, rising to 15% in 2022; while PLOS articles are more likely to link to an associated public dataset than comparable articles published elsewhere, when viewed over time, a generally positive trend is seen in repository usage and data sharing, by any method, overall; code-sharing rates among PLOS and comparator articles are similar, rising from 6% of articles in 2019 to 16% of articles in 2022. The dataset also reveals, for the first time, insights into which research projects generate code and software and those which do not; and PLOS articles are more likely to have an associated preprint than comparable articles published elsewhere. Overall, 21% of PLOS articles in the dataset have an associated preprint, compared to 19% of comparator articles.
In September, PLOS provided a summary of plans and some preliminary results. The first complete release of the data explores only a fraction of the many ways in which the dataset can be analyzed to understand Open Science practices. For example, readers can delve into data- and code-sharing practices, differentiating between sharing as Supporting Information (SI) or in a repository. Data on research topics and geography can also be used to explore trends in different research communities. In the future PLOS plans to expand on this dataset with new data points and ways of presenting them, additional publication years, and new indicators related to other aspects of Open Science practice.
Click here to read the original press release.
