Cochrane has launched a study to test whether AI tools can support or enhance evidence synthesis. The initiative uses an adaptive platform study design and adheres to criteria intended to uphold Cochrane’s established standards for evidence. The study is positioned as an independent evaluation of artificial intelligence tools across defined stages of the evidence synthesis lifecycle.
AI technologies are advancing rapidly, with a growing number of studies and preprints examining whether elements of a Cochrane review can be replicated in a fraction of the time. The release notes that many existing evaluations have methodological limitations and often rely on studies conducted by tool developers. Cochrane is seeking to introduce independent, prospective, and reproducible methods into this area of research.
The study will evaluate how AI tools could support or enhance key stages including literature screening and data extraction. It is being led by the AI Methods Group. The initiative reflects the view that speed alone does not define trust in evidence synthesis, which relies on transparent and methodologically rigorous processes.
In November 2025, Cochrane welcomed proposals from developers of AI tools aligned with the evidence synthesis lifecycle. A total of 48 proposals were received, followed by an internal assessment that resulted in a shortlist. Submissions were screened for alignment with RAISE principles, Cochrane’s mission, vision and values, maturity of release, affordability, and compliance with data protection and copyright standards.
Shortlisted tools were ranked by self-nominated members of the AI Methods Group based on evaluation evidence, performance claims, scope of functionality, human oversight levels, and reliance on user-defined prompts. The process resulted in two shortlisted tools, with five additional tools placed on a reserve list for potential inclusion.
The platform study design allows multiple interventions, including AI tools and a dual-human standard, to be evaluated under a single protocol. Performance metrics and uncertainty thresholds will be defined in advance, and new tools may be added while ineffective ones are removed over time.
The AI tools will be tested across approximately 15 Cochrane review updates, with results compared to traditional methods through author teams. Author teams are being finalized, with two teams prepared to test the protocol prior to public release. Completion of the platform study is targeted for the second half of 2026, with results to be written up afterward.
Click here to read the original press release.
