Elsevier, a global leader in medical information and data analytics, has announced a groundbreaking evaluation framework designed to assess the performance and safety of generative AI-powered clinical reference tools. Developed for all Elsevier Health generative AI solutions, including its flagship ClinicalKey AI platform, the framework establishes a new benchmark for responsible AI integration in healthcare and will be highlighted in a forthcoming issue of the Open Access Journal of the American Medical Informatics Association (JAMIA).
This comprehensive framework, created with guidance from clinical subject matter experts across various specialties, evaluates AI-generated responses across five key dimensions: query comprehension, response helpfulness, correctness, completeness, and potential for clinical harm. It aims to ensure that AI-powered tools deliver accurate, relevant, and up-to-date information that meets the needs of healthcare professionals at the point of care.
In a recent evaluation of ClinicalKey AI, Elsevier collaborated with a panel of 41 board-certified physicians and clinical pharmacists who rigorously assessed the tool’s responses to a diverse range of clinical queries. This evaluation covered 426 query-response pairs and demonstrated strong performance, with 94.4% of responses rated as helpful, 95.5% judged as completely correct, and only 0.47% flagged for potential improvements.
Elsevier is committed to responsible implementation of AI across its portfolio of solutions and is actively involved in industry-wide initiatives. As a partner in the Coalition for Health AI, the company contributes to the development of standards that promote responsible AI deployment in healthcare.
The release of this evaluation framework marks a significant advance in the responsible use of AI in clinical decision-making, laying the foundation for more efficient, accurate, and patient-centered healthcare delivery.
Click here to read the original press release.
