2023 AI for Population Health and Health Systems Implementation Grant

Decision-makers need access to high-quality studies to determine which interventions and policies should be used. To summarize all medical literature on a given topic, Systematic Reviews (SRs) are used. Currently, two tools are used to manually assess the quality of SRs: AMSTAR and ROBIS. However, there is currently no automated tool to appraise SR quality.

Our project, WISEST (Which Systematic Evidence Synthesis is besT), addresses this gap by using items from the AMSTAR and ROBIS tools as features (quality indicators) in our models. WISEST will also use methods features and SR results to help users compare SRs on the same topic. WISEST will transparently provide the rationale behind the AI’s comparison of features, choice of the best SR(s), and allow users to make their own decision based on the AI's output.

Dr. Andrea Tricco

Dr. Andrea Tricco is the Tier 2 Canada Research Chair in Knowledge Synthesis; Scientist and Director of the Knowledge Synthesis Team in the Knowledge Translation Program, Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Unity Health Toronto. In addition, she is an Associate Professor at the University of Toronto in the Dalla Lana School of Public Health & Institute of Health Policy, Management, and Evaluation. Dr. Tricco is also the Co-Director & Adjunct Associate Professor for the Queen’s Collaboration for Health Care Quality JBI (formerly Joanna Briggs Institute) Centre of Excellence at Queen’s University and Nominated Principal Investigator of the Canadian Institutes of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR) Evidence Alliance. 

Proposal

Our project, WISEST (Which Systematic Evidence Synthesis is best), will use items from the AMSTAR and ROBIS tools as features (quality indicators) in our models. In addition, WISEST will use methods, features, and SR results to help users compare SRs on the same topic. WISEST will transparently provide the rationale behind the AI’s comparison of features, choice of the best SR(s), and allow users to make their own decisions based on the AI's output.

The team plans to use the grant to:
• Create a generalizable and labelled dataset of over 10,000 SRs;
• Test and train models and compare their accuracy.

WISEST will be based on empirical research and  continuously monitored by experts for accuracy. The code and supporting text for the AI outputs will be transparently documented. The research team's expected outcomes include:
• an open-access tool;
• publication;
• presentations; and 
• workshops.