Saeed Asiri

Artificial Intelligence Specialization · Cohort 2025

Leveraging LLMs for Automated Peer Review of Randomized Controlled Trials (RCTs)

Supervisor Fralick has developed a framework of six domains of study design that can affect the internal validity of randomized controlled trials (RCTs), encapsulated by the acronym PHOBIA: Placebo controlled? How was it funded? Outcome clinically valid? Blinded? Intention-to-Treat? A lot of centres and patients included? When evaluating an RCT, these 6 elements are crucial considerations. The current paradigm leaves reviewers to parse these details from the manuscript, which is inefficient, time-consuming, risks bias, and lacks quality control. All RCTs require registration on a publicly available clinical trial registry, meaning key aspects of their design are readily available. This project will apply supervised machine learning (ML) and two large language models (LLMs) for automating part of the peer review process. The data from the RCT will be parsed. Then, LLM 1 (Summarizer) will extract key information related to the PHOBIA framework. LLM 2 (Validator) will validate the summary by checking it against the original study content. Performance of the dual-LLM system will be evaluated according to the following metrics: hallucination detection, consistency, speed, and helpfulness. A detailed comparison of the system’s reviews with traditional human reviews will assess whether the LLMs can reliably augment the peer review process for

Mentor: Michael Fralick

Read the final report (PDF)

See it in the program report (PDF)