Upcoming Seminars
November 15, 2024, 11 am CST
Using GPT API Models as Second Screeners of Titles and Abstracts in High-Quality Systematic Reviews
-
Speaker: Dr. Mikkel Vembye, Danish Center for Social Science Research
-
Description: Independent human double screening of titles and abstracts is a critical step to ensure the quality of systematic reviews and meta-analyses. However, double screening is a resource-demanding procedure that slows down the review process. In this presentation, I will demonstrate how reviewers reliably can implement OpenAI’s GPT models as second screeners of title and abstracts to mitigate this resource demand. I will provide a practical demonstration of how to conduct title and abstract screening with GPT API models through the newly developed R package AIscreenR and highlight how the approach differs from simply using the ChatGPT interface. I will discuss implementation requirements for ensuring performance that is at least on par with typical human screening. To facilitate this, I will introduce a new screening benchmark scheme that is based on common human screening performance in 22 large-scale systematic reviews. As the initial proof of concept, I will present results of using GPT API screening in three large classification experiments. I will conclude by discussing limitations and future directions for this screening method and how it can be combined with traditional (semi-)automated screening tools. The background for this presentation can be found at https://osf.io/preprints/osf/yrhzm.
December 13, 2024, 11 am CST
Friday the 13th & Single Case Experimental Designs: Allegedly Unlucky Encounters for Meta-Analysts
-
Speakers: Dr. John Ferron, University of South Florida, and Dr. Megan Kirby, Glazer Children’s Museum