Cognita Imaging has secured a $1.29 million FDA research contract to test a novel evaluation method for generative AI in radiology. The 18-month project, starting June 22, 2026, will assess whether a 'jury' of large language models can effectively evaluate AI-generated radiology reports. This initiative addresses the challenges of validating complex, open-ended AI outputs amid a national radiologist shortage. Cognita is also developing a breakthrough-designated chest X-ray vision-language model.
FDA LLM evaluation contract
- ▪The Food and Drug Administration research contract with Cognita Imaging commenced on June 22, 2026, and is scheduled to run for an 18-month period.
- ▪Cognita Imaging received a $1.29 million research contract from the Food and Drug Administration to test a new method of evaluating artificial intelligence-generated radiology reports.
- ▪Cognita Imaging, which was founded in 2024, was acquired in 2025 by Radiology Partners, the largest radiology practice in the United States.
Generative AI validation challenges
- ▪Evaluating generative artificial intelligence models that produce full-text radiology reports is challenging because they can represent hundreds of potential findings in an open-ended output.
- ▪The United States is experiencing a shortage of radiologists, which has resulted in longer working hours and larger case backlogs.
- ▪Generative artificial intelligence models in radiology face risks of hallucinations, which are plausible-sounding but incorrect details, and omissions, where critical findings are left out.
- ▪Most artificial intelligence tools currently authorized by the Food and Drug Administration in radiology are one-off solutions designed to detect specific findings like blood clots or collapsed lungs.
LLM jury testing methodology
- ▪Cognita Imaging is collaborating with the Center for Devices and Radiological Health's regulatory science team to test using multiple large language models as a jury to evaluate AI radiology reports.
- ▪Cognita Imaging will provide the Food and Drug Administration with a final report, software code, guidance for building LLM juries, validation cohort analyses, and radiologist-reviewed discrepancy studies.
- ▪Under the Cognita Imaging framework, radiologists will review discrepancies when there are clinically significant disagreements between the AI-generated report, the LLM jury, or the original radiologist's report.
- ▪Cognita Imaging will build and test its evaluation framework on 1 million patient exams from a large United States cohort to analyze performance across diverse patient groups and equipment.
Cognita breakthrough device designation
- ▪Cognita Imaging received a Food and Drug Administration breakthrough device designation in early 2026 for a vision-language model that interprets chest X-rays and generates reports.
- ▪Cognita Imaging is working with the Food and Drug Administration to design and implement a clinical study to evaluate the accuracy, benefits, and risks of its chest X-ray vision-language model.
FDA generative AI regulation
- ▪Cognita Imaging co-founder Akshay Chaudhari stated he intends for the project's findings to be generalizable so the Food and Drug Administration can use them in future regulations and guidance.
- ▪The Food and Drug Administration is increasing its scrutiny of generative artificial intelligence medical devices through breakthrough designations, research contracts, and an August 2026 discussion paper.
Debatable claims
- ▪The FDA should permit LLM juries to evaluate generative medical AI
- ▪Generative AI poses unacceptable safety risks when used to generate medical content like radiology reports
Story comments
Loading comments…