知镜 Cognispect · 医学文献与研究工作台

Benchmark Evaluation of Vision-Enabled Large Language Models in Mammography: A Case-Control Study.

Sinci KA, Aksit M, Gursoy Bulut M.
Journal of breast imaging · 2026-10-21 · PMID 42905566

文献摘要

Objective To compare the diagnostic performance of five contemporary vision-enabled large language models (LLMs) in mammography interpretation using a case-control benchmark design. Methods A total of 200 cases were analyzed using a quota-balanced case-control design (100 malignant, BI-RADS 4-5; 100 controls, BI-RADS 1-2). All models (ChatGPT-5T, ChatGPT-5I, ChatGPT-4o, Gemini 2.5 Pro, and Gemini 2.5 Flash) received standard mammography images. Models were evaluated for lesion detection, laterality, quadrant localization, lesion type, and malignant-benign discrimination. Performance was assessed using accuracy, sensitivity, specificity, balanced accuracy, Cohen's κ, and area under the receiver operating characteristic curve (AUC). Breast density was recorded for all cases, while histopathological subtype and lesion size were documented for malignant cases. Results Lesion detection accuracy ranged from 79% to 91%, with high malignant sensitivity (92%-100%) but lower specificity in normal cases (24%-82%). ChatGPT-5T performed best in laterality (83%; κ = 0.70) and quadrant localization (73%; κ = 0.62). Lesion type classification was modest (74%-85%). In malignant-benign discrimination, ChatGPT-5T achieved the highest balanced accuracy (87%), while ChatGPT-4o showed poor specificity (28%). The study cohort was predominantly composed of lower-density breasts, larger lesions, and invasive ductal carcinoma as the most common subtype. Conclusion Vision-enabled LLMs demonstrated high sensitivity for breast lesion detection and classification tasks. More deliberative models, particularly ChatGPT-5T, consistently outperformed faster counterparts, especially in spatial reasoning tasks. However, reduced specificity in normal cases underscores that current LLM outputs should remain under radiologist supervision as supportive second-reader tools rather than standalone diagnostic systems. Given the quota-balanced design and cohort composition, results should be interpreted as comparative benchmarking rather than real-world diagnostic performance.

在知镜精读原文来源DOI 原文

知镜提供医学文献检索、阅读笔记、人体图谱、统计绘图与研究复现工具。