DIAGNOSTIC PERFORMANCE OF MULTIMODAL LARGE LANGUAGE MODELS FOR PROXIMAL FEMORAL FRACTURE DETECTION ON AP HIP RADIOGRAPHS: A RETROSPECTIVE SINGLE-CENTRE STUDY

¹ Vita-Salute San Raffaele University, Orthopedics and Traumatology Department, Milan, Italy
² IRCCS San Raffaele Hospital, Orthopedics and Traumatology Department, Milan, Italy

Correspondence to:

European Journal of Musculoskeletal Diseases 2026 May-August 15(2): 67-75
https://doi.org/10.82093/ejmd/2026v15iss2_1

Received: 2 May 2026 Accepted: 5 June 2026

Download PDF

Abstract

Multimodal large language models (MLLMs) capable of analyzing both images and text represent a new class of artificial intelligence tools with potential relevance for clinical imaging. Their diagnostic performance on real-world radiographs, particularly for high-stakes conditions such as proximal femoral fractures, has not yet been systematically evaluated. The purpose of this retrospective study is to determine whether contemporary MLLMs can detect proximal femoral fractures on anteroposterior (AP) hip radiographs with diagnostic accuracy non-inferior to that of an experienced musculoskeletal radiologist, and to assess potential efficiency gains in reporting time. In this retrospective, single-center diagnostic accuracy study, which adhered to STARD 2015, 299 AP hip radiographs (200 fracture-positive; 99 fracture-negative) were evaluated. Four commercially available MLLMs (GPT-5, Gemini-2.5, LLaMA-4, Kimi-VLA3B) were tested using a fixed prompt, and their performance was compared with that of a fellowship-trained radiologist. Non-inferiority for AUC (margin -0.05), superiority for F1-score (margin +0.03), and >50% reduction in time-to-report were prespecified. Decision-curve analysis and an exploratory cost model were performed. The fracture-positive subset (n = 200) was used for efficiency and economic analyses. The radiologist achieved an AUC of 0.919. GPT-5 (AUC 0.950; Δ = +0.031), Gemini-2.5 (0.946; Δ = +0.027), and LLaMA-4 (0.941; Δ = +0.022) met the non-inferiority criterion, whereas Kimi-VLA3B did not (0.872; Δ = -0.047). GPT-5 and Gemini-2.5 demonstrated superior F1-scores relative to the radiologist (0.973 and 0.967 vs 0.933), while LLaMA-4 was not significantly different (0.948) and Kimi-VLA3B performed worse (0.912). All models markedly reduced reporting time, with GPT-5 requiring 15.0 ± 2.9 seconds versus 308.0 ± 64.8 seconds for the radiologist (>90% reduction). Decision-curve analysis showed the highest net benefit for GPT-5 across threshold probabilities of 0.05-0.30. In exploratory economic analysis, GPT-5 and Gemini-2.5 produced positive cost estimates within the fracture-positive cohort, while LLaMA-4 was cost-neutral and Kimi-VLA3B unfavourable. Three MLLMs demonstrated non-inferior diagnostic performance compared with an expert radiologist for proximal femoral fracture detection, with GPT-5 and Gemini-2.5 achieving superior composite accuracy and all models providing substantial efficiency gains. Although limited by its single-center, prevalence-enriched design, this study provides early evidence supporting the potential role of MLLMs as adjunctive tools in emergency radiographic workflows.

Keywords: artificial intelligence, diagnostic accuracy, hip fracture, multimodal large language models, orthopedic AI