Evaluating fracture-detection AI: the questions to ask

Independent studies, published in peer-reviewed journals, that compare several software products on the same images (head-to-head studies). Prefer consecutive series of real patients, for example every emergency visit over a period, to image sets enriched in fractures. Check who funded the study and who set the reference standard.

Sensitivity is the share of real fractures the AI flags; specificity, the share of fracture-free exams it leaves without an alert; accuracy, the share of correct answers overall. The AUC sums up performance across all possible thresholds. Tell the standalone evaluation of the AI apart from the reader study, which measures what radiologists gain when they read with the AI: the second figure is the one that describes real use.

A global figure hides large gaps. In the Zurich multivendor study (Pediatric Radiology, 2025), Milvue's specificity ranges from 82.4% on the elbow to 98.8% on the lower leg. Ask for sensitivity and specificity on the regions you read most: wrist, ankle, elbow, hip, ribs.

Growing bone, cartilage and normal variants mislead models trained on adults. Check the population covered by the clearance (minimum age, in each territory) and whether independent pediatric studies exist, with results by region. Milvue's fracture detection is CE marked with no age limit, FDA cleared from birth and licensed by Health Canada with no age limit.

In Europe, CE marking under the MDR 2017/745, the device class and the notified body. In the United States, the 510(k) number, searchable in the FDA public database. In Canada, the Health Canada licence. In the United Kingdom, the NICE recommendation: in January 2025, the HTG739 assessment listed four fracture-detection technologies that can be used in the NHS during the evidence generation period, among them Milvue's TechCare Alert.

Results should reach the existing PACS, on the image, with no workstation change: DICOM in and out, IHE profiles. Look at the delay from acquisition to result, how suspicious exams move up the worklist, and how the tool shows uncertainty.

Run a retrospective series of your own consecutive exams through the software, with a reference set by your radiologists. Count missed fractures, and also false alerts per reading day: that is the load your teams will live with.

In an independent emergency department study comparing three commercial algorithms on 1,210 patients (Bousson et al., Academic Radiology, 2023), Milvue reached 90.1% accuracy, versus 88.8% and 71.0%: the highest fracture-detection accuracy in published independent head-to-head studies. The reader study of the FDA submission shows +22% reader sensitivity in adults and +20% in children. TechCare Alert is a Class IIa medical device in Europe, FDA cleared (TechCare Trauma, K242171) and licensed by Health Canada (SmartTrauma), with native RIS and PACS integration.

Book a demo