AI vs Radiologists: can AI surpass doctors in reporting accuracy?

A study by the Centre for Advanced Research in Imaging, Neuroscience and Genomics (CARING) found that nearly 80% of AI generated chest X-ray case reports were as accurate as produced by a radiology specialist. Furthermore, AI reports were more accurate in 5% of analyzed patient cases. The study was undertaken using Oxipit ChestEye AI imaging suite. The full study titled The full study titled “Judging the Accuracy and Clinical Validity of Deep Learning-Generated Test Reports” will be presented at the annual Radiological Society of North America conference in Chicago later this year.

The study led by Dr Vasanth Venugopal aimed to evaluate the accuracy and clinical validity of AI-generated reports of chest X-rays. Additional aim of the study was to identify elements of “hedging” - a practice of using inconclusive, defensive, non-committal diagnosis statements in order to hedge against mistakes or potential legal threats - in clinical reports.

A set of 297 retrospective chest X-ray images was used in the study. The images were analyzed by Oxipit ChestEye and AI generated reports compared to the ones produced by a radiologist. Final report comparison and diagnosis validation was conducted by a radiologist with 9 years of clinical experience.

In 236 (79%) cases, algorithm-generated reports were found to be as accurate as the radiologists' reports. In 16 (5%) cases, algorithm generated reports were found to be either more accurate or more clinically appropriate. In 18 (6%) cases, the algorithm made significant diagnostic errors and in 27 (9%) cases, the algorithm-generated reports were found to be clinically inappropriate or insufficient even though the significant findings were correctly identified and localised.

“Oxipit ChestEye was designed as a productivity and second-opinion tool to aid the work of radiologists. The evaluation that nearly 80% of preliminary reports generated by our software were clinically accurate to be deemed as final reports, and in 5% cases - even more accurate than the ones produced by a radiologist, is an inspiring validation of current capabilities of AI diagnostics” - notes Dr N. Ramanauskas.

In the words of Dr N. Ramanauskas, even two radiologists can have differing opinions over a particular X-ray report.

“Therefore aiming for 100% correlation to radiologist reports is unrealistic. However, with constant improvement of deep learning algorithms and innovations in medical imaging field, a radiologist working with the help of AI can help to reduce interpersonal subjectivity and greatly increase the quality of the reports while helping the radiologists become more efficient.” - notes Dr N. Ramanauskas.

Oxipit ChestEye imaging suite encompasses a fully automatic computer aided diagnosis (CAD) platform which supports 75 radiological findings, covering over 90% of radiological cases presented to radiologists on a daily basis. ChestEye produces a standardized preliminary text report that incorporates all the radiologically relevant information present in a chest X-ray image, speeding up case description and minimizing the risk of overlooked secondary findings. Internal platform trials showed 30% time saved per patient case and reduced error rate by up to 50%.

The study also found that preliminary reports produced by Oxipit ChestEye AI platform can reduce hedging and defensive reporting statements in radiologist reports, resulting in clearer more actionable diagnosis descriptions.

“Subjectivity is part of human nature. Sometimes radiologists might be fairly confident that there is no pathology in the analyzed case, but they choose to include inconclusive vague diagnosis statements to hedge against potential mistakes or legal threats. On the contrary, AI produces clear, unbiased reports that can help to reassure radiologist in their opinion. A combination of both produces clearer, actionable reports to make treatment decisions, as well as helps to avoid unnecessary procedures resulting from evasive and non-committal reports” - notes Oxipit’ Chief Medical Officer Dr Naglis Ramanauskas.

Oxipit
Saulėtekio al. 15, LT-10224 Vilnius
Lithuania
info@oxipit.ai

ABOUT CARING
CARING (Centre for Advanced Research in Imaging, Neuroscience and Genomics) is a research & development center focused on performing cutting-edge scientific and clinical research and helping radiology and genomics companies develop world-class clinically relevant products. CARING is a wing of Mahajan Imaging - a leading high-end medical imaging service provider in New Delhi, India. Research and development partners of Mahajan Imaging include Massachusetts Institute of Technology (MIT), All India Institute of Medical Sciences and Indian Institute of Technology (IIT).

ABOUT OXIPIT
Oxipit is a computer vision software startup specialized in medical imaging. With a team of award-winning data scientists and medical doctors, the company aims to introduce innovative Artificial Intelligence/Deep Learning breakthroughs to everyday clinical practice. Oxipit are the authors of CE certified multi-award winning ChestEye radiology imaging suite.

This release was published on openPR.