AI in Radiology: Chest X-Rays

Radiology has attracted the most attention for the application of AI. This has raised the question about whether or not radiologists should worry about their future.

Geoffrey Hinton, the Godfather of Deep Learning, made the bold claim in 2016 that “It’s quite obvious that we should stop training radiologists.” Andrew Ng, another AI pioneer, pondered about the future of radiologists in a twitter post, prompting a quick response from Eric Topol.

Let’s take a deeper dive into the application of AI for the interpretation of chest x-rays and see what value they have in radiology.

What are Chest X-rays?
Chest X-rays are the most common type of medical scan and use ionizing radiation (X-rays) to generate images of the chest for diagnosis of common medical conditions like pneumonia, heart failure, and bone fractures.

There are five main shades of black, white, and gray visible on a chest x-ray. Areas on the chest that are less dense such as air appear black since they attenuate less X-ray signal. Bones absorb more signal and appear more white.

How are Chest X-rays analyzed?
A normal posteroanterior chest x-ray looks like the following image:

Normal posteroanterior (PA) chest radiograph (X-ray).jpg

In medical school, we are taught to analyze these images in a systematic way, summarized by the ABCDEs:
A: Is the airway deviated or enlarged?
B: Are there any broken bones?
C: Is the heart (cardiac silhouette) enlarged?
D: Is the diaphragm normal?
E: Is there any edema or effusion?

What does a radiologist report look like?
When doctors order chest x-rays, they provide a reason for the exam and radiologists subsequently provide their impressions. For example, here is a report from a patient with worsening acute respiratory illness:

With this information, doctors confirmed the central line is placed in the right area, have evidence that the patient’s disease is worsening, and now have a couple potential diagnoses to guide treatment management.

How well does AI analyze chest x-rays?
Many high performing AI algorithms are trained from open, public databases. One large data set comes from the NIH Chest X-Ray database, comprising 112,120 images from 30,805 unique patients with 14 different diagnoses labeled mined from radiology reports. The data is open to public use and can be accessed here: . Another significant data set comes from CheXpert, a database of 224,316 images from 65,240 patients at Stanford Hospital also with 14 different diagnoses labeled from radiology reports (

Using these datasets, researchers have designed deep neural network algorithms to classify and localize disease. For example, Li et al ( was able to correctly classify these 14 different diseases with an accuracy (measured by AUC or area under the curve) ranging from 0.67 for pneumonia to 0.87 for cardiomegaly and effusion. Below is an image from their study showing the visualization of where the neural network identified disease in a patient.

Using the same data set, Rajpurkar, Irvin et al. designed a neural network and compared their accuracy in classifying these 14 diseases to analyses from nine radiologists ( The accuracy of AI was near equivalent to that of the radiologists:

Regardless of any naysayer, these results are incredible research achievements. These researchers deserve enormous recognition and applause for embarking on such ambitious projects.

How will AI affect radiologists in chest x-ray interpretation? 
The billion dollar question that still remains however is how AI will affect clinical medicine.

Eric Topol makes the astute argument that “validation of the performance of an algorithm in terms of its accuracy is not equivalent to demonstrating clinical efficacy.” An algorithm “with an AUC of 0.99 is not worth very much if it is not proven to improve clinical outcomes.” He ends by stating that “the field clearly is far from demonstrating very high and reproducible machine accuracy, let alone clinical utility, for most medical scans and images in the real-world clinical environment.”

In my view, Dr. Topol is 100% spot on. Regardless of these incredible research achievements, AI at present cannot provide the variety of analyses we request from radiologists, cannot provide interpretation for why a patient’s disease is worsening, and cannot guide the next steps of clinical diagnoses and treatment management. In the hospital, the radiologists are the cornerstone of disease diagnosis and are too valuable to replace with only a machine (at least in the near future).

A lot of the purported benefits of AI in radiology so far are theoretical and remain in the discussion sections of research papers. In brief, authors state that these algorithms may help triage patient disease severity, improve speed and precision of radiologists, and help in low resource settings.

1. Triage
Someone has to prove that AI can help in triage better than humans. A lot of triage in the hospital is based on the emergency room doctors and nurses judgment from a multitude of factors, including patient history, physical examination, laboratory findings, and imaging. Triage based on imaging alone is not enough.
2. Time and efficiency
Would AI help save time and improve efficiency? Or would they be the opposite if they provide a prediction that differs from a human? Who would you trust when a computer provides an interpretation different than a radiologist one? Perhaps AI can help create preliminary, automated reads of X-ray images before a radiologist performs a formal analysis. Will radiologists be better able to analyze images if they were all pre-read with AI? I hope so.
3. Utility in low resource settings
Why would a low resource setting invest their limited money in something of questionable utility like AI? What added benefit is AI over the interpretation of imaging from a general practitioner in these settings?

No one knows the answers to these questions. Ultimately, someone has to put their money down and prove that AI will improve clinical outcomes. AI does not need to be superior to a radiologist, but at the very least it must demonstrate value in some aspect of clinical care. Until then, AI will not even come close to replacing radiologists.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s