Researchers trying to improve health care with artificial intelligence Usually submit their algorithms to some form of automatic medical school. The software learns from doctors by digesting thousands or millions of x-rays or other data labeled by expert humans until it can accurately report suspicious moles or lungs showing signs of Covid-19 by himself.
A study published this month took a different approach – training algorithms to read knee x-rays for arthritis using patients as AI arbiters of truth instead of doctors. The results revealed that radiologists can have literal blind spots when it comes to reading x-rays of black patients.
Algorithms trained on patient reports did a better job than doctors at accounting for the pain experienced by black patients, apparently uncovering patterns of disease in images that humans usually ignore.
“This sends a signal to radiologists and other doctors that we may need to re-evaluate our current strategies,” says Said Ibrahim, a professor at Weill Cornell Medicine in New York, who studies health inequalities and has no participated in the study.
Algorithms designed to reveal what doctors don’t see, instead of emulating their knowledge, could make healthcare more equitable. In one comment About the new study, Ibrahim suggested it could help reduce disparities in people who undergo surgery for arthritis. African-American patients are about 40% less likely than others to receive knee replacement surgery, he says, even though they’re at least as likely to have osteoarthritis. Differences in income and insurance likely play a role, as do differences in diagnosis.
Ziad Obermeyer, author of the study and professor at the School of Public Health at the University of California at Berkeley, was inspired by the use of AI to probe what radiologists did not see in a medical puzzle. Data from a long-standing national health institute study on osteoarthritis of the knee showed that black patients and low-income people reported more pain than other patients whose radiologists had a similar score. The differences may arise from physical factors unknown to the knowledge keepers of the knee, or from psychological and social differences – but how do you tell them apart?
Obermeyer and researchers at Stanford, Harvard, and the University of Chicago created computer vision software using NIH data to investigate what human doctors might be missing. They programmed algorithms to predict a patient’s pain level from an x-ray. In tens of thousands of images, the software has discovered pixel patterns that correlate with pain.
When he receives an x-ray that he has never seen before, the software uses these patterns to predict the pain a patient would report feeling. These predictions were more closely related to patient pain than the scores assigned to radiologists by knee radiologists, especially for black patients. This suggests that the algorithms had learned to detect signs of illness that radiologists did not have. “The algorithm saw things beyond what radiologists saw – things that are more common causes of pain in black patients,” says Obermeyer.
History may explain why radiologists are not as proficient at assessing knee pain in black patients. The standard ranking used today comes from a small 1957 study in a mill town in northern England with a less diverse population than the modern United States. Doctors used what they saw to design a way to assess the severity of osteoarthritis based on observations such as shrunken cartilage. X-ray equipment, lifestyles and many other factors have changed a lot since then. “It’s no surprise that this fails to capture what doctors are seeing in the clinic today,” says Obermeyer.
The study is remarkable not only for showing what happens when AI is driven by patient feedback instead of expert opinions, but because medical algorithms have more often been seen as a cause of bias, not as a remedy. In 2019, Obermeyer and colleagues showed that an algorithm guiding the care of millions of American patients white priority over blacks for assistance in complex conditions such as diabetes.