| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Commentary |
1 Department of Radiology, Rush North Shore Medical Center, 9600 Gross Point Rd., Skokie, IL 60076.
Received September 25, 2006; accepted after revision October 4, 2006.
Address correspondence to L. Berlin.
Keywords: general radiology practice • medical-legal issues
So wrote L. Henry Garland [1] in his classic pioneering article on diagnostic errors published in the American Journal of Roentgenology. Garland went on to relate that when one of his friends, a well-known professor of radiology, learned that Garland's research disclosed that radiologists missed about one third of roentgenologically positive findings, the friend expressed the hope that Garland would discontinue his "investigations in this field because they were so morale-disturbing." When other radiologists were confronted with this data, continued Garland, their usual reaction was "Well, in my everyday work this does not apply; I would do better than those busy investigators" [1].
Today, 48 years after Garland's 1959 article [1] and, in fact, 58 years after Garland first wrote about radiologic errors [2], it is appropriate to examine Garland's initial findings in greater detail; to review and summarize the myriad studies published over the next half century that have tested Garland's conclusions; and, finally, to ponder whether radiologists have made, or have failed to make, progress in reducing error rates in radiologic interpretation.
But first, let us ask ourselves, "Who was this man, Leo Henry "Harry" Garland, who had the audacity to inform a then-disbelieving radiologic community that radiology interpretations were fraught with error?
L. Henry Garland
L. Henry Garland was born in Ireland in 1903. The son of a general medical practitioner in Dublin, Garland immigrated to the United States in 1925 and became a U.S. citizen in 1934. During a 41-year radiology career, Garland held leadership positions at San Francisco General Hospital, Stanford Medical School, the U.S. Navy, the U.S. Veterans Administration, and the Letterman Army Hospital. Garland served as president of the American College of Radiology, the Radiological Society of North America, the California Academy of Medicine, and the Northwest Medical Association. He maintained active memberships in the American Roentgen Ray Society and numerous other medical and nonmedical associations.
In 1954, Garland received the Bronze Medal for Distinguished Service from the American Cancer Society and in 1960, the Gold Medal of the American College of Radiology. Two years later, he was granted an Honorary Fellowship in the Royal College of Surgeons in Ireland. Colleagues [3-6] have described Garland as a "thin, gnarly, sharpfeatured" individual, "with steel-gray hair, flashing brown eyes, a strong conviction, and keen wit," who was "scholarly, widely interpreted, well informed," and "so bright his intelligence could not be tested." Garland was "a superlative teacher, brilliant clinician, and investigator" and a person who gave radiology his "enthusiasm and genius, making it a cornerstone of modern medicine." They also pointed out, however, that Garland possessed a "sharp tongue and a fierce determination to his every act," could be a "thorny person if he disagreed with you" [4], occasionally was "a bully, riding over students, residents, and anyone else who could not or did not stand up to him," and was "caustic and domineering" [5]. Garland "never hesitated to join in a battle if one were in progress or to start one if a battle needed starting" [3]. He died of cancer on October 31, 1966.
L. Henry Garland was the author or coauthor of more than 150 scientific articles on many subjects encompassing both diagnostic and therapeutic radiology. It is his articles on radiologic errors and accuracy, however, for which Garland is most remembered and still frequently quoted today.
Garland's Scientific Studies on Radiologic Errors
In the 1959 AJR article on which this commentary focuses, Garland [1] introduced the subject of the accuracy of diagnostic procedures by summarizing investigations that revealed a "surprising" degree of inaccuracy in many nonradiologic clinical and laboratory tests. Garland referenced studies that found a 34% error rate in the diagnosis of myocardial infarction, an only 15% agreement among eight experienced internists on the presence of "the most simple signs" of emphysema when examining the chests of patients affected with that disease, a marked disparity in clinical evaluation of 1,000 school children of the indications for tonsillectomy, an agreement rate of only 7% among five experienced pediatricians determining clinically whether children were suffering from malnutrition, a 20% error rate in the interpretation of ECGs, a 28% error rate among 59 different hospital clinical laboratories in reporting the results of chemical analyses, and a 28% error rate among clinical laboratories in measuring the erythrocyte count.
Garland [1] then turned his attention to errors in radiologic diagnosis. The research projects from which Garland calculated error rates began with a 1944 study that was made to compare the diagnostic value of various radiologic recording media then used in mass surveys of the populace to detect pulmonary tuberculosis. Data were generated from interpretations rendered by experienced radiologists of 5,000 radiographic studies recorded both on 35-mm film and 14 x 17 inch (36 x 43 cm) film. Interindividual disagreement among the expert reviewers averaged 30%; intraindividual disagreement (one reviewer with himself later) averaged 21%. Interestingly, Garland had written or cowritten four articles on the subject of radiologic errors during the decade immediately before the 1959 AJR article: the first in 1949 in Radiology [2]; the second in 1950 in the AJR [7]; and two more in 1952, one in Radiology [8] and the other in The Lancet [9]. The studies on which Garland based his conclusions had also been the subject of articles by other researchers [10-12].
In the 1959 article, Garland [1] discussed reasons for the missing of lesions on chest radiographs. "There are times when an experienced physician `sees' a visible lesion clearly and times when he does not," explained Garland. Garland attributed this phenomenon to the "still unexplained human equation." Garland concluded his article by reiterating that experienced radiologists will miss about 30% of chest radiographs positive for radiologic evidence of disease, will overinterpret about 2% of those that are actually negative for disease, and will disagree with themselves 20% of the time. Today, these figures are readily acknowledged, but in 1959 they astonished, if not shocked, virtually all radiologists.
Garland's pioneering work reporting and analyzing radiologic error rates provided the impetus for a multitude of researchers to undertake their own studies regarding the accuracy of radiologic interpretations. A brief review of scientific articles published over the subsequent 48 years reveals confirmation of Garland's findings with an almost unbelievably high degree of consistency.
Garland's Studies on Accuracy Confirmed
In the decades after Garland's classic article, a number of investigators replicated Garland's findings relative to film radiographic studies [13-23]. In a 1976 study at the University of Missouri, an error rate of 30% was reported among staff radiologists in their interpretation of chest radiographs, bone studies, gastrointestinal series, and special procedures, [24]. Other researchers found that as many as 20% of colonic tumors were missed on lower gastrointestinal examinations [25-29]. Herman et al. [30] reported that a group of Harvard University radiologists disagreed on the interpretation of chest radiographs as much as 56% of the time. Additional studies conducted by researchers at major academic medical centers disclosed that from 26% to 90% of all lung carcinomas were missed by radiologists interpreting chest radiographs [31-33].
Numerous reports have documented similarly high error and disagreement rates for myriad other techniques used in radiologic practice, including sonography [34, 35], angiography [36], MR angiography [37], MR when evaluating lumbar disk herniation [38], thallium radionuclide heart scans [39], MR when evaluating rotator cuff injury [40], and MR when evaluating prostatic cancer [41]. More recent studies have confirmed a 35% error rate among radiologists interpreting radiologic studies obtained in patients who had undergone trauma [42-44].
Statistics disclosing inaccuracies in the interpretation of mammograms are startling [45-52]. A 1994 study at Yale University School of Medicine found on retrospective review of mammograms originally interpreted by experienced radiologists as normal that from 15% to 63% of breast carcinomas had been overlooked at the initial interpretations [53]. A University of Arizona study found that in 75% of mammograms initially interpreted as normal, breast carcinomas were seen on retrospective evaluation [54].
Evidence of Errors in Clinical Medicine
Garland's [1] 1959 revelations about the incidence of errors in clinical medicine have also been confirmed and expanded on by subsequent researchers. Agreement among academic faculty physicians performing physical examinations for spleen enlargement [55], liver enlargement [56], abdominal ascites [57], acute otitis media [58], and other assorted physical findings [59, 60] has been shown to be poor. Large autopsy studies have also uncovered frequent clinical errors and misdiagnoses, with error rates as high as 47% [61-63].
A recent Wall Street Journal article [64] reported errors ranging from 25% to 49% in pathologists' interpretations of biopsy specimens, and a recent article in a pathology journal reported a 24% error rate in laboratory results [65].
Error Rate Defined—And Its Impact on Patient Care
Garland [1] explained in his 1959 article that the error rate can be calculated in two different ways depending on the denominator used:
If a series of 100 roentgenograms contains 10 positive and 90 negative films, and a reviewer misses three of the positive films and overreads two of the negative films, he may be regarded as having only a 5% error. On the other hand, since the series of 100 roentgenograms is being examined to detect patients with disease, the reviewer who misses three of the 10 positive films has an error rate of 30%. Coupled with an overreading of two of the 90 negative films, the combined error rate in the example mentioned is about 32%.
In virtually all of the studies of error rates to which this commentary has thus far referred, the denominator consists of a preselected number of abnormal radiologic studies. Thus, if a radiologist participating in a research project is given 100 radiographs known to be abnormal and misses 30 of them, the error rate is obviously 30%. This should not be construed as indicating that radiologists have an average 30% error rate in their everyday practices. Several studies have measured error rates of radiologists by determining how many errors were committed among a large number of radiologic studies interpreted by the radiologist in a practice situation over a selected period of time—in other words, with a different denominator.
Siegle et al. [66] in 1998 reviewed imaging interpretations rendered in the radiology departments of six community hospitals affiliated with The University of Texas and found a 4.4% mean rate of interpretation error. Borgstede et al. [67] reviewed the performance of more than 250 radiologists who had interpreted more than 20,000 examinations as part of the clinical testing of a performance-improvement product, RADPEER (American College of Radiology). They found an all-case disagreement rate of 3% to 3.5%. Soffa et al. [68] reviewed the results of a quality improvement study conducted among 26 radiologists who interpreted 6,703 cases and found an overall disagreement rate of 3.48%. In other words, if the denominator consists only of radiology studies that harbor abnormalities, the error rate averages 30%; if the denominator consists of an everyday mixture of abnormal and normal cases, as is usually found in daily practices, the error rate averages 3.5% to 4%.
Although Garland [1] did not discuss the extent to which radiologic errors compromised the medical care of patients, it should be emphasized here that none of the studies referred to in this commentary reflect the degree (if any) to which patient care is jeopardized because of reviewer misinterpretation. "Extrapolation of reviewer error to medical care is complex," stated Herman et al. [30]. Although some radiologic errors may indeed result in serious injury or mismanagement of a patient, many others are either corrected quickly or, fortunately, not clinically important and thus exert no adverse effect on the management of the patient.
Reducing Radiologic Errors
Garland [1] did not focus on methods of reducing radiologic error other than to write that errors "are correctable with care...training and experience...increased use of consultation...and...continued attempts at elucidation and correction of the factors involved." It is beyond the purview of this commentary to discuss in detail methods of reducing radiologic error. Nonetheless, reviewers will find at the end of this commentary references that focus on the influence of clinical history [69-77], comparison with previous radiologic studies [78-81], time spent interpreting studies [82-85], double interpretation [86-88], volume of cases interpreted per unit of time [82, 89], and other relevant factors that can reduce the occurrence of errors in diagnostic radiology [89-92].
Digital Radiography and Computer-Aided Detection
Fifteen years ago, Greene [93] observed:
In computed digital radiography, we now have a powerful raw diagnostic tool of unprecedented power with which to apply psychophysical principles to the task of detecting subtle lung nodules. Digital technology can potentially improve screening by providing compatibility for computer-assisted detection of lesions and by permitting flexible manipulation of gray-scale and edge enhancement of the images presented to the radiologist.
Over the past decade, innumerable articles reporting on and discussing the results of these new technologies have appeared in the radiology literature [94]. A series of recent articles [95-98] extensively reviewed major developments in digital radiography and computer-aided detection (CAD). Several conclusions can be drawn from these articles and those referenced in their bibliographies. Clearly, CAD improves sensitivity and decreases variability of interpretation by raising the level of the radiologist's suspicion in examinations in which a potential abnormality has been highlighted.
CAD has been shown to be of value in assisting radiologists in reducing radiologic errors and improving radiologic interpretation, particularly in mammography [99, 100]. However, whether digital radiography and CAD will once and for all substantially reduce radiologic error, the rate of which has not changed in the nearly half century since Garland's classic article [1] appeared, is yet to be determined. Let us be mindful of the observations made by one radiology researcher (Jasinski R, unpublished manuscript):
Technology doesn't solve, but only displaces, the problem of perceptual error to a new and different technology, offering the opportunity to make a whole new, and maybe longer, list of mistakes. You can't buy excellence in a box, although you can keep buying newer and more expensive boxes.
Even if good methods to improve perceptual accuracy are found, there might be obstacles to implementation. The reality of radiologic error seems to persist unchanged over decades, impervious to improvement. When education is used to remedy some errors, leaders may go on to make new and different errors or resume making errors that they had in the past gotten beyond.
Radiologic Errors: Past, Present, and Future
Even experienced physicians are found to have a measurable degree of "observer error" due apparently to the so-called human equation.... This is a baffling problem.... In evaluating pairs of serial roentgenograms, one experienced physician is apt to disagree with another in about one third of the cases and (on review) to disagree with himself in one fifth of them.... Comparable degrees of error occur in many forms of clinical practice.
So concluded L. Henry Garland [1] in his 1959 article.
Ten years later, Tuddenham [101] observed:
So long as human beings are responsible for roentgen interpretation, the process will be subject to the variability of human perception, and "reviewer error" due to the interpreter's failure to perceive critical detail will remain a problem of importance to the clinician.... The processes governing search behavior and mediating visual perception are correspondingly complicated, and our knowledge of them is fragmentary. Enough is known, however, to suggest that errors of perception are, for the most part, not the result of carelessness or willful bias on the part of the radiologist but rather a consequence of the physiologic process of perception.... Errors of perception are an unavoidable hazard of "human condition".... The ultimate solution to the problem of "reviewer error" is not yet clear.
British radiologist P. J. A. Robinson [102] has commented similarly: "Although technology has made enormous progress in the last century, there is no evidence for similar improvement in the performance of the human eye and brain."
In 2004, author Malcolm Gladwell [103], writing in The New Yorker, made the following insightful observation regarding uncertainty, one of the causes of errors in radiologic interpretation:
The reason a radiologist is required to assume that the overwhelming number of ambiguous things are normal, in other words, is that the overwhelming number of ambiguous things really are normal. Radiologists are, in this sense, a lot like baggage screeners at airports. The chances are that the dark mass in the middle of the suitcase isn't a bomb because you've seen a thousand dark masses like it in suitcases before, and none of those were bombs—and if you flag every suitcase with something ambiguous in it, no one would ever make his flight. But that, of course, doesn't mean that it isn't a bomb. All you have to go on is what it looks like on the X-ray screen—and the screen seldom gives you quite enough information.
In the 10-year period beginning in 1949 with publication of his presidential address delivered at the 1948 annual meeting of the Radiological Society of North America [2] and culminating with his classic 1959 AJR article [1], L. Henry Garland in five published articles [1, 2, 7-9] almost single-handedly awakened the radiology community and focused its attention on the fact now known and accepted by all, but then not previously known and accepted only with great reluctance, that a substantial degree of observer error is prevalent in radiologic interpretation.
The pioneering work of Garland [1] and his associates has been affirmed and reaffirmed by hundreds of researchers in the ensuing half century. Tuddenham's [101] 1969 lament that "the ultimate solution to the problem of `reader errors' is not yet clear" still remains not yet clear in 2007. Garland ended his classic 1959 AJR article by exhorting future radiologists to continue "attempts at elucidation and correction of the factors involved" in causing radiologic errors. Although spearheaded and urged on by Garland's charge, radiologists have not yet been successful in elucidating and correcting the factors involved in causing radiologic errors. Their efforts to do so will undoubtedly continue for many years to come.
References
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  C. G. Filippi, B. Schneider, H. N. Burbank, G. F. Alsofrom, G. Linnell, and B. Ratkovits Discrepancy Rates of Radiology Resident Interpretations of On-Call Neuroradiology MR Imaging Studies Radiology, December 1, 2008; 249(3): 972 - 979. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. W. Jasinski Citing Web Articles Am. J. Roentgenol., January 1, 2008; 190(1): W76 - W76. [Full Text] [PDF]
|
|
|
|
 |
|
 H. Singh, S. Sethi, M. Raber, and L. A. Petersen Errors in Cancer Diagnosis: Current Understanding and Future Directions J. Clin. Oncol., November 1, 2007; 25(31): 5009 - 5018. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Berlin Radiologic Errors and Malpractice: A Blurry Distinction Am. J. Roentgenol., September 1, 2007; 189(3): 517 - 522. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
