| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1
Department of Radiology, The University of Arizona, 1501 N. Campbell Ave.,
Tucson, AZ 85724-5067.
2
Present address: Department of Radiology, San Francisco General Hospital, 100
Potrero Ave., San Francisco, CA 94110.
3
Department of Radiology, Veterans Affairs Medical Center, 3601 S. 6th Ave.,
Tucson, AZ 85723.
Received February 14, 2000;
accepted after revision June 20, 2000.
Address correspondence to W. K. Erly.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Thirty-five solitary pulmonary nodules that were examined by both posteroanterior and lateral chest radiography and on thin-section CT were retrospectively identified. Fourteen radiologists blinded to the results of CT assessed the nodules for the presence or absence of calcification using chest radiographs alone. The radiologists then assigned one of six values on the basis of their confidence in that assessment. The accuracy and confidence values for each nodule were analyzed on the basis of the presence or absence of calcification as seen on CT. Receiver operating characteristic (ROC) curves were generated.
RESULTS. The positive predictive value of a "definitely calcified" assessment was 0.93. Combining all levels of radiologists' confidence, the sensitivity of the chest radiograph in the detection of calcium was 0.50 and the specificity was 0.87. There was no difference in the confidence levels reported between the calcified and noncalcified nodules, and there was no correlation of nodule size with accuracy or confidence level.
CONCLUSION. The ability of radiologists to detect calcium in a solitary pulmonary nodule by chest radiography was low, as defined by the ROC data. Of the "definitely calcified" nodules, up to 7% may not be calcified and may be potentially malignant. Without documentation of long-term stability, a low threshold for recommending CT may be appropriate.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Aside from stability on sequential chest radiographs over a 2-year period, the single most reliable evidence that a solitary pulmonary nodule is benign is the presence of calcification [1, 2]. Many texts state that it is difficult to document reliably the existence of calcification on a conventional radiograph of the chest and that in the absence of obvious calcification, additional workup is indicated. If the pattern of calcification is homogeneous within the nodule, the interpretation of its presence or absence may be particularly subjective [3]. The ability to accurately define the existence of calcification within a solitary pulmonary nodule is vital to an appropriate and timely workup. To our knowledge, no prior study has tested the accuracy of the detection of nodule calcification on conventional radiographs of the chest, with thin-cut CT as the gold standard. Our purpose was to determine the accuracy of conventional radiographs in the evaluation of the solitary pulmonary nodule for the presence or absence of calcification.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The nodules were marked on both the posteroanterior and lateral images of the chest before evaluation. Fourteen board certified radiologists who were blinded to the CT findings were asked to review the chest radiographs and assign a number to each nodule on the basis of their confidence in determining the presence or absence of calcification. A traditional six-point receiver operating characteristic (ROC) scale was used. If the reviewers believed a solitary pulmonary nodule to be calcified, they were asked to assign a number on the basis of their confidence as follows: 6, definitely calcified; 5, probably calcified; and 4, possibly calcified. Similarly, if the reviewers believed a solitary pulmonary nodule to be noncalcified, they were asked to assign a number on the basis of their confidence as follows: 1, definitely not calcified; 2, probably not calcified; and 3, possibly not calcified. The reviewers were asked not to compare nodules, and each viewed the cases in the same sequence.
The numbers for each nodule were analyzed on the basis of the presence or absence of calcium as defined on CT. ROC curves were subsequently generated.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The average time between conventional radiographs and CT was 42 days, with the shortest interval being 2 days and the longest, 300 days. In the case of the 300-day interval, CT was the initial examination, and the nodule was completely calcified on that CT.
To assess overall diagnostic performance, the confidence data were used in ROC analysis to generate areas beneath the curves (Az). The individual Az values for each observer, the respective subspecialties, and years of practice experience can be found in Table 1. There were no significant differences in the Az as a function of years of experience (F test = 0.06, p = 0.982, df = 3,10), so the data were pooled. The average Az was 0.751. From these data the standard deviation (0.088) and range (0.306) were calculated.
|
The data were also analyzed to determine sensitivity and specificity values. For this analysis the ratings of 1, 2, and 3 for the noncalcified nodules proven on CT were true-negatives, and the 4, 5, and 6 ratings were false-positives. For the calcified nodules, 1, 2, and 3 were false-negatives, and 4, 5, and 6 were true-positives. The overall sensitivity was 0.50, and the overall specificity was 0.87. The overall positive predictive value was 0.81, and the negative predictive value was 0.59. The positive predictive value of a "definitely calcified" classification was 0.93.
Of the 224 noncalcified nodules (16 noncalcified nodules x 14 reviewers), a rating of "definitely not calcified" was rendered 67 times (30%). Of the 266 calcified nodule encounters (19 calcified nodules x 14 reviewers), a rating of "definitely calcified" was rendered 50 times (19%). The average rating was in the "probably" range for both types of nodules. There was no significant difference in the confidence level reported between the calcified and noncalcified nodules.
There were four instances in which a noncalcified nodule was judged to be "definitely calcified" (false-positive). Two (12.5%) of the 16 noncalcified nodules were misclassified by at least one observer. Three (21%) of the 14 observers classified at least one of the noncalcified nodules as definitely calcified. Three observers misclassified one nodule (Fig. 1A,1B); one of these observers erred on a second nodule (Fig. 2A,2B). (The observers all had more than 5 years' experience in radiology: two were involved primarily with body imaging, and one was a thoracic radiologist. All radiologists were actively reviewing chest radiographs as part of their clinical duties at the time of the interpretation.) One nodule was stable at 2-year follow-up and is presumed to be benign; the other was a squamous cell carcinoma. Both nodules were closely related to ribs. The contribution of the ribs to the relative density of the lesions may have caused the misclassification.
|
|
|
|
There was no correlation between confidence and nodule size (correlation coefficient [r]=0.22), and there was no correlation between decision accuracy and nodule size (r = 0.32). Observed frequencies for each type of response (true-positive, true-negative, false-positive, and false-negative) were compared with nodule location to calculate percentage of accuracy (Table 2) for each lobe of the lung. The percentage of accuracy for the right upper lobe was 7%, for the right middle lobe, 52%; for the right lower lobe, 36%; for the left upper lobe, 65%; and for the left lower lobe, 48%. The percentage of accuracy for the peripherally calcified nodule was 14%; for the laminated nodule, 64%; for the solidly calcified nodules, 59%; and for the centrally calcified nodules, 42%.
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CT was chosen to function as the gold standard because it is the modality commonly used to discriminate between indeterminate- and benign-appearing solitary pulmonary nodules and because resection or biopsy of a solitary pulmonary nodule characterized as benign on CT is generally not clinically indicated. CT has been shown to be an extremely accurate modality in differentiating benign from indeterminate solitary pulmonary nodules [4]. Our focus was on the issue of whether the nodule contained calcium on CT, not on the absolute presence or absence of microscopic calcification. The measurement of Hounsfield numbers was not performed because the measurement of such numbers, even with reference phantoms, has been shown to be unreliable [5, 6].
The mean Az for the ROC curves generated by the 14 observers was 0.763, with a range of 0.3062. An Az value of 0.50 is statistically equal to chance, and a value of 1.0, statistically equal to absolute diagnostic accuracy. There is no generally accepted Az value in the literature defining an accurate or good test [7]. One can only say that a value close to 0.50 is reflective of a less accurate test, whereas a value close to 1.0 is reflective of a more accurate test. Our interpretation of the Az value for the overall detection of calcification within the solitary pulmonary nodule on posteroanterior and lateral radiographs of the chest is that for our study, the value is reflective of a merely average test that was somewhat accurate.
The large range of Az values among the observers shows that there were broad differences in the accuracies generated by the various observers. Accuracy did not seem to correlate with years of practice experience (r = 0.373). The type of subspecialty training also did not seem to have a major effect on accuracy.
Sensitivity, as defined in our study, was the likelihood that a calcified solitary pulmonary nodule would be called calcified by the observer. Specificity, as defined in our study, was the likelihood that a noncalcified solitary pulmonary nodule would be called not calcified by the observer. Given that most malignant solitary pulmonary nodules are noncalcified both histologically [8] and on CT [4, 9], we believe that it is probably beneficial that our observers generated a higher specificity (0.87) than sensitivity (0.50). The sensitivity may be relatively low because a given observer may not wish to call a potentially malignant lesion calcified, so he or she underdiagnoses the presence of calcium. There is some support for this hypothesis in our data in that the percentage of "definitely not calcified" responses for the noncalcified solitary pulmonary nodules (30%) was higher than the percentage of "definitely calcified" responses for the calcified solitary pulmonary nodules (19%). The fact that the overall positive predictive value (0.81) was greater than the negative predictive value (0.59) also supports the observation that our observers were undercalling the presence of calcium. No treatment decisions were made on the basis of these interpretations. It may be that the judgement of "definitely calcified" would not have been made as frequently had this been the case. This circumstance could result in a lower sensitivity and higher specificity for the detection of calcium in clinical practice than those observed in this study.
Generally, if a radiologist is unsure if a nodule contains calcium, CT is performed. Therefore, the important decision made on the basis of the radiograph is whether the nodule is definitely calcified because if there is some uncertainty, additional workup is warranted. The data in this study indicate that when radiologists describe a nodule as "definitely calcified" (and therefore benign), they are correct 93% of the time (positive predictive value of 0.93). In this series, 7% of solitary pulmonary nodules called definitely calcified may not be calcified and thereby potentially malignant. Depending on the prevalence of benign granulomatous disease, Zerhouni et al. [10] believe as many as 60% of nodules found noncalcified on CT may be malignant, depending on the prevalence of benign granulomatous disease in the subject populations. Therefore, on the basis of the data in this study, an average radiologist may include up to four malignancies in every 100 solitary pulmonary nodules believed to be "definitely calcified" on a conventional radiograph. Radiologists should therefore use caution when attempting to assess the presence or absence of calcification within a solitary pulmonary nodule on standard posteroanterior and lateral radiographs of the chest.
In retrospect, both nodules that were misclassified as "definitely calcified" projected over a rib. It seems likely that the increased density of the rib played some role in the misclassifications. If the observers were to classify all nodules that project over ribs as indeterminate (categories 1-5), there would have been no misclassifications. This classification would have increased the measured specificity of chest radiography to 1.00—a perfect result. However, to create a perfect result ignores the fact that radiologists are human, and therefore their interpretations are often imperfect. Errors that in retrospect may have been avoidable are often made. At our institution, the global reimbursement for chest CT without contrast material is $277, and $34 for a two-view chest radiograph. To document stability, we follow nodules at 3, 6, 12, 18, and 24 months after identification. If oblique radiographs are also ordered, the total cost for the follow-up examinations is $204. For these reasons, we believe that without documentation of long-term stability, nodules similar to those included in this study (>5 mm without other signs of granulomatous infection) warrant CT. We believe that the negative impact of misdiagnosing calcium in these patients outweighs the $73 in cost differential between CT and oblique radiographs with 2-year follow-up.
There was no correlation between nodule size and decision confidence or accuracy. The smallest nodule included in this study was 6 mm in diameter, and the average size was 13 mm. This study does not address the accuracy of chest radiography in identifying nodules 5 mm or smaller. It may be that the posteroanterior and lateral chest radiographs are more accurate in these circumstances because the smaller a nodule, the denser it must be to be conspicuous.
This study also did not address other signs of a benign process that might alter evaluation. For example, in the setting of a fibrotic process in the apices, the judgment of "definitely calcified" in a small apical nodule may have a higher positive predictive value than that observed in this study. Similarly, if there were other evidence of granulomatous infection such as calcified hilar or mediastinal lymph nodes, the positive predictive value of the "definitely calcified" judgment may rise. "Obviously calcified" nodules were less likely to be included in this study than those more difficult to classify on chest radiography. This omission was not specifically by design but occurred partly because of the relative infrequency of these nodules and partly because of our requirement of a CT through the nodule within 1 year of the chest radiograph. To be included in this study, CT should have been performed on a "clearly benign" nodule for reasons other than to examine the nodule. However, to correct for this bias, patients included in this study were identified by reviewing the CT records only. This system allowed the inclusion of incidentally noted nodules that may have been judged "clearly benign" by chest radiography. Although the study design may allow a slight bias toward nodules that were more difficult to classify, we believe this bias does not alter the relative data and their diagnostic implications.
The accuracy for solitary pulmonary nodules in the right upper lobe was markedly less than that for solitary pulmonary nodules in other locations. The reason is uncertain and may be related to statistical variance rather than to some anatomic basis.
In summary, the overall ability of chest radiography to detect calcification in a solitary pulmonary nodule is low. In the absence of other signs of granulomatous infection, the positive prodictive value of a "definitely calcified" assessment is 0.93. Of these "definitely calcified" nodules, up to 4% may be malignant. Without documentation of long-term radiographic stability, CT may be appropriate.
Acknowledgments
We thank Kristina Lärka for editorial
assistance.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  A O Soubani The evaluation and management of the solitary pulmonary nodule Postgrad. Med. J., September 1, 2008; 84(995): 459 - 466. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. M. Marom "CT of the Solitary Pulmonary Nodule"--A Commentary Am. J. Roentgenol., May 1, 2008; 190(5): 1154 - 1155. [Full Text] [PDF]
|
|
|
|
|
|
E. A. Krupinski, W. C. Black, and F. S. Chew Methodology and Application of Prospective Reader Studies: Self-Assessment Module Am. J. Roentgenol., March 1, 2008; 190(3_Supplement): S29 - S34. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. P. Ko, C. C. Roberts, W. G. Berger, and F. S. Chew Imaging Evaluation of the Solitary Pulmonary Nodule: Self-Assessment Module Am. J. Roentgenol., March 1, 2007; 188(3_Supplement): S1 - S4. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. T. Winer-Muram The solitary pulmonary nodule. Radiology, April 1, 2006; 239(1): 34 - 49. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. Fischbach, T. Freund, R. Rottgen, U. Engert, R. Felix, and J. Ricke Dual-Energy Chest Radiography with a Flat-Panel Digital Detector: Revealing Calcified Chest Abnormalities Am. J. Roentgenol., December 1, 2003; 181(6): 1519 - 1524. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Ost, A. M. Fein, and S. H. Feinsilver The Solitary Pulmonary Nodule N. Engl. J. Med., June 19, 2003; 348(25): 2535 - 2542. [Full Text] [PDF]
|
|
|
|
 |
|
 B. B. Tan, K. R. Flaherty, E. A. Kazerooni, and M. D. Iannettoni The Solitary Pulmonary Nodule Chest, January 1, 2003; 123(1_suppl): 89S - 96S. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
