| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Original Research |
1 Department of Radiology, University of Occupational and Environmental Health,
School of Medicine, 1-1 Iseigaoka, Yahatanishi-ku, Kitakyushu 807-8555,
Japan.
2 Department of Radiology, School of Health Sciences, Kumamoto University,
Kumamoto, Japan.
3 Department of Radiology, Kurt Rossmann Laboratories for Radiologic Image
Research, The University of Chicago, Chicago, IL.
Received July 22, 2005;
accepted after revision October 6, 2005.
Partially supported by grants CA 62625 and CA 98119 from the United States
Public Health Service.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Thirty cases with newly developed chest abnormalities on chest radiographs and 90 negative cases were selected. All chest radiographs were obtained with a computed radiography system. For the 90 negative cases, subtraction images were classified into two groups: 33 clean images without misregistration artifacts and 57 images with some misregistration artifacts. Eight radiologists (four board-certified radiologists and four radiology residents) participated in observer tests and interpreted the original radiographs without and with subtraction images using an independent test method. The reviewing time for each radiologist was recorded in each case. The observers' performance was evaluated by use of receiver operating characteristic (ROC) analysis.
RESULTS. When subtraction images were available, the mean reviewing time per case was reduced significantly from 13.6 to 10.8 seconds for the cases with newly developed abnormalities (p < 0.001) and from 29.8 to 14.1 seconds for negative cases (p < 0.001). The reduction in the mean reviewing time with subtraction images was greater for clean images than for images with artifacts (17.7 vs 14.5 seconds, p < 0.001). The average mean area under the ROC curve value increased significantly from 0.942 without subtraction images to 0.988 with subtraction images (p = 0.025). There were significant differences in the sensitivity (0.963 with and 0.888 without the subtraction images) and the specificity (0.976 with and 0.899 without the subtraction images) (p < 0.001).
CONCLUSION. The temporal subtraction technique can reduce reviewing time and also improve diagnostic accuracy in the interpretation of chest radiographs.
Keywords: diagnostic radiology • digital radiography • lung disease • receiver operating characteristic (ROC) • reviewing time • temporal subtraction
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Our purpose in this study was to compare the reviewing time and diagnostic accuracy in the interpretation of radiographs without and with subtraction images and to examine whether the subtraction technique can contribute to improving radiologists' performance.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Technical details of the temporal subtraction method have been published previously [1, 7, 8]. The previous image was shifted and rotated to correct for variations in patient positioning. After global matching was performed with low-resolution images, a number of template regions of interest (ROIs) and the corresponding search area ROIs were selected from the previous and current images, respectively. Local matching of template ROIs with the corresponding ROIs and the shift values was determined for all pairs of selected ROIs by use of a cross-correlation technique. The previous image was nonlinearly warped according to local shift vectors. For further reduction of misregistration artifacts in the subtraction image, local matching and image warping were performed again with the first warped previous image and the current image. Finally, the subtraction image was obtained by subtraction of the second warped previous image from a current one.
Case Selection and Classification
The study was approved by our institutional review board, and written
informed consent was not required. At retrospective review of the thoracic CT
file between January 2000 and March 2003, we selected 30 consecutive cases
that had only one focal chest abnormality (mean size, 3.6 cm; range, 0.7-4.5
cm) on the basis of the following selection criteria: the diagnoses were
proven; current and previous chest radiographs were available; current chest
radiographs had only one focal chest abnormality that had newly developed when
compared with previous radiographs; and a focal chest abnormality was defined
as less than 50 mm in diameter identified on current chest radiographs.
The final diagnoses were 16 primary lung cancers, three pulmonary metastases, one mediastinal mass, one pleural effusion, and nine organizing pneumonias. Pathologic proof was obtained from surgical resection or fine-needle biopsy for all primary lung cancers (eight adenocarcinomas, six squamous cell carcinomas, and two small cell carcinomas), three solitary pulmonary metastases, and one mediastinal mass (inflammatory lymph node swelling). The primary tumors in the pulmonary metastases were colorectal carcinoma (n = 1) and renal cell carcinoma (n = 2). The diagnoses of one pleural effusion and nine organizing pneumonias were obtained by clinical follow-up. There were nine women and 21 men who ranged in age from 44 to 91 years (mean age, 69.2 years).
At retrospective review of a chest CT file between January 2000 and March 2003, 90 consecutive cases were selected for negative cases on the basis of the following selection criteria: current and previous chest radiographs were available, there were no perceptible interval changes between current and previous chest radiographs, and CT studies obtained at the same time as these radiographs also showed no interval change. Therefore, the negative cases included calcified nodules (two cases), strandlike opacities (four cases), and idiopathic pulmonary fibrosis (two cases) without perceptible interval changes. There were 40 women and 50 men who ranged in age from 17 to 81 years (mean age, 57.8 years). The quality of the subtraction images of the 90 negative cases was classified subjectively by two certified radiologists into two groups: 33 (36.7%) clean images without misregistration artifacts (Fig. 1A, 1B, and 1C) and 57 (63.3%) images with some misregistration artifacts (Figs. 2A, 2B, and 2C).
|
|
|
|
|
|
Display of Images
For the observer performance study, the current and previous radiographs
were mounted on a conventional viewbox as radiographs. Next to the viewbox,
the subtraction image was displayed on a liquid crystal display monitor (1,280
x 1,024 lines, 15-inch [38-cm] diagonal screen [RDT-154H, Mitsubishi]).
The observers were permitted to manipulate the monitor brightness and contrast
with a track ball or push buttons.
Observer Performance Study
Eight radiologists, four board-certified radiologists and four radiology
residents with 2-5 years of training, participated as observers. These
observers had undergone sufficient training on the subtraction images in daily
routine use. Before the test, the observers were informed that the purpose of
this study was to evaluate the radiologists' performance in detecting newly
developed abnormalities without and with the subtraction image; the role of
the subtraction image was to provide a second opinion; and this study included
120 cases that consisted of newly developed abnormalities (only one focal
abnormality per image) and normal images. The observers were blinded to the
number of cases with abnormalities and the characteristics of chest
abnormalities.
For the observer performance study, an independent test was used [2, 9]. The independent test consisted of two series of sessions, one with previous and current radiographs alone and the other with a pair of radiographs and a subtraction image. To reduce learning effects, the interval between reviewing sessions was at least 2 weeks. In the first session, half the observers (two residents and two certified radiologists) interpreted images without subtraction images, and the other half interpreted them with subtraction images.
In the second session, each half of the observers interpreted images in a series that differed from the first session. A continuous rating scale with a linemarking method [4, 9] was used for recording each observer's confidence level regarding the presence or absence of a newly developed abnormality. In each series, the observer marked his or her confidence level on a 7-cm line. Then the observers located the most likely position of newly developed abnormalities on each radiograph. The 30 positive cases were mixed with the 90 negative cases by use of a computer randomization method for each observer. The observers were informed that the reviewing time was not limited, but the actual reviewing time for each observer was recorded in each case. A research assistant measured the reviewing time with a stopwatch from the moment that the radiologist first viewed the images (when the images were displayed) to the moment that the radiologist marked his or her confidence level on the line. The time required to display or hang the images was not included in this measurement.
Data Analysis
The reviewing times with and without the subtraction images were compared
separately for the negative and positive cases. In addition, we evaluated the
relationship between the quality of images and the reviewing time. The
statistical significance of differences in the reviewing time was determined
by use of a two-tailed paired Student's t test.
Receiver operating characteristic (ROC) analysis was used for comparison of observer performance in the detection of chest abnormalities without and with the subtraction images. The area under the ROC curve (Az) was calculated by use of the computer program LABROC5 provided by Metz et al. [10]. The statistical significance of the difference in Az without and with the subtraction images was estimated by use of the Dorfman-Berbaum-Metz method [11]. In this study, the observer was required to identify the most likely position of newly developed abnormalities on each radiograph. Localizations within a true lesion were scored as true-positive events, and all other events were scored as false-positive. The data obtained in this way were used only for determination of the sensitivity and specificity. The differences in the sensitivity and the specificity for localization with and without subtraction images were analyzed with a Student's t test.
For all tests used, a p value of less than 0.05 was considered to indicate a statistically significant difference.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
|
The effect of the quality of the subtraction images on the mean reviewing time per case was also examined (Figs. 1A, 1B, 1C, 2A, 2B, and 2C). The mean reviewing time per case with the subtraction images was significantly shorter (11.9 and 15.3 seconds, p < 0.001) compared with that without the subtraction images (29.4 and 30.0 seconds) for 33 clean images (Fig. 1C) and 57 images with some misregistration artifacts (Fig. 2C), respectively. The reduction in the mean reviewing time with subtraction images was greater for the clean images than for images with some misregistration artifacts (17.7 vs 14.5 seconds).
Evaluation of Diagnostic Accuracy
Average ROC curves showed a significant improvement in the accuracy of
detection of interval changes with the temporal subtraction images
(Fig. 3). The diagnostic
performance of each reviewer in this study is shown in
Table 2. The average
Az value increased significantly from 0.942 without to
0.988 with the subtraction images (p = 0.025, Dorfman-Berbaum-Metz
method). There were significant differences in the sensitivity (0.963 with the
subtraction images, 0.888 without the subtraction images, p <
0.001) and the specificity (0.976 with the subtraction images, 0.899 without
the subtraction images, p < 0.001)
(Table 3).
|
|
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In our observer performance study, we analyzed the reviewing times separately for positive and negative cases. The reviewing time decreased for both positive and negative cases when temporal subtraction images were used. However, the reduction in the reviewing time was less for positive cases than for negative cases. One reason for this result seems to be that the observers knew that there was only one abnormality involved per image in the test cases. In positive cases, therefore, the observers did not spend too much time on the interpretation of radiographs whether with or without the subtraction images if they could easily detect an abnormality.
In negative cases, on the other hand, our results revealed a 52.7% reduction in the mean reviewing time with subtraction images. This dramatic result may be due to hesitation in diagnosing normal cases with confidence in the conventional interpretation of radiographs. Another reason for this result seems to be that the observers underwent sufficient training on subtraction images in daily routine use. In clinical settings, we believe that the extent to which subtraction images are used to reduce the reviewing time in the interpretation of radiographs ultimately depends on the degree to which radiologists understand the performance of the temporal subtraction system.
We separately analyzed the reviewing time obtained from the certified radiologists and the radiology residents. The results of our observer study indicated that the difference between residents and certified radiologists was not significant for the reduction in the mean reviewing time with subtraction images. It is important to note that the effect of using the temporal subtraction images was also beneficial for certified radiologists.
Misregistration artifacts are caused by a mismatch of normal anatomic structures in current and previous images. To determine whether the effect of the subtraction images on reviewing time was dependent on the quality of images, we analyzed separately the reviewing time obtained for subtraction images without and with misregistration artifacts. Our results indicated that the reduction in the reviewing time obtained from clean images without misregistration artifacts was greater than that with some misregistration artifacts. However, it is most important to note that the mean reviewing time even with misregistration artifacts was reduced significantly by 49%. It is not difficult for observers to distinguish between misregistration artifacts and chest abnormalities; therefore, it seemed that the reviewing time was not influenced detrimentally by some misregistration artifacts in the subtraction images (Figs. 2A, 2B, and 2C).
In our study, we attempted to address some limitations in previous reports regarding the analysis of the effect of subtraction images. Many previous studies have evaluated the subtraction images by observer performance studies with ROC analysis [1-6]. In ROC analysis, the observers were not required to indicate the location of a chest abnormality; therefore, it is not possible to know whether the radiologists specified the locations of true abnormalities. In the present study, it was possible to calculate the sensitivity and specificity by taking the location into consideration because we asked the observer to identify the most likely positions of abnormalities. Our results showed that both the sensitivity and specificity for detection of chest abnormalities were increased significantly by the use of subtraction images. The improved sensitivity indicates that subtraction images can prevent observers from missing chest abnormalities.
Similarly, the improved specificity means a decrease in false-positives by chest radiographs alone. This result indicates that the temporal subtraction technique has the potential to help avoid unnecessary CT examinations. Johkoh et al. [3] reported that temporal subtraction images did not improve the detection rate of solitary pulmonary nodules by chest radiologists, probably because many easy cases were included in their observer performance study. Since our observer performance study contained many cases with subtle abnormalities, we suppose that our study could reveal the usefulness of temporal subtraction images not only for residents but also for chest radiologists. In addition, the observers were blinded to the number of cases with abnormalities and the characteristics of chest abnormalities, which may simulate real clinical settings more appropriately when evaluating the effect of the temporal subtraction technique.
A limitation of our study design is that only one focal chest abnormality was evaluated in the analysis of the sensitivity and specificity. If diffuse lung disease or multifocal opacities had been included in our cases, we could not have required the observers to indicate the location of a chest abnormality. In our observer performance study, the radiologists' attention was specifically focused on the task of detecting one focal chest abnormality. In clinical situations, however, radiologists must often consider possible additional abnormalities. Therefore, the usefulness of subtraction images in regard to the reviewing time for diagnosing abnormal chest radiographs may be far greater in clinical situations than in this idealized observer test. In clinical situations, radiologists often spend time rearranging the order of films, retrieving additional comparison studies, and displaying or hanging the images. The time required for a number of work-flow steps related to the interpretation of radiographs was excluded from the reviewing time in our observer performance study.
In conclusion, we performed an observer study to evaluate the effect of the temporal subtraction technique on reviewing time and diagnostic accuracy in the interpretation of chest radiographs. Our results reveal that the temporal subtraction technique can contribute to improvements in radiologists' performance and diagnostic accuracy in the detection of a focal abnormality. Further evaluation will be necessary for establishing the role of the temporal subtraction technique in chest radiography in actual clinical situations.
Acknowledgments
We are grateful to Takayuki Ohguri for his contribution to this study.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  F. Girvin and J. P. Ko Pulmonary Nodules: Detection, Assessment, and CAD Am. J. Roentgenol., October 1, 2008; 191(4): 1057 - 1069. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. Li, R. Engelmann, K. Doi, and H. MacMahon Improved Detection of Small Lung Cancers with Dual-Energy Subtraction Chest Radiography Am. J. Roentgenol., April 1, 2008; 190(4): 886 - 891. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Kakeda, Y. Korogi, H. Arimura, T. Hirai, S. Katsuragawa, T. Aoki, and K. Doi Diagnostic Accuracy and Reading Time to Detect Intracranial Aneurysms on MR Angiography Using a Computer-Aided Diagnosis System Am. J. Roentgenol., February 1, 2008; 190(2): 459 - 465. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
