| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Perspective |
1 Department of Quantitative Health Sciences, Wb4, The Cleveland Clinic, 9500
Euclid Ave., Cleveland, OH 44195.
2 Division of Radiology, The Cleveland Clinic, Cleveland, OH.
3 Department of Cardiovascular Medicine, The Cleveland Clinic, Cleveland,
OH.
4 Department of Hematology/Medical Oncology, The Cleveland Clinic, Cleveland,
OH.
Received September 29, 2006; accepted after revision February 17, 2007.
Address correspondence to N. A. Obuchowski.
OBJECTIVE. Randomized clinical trials (RCTs) using disease-specific mortality as the primary outcome are the gold standard for evaluating the efficacy of screening tests. These trials require thousands of subjects and 8–10 years of follow-up; often the imaging technology has changed by the end of the trial.
CONCLUSION. We propose the incidence of symptomatic disease as an alternative to disease-specific mortality. This endpoint is sensitive to the benefit of screening, correlates with patients' quality of life and societal costs, and can dramatically reduce the sample size and follow-up requirements of RCTs.
Keywords: advanced symptomatic disease • disease-specific mortality • randomized clinical trials • screening
Despite major breakthroughs in the treatment of cancer and heart disease, these two disease entities remain the major causes of death in the United States and Western Europe. It is increasingly recognized that further reduction in morbidity and mortality can only be achieved by early detection and prevention of disease progression. In fact, over the past 20 years there have been remarkable advances in imaging technology, allowing detection of early disease, or disease risk factors, before the onset of signs and symptoms.
Yet, it is surprisingly difficult to show the efficacy of screening tests. Randomized clinical trials (RCTs) are considered the gold standard for evaluating the efficacy of screening tests, with disease-specific mortality as the most commonly used outcome measure. This outcome measure encompasses the most critical patient outcome (i.e., death), and because it focuses on the disease that is the target of screening, it is more sensitive to the benefits of screening than is all-cause mortality (i.e., all deaths). However, RCTs of screening tests typically require tens of thousands of subjects, 8–10 years of follow-up, and cost tens of millions of dollars [1, 2]. Furthermore, by the end of the trial the imaging technology has often been improved; thus, the results may underestimate the benefit of screening, and the clinical applicability of the results may be jeopardized.
Because of the increasing speed of new technological developments and expanding therapeutic options, there is, therefore, a need for valid, alternative clinical endpoints that could be used in shorter-term, smaller-sized RCTs for assessing the efficacy of screening tests. We propose the incidence of advanced symptomatic disease as a valid alternative to disease-specific mortality. The incidence of advanced symptomatic disease will, in many situations, meet the criteria for a valid surrogate endpoint; however, regardless of its validity as a surrogate endpoint, it is a true clinical endpoint that is sensitive to the benefit of screening, correlates strongly to patients' quality of life and societal costs, and reduces the sample size and follow-up requirements of RCTs.
Clinical Endpoints in RCTs of Screening Tests
We consider a two-arm RCT of asymptomatic subjects; the goal of the study is to evaluate the efficacy of screening. Subjects in the screening arm undergo a form of screening (e.g., chest CT, mammography, or calcium scoring) for a particular target disease or risk factor (e.g., lung cancer, breast cancer, or atherosclerosis). If the screening test detects an abnormality, then additional follow-up tests are performed and early treatment or risk reduction [3] is begun. Subjects in the control arm do not undergo this form of screening. We examine the hypothesis that this form of screening is beneficial to subjects in detecting early disease and consequently preventing or delaying the development of advanced disease [4].
Disease-Specific Mortality
The most commonly used outcome in these trials is disease-specific mortality. It is determined at a particular follow-up time t (for example, t = 3 years from the start of the study). It is calculated by counting the number of subjects who die from the target disease before time t [5] (see Appendix 1 for details.). Typically, subjects who die as a result of a complication from treatment for the disease (e.g., death during a surgical resection of a tumor) are also included.
The advantages of disease-specific mortality as an endpoint are that it can be ascertained through public records even in patients who are lost to follow-up, it is the most critical patient outcome, and it is sensitive to the benefits of screening. A disadvantage is that it does not incorporate some negative effects of screening, such as inappropriate treatment after a false-positive test result, detection and treatment of pseudodisease, and long-term effects of radiation. A second limitation is that it is sensitive to misclassification biases, such as "sticky-diagnosis bias" and "slippery-linkage bias" [6]. Thus, study results based on disease-specific mortality may be distorted because of underdetection and biased ascertainment of disease [7]. Third, RCTs using disease-specific mortality as the endpoint require extremely large sample sizes and long follow-up periods.
Incidence of Advanced Symptomatic Disease
The incidence of advanced symptomatic disease is calculated by determining the number of subjects who, before time t, develop one or more signs or symptoms of advanced disease (including death) that, on further diagnostic testing, are attributable to advanced target disease. A clear operational definition for the presence or absence of symptoms and signs of advanced disease must be established before the start of the study. These signs and symptoms should be indicative of advanced disease. For example, in breast cancer screening, the signs and symptoms should correlate with metastatic disease, not signs and symptoms of the primary tumor. The rationale is that breast cancer without metastases has a very high cure rate; thus, signs or symptoms of local disease would not correlate well with mortality or diminished quality of life.
Consider screening for lung cancer. Chronic cough, hemoptysis, chronic chest pain, shortness of breath, wheezing, and recurrent bronchitis or pneumonia are common signs and symptoms of advanced disease. These signs and symptoms are not, however, specific to lung cancer. An operational definition for advanced lung cancer would take into account the subjects' reported presence, severity, and duration of these symptoms, recent changes in symptoms, and the correlation between symptoms and findings on diagnostic (not screening) imaging examinations. An outcome review committee, blinded to subjects' study arm allocation, would evaluate patient-reported signs and symptoms over time based on responses to symptoms questionnaires and then determine whether study subjects had advanced symptomatic disease. Note that the operational definition of advanced disease may include symptoms resulting from treatment for lung cancer; it is appropriate, however, to include these symptoms because this situation is analogous to the inclusion of treatment deaths in calculating disease-specific mortality.
In coronary heart disease prevention trials, myocardial infarction and sudden death due to ischemic heart disease might be considered appropriate symptoms. Operational definitions for these symptoms and a blinded outcome review committee would again be necessary.
The incidence of advanced symptomatic disease is most useful as an alternative to disease-specific mortality when the symptoms appear much earlier than death, if disease-specific death occurs at all. This condition occurs in two interrelated situations: The first is when the survival time between symptoms and death is long. In this situation, an RCT using the incidence of advanced symptomatic disease will require a shorter follow-up than an RCT that must follow subjects until death. For example, subjects who develop symptoms of colon or breast cancer typically have several years of survival (approximately 20–22 months for colon cancer and 3 years for breast cancer).
The second is when the percentage of subjects with disease-specific symptoms who die from other causes is high. This situation occurs when there is a high cure rate of the target disease even after advanced symptoms appear or when the survival time after symptoms is long enough that the subject succumbs to competing causes of death. In either case, there will be many more events in an RCT using the incidence of advanced symptomatic disease compared with an RCT focused on terminal events. Thus, the RCT using the incidence of advanced symptomatic disease will require fewer subjects.
Advantages of the Incidence of Advanced Symptomatic Disease Endpoint
The advantages of using the incidence of advanced symptomatic disease as the primary endpoint in RCTs of screening tests are the following: First, its use reduces the required sample size and length of follow-up, thus reducing the costs of RCTs compared with trials using disease-specific mortality. This is illustrated by two examples in a later section. Second, the prevention of symptoms in an asymptomatic population is, of itself, evidence of benefit to the population. Thus, showing reduction in the frequency of the development of symptoms is evidence of screening efficacy [8]. The Multi-Ethnic Study of Atherosclerosis (MESA) investigators [7] share this viewpoint, in which the progression of subclinical disease is correlated with clinical events, defined as coronary heart disease deaths and nonfatal myocardial infarctions. Third, the incidence of advanced symptomatic disease, if defined appropriately, will correlate with reduced quality of life and also with larger medical and societal expenses. A quality of life instrument can be administered throughout the trial to assess the utility values of subjects with advanced symptomatic disease, some of whom will progress to death, and cost–utility analyses can be performed. Fourth, as with disease-specific mortality, the incidence of advanced symptomatic disease is not inherently biased by lead time, length, or overdiagnosis as is the length of survival from the time of diagnosis [4].
Disadvantages of the Incidence of Advanced Symptomatic Disease Endpoint
Some of the disadvantages of the incidence of advanced symptomatic disease are analogous to the disadvantages of disease-specific mortality. As with disease-specific mortality, the incidence of advanced symptomatic disease does not incorporate some negative effects of screening, such as inappropriate treatment after a false-positive test result and treatment of pseudodisease. It is also sensitive to misclassification bias (ascertainment bias)—that is, symptoms may be more likely attributed to the target disease in the screened than in the control group (sticky-diagnosis bias). Patient-reported symptoms are particularly vulnerable to this bias because screened patients with screening-detected findings will likely be more aware of their symptoms. It is important, therefore, to minimize this bias by regularly scheduled follow-up assessment of all study subjects' signs and symptoms using reliable symptoms questionnaires along with strict operational definitions of advanced symptomatic disease and using a blinded outcome review committee. Note that misclassification bias reduces the gains in power achieved by using symptoms as an endpoint. An example of the reduction in power is given in Appendix 2.
Another limitation is that the incidence of advanced symptomatic disease is affected by subjects lost to follow-up. In studies where there is a high rate of subjects lost to follow up, the incidence of advanced symptomatic disease will have reduced power because symptom events may be censored. Because death can be identified and dated via public records even if the subject is lost to follow-up, we consider disease-specific mortality as an "event" in the calculation of the incidence of advanced symptomatic disease. Thus, the power of RCTs using the incidence of advanced symptomatic disease will have at least as much power as trials using disease-specific mortality.
Last, use of this endpoint in an RCT requires a clear operational definition of advanced symptomatic disease. This can be challenging if the signs and symptoms are diverse and non-specific. Validated, reliable symptoms questionnaires, when available, should be used to document signs and symptoms over time. Changes in signs and symptoms must be correlated with diagnostic findings in an unbiased manner. The operational definition of advanced symptomatic disease must be established during the design phase of the study. Unlike mortality, there is the risk that different RCTs of the same disease could use different definitions of advanced symptomatic disease, perhaps complicating future meta-analyses.
The Incidence of Advanced Symptomatic Disease as a Surrogate Endpoint
In this section we assess the validity of the incidence of advanced symptomatic disease as a surrogate endpoint and discuss statistical properties of valid surrogate endpoints.
Criteria for Surrogate Endpoints
Suppose we consider disease-specific mortality rate as the true endpoint in
an RCT. Then the incidence of advanced symptomatic disease is a valid
surrogate endpoint for disease-specific mortality rate if it meets the
following two criteria [9]:
First, subjects always experience symptoms of the target disease before experiencing death from the target disease. This criterion is always met except when death may be the first clinical symptom of the target disease. For example, a fatal myocardial infarction might be the first indication of coronary heart disease. We propose that disease-specific death be considered an event in calculating the incidence of advanced symptomatic disease.
Second, screened subjects who experience symptoms have the same probability of death from the target disease as unscreened subjects who experience the same symptoms. In other words, an asymptomatic subject may be screened and receive early treatment, but if the disease continues to progress to advanced disease or if it recurs and produces symptoms of advanced disease, then that subject's history of screening and early treatment has no positive or negative effect on the mortality rate. Before an RCT, the validity of this criterion could be evaluated using existing data from the literature. Specifically, data from nonrandomized registries can be used to estimate the mortality rate of subjects who became symptomatic despite past screening. This mortality rate would then be compared with published disease-specific age-matched cure rates.
Properties of Surrogate Endpoints
Clearly, not all subjects with symptoms will die from the target disease.
Thus, the absolute risk reduction in an RCT using the incidence of advanced
symptomatic disease will not be the same as the absolute risk reduction in an
RCT using disease-specific mortality. If, however, the incidence of advanced
symptomatic disease is a valid surrogate endpoint for disease-specific
mortality, then the risk ratio estimated from a trial using the incidence of
advanced symptomatic disease will be the same as the risk ratio estimated from
a trial using disease-specific mortality. The risk ratio is the ratio of the
incidence of advanced symptomatic disease for screened to unscreened
subjects.
The use of surrogate endpoints in RCTs is somewhat controversial [10]. There is the risk that a positive effect on a surrogate endpoint will ultimately be shown to be detrimental to the subjects' clinical outcome. For example, two major antiarrhythmic drugs, encainide and flecainide, were shown to reduce arrhythmia yet found later to cause a threefold increase in overall mortality [11]. Conversely, there is the possibility that the surrogate endpoint will fail to show benefit when there is clinical benefit. This can occur when there are multiple pathways in which the therapy affects the true endpoint, and the surrogate is not part of every pathway. The incidence of advanced symptomatic disease is not subject to these criticisms because it is intermediate to the endpoint of death. It is itself a legitimate "clinical endpoint," defined as "a characteristic or variable that reflects how a patient feels, functions, or survives" [12].
Application to Completed and Ongoing RCTs
We chose a completed RCT and an ongoing RCT evaluating the efficacy of a
screening program as examples to illustrate the potential effect of the
proposed endpoint on the reduction of study resources and increase in study
power.
The Malmo mammography screening trial [2] involved 21,088 women invited for mammographic screening at baseline and five incident screens and 21,195 controls. The study results were published after 11 years, at which time the authors found a statistically insignificant reduction in breast-cancer mortality of 17% (risk ratio of 0.825). There were 69 breast-cancer deaths in the screened group and 84 deaths in the control group (p = 0.237). The observed risk ratio was 69 of 21,088 / 84 of 21,195 = 0.825. If an RCT using the incidence of advanced symptomatic disease as its endpoint had been performed, then there would have been many more study events than the 153 cancer deaths.
The published article reports that there were 588 breast cancers detected in the screened group (64% detected at screening, 17% diagnosed in the intervals between screenings, and 18% diagnosed in women noncompliant with screening), compared with 447 breast cancers detected in the unscreened group. The number of cases of advanced symptomatic breast cancer in this study is unknown; however, we speculate here about plausible numbers of advanced symptomatic breast cancers for screened and control subjects. The minimum number of advanced symptomatic breast cancers needed in this study to reach statistical significance is 193 screened cases and 235 control cases.
These numbers could be achieved under the following scenario: 53% of unscreened subjects with diagnosed breast cancer either presented or developed advanced signs or symptoms of breast cancer (which seems plausible because all 447 of these unscreened subjects likely presented with some signs or symptoms at the time of diagnosis) and, correspondingly, 33% (or fewer) of the screened subjects with diagnosed breast cancer either presented or developed advanced signs or symptoms (which also seems plausible because 206 of these 588 cases were not detected at screening; thus, only 206 likely presented with some signs or symptoms—53% of the 206 detected cancers would be about 109 cases of advanced symptomatic disease). Under this scenario, an RCT using the incidence of advanced symptomatic disease as its endpoint would have yielded a statistically significant (i.e., p < 0.05) finding (see details of this hypothetical analysis in Appendix 3).
The ongoing National Lung Screening Trial (NLST) [1] randomized 50,000 smokers with equal allocation to either screening with CT or screening with a chest radiograph (25,000 per arm); there were two annual incident screens. Based on the NLST study protocol and using estimates of the incidence of lung cancer, lung cancer mortality, and the accuracy of CT from a published cost-effectiveness analysis [13] (see Appendix 4 for details), we used a Markov model to simulate this RCT. We estimated that at 5 years with 25,000 subjects per arm, the power using disease-specific mortality is 82%. On the other hand, under a best-case scenario, if this RCT used the incidence of advanced symptomatic disease as its primary endpoint, then a similar power (i.e., 83%) could be achieved at 4 years with 15,000 subjects per arm.
Conclusion
The purpose of screening is to detect early disease to prevent or delay the development of advanced disease [4]. The two major causes of death in the United States and Western Europe, heart disease and cancer, are preventable or treatable if diagnosed early, optimally years before symptoms occur. Because of the long, latent, subclinical disease development, the disease-specific mortality rate, although capturing the benefit of screening, does so at high cost in terms of time, resources, and money. These difficulties in designing screening trials can have several negative effects. On the one hand, some screening tests will never be appropriately evaluated and valid screening and prevention strategies will be discarded because of a lack of resources for the performance of an RCT with mortality endpoints. On the other hand, popular tests will be performed without proper scientific evaluation despite possible detrimental effects. For example, no RCTs have been published to evaluate the efficacy of whole-body screening, yet in 2003 there were 161 sites in this country offering whole-body screening to people willing to pay for this test [14], up from 88 sites only 2 years earlier [15]. Similarly, contrast-enhanced coronary CT angiography (CTA) is currently promoted for a variety of clinical indications including screening for coronary artery disease without sufficient evidence-based data [16].
An important advantage of an RCT using the incidence of advanced symptomatic disease is that the benefit of screening, if any, is observed earlier than in an RCT using disease-specific mortality. As other authors have pointed out [17, 18], at the beginning of an RCT for a screening test, the mortality rates of screened and unscreened subjects are often similar or even favorable to no screening. This is because the benefits of screening have not yet been realized. However, with repeated screening and sufficient follow-up, the reduction in mortality rate due to screening is eventually realized. Because disease-specific symptoms can occur substantially earlier than death, there is great potential for savings in follow-up time.
The disadvantages of advanced symptomatic disease as an endpoint are that it requires a strict operational definition of advanced symptomatic disease that must be established before the study begins, and a blinded outcome review committee will usually be required.
The incidence of advanced symptomatic disease offers the opportunity to study new screening tests in a randomized controlled setting with fewer subjects and shorter follow-up. The ability to provide study results with a more prevalent endpoint approach would yield timely results and is particularly critical in the fast-changing environment of medical imaging. Further investigation of the incidence of advanced symptomatic disease as an endpoint in RCTs should be undertaken.
APPENDIX 1. Disease-Specific Mortality Rate
Disease-specific mortality rate is calculated as:
 |
We assume that all events would be identified; that is, if a subject was lost to follow-up and if an event occurred after the subject was lost to follow-up, then the event (i.e., death) would be determined via public death records. The number of person-years at risk is equal to t' – e, where t' is the smaller of (t, time at which the subject died), and e is the number of study months before the subject was enrolled into the study (i.e., e =0 at the start of the study).
APPENDIX 2. Incidence of Advanced Symptomatic Disease
The incidence rate of advanced symptomatic disease is calculated as:
 |
Not all events are observed because of loss to follow-up. Thus, the number of person-years at risk is equal to t'–e, where t' is the smaller of (time at which the subject experienced symptoms of the target disease, time at which the subject was lost to follow-up, or t).
APPENDIX 3. Malmo Screening Study
The published study [2] gives the number of breast-cancer deaths for each year since the start of the study for both subjects invited for screening and those not invited. It also provides the total number of breast cancers detected in each study arm. The risk ratio based on the disease-specific mortality rates is 69 / 21,088 to 84 / 21,195, or 0.825, which does not reach statistical significance (p = 0.237, chi-square test). For the same risk ratio of 0.825, we considered the number of advanced symptomatic disease events needed to yield statistical significance. The minimum numbers of events needed for statistical significance are 193 of 21,088 screened subjects and 235 of 21,195 controls. Thus, if the number of subjects experiencing advanced symptomatic disease in the screened and unscreened groups remained in the same ratio as the breast cancer deaths (a valid assumption for a surrogate endpoint) and if at least 235 of the 447 detected breast cancers in the unscreened group reached advanced symptomatic disease, and if 193 (or fewer) of the 588 detected breast cancers in the screened group reached advanced symptomatic disease, then an RCT based on the incidence of advanced symptomatic disease would have reached statistical significance. These numbers of events would occur if 53% of non–screen-detected breast cancers presented or developed advanced signs or symptoms and no more than 22% of screen-detected breast cancers developed advanced disease.
APPENDIX 4. National Lung Screening Trial (NLST)
To simulate the ongoing NLST, the following transition probabilities for the Markov model were assigned: annual incidence rate for developing preclinical disease = 0.0043 [9], early treatment cure rate = 0.7 [9], and treatment cure rate after symptoms appear = 0.07 [13]. The average survival time after symptoms of advanced disease was 14 months in this model.
The sensitivity and specificity of CT were assumed to be 0.93 and 0.81, respectively [13]. There was a prevalence screen and two annual incidence screens [1]; the compliance rate was assumed to be 0.85 for both incident screens [1]. To achieve the mortality rates expected in the trial [1], the prevalence of undetectable preclinical disease at the first screen was set to 0.04, the prevalence of detectable preclinical disease before the critical point was set to 0.03, and the prevalence of detectable preclinical disease after the critical point was set to 0.01. We assumed that 15% of study subjects had small-cell lung cancer and 85% had non–small-cell lung cancer [13]. We assumed 4.5% 30-day mortality after surgical resection [9], 14% risk of complications with follow-up diagnostic testing, and 2% risk of death from these complications [13]. We assumed an annual lost to follow-up rate of 15% for controls and 10% for screened subjects.
With a sample size of 25,000 per study arm per RCT, we estimated the power using lung-cancer specific mortality (including deaths that occur during diagnostic testing or treatment) to be zero at 3 years and 82% at 5 years. With a sample size of 15,000 per study arm, the power using the incidence of advanced symptomatic disease is estimated to be 0.50 at 3 years, 0.83 at 4 years, and 0.94 at 5 years. We also considered the reduction in power due to misclassification bias. If 5% of study subjects experience treatment-related symptoms similar to the disease-specific symptoms, then the power of the study is reduced to 0.13 at 3 years, 0.47 at 4 years, and 0.76 at 5 years.
References
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
