Research News: 2002 - 2003 - 2004

Outcomes Research and Evidenced-Based Radiology
Author: Rebecca Smith-Bindman, MD

Outcomes research focuses on the results of health care decisions. It addresses how interventions and treatments impact important patient outcomes, such as morbidity, mortality, and quality of life, rather than more immediate and isolated results, such as whether a lesion is detected or whether a tumor shrinks following treatment. Put simply, outcomes research tries to ascertain how patients are helped and harmed by what we do to them, so that we, and they, can make more informed decisions.

This focus on outcomes is relatively new in the field of radiology, but it is clearly just as important to know how a new test affects patient outcomes, as it is to know whether that test improves diagnostic accuracy. For example, while a screening test such as spiral CT may detect lung cancer cases prior to the time they would have become symptomatic, it may potentially harm patients if, through false positives, it leads to large numbers of unnecessary open lung biopsies in patients without cancer, or if it has no impact on those with cancer because the detected cancers would not have caused symptoms in the patient's lifetime.

Outcomes research requires thinking about radiology as it affects patients in the context of their disease, other diagnostic tests, and treatment interventions. Typically, outcomes research requires large numbers of subjects and different types of study designs from those common in radiology research. The members of UCSF's Radiology Outcomes Research Laboratory (RORL) bring diversity of experience and training, not only in medicine and radiology, but also in epidemiology, biostatistics, demography, anthropology, public administration and policy, and population health. We also actively collaborate with investigators from the UCSF Departments of Medicine, Obstetrics and Gynecology, Pediatrics, and Epidemiology and Biostatistics.

Under the direction of Rebecca Smith-Bindman, MD, current RORL research concentrates on the impact of radiological imaging on women's health. Recent studies have addressed: (1) the use of prenatal ultrasound for diagnosis of birth defects, chromosomal abnormalities, and reproductive outcomes; and (2) screening mammography, including screening strategies and programs, physician predictors of mammographic accuracy, and optimal ages for stopping mammographic screening.

Ultrasound for the Detection of Down Syndrome Fetuses
Second trimester ultrasound is widely used to detect Down Syndrome (DS) fetuses, but the accuracy of this method is unknown. The RORL recently published a systematic review in the Journal of the American Medical Association that found the existing published literature does not support the use of prenatal ultrasound as a screening test for DS. The review suggested that in otherwise low-risk women, if ultrasound is used as a screening test, it will lead to more losses of healthy fetuses as an expected complication risk of amniocenteses (triggered by the false positive ultrasound) than cases of DS detected. To empirically estimate the performance of prenatal ultrasound as a screening test for DS and other chromosomal abnormalities, the RORL is currently collaborating with the California Maternal Serum Expanded AFP Program, a state- mandated DS screening program, to prospectively evaluate the role of ultrasound in women who participate in this program. We have recently collected the ultrasound results on the cohort of 19,694 women who participated in this program between 1999-2000 by using a standardized data form administered in 29 sites in California. The ultrasound results will be compared with serum biochemical screening results and birth outcomes to determine if ultrasound adds additional information to that obtained by serum testing alone. Related studies using these data will study the impact of ultrasound on the detection of a broad range of birth defects, as this sample also includes large numbers of cases of neural tube defects (n=250), severe growth restriction (>1000), and other defects. Also using these data, the RORL will investigate associations between physician characteristics, such as specialty (radiology versus perinatology) and training, and the accuracy of interpretation of prenatal ultrasound. These studies will provide information that women and clinicians can use when deciding about prenatal testing.

The RORL recently completed an analysis to determine if patient outcomes in the United Kingdom vary by the types of methods used to screen for DS. This study of all of England and Wales for ten years, including 5,980,519 pregnant women, 335,184 invasive prenatal tests, and 12,047 DS diagnoses, compared population outcomes for local areas that predominantly screened for DS via (a) serum biochemistry, (b) first trimester nuchal translucency, and (c) maternal age. Our main outcomes measures were effectiveness (percentage of cases prenatally diagnosed) and efficiency (number of invasive prenatal tests [amniocentesis and chorionic villus sampling] performed to diagnose these cases). Areas that used serum biochemistry or nuchal translucency screening detected 50% more DS cases before birth than areas that used advanced maternal age as their dominant method of screening (52% and 49% vs. 35% respectively) and had lower live-born DS rates. Screening with serum biochemistry or nuchal translucency also was associated with the performance of fewer invasive procedures to diagnose each DS case than screening based on maternal age (62 and 61 vs. 90 invasive procedures per DS case detected respectively). Thus, screening for DS based on serum biochemistry or nuchal translucency is more effective and efficient than screening based on advanced maternal age.

Surprisingly, most professional guidelines in the U.S. (American College of Obstetrics and Gynecology, American College of Medical Genetics, and U.S. Preventive Task Force, etc.) continue to endorse advanced maternal age screening for DS. The RORL has started a similar study in the U.S. to evaluate amniocentesis rates, prenatal DS detection rates, and prevalence of live-born DS rates. They will collect data from hundreds of cytogenetic laboratories and U.S. State Birth Defect Registries to try to better understand current practices related to prenatal screening for birth defects, as well as differences that are seen in actual clinical practice based on the screening methods used.

Screening Mammography
Although randomized trials of screening mammography have shown a 30% reduction in breast cancer mortality among screened women, women older than age 74 were not included in any of the trials. Thus, the optimal age at which to stop mammographic screening remains unknown. The RORL published an analysis in the American Journal of Medicine that evaluated the effectiveness of screening mammography in women ages 70-79, among approximately 700,000 women in California, in whom 6,000 were diagnosed with breast cancer. They found a substantial benefit of mammography in older women, but raised questions to the degree to which older women were being over-diagnosed with breast cancer, analogous to prostate cancer in elderly men.

A number of studies in the RORL research projects are evaluating the impact of screening mammography on patient outcomes in elderly women (such as stage of disease at diagnosis, breast cancer treatments, and breast cancer and total mortality). These studies will use national Medicare claims data linked to national tumor registry data from the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute (NCI). These data include screening mammography utilization and outcomes on 70,000 women diagnosed with breast cancer from 1991-1996 from across the U.S., and their age-matched controls, and it is hoped these studies will provide important information for physicians and patients to use when deciding about what age to cease mammographic screening. As part of this analysis, we are analyzing differences in outcomes of screening mammography based on patient demographics and co-morbidity to determine which women have the greatest potential benefit of undergoing mammography. A separate study will focus on causes of racial disparity (higher rates of breast cancer mortality in the African-American population) in breast cancer mortality.

Despite the clear benefits of screening mammography, its accuracy is only modest; it produces large numbers of false positive examinations (5-20%) leading to high rates of biopsy, and misses 10-20% of true cancers. Additionally, wide differences across physicians have been reported. Several studies in the RORL are analyzing physician predictors of the accuracy of interpretation of screening and diagnostic mammography. One study uses data from the Breast Cancer Surveillance Consortium (BCSC), an NCI-funded consortium of mammography registries in seven U.S. states. These data describing the prospective interpretation of over 1,000,000 mammograms, including approximately 5,000 diagnosed with breast cancer, have been linked with tumor registry data and data from the American Medical Association Master File on physician characteristics. The RORL is currently evaluating physician predictors of mammographic accuracy (e.g., annual volume of interpretation and years since training) among the approximately 300 physicians who interpreted these examinations to try to better understand how to improve the accuracy of mammography.

The RORL is nearing completion of a comparative study of differences in the accuracy of screening mammography, and methods used to diagnose cancer, between the United States and the United Kingdom. This collaborative effort between the RORL, the BCSC, the Centers for Disease Control and Prevention, and the U.K. National Health Services Breast Screening Program, includes nearly 6 million mammograms obtained between 1996-1999. Smith-Bindman recently presented the results of this analysis at a Global Summit on Mammography held in Milan, Italy. The dramatic results found that recall and open surgical biopsy rates were two-fold higher in the U.S. compared to the U.K., without significant differences in the cancer detection rates. Efforts to improve mammographic screening in the U.S. clearly should aim at lowering recall rates without substantially reducing cancer detection rates.

Summary
The tremendous upsurge of diagnostic testing over the last decade has made it increasingly difficult and challenging to make appropriate decisions about how to interpret tests, how to choose between different tests, and how to determine when imaging is indicated. Additionally, health care payers, providers and patients increasingly require evidence that imaging is useful, cost efficient, and beneficial. Outcomes research will guide these decisions, and if radiologists want to influence how and when these new technologies are used, they must become active participants in understanding how such tests impact patients. Through outcomes research, the RORL will shape and advance the use of radiology by informing clinicians on currently optimal practices and by developing research techniques and strategies that will truly enhance patient health and quality of life.

At present, we have only scratched the surface of what outcomes research and evidence- based radiology can achieve. The Radiology Outcomes Research Laboratory, through the generous support of the Department of Radiology, can provide the space, tools, and guidance for medical students, residents, fellows, and faculty members interested in working with our group to evaluate outcomes related to diagnostic testing.