| Position and Concept Papers |  | ||
Colorado Medical Society Physician Specific Data CONCEPT PAPER |
INTRODUCTION
The Colorado Medical Society (CMS) supports the use of information to improve the quality of health care for Coloradans. We believe that those involved in health care decisions, specifically the purchasers, consumers, and providers of care, should have access to information that allows for informed decisions and continual quality improvement. We define quality along three axes: quality of care (effectiveness of care in affecting outcomes); quality of service (access to appropriate services and patient satisfaction); and quality of cost (efficiency of health care delivery).
The purpose of this position paper is to provide the Colorado State Legislature, other public organizations, and other interested audiences with the Medical Society's appraisal of the feasibility, utility, limitations, and hazards of the collection and publication of physician-specific data by public organizations. Also outlined is our recommendation of how public and private groups, with participation of the CMS, should proceed with the responsible and useful analysis of physician-specific data.
OBJECTIVES FOR PHYSICIAN SPECIFIC DATA
Information is a very powerful tool that can be used wisely and productively. It can also be used either intentionally or unintentionally in non-productive and harmful ways. We believe that when considering the use of physician-specific data, it is essential that policy-makers have a clear understanding of their own goals and objectives for such a process. Without clear goals and objectives, the risk of the use of information having untoward effects is much too great to be outweighed by uncertain benefits. This document outlines the potential utility of collecting and publishing physician-specific data in meeting these different objectives.
What are the potential objectives for reporting physician specific data? We recognize that physician specific data could mean the use of an individual physician's data, or the use of logical aggregates of several physicians' data. Potential objectives for using individual physician's data include:
BASIC PRINCIPLES FOR PUBLIC REPORTING OF PHYSICIAN-SPECIFIC DATA
Before collecting, analyzing or publishing any physician specific data, the following questions must be asked: 1) What question is it designated to answer? 2) Is the appropriate data available, accurate and valid? 3) Is the severity adjustment and other analysis sufficiently sophisticated to report valid results on such small numbers? and 4) What will the consequences be to good physicians if the data is not accurate and valid?
The data used for analyzing physician-specific performance should be accurate, complete, and valid. When reporting gets to the level of the individual physician, it is unconscionable to allow for errors in the data to claim the career of any physician as a sacrifice for improving the data. Data errors may be diluted when reported by hospital or other units of aggregation, but this will not be the case for individual physician profiling. While a rich data source, administrative databases have important liabilities, especially when used to track physician specific information. Data entry and medical record abstraction is inconsistent across hospitals and has not been studied widely in outpatient settings. There is no agreed upon criterion for a data element even as seemingly simple as the date of admission and discharge. Where does the potential for error exist?
A. Attribution of Care
Assigning responsibility for care is at times straightforward and noncontroversial; for example, it is usually the primary care physician who is responsible for health maintenance screening tests such as mammograms or Pap smears although a single, acute care visit by a patient to a physician may not imply this relationship.
A significant caveat to administrative data is the fact that hospital or outpatient medical record abstraction systems are not sufficiently sophisticated to identify which physician, of those having treated the patient, is responsible for the outcome of the patient. For instance, there is always confusion and little attention paid to who is the attending physician of record versus who admitted the patient and who is responsible for care after the admission. In part, this is because the incentives for payment do not rely on the identification of the physician of record. Because they are designed for billing, they are focused on what was done, not who did it. Thus, while it is fairly simple to assign an episode of care to a hospital, it is difficult to assign an episode to the physician who is truly responsible for the outcome. In a complicated case (the usual in most hospitalized patients), the primary physician may not be responsible for the outcome of several consulting physicians' treatment.
Additionally, the general concept that physicians are singly responsible for outcomes has been proven to be wrong. Patient outcomes are the result of a process in which the physician is one part. In all hospital settings, the physician is operating as a member of a multidisciplinary team; for example, hospitalized patients receive care from nurses, consulting physicians, and various technicians. In these cases, a systems approach is often better in evaluating care (i.e., evaluating the institution). Also to be considered are the individual patient's lifestyle decisions, for which the physicians and hospitals cannot be held accountable.
B. Diagnosis
Inpatient health care outcome and cost data are derived from hospital administrative databases or inpatient claims data and hospital severity adjusted outcomes systems. While inpatient diagnostic coding problems have been more widely studied than outpatient diagnoses, both suffer from difficulties including: (1) such coding systems often do not capture clinically relevant data, (2) the accuracy and completeness of data intended for billing may not be suitable for clinical evaluations, and (3) there are substantial differences in reliability and validity between different coding entities.
There are research techniques used to improve data reliability. These techniques include asking questions of the data that don't rely on fine coding distinctions, requiring that diagnoses be submitted by more than one source, or requiring that diagnoses be accompanied by a prescription drug or procedure that confirms the diagnosis. These research techniques require sophisticated software and are currently used almost exclusively in academic research centers. However, even when such techniques are used to confirm a specific diagnosis, there are clinical conditions where the diagnostic coding is inadequate to address a clinical situation [Joillis].
There are multiple examples of where current diagnostic coding does not capture the spectrum of clinical presentations and different treatment strategies. For instance, the diagnostic coding for carotid stenoses available in databases will not differentiate who has a greater than 70 percent blockage and is symptomatic (a good surgical candidate) versus a patient who has a 30 percent blockage and no symptoms (for whom surgery is not recommended).
Claims data for patients' sociodemographic characteristics and procedures (such as mammograms) may be more reliable and complete than diagnoses. Such data may be used for looking at individual physician's procedure rates, but such a report is not useful unless it could somehow be adjusted for the physician's practice content, areas of expertise, and referral base or individual patient preference.
C. Outcomes
1. Mortality - At the hospital level, mortality data contribute to the body of information that may be used in the evaluation of outcomes. Where mortality outcomes are higher or lower than expected, hospital staff may be able to use this information to identify quality improvement projects. Mortality data are readily available and are generally accurate.
At the individual physician level, however, mortality data are less helpful. Mortality is a relatively rare event, especially if one considers all patients treated by a given physician over any time period. Consider the physician who has deaths among his patients during a 6 month period, and then two of his patients die in month 7. It is conceivable that the physician would be identified as having "excessive" mortality that month when, in fact, the deaths were unavoidable - perhaps even expected. Such information could produce undeserved damage to a competent physician's reputation and could conceivably fail to identify an incompetent physician.
The complexities of accurately defining co-morbidities and capturing this information from the medical record makes it virtually impossible to adjust for severity of illness when we consider mortality data. A physician who admits only the most severely ill patients may have a proportionately higher in-hospital mortality rate compared to the physician admitting less ill patients.
Finally, to reiterate the problem with assigning responsibility, for a death it is not at all unusual to find three or more physicians participating in the care of an individual patient; if the patient dies, to which physician should his or her death be assigned? Even if the treating physician is correctly identified, this information is commonly miscoded in large databases. Also, we must recognize that the physician may not be directly responsible for every adverse event that befalls a patient under his or her care; nursing and ancillary personnel, the physical plant, and the activities of other individuals in the hospital also affect a patient's morbidity and mortality. It is not appropriate to attribute these events to the patient's physician, when they are beyond his or her control.
2. Morbidity - Many of the concerns raised for mortality are also relevant here. One of the major problems with attempts to measure physician performance with regard to morbidity outcomes centers around defining "morbidity". The post-operative phlebitis that one doctor would classify as a complication of surgery another might classify as an unavoidable adverse event. As we have problems even defining morbidity, we are unlikely to do well at measuring it. The use of cross-sectional data compounds the problems described above insofar as such data do not allow identification of temporal relationships between events. In other words, co-morbidities may be incorrectly identified as complications.
D. Patient Satisfaction
Patients can provide data that is not routinely collected in claims data or medical records. Limitations to their use, according to Lambird, are subjectivity and faulty recall. Physicians fear that such surveys may reward poorer practices such as giving antibiotics on demand for viral infections, willingness to prescribe narcotics or other drugs of potential abuse, or diagnostic work-ups that are not justified. Most of these objections might be overcome if the questions were less open-ended and focused more on the quality of the interactions. For example, rather than asking if a patient is "satisfied", such surveys might ask how the physician handled differences in opinion (e.g., did the physician explain why they were not complying with a patient's request? did the physician offer alternatives? did the physician have a plan if the offered therapy was not successful?). Likewise, instead of general questions about waiting time, a survey might ask what the physician did if a waiting time was anticipated (e.g., notify the patient of being behind schedule? offer to reschedule? etc.). However the questions are asked, patient satisfaction data must be evaluated in light of the appropriateness of the care provided.
E. Analysis
1. Sample Size, Rate Stability, and Rare Events - When rates are reported, there should be a minimum number of patients in the physician's sample to ensure that the rates are meaningful and stable. A 100 percent rate based on one patient, or a 50 percent rate based on two patients is likely to reflect individual patient characteristics rather than physician practice patterns.
Even with larger samples, a rare bad outcome or other rare event that could not have been prevented can produce a statistically significant result that could inappropriately be used to unfairly judge the quality of care of a physician. There are many examples in clinical medicine: a true knot in the umbilical cord is an extremely rare condition that can lead to a fetal demise in a term pregnancy yet gives no warning of its presence and, if not impossible, is extremely difficult to diagnose prior to fetal
demise. Fetal demise at term is similarly rare. The fetal demise rate of an obstetrical care provider caring for a pregnancy with this problem would be significantly higher than his or her peers, for an event that was unavoidable. A Berry aneurysm is a very rare cause of headaches in young people; if it bursts, the person will die. A physician who cares for such a young person could have his or her mortality rates compare unfavorably to peers, again due to an outcome that could not be avoided.
2. Severity and Other Rate Adjustment - Outcome rates should generally be adjusted for factors that independently affect the outcome. Such factors may be age, gender, stage or severity of the illness, and any concurrent illnesses. For example, a study released information on "unnecessary" hospitalizations for conditions which should be amenable to outpatient care [JAMA]. The study looked at conditions such as pneumonia. The study did not account for patients with concurrent illnesses. Thus, patients with pneumonia who had emphysema, used oxygen at home, or were on immunosuppressive drugs were counted in the same category as a 23 year old otherwise healthy patient. Clearly, such comparisons are not appropriate.
Severity of illness can also affect an outcome rate -- either with individual patients or within a whole practice (where it is referred to as case mix). An example of individual severity is that the five-year survival outcome is quite different for breast cancer detected as a small lump with no lymph node involvement from a cancer which when detected has spread not only to the local lymph nodes but to the liver. A physician's practice may also have a "severity factor" or "case mix". Case mix differences may be due to specialty, referral patterns within a region, and/or availability of certain services at the affiliated hospital. For example, both a family practitioner and a rheumatologist may care for patients with rheumatoid arthritis; however, the rheumatologist is more likely to take care of those patients who have disease beyond joint involvement (e.g., rheumatoid involvement of the lung or heart) or need reconstructive surgery. Likewise, there are obstetricians who receive regional referrals for high risk pregnant patients because their hospital has an intensive care nursery. Additionally, there are orthopedists who specialize in total hip revisions and receive regional referrals for such complicated cases.
There are many severity systems available today. None are flawless. They fall into two categories: administrative and medical record systems. Administrative systems (like that used in the Cincinnati outcomes project) are based on discharge diagnoses and are thus affected by: the fallibility of coders and DRG Groupers; the number of fields available for diagnoses; the sequence of diagnoses; coding or not coding co-morbidities or complications. Procedures and complications, as a result, tend to justify themselves. Severity may be increased by complications and is often a result of how the patient looked at discharge. The systems can be "gamed." Conversely, medical record abstraction systems examine clinical information in the record by strict algorithms. They measure the severity of illness of the patient at admission as well as how they fared during the hospital stay (better, worse or unchanged). Complications do not contribute to admission severity but are accounted for in morbidity (what happened after admission). Complications are construed as morbidity, not as severity.
Both types of severity adjustment systems however, carry over the same caveat that are seen with administrative data as it applies to the assignment of the responsible physician-inaccuracies. Both types of severity systems also become less valid when dealing with the small numbers of patients that a physician may admit in a given period. There are limitations in the severity adjustment for all currently available severity adjustment systems. Factors which clearly impact outcomes such as quality of life, socioeconomic status, and psychological status at the time of admission are not included in severity adjustment.
3. Outcomes to Be Analyzed - Performance measures should be linked to better patient outcomes. The use of indicators is intended to use a simple, single measurement in time to capture quality of care. However, indicators with their simplistic approach run the risk of rewarding bad practice. For example, one indicator that was developed measured anemia after total hip replacement with the belief that anemia signaled longer, or prolonged surgical time. However, physicians who did not fail the indicator were found to be routinely (without regard to symptoms) transfusing patients. While there was no evidence that patients with a mild anemia had poorer outcomes or prolonged hospital stays, those receiving transfusions were put at risk of transfusion reactions or transfusion-transmitted hepatitis [Soumerai].
In addition, the standards of comparison used in physician-specific data should be based on scientific evidence of what constitutes excellent care. Standards should not depend on comparing utilization rates to those of other physicians and looking for statistical rather than clinical differences. By way of example, a third party payer in Washington State was concerned with the expense of using one antibiotic compared with another and generated a utilization report of physicians using the more expensive antibiotic. They wrote "outlier" physicians about their "over-utilization" of the more expensive drug. In fact, the physicians who differed statistically from others, were using the drug appropriately in the counties of the state where there was an increase in resistant strains of microorganisms that were causing ear infections.
We do not currently have strong scientific evidence to cover all aspects of patient care as often our technology has outstripped our technological assessment. An example of this is that when and how often we should use fetal ultrasound is still under investigation. Likewise, we have different recommendations for who and how often screening mammograms should be performed. However, as evidence accumulates linking certain types of care with improved patient outcomes, this evidence can be used to form standards of care on which performance can be evaluated. Patient care issues which do not have strong quality of evidence should be left to researchers and are not good candidates for physician-specific performance reports.
AVAILABILITY OF DATA
In the past in Colorado, hospital data were readily available and reported by the Colorado Health Data Commission. The Commission published a severity-adjusted hospital outcomes report. All hospitals record a physician identifier for admissions; all that would be required for physician profiling for hospital care would be the use of a common identifier.
A great deal of care provided by most physicians, however, occurs outside of the hospital, where existing data sets become more problematic. In HMO's such as Kaiser Permanente, good data on utilization of services are available, but processes for measuring severity of illness, patient outcomes, and even diagnosis and procedures are only now in the preliminary stages. Other managed care plans rely on billing information. Such information depends on coding at the physician's office, has no severity adjustment available, and again, cannot be linked to outcomes at this time. Other systems, such as commercial insurance/indemnity plans, are known to have even larger problems with providing accurate and valid data. The implementation of a severity scoring system for outpatient data, even if technically feasible, would not be economically reasonable for most if not all physicians, especially those providing primary care.
CAN PHYSICIAN SPECIFIC DATA BE USED TO MEET POTENTIAL OBJECTIVES?
Objective 1: To identify "bad" physicians - There are already multiple systems in place which we would expect to work better than the use of available data. These include the Colorado Board of Medical Examiners, (who publish publicly their lists of censured physicians), the credentialing committees of all hospitals, CFMC, the National Practitioner Data Bank, and the quality assurance committees and credentialing/privileges processes of all hospitals. We do not believe there would be added utility in publishing available physician-specific data, and there are potential serious adverse effects of publishing such data.
Objective 2: To rate the quality of physicians - We do not believe that the current available measures are useful in rating the quality of physician care. As physicians rating the care given by our peers, we require direct observation of the physician so we can take into account several factors: the physician's medical abilities (which include problem solving ability, information processing, and procedural skills), the appropriateness of the physician's clinical management, and the physician's interpersonal skills and concern for patients. We are looking for excellence in medical judgement, patient satisfaction, appropriateness, quality, and outcomes (beyond mortality). None of these characteristics are available through the objective analysis of routinely collected health data.
From medical school through active practice, physicians form strong opinions about how "good" another physicians is. While we use standardized exams in school, training and for Board certification, we usually view such achievements as confirmatory of opinions formed from watching other physicians -- their approach to a problem, their problem solving techniques, and the way they respond to patients. We share our evaluations about other physicians with our patients through our referral practices. However, this art of evaluation leaves us equally vulnerable when asked to judge physicians with whom we do not have experience. We believe there is a need to develop objective standards of excellence, and that physicians should actively help in this development, to be shared with the public. However, we also believe that the current science of evaluating physicians using indicators or profiles is in its infancy, and that few if any physicians would rely on the proposed physician-specific indicators to choose a physician for themselves, their family, their friends, or their patients.
Objective 3: To rate the cost effectiveness of physicians - This relies on being able to measure quality and then decide whether variations in cost reflect variations in quality. Perhaps even more at issue is, what would be done with this information? If we did manage to rate physicians by quality, and higher ranking physicians got more business and charged more, what would that mean for the patient who could only afford the less expensive physician?
Objective 4: To provide information on practice style and scope - There may well be utility in gathering and publishing this information as a public service. This is being done now as a marketing technique by hospitals, and a more objective measurement, review and publication process could serve a public need.
Objective 5: To aid in developing practice guidelines - This is an exciting potential objective for the use of physician specific data, linked to outcomes. A good example of such a process is seen in efforts in the area of Worker's Compensation. There are several cost containment projects in Colorado and other states relating to use of data on case costs. The primary purpose is to establish the cost effectiveness of the medical case management model. The Colorado Compensation Insurance Authority (CCIA) established a Preferred Provider Network in 1988, whose primary purpose was to reduce medical and compensation payments through a managed care network. Data from a 1992 analysis demonstrated a 30% savings on overall costs per case for time loss injuries using the PPN as compared to injured workers cared for outside the network. The CCIA used its claims paid data for the analysis, and the individual physician network provider groups were identified by provider numbers. Physician specific data was not used in the analysis. Patient and employer satisfaction surveys were conducted on those injured workers whose claims were included in the study. Each member of the PPN received information on the aggregate data from all PPN members and an individual analysis for self comparison.
In Oregon, the State Accident Insurance Fund (SAIF) has engaged a consultant to study claims data for treatment and outcome pattern analysis. The study includes faculty and staff time from the Oregon Health Sciences University who are currently analyzing an outcome survey on carpal tunnel treatment. The SAIF corporation has an advisory board made up of physicians and workers' compensation agencies from Oregon, California, Washington, Utah and Colorado. The board has had considerable input via written communication and met for two days in October, 1993 in Salem, Oregon to review and advise on data already produced from the system. The general goal is to use the patterning data to help establish treatment guidelines for the most common injuries and illnesses.
The Colorado Division of Workers' Compensation (DOWC) has developed a set of treatment guidelines for low back pain and went on to develop consensus guidelines for a number of other injuries. CFMC and the DOWC negotiated for the CFMC to develop some tools or prompters to facilitate easier use of the low back pain guidelines. Eventually, the two organizations will then collaborate to analyze outcome information on length of temporary total disability and per case costs to attempt to gauge the effectiveness of the guidelines. They will be using insurance claims data and patient satisfaction surveys to evaluate the outcomes.
These project point out the necessary effort and sophistication of such and effort, and how the data, aggregated across groups of physicians and associated with outcomes, can have provide Useful information.
Objective 6: To feed data back to physicians to promote quality improvement - With respect to the use of physician specific data in quality assurance, this is an attainable and worthwhile objective. Physicians, while perhaps anxious about addressing peer comparisons, have been found in several situations (for example, the Maine Medical Assessment Foundation model) to be receptive to such information, and the release of such data to physicians has been associated with favorable quality improvement results. We support the use of individual data to this end, recognizing that where it has worked, the data has been released privately to the physician. We believe this is the proper route, not because we wish to keep information from the public, but because the data cannot be used in isolation to draw the appropriate conclusions. The data will raise questions, however, that the physician can use to analyze and improve his or her own practice.
Publication of hospital and physician specific data has occurred in New York and Pennsylvania. The New York Health Department did not feel the physician specific data they had collected would be useful in the public domain and that it was not sufficiently valid to publish. The data, however, was obtained and published by Newsday under the Freedom of Information Act. There has been demonstrated improvement in charges and outcomes documented following publication in New York. However, those improvements have been attributed to the publication of aggregate hospital data, not to physician specific data. That is, addition of physician specific data did little to improve outcomes because publication was not intended to answer a question. In fact, publication may have damaged overall health care in the state by dislocating the careers of excellent cardiac surgeons.
Several coalitions across the United States have been leaders in the use of aggregate outcome data with impressive results. In Cleveland, the 4th most expensive place to obtain health care in the United States, the hospital association, businesses and physicians now publish outcomes and patient satisfaction results twice a year (Cleveland Health Quality Choice). Physician specific data was provided to the individual medical staffs and hospitals in order for them to use the information to continually improve their processes and outcomes. The results in reducing charges and improving outcomes have been impressive. In Cincinnati, 4 major businesses whose health care costs were inflating in double digits, collaborated with 15 Greater Cincinnati hospitals to look at resource and mortality outcomes to reduce health care costs and improve quality. Data were available to the businesses regarding hospital data, but physician specific data were available only to the hospitals to use internally for process improvement. In the first year of this project, the savings to the city were estimated to be $70 million dollars with a small improvement in outcomes. There was demonstrated improvement in the care of patients of the physician outliers through use of physician specific data in peer groups.
Physician profiling in the peer review setting is recognized as an important tool in continuous quality improvement within the hospital or provider environment. It is clear that physicians, in an era of cost containment and competition, will work to improve, given their own data and peer comparisons.
CONCLUSIONS
We as a medical society strongly support the publication of aggregate data on outcomes since they may point to a problem with the process of care. We also support collection and dissemination of physician specific data to hospitals and physicians to use to analyze processes and improve outcomes.
We believe that health data, when adjusted for severity of can be used to ask questions and promote analysis. Within the hospital/medical staff walls this data can be analyzed, actual charts can be reviewed and the best practices identified. This gives rise to information with which physician can develop guidelines. On the other hand, making this information public serves to put physician in a defensive mode rather than a proactive mode, without meeting either a well-defined or achievable objective. Public data that depends on an incomplete and inappropriate set of reported outcomes will not improve health care and could result in adverse patient selection by practicing physicians.
We welcome the ability to look at physician specific data and it's use in improving the practice of medicine. Publication of physician specific data should be done for a purpose. We support public physician specific data only if there is a question that physician specific data will answer. However, the data must be pure so as not to risk the reputation of good physicians. Criteria for assignment of responsible physician, data entry in administrative systems and medical records abstraction, must be standardized and consistent across all hospitals for physician specific data requested by the State to be valid.
We would support collection of physician specific data with grouping of logical aggregates and with reporting of elements that will be useful in judging the quality of care provided by the aggregate. Reporting the names of the physicians in the aggregate would be appropriate. Such reports are likely to be useful in judging care while minimizing the risks of individual reports described above.
There are currently examples of such aggregates and data sets to use as models. Probably the most important and influential effort to date has come from a group of American corporations who have been working with managed care organizations and the National Committee on Quality Assurance (NCQA) to develop a set of measures of quality known as HEDIS (Health Plan Employer Data and Information Set). This set of measures is designed for employers to judge the quality and cost of managed health care plans as they choose which to offer to their employees. HEDIS represents a core set of performance measures that cover quality, access and patient satisfaction, patient population and utilization, and finance.
We believe that the best system designed to use information to improve the quality and decrease the cost of medical care would use physician specific data in two ways:
We are anxious to offer the collaboration of the Colorado Medical Society
in a partnership with public entities that are interested in pursuing this
area of effort.
| ||||