I've gotta say it: When looking through value-added data on teachers in the past -- especially when noting vast differences between teachers in the same school -- I've often wondered (idly, I thought) what would happen if these data were ever made public.
Still, when the Los Angeles Times actually did that recently, I was one of many educators around the country caught by surprise and without a clear point of view. In the effort to assure all kids, but especially our most vulnerable, the quality teachers they need and deserve, would publicly accessible data by teacher name help -- or would it hurt?
Instead of jumping to stake out a position, we at The Education Trust decided to look into what we could learn from the health care field where some data have been available on physicians for years.
We're not yet done learning. What we've gleaned so far, though, is interesting enough to share more broadly to help others who are thinking through this question. Yes, we all know there are big differences between doctors and teachers, beginning with pay and prestige. But it turns out that there are also many parallels.
The Role of the MediaOne of those parallels is the role that media outlets played in moving the issue of public data to the fore. In education, of course, the year was 2010 and the paper was the Los Angeles Times. As for health care, things got rolling much earlier. In 1977, the Department of Health, Education and Welfare -- now the Department of Health and Human Services (HHS) -- released a list of Medicare reimbursement amounts, by physician, that was disseminated widely.
On hearing this news, the Florida Medical Association, later joined by the American Medical Association (AMA), went into overdrive, and sued to block the department and the media from publishing the data. These groups argued that releasing such data and letting the public see how much an individual doctor was paid by Medicare was an unwarranted invasion of privacy.
Interestingly, the AMA won, and the decision in that case has been for nearly three decades interpreted as barring the public release of such data -- at least by the federal government. In 2009, two different federal appeals courts affirmed the decision. But though the courts precluded the federal government from doing much, the movement toward collecting and reporting data on physicians has continued to grow.
Professional ResistanceWith some wonderful exceptions -- apparently the association of thoracic surgeons collects all sorts of individual-level data and members use that database for benchmarking -- doctors have mostly opposed these efforts. Their reasons resemble those we hear from educators: "the data aren't good quality," "if the outcomes are poor, it may be the patient's fault," "when we treat them in groups, who are you going to hold accountable?" and "some of us serve more challenging patients, so the results aren't comparable."
Over time, however, many have come around. They've even established their own vehicle -- the Physician Consortium for Performance Improvement -- to work on metrics.
Pressure from Consumers and PurchasersOne of the reasons more doctors are coming around is that some very powerful interests are working together and pushing very hard for more transparent physician-level data. Consumer organizations -- like Angie's List and Consumer Reports, but also associations like the American Diabetes Association and the American Heart Association/American Stroke Association -- are at the forefront of the drive for public data. Indeed, if you Google "physician ratings," more than 2.7 million references come up, including lots of "rate your doctor" sites where those seeking medical help can learn what other patients think, as well as publicly available information on board certifications and the like. But the consumer organizations seek more than simple reports on patient satisfaction: They want their members to have honest information about error and complication rates, costs, and much more.
The big insurance companies, too, collect a lot of data on individual physicians and have sites that give their plan members access to at least some of that data. They also use the data themselves in tiering doctors for reimbursement purposes: Subscribers get free or near-free access to doctors with relatively low charges and relatively low complication rates, and have to pay more for others.
And these forces -- consumers and purchasers, interestingly including groups like the AFL-CIO -- have teamed up to create a Consumer/Purchaser Disclosure Project, which has fought state, regional, and national battles for greater public access to this information.
Some ProgressAdvocates for more transparent physician-level data made important but limited progress during the 1990s and the early part of this decade. Victories have included mandatory public reporting of physician-level data in some states. New York, Massachusetts, Pennsylvania, New Jersey, and California, for example, have a good deal of information available on heart surgeons by name, including, typically, the numbers of procedures performed, average length of hospital stay, mortality rates, hospital readmission rates, and so on. In Pennsylvania, consumers also have access to data by surgeon name on hip and knee replacements, including volume, complication, and readmission rates.
But consumers in other states are still mostly limited to information from "rate your physician" initiatives, supplemented by more objective data from voluntary physician "recognition" programs, wherein organizations like the National Committee for Quality Assurance recognize doctors who provide evidence that they meet a certain standard of patient care.
The movement toward greater transparency got a huge shove forward, though, with passage of the recent health care reform legislation. HHS is now charged with creating, in addition to the "Hospital Compare" and "Nursing Home Compare" functions on its Web site, a new "Physicians Compare" function. The department also must release Medicare claims dates to "qualified entities" that can generate aggregated information for public reporting.
Over time, the site will include four types of data: procedural data, like how many patients presenting symptoms of diabetes get the test considered best practice for managing diabetes; structural data, like whether the physicians have electronic records and use e-prescriptions; outcome data, like mortality and complication rates; and cost data.
True, the physician-level data in the HHS system will be limited to Medicare-funded health care. But still, this could be a big step forward for consumers.
What are the Key Lessons to Date?The experts I have talked with so far made several suggestions worth thinking about.
First, they all reiterated the need for consensus-based processes on what and how to measure. Indeed, in health care, there is a public-private body called the National Quality Forum (NQF) that was created by presidential executive order in the late 1990s to oversee consensus-based processes to develop physician-specific metrics.
Second, though broad stakeholder involvement in this process is considered essential, experts seem to agree that majority power has to be tipped toward the consumer side -- that, otherwise, organizations representing doctors will go in the back room and nothing useful will emerge. To this end, NQF has a majority representing consumers and purchasers.Third, questions of attribution might be even trickier than we think. For example, doctors -- like teachers -- often practice in teams or groups. When a patient deteriorates, who gets blamed? Similarly, doctors -- like teachers -- "treat" patients with varying levels of personal resources to follow through on "treatments." And doctors -- like teachers -- often serve patients who are also served by others (for instance, the after school reading tutor).
Fourth, participants in the indicator development process believe the core metrics must be grounded in evidence and the NQF has very exacting rules of evidence. Moreover, metrics must generally serve two purposes: improving practice and informing choice. That said, the evidence about the impact of this kind of transparency on either choice or improvement is rather mixed. Consumers, it seems, don't use available data to choose physicians nearly as much as they do, say, when they buy a car or a major appliance -- or at least not yet. But there is pretty solid evidence, at least at the hospital level, that data are more likely to be used for improvement when they are publicly available.
Fifth, speaking of institutional use, it does seem clear that institutions -- hospitals, nursing homes, insurance plans, and the like -- do make use of physician-specific data. If you join a large health plan, for example, the "star ratings" you see on participating physicians are often based on the data collected by health plan administrators over time. So, too, is the plan's decision about whether that particular physician will be a "co-pay only" choice, or whether you will be reimbursed at only the 80 percent level. Sure, what those physicians "charge" figures in these calculations, too. But even a low-charging doctor is hardly a bargain if his complication and readmission rates are high.
Finally, at least some of the experts I talked with are worried (as I am) about possible unintended consequences of increased transparency. They raise the same exact question that reverberates in our heads here at The Education Trust: If people actually know who the best doctors (or teachers) are, what reason do we have to believe that poor kids will actually get more of them? Indeed, it seems possible that the exact opposite could transpire. In at least one recent study in the Journal of the American Medical Association, researchers at Boston General warn that the close association between physician ratings and the race and income of their patients could serve to widen health disparities.
Needless to say, there's a lot of food for thought here. We intend to keep digging, and hope others will too.
As they say on the health care side, all of this is too important to get wrong.