Roundtable on Expanding Capacity for Comparative Effectiveness Research in the United States

Facilitator

Sean Tunis, M.D., MSc., Director, The Center for Medical Technology Policy, a small nonprofit firm; formerly, at Centers for Medicare and Medicaid Services (CMS).

Panel Members

• Carolyn Clancy, M.D., Director, Agency for Healthcare Research and Quality (AHRQ); previously she was Director of the Agency's Center for Outcomes and Effectiveness Research.

• W. David Helms, Ph.D., President and Chief Executive Officer (CEO) of AcademyHealth. He also serves as President and CEO of the Coalition for Health Services, AcademyHealth's advocacy arm.

• J. Michael McGinnis, M.D., M.PP., Senior Vice President and Director of the Health Group at the Robert Wood Johnson Foundation. Previously he served as Assistant Surgeon General, Deputy Assistant Secretary for Health and Director of the U.S. Office of Disease Prevention and Health Promotion.

• Steven D. Pearson, M.D., M.Sc., FRCP. President, Institute for Clinical and Economic Review (ICER) at Harvard Medical School; Senior Fellow, America's Health Insurance Plans in Washington, DC.

Sean Tunis: There is growing interest in building capacity for comparative effectiveness research in the United States. Many people, including but not limited to panelists at this roundtable, have contributed to the increasing enthusiasm and growing consensus that something needs to be done to further build capacity for comparative effectiveness research. Here are a few examples. The Institute of Medicine's (IOM) Roundtable on Evidence-Based Medicine, under Michael McGinnis's leadership, has been facilitating this conversation. A proposal by America's Health Insurance Plans (AHIP) on comparative effectiveness, which Steven Pearson has been instrumental in helping put together, has received a lot of attention on Capitol Hill and elsewhere. Others such as Gail Wilensky (2006) have called for a massive increase in investment in comparative effectiveness.

Similarly, several groups have been active on the issue of comparative effectiveness, including the Blue Cross and Blue Shield Association, the Medicare Payment Advisory Commission, and the Congressional Budget Office. Congressman Tom Allen (D-ME) recently drafted legislation that specified resources to build capacity for comparative effectiveness (H.R. 2184, the Enhanced Health Care for All Act of 2007). Comparative effectiveness surfaced early on in the health proposals of presidential candidates. All these activities reflect an urgent and profound interest in comparative effectiveness research.

In this roundtable discussion, we will try to clarify what falls under the umbrella of comparative effectiveness. One proposed definition of comparative effectiveness (sometimes called comparative clinical effectiveness) is that it is research that compares the benefits and risks of health care option A to health care option B, where option A and B will usually be a drug, device, or procedure. So the first question for our panelists is, Do you agree with this definition, and if not, how would you change it?

Carolyn Clancy: Your question raises several issues. First, you said comparative clinical effectiveness, and I do think that one of the potential flash points is whether we're talking about clinical effectiveness alone, clinical effectiveness with some information about costs, or clinical effectiveness coupled with cost effectiveness.

A second issue relates to what we mean by “options.” Many interventions in medicine, whether about diagnosis or treatment, are linked to a series of contingent actions, that is, as part of a strategy or chain of events. We need more clarity around the term “option.”

And then, frankly, I think the big issue in setting priorities for this enterprise relates to whether we mean an array of all possible interventions (including no intervention), or are we focusing on specific types of interventions?

Now, to that extent, I would disagree with your definition, because it seems to imply that research must focus on the comparative effectiveness of drugs, or devices, or surgical procedures. As a clinician, I believe that the decisions facing many clinicians and patients actually are, “Given this set of circumstances, what can I do?” Often, clinicians and patients have many different and sequential options embedded in the choices that they have to make. So, when I turn to a definition, I usually turn to the legislative authority that AHRQ has under the Medicare Modernization Act (MMA) (see http://www.hhs.gov/asl/testify/2007/06/t20070612a.html for details). The MMA offers a start and defines health care services broadly based on input from the public and private sectors.

David Helms: AcademyHealth has been preparing for the opportunity to debate this issue publicly, and we have a report on comparative effectiveness options that offers some principles (AcademyHealth 2005). I endorse the idea that we ought to be thinking broadly. This means comparing the range of options for addressing a condition, whether that's a pharmaceutical, a device, a medical therapy, or a combination of those compared with a procedure. I'm pleased to endorse that part of the definition.

Michael McGinnis: We used a very simple definition of comparative effectiveness studies in the IOM work in which we've been engaged. It essentially goes something like, “The comparison of one diagnostic or treatment option to one or more others.”

This definition suggests a departure from the one Sean mentioned, in that it allows latitude to compare, whether in diagnostic or treatment categories, across different kinds of approaches. For example, clinicians want to compare the extent to which a computed tomography scan is preferable or not to cardiac catheterization in diagnosing coronary artery disease. Or for example, in treatment; when it comes to adolescent obesity, clinicians do not want to compare only across bariatric surgery approaches; they want to have a sense of the relative effectiveness of bariatric surgery versus lifestyle modification.

Steven Pearson: I'll try to be a little bit provocative. I think that in looking at that definition, which is “benefits and risks of option A to B,” it's seductive but dangerous to leave out costs.

We are under an illusion that comparative effectiveness is something easy, where we'll go out and do a trial. Option A will be an easy winner over option B, and clinicians and patients will know instantly what to do. A magic, invisible hand will affect the medical system, and everybody will shift to the superior option.

How often have we actually seen that happen? Not that often. Most of the time, there are eight randomized clinical trials that show conflicting results, and some observational data that seem to point in one direction. One option is more expensive than another but may have downstream effects that may make it more cost effective. And decision makers, even with a very fine systematic review, are still somewhat perplexed. Meanwhile, manufacturers rev up, honestly, the marketing arm and try to influence decisions.

And so I want to propose that we carefully allocate new resources so that we get out of comparative effectiveness the value that is needed. So, if we're going to compare benefits and risks, what's that going to look like? How are we going to weigh them?

I believe that modeling is one of the few ways to help decision makers take into account the different harms and benefits that are being compared, especially between drugs and procedures. Drugs and procedures will have different types of side effects, different types of benefits, and they have to be weighed. To do so helpfully requires a model.

Secondly, the issue of cost is important because it's easy to fixate on the upfront acquisition costs, when what we really want to know—all of us—is what's the effect on the health care system? If something costs a lot up front but has downstream effects that reduce utilization and costs for a variety of reasons, we really want to know that.

Even just saying that we're going to look at costs can be dangerous. Instead, we want to look at cost effectiveness, and we want to do it in a transparent and rigorous way to help decision makers.

Another area of contention is whether comparative effectiveness research should focus on brand-new drugs and devices that are expensive. Should comparative effectiveness examine health services more broadly? Or should it look at what health plans do with their policies and benefit designs? The wider we draw that circle, the more diffuse the impact, making prioritization more difficult. If we leave it too broad, we may lose the focus necessary to gain some benefit from this work.

Sean Tunis: I heard consistently across this group that comparative effectiveness studies should not be restricted to head-to-head comparisons of drug A versus B. At a minimum, these studies need to look at the range of alternatives and combinations. My understanding of some of the original round of comparative effectiveness studies was that they were very much to do with wanting to have information on drug A versus B (within class comparisons). I think a lot of folks on Capitol Hill still believe that's what comparative effectiveness research is.

Are studies of the impact of quality improvement programs, disease management programs, and pay-for-performance programs inside or outside the boundaries of comparative effectiveness research? And are other potential studies, like comparing different kinds of benefit designs, or treatment outcomes across practitioners, within the scope of comparative effectiveness research?

Steve Pearson: Those other topics are important, but to me, they belong still in the domain of health services research. I view internal validity questions that compare and review specific options as the real “bang for the buck” in comparative effectiveness research. So, while there is much more we need to know about pay-for-performance, and quality improvement, for example, I think it's more useful to leave those topics outside the tent for now.

Michael McGinnis: I would agree with that. The issue is essentially, “How crisp is the intervention?” Comparative effectiveness conveys a sense that you're looking at very specifically defined, easily characterized interventions. Now, having said that, I'll blur my own point by taking the challenge of the second part of Sean's question: the question about comparing either benefit designs or treatment outcomes across different practitioner types.

I think it is legitimate as a focus for comparative effectiveness research to look at comparative outcomes across different practitioner types. I would view that as another look at, in effect, the level of intensity variation.

David Helms: This is a very important issue for the field of health services research. One of the concerns we've had all along is that comparative effectiveness could be seen as the “new ticket to ride,” meaning the new great panacea for solving America's health care problems. If we take this view, however, we would leave a very under-funded field of general health services research even further in the background.

To me, if we've done a good job of studying the comparative effectiveness of different ways of treating a condition, or an intervention, we aren't done with the research needed. We would still need to know the extent to which practitioners are aware of this finding, how well that finding is being integrated or adopted by the delivery system, and if they're using it effectively in improving health care.

AcademyHealth has been stressing in its advocacy that this new emphasis on comparative effectiveness research should not be viewed as a further opportunity to reduce the emphasis and priority that is given to basic health services research.

Carolyn Clancy: I think the only context in which there's a very sharp, bright line between the question and kinds of interventions you posed—and focusing on what's in a clinical box, for lack of a better term—is the context of a clinical trial, which essentially assesses efficacy, where you can control for a lot of contextual factors. I think it is inevitable that some of the information we're going to be drawing on will come from observational studies as we get more and better data from health care systems themselves.

There are two areas where this boundary becomes problematic. First, if we're only talking about producing new findings and new information without a clear eye on the demand from the time we launch the research, and frame the questions, we will have missed a huge opportunity. So, we've got to be focused on “what are the outputs?” I believe that separating comparative effectiveness from health services research would be a huge mistake.

Sean Tunis: Next, we're going to focus on the tools or methods of comparative effectiveness research. There are generally four categories of tools or methods used for comparative effectiveness research: (1) prospective clinical studies, which include clinical registries, head-to-head trials, pragmatic trials, and adaptive trials; (2) retrospective studies using administrative or electronic health record data; (3) decision models with or without cost information; and (4) systematic reviews.

The next question for our panelists is, “Would you make any amendments or corrections to those four buckets? If you had a billion new dollars to spend, how would you allocate it across those four buckets? And if you only had $50 million to spend, how would you allocate it across those buckets?”

David Helms: I would endorse all of the above. We believe that comparative effectiveness research is certainly more than randomized-control trials. Our advocacy has emphasized the need to make significant investments in the research infrastructure, including refining and improving the methods and developing improved data systems so that we get more reliable results from using registries, administrative data, or even electronic health records.

Systematic review is a big part of how we consider the array of tools. We should think not just about new research and the methods that will be required but also about what is most important, what information is needed most by patients, providers, and payers. What array of existing methods will get us the information that will be needed to improve health and health care?

Sean Tunis: And how would you allocate $50 million to ensure the greatest net impact on improving public health and the greatest improvement on efficiency of care?

David Helms: I think the priority-setting process for what studies get picked will be crucial here. By relying on stakeholders to help set priorities, we can use their input as well to identify the easier-to-do, “low-hanging” fruit that would be initial priorities so that we get some results out quickly.

I'd be looking at high-cost and/or high-volume procedures. The emphasis that AHRQ has given to systematic reviews is important. If I had $50 million, I wouldn't put it in even a small number of clinical trials. I'd probably continue the emphasis on systematic reviews. And I might invest some of those funds in methods and data.

I hope we ultimately get more than $50 million. As our audience may recall, this was the amount that was authorized in the MMA section 1013. So if we had a billion dollars appropriated, then we could fund the range of activities needed including some comparative head-to-head trials as well as observational studies, research syntheses, and investments in the research infrastructure.

Michael McGinnis: The four buckets look perfectly reasonable to me. I think it's a nice inventory and taxonomy of the opportunities. With respect to the use of the monies, I agree with David that if you had a billion dollars right now, obviously you'd want to do more head-to-head trials.

The methods we have at hand would allow us to do that; we can take on some of the highest-priority issues. In addition, you'd want to set aside some funds to work on enhancing the methods. Clearly what we're working toward as a society—our vision—is a time when we can take advantage of a 300-million-person, fully interoperable, electronic health database. That means that we need to be working now on the implications and approaches to using databases of that magnitude. So, one would want to set aside some sizable investment for that effort.

If we only had $50 million, I think that the way AHRQ has used the monies under the MMA is nicely structured. You'd want to focus on systematic reviews, ramping up in the areas that are most important. You'd want to fund some registries that would give you a head start on information about some of those choices that have to be made. You would allocate some funding for methodological enhancements. Finally, I would recommend some analyses that help make the case on the importance of this kind of investment generally.

Steven Pearson: Most nations are looking at systematic review and decision modeling to help with decision making for the health system, particularly with new technologies. Much like health plans in this country, health systems internationally have to make an up-front decision when something new comes through the pipeline. Internationally, investing in this type of information—up-front reviews of information to assist with this type of decision making—has gained momentum.

I am far from unbiased because the institute that I direct is heavily invested in doing decision modeling and systematic reviews as an approach to helping decision makers.

Any time you start dangling billions of dollars out there, everybody has wonderful ideas about prioritization. But I agree that comparative effectiveness research must show a return on the investment. Policy makers believe that comparative effectiveness research will be useful, based on international examples and some recent literature. But we need to pick some easy, quick “wins” if we can find them, and in Washington that means some that save money. To launch a 10-year, wonderful head-to-head trial as the first initiative, even if it costs $50 million, isn't going to be feasible. Hopefully, as the money grows, there'll be the time and the resources to do head-to-head trials.

Carolyn Clancy: The minute people hear “clinical trials,” they think, “Wow, big dollar figures,” and they're right. The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial cost a $100 million. So a billion dollars might buy you 10 clinical trials. These figures are daunting and suggest the importance of picking the right trials.

I'd like to make two additional points. First, we need to ramp up the methods and codify standardization of some practices in order to strengthen observational methods. A variety of studies need this standardization, not just registries. I would agree with Steve's comment about models being very helpful for balancing benefits and harms. We need to work on making models more user friendly.

Two themes need to come out of a resource allocation discussion. First, resource allocation should be a collaborative enterprise that clearly identifies studies that all stakeholders think are really important. This collaborative process is probably at least as important as the resource allocation itself.

Second, we need a balanced portfolio approach that ensures the inclusion of longer-term studies. What are the sustainable capacity and the infrastructure that this field will need? If we pursue only short-term benefits, we'll continue to study hospitalizations for congestive heart failure and similar types of problems, because we know we can knock those out.

Sean Tunis: I believe that 60 to 80 percent of the money should be spent on better, simpler, faster prospective clinical studies, perhaps head to head. Having spent 5 years trying to make coverage decisions for Medicare, I'm struck by the fact that not once did we ever read a technology assessment or a coverage review that called for a retrospective study. The gaps in the evidence that we faced were all for lack of prospective, well-designed, comparative studies. One of my concerns is that we could potentially produce large volumes of information that will not be as helpful to decision makers as needed and will not result in the impact on health and efficiency that we desire.

Carolyn Clancy: A key part of building this enterprise, in addition to a collaborative process with multiple stakeholders, is thinking about how the clinical trials, systematic reviews, and other studies interrelate.

Sean Tunis: The question is, “Should cost, cost effectiveness, and comparative value be considered part of the definition and part of the portfolio of a new federal capacity?”

If not, what are your specific concerns about including costs in the mission or charter of this organization? And if you do think they should be included, should that information be used in decision making?

Michael McGinnis: The issue of whether or not cost effectiveness is in the formal charter of an entity of this sort is fundamentally a political notion that will be discussed and brokered. The entity will do or not do whatever emerges at the end of the day. But let's assume that cost effectiveness is not part of the formal charter. If that's the case, a cottage industry to address cost-effectiveness considerations will emerge. Such an industry probably won't remain “cottage” for long.

So, I think the notion of whether cost effectiveness is formally included in a Congressionally established entity is a bit moot. There'll be a lot of focus on these issues regardless.

David Helms: The field of health services research believes it has a good deal to contribute around the answer to the cost-effectiveness issue, and we would certainly hope that cost is part of the consideration of comparative value. I agree with Michael that this is going to be a political decision and may be argued out at above our pay grade here. That said, I do think that it will be important for all of us to make it clear that comparative effectiveness is used now by other nations and by health plans in this country.

Cost-effectiveness analysis should be part of the information that's put in the public domain. But unlike a single-payer system such as England's, where the finding is determinative for what gets applied through the clinical guidelines and decision process, I would see multipayers making different judgments about how they use the information.

Carolyn Clancy: This is probably one of the more politically charged aspects of this whole discussion, and there's a lot of urgency. So on the one hand, we're having this conversation because of concerns about rising health care expenditures. On the other hand, the minute you link this to discussions about coverage, then people start coming out of the woodwork very worried about, “Does this give me the right to say no?” Or, “Does this give me (on the other side) an opportunity and an absolute mandate to figure out what structure can keep them from saying no?”

Politically, it's going to be much safer to keep arm's length from all this, even though, as you said, Michael, a cottage industry focused on cost-effectiveness analysis may emerge. Part of the conversation for every individual trying to figure out “What's right for me?” is to figure out “What can I pay for?”

Having said that, on the technical side, much of our framework for thinking about cost effectiveness is an up/down “Do we include this in the benefit package or not?” rather than a more sophisticated approach. A more complex and difficult-to-administer approach would be to ask, “How do we modulate this through differential reimbursement or differential cost sharing and so forth?” Again, it will be easier to build a political consensus if those questions are at a bit of a distance.

Steven Pearson: There's no doubt that the political issues around the words “cost effectiveness” are inescapable. There are clear ways that the information from modeling and cost-effectiveness analysis can be used not to reflect an “up” or “down” coverage decision, but to lead to approaches that we're comfortable with—the idea of tiering, for instance. Not long ago the idea of tiering drugs seemed revolutionary and difficult.

I wanted to address this idea of cost effectiveness being at arm's length from clinical effectiveness. While it sounds politically expedient, I don't see how that would work as well, honestly, as it would if it were not at arm's length. If we're going to use modeling to weigh the harms and benefits, are we going to pretend that that's not the same model that's being used for cost effectiveness? This approach would risk a lack of legitimacy and transparency. By centralizing cost effectiveness and making it more transparent, we will have a more honest discussion about modeling issues. Centralizing this function would also lend legitimacy and transparency to the results, something we lack right now.

Carolyn Clancy: When you talk about transparency, that does sound pretty compelling. One of the technical aspects that I think is extremely sensitive, however, relates to various pricing strategies. Rebates for pharmaceuticals come to mind. In the short term, I don't believe that manufacturers would be willing to be transparent about that, which means that the inputs into modeling costs and effectiveness together become suspect.

Steven Pearson: Everyone's going to have to give a little bit if the idea of transparency is going to work. Weighing evidence is a complicated issue and one that is difficult for stakeholders to appreciate. I suggest that the shorthand answer to the concern about different costs is, again, a hybrid approach in which the model itself is developed transparently.

Sean Tunis: We're going to now open the discussion to the audience.

Stuart Gutterman (The Commonwealth Fund): It seems to me that evidence on effectiveness could have a lot of impact on what the cost part of the equation was. It'd be difficult to determine the cost part of that cost-effectiveness equation until you knew what the effectiveness was. Then that could play out in the market and determine what the price was.

If one drug produced twice the benefit of another drug, that would have a lot to say about what manufacturers could sell those drugs for. And so, if you took the initial price as the number in that analysis, you'd be presuming something that wouldn't necessarily be the long-term cost of the drug or procedure you were evaluating.

Also, I wince when I hear the term “transparency,” especially when I hear discussions of compromises on transparency. I'm thinking about writing an Op Ed piece that I have the title to already. It's called “Transparency You Can See Through” because it seems that transparency is a situation where you can see everything. One of the things we need to see is how valuable a particular thing is before we can tell what its market price should be.

Sean Tunis: Thank you for that comment, Stu. You've identified the reason that I was noting the cost-effectiveness component of this will sort of become inevitable after the effectiveness is focused on. There's just no question that people are going to be undertaking those kinds of analyses.

Carolyn Clancy: The other point that I would make here is that we're the wrong people to be having a conversation about whether costs and cost effectiveness should be part of the work undertaken by a comparative effectiveness entity. What many of us in this room would regard as inherently reasonable, others are very worried about. The central worry can be distilled into a heading of antiinnovation. Even talking about reference pricing, which many people think is a reasonable way to go, would be interpreted by some as antithetical to innovation. By reference pricing, I mean benchmarking what either out-of-pocket costs or reimbursement are going to be by what is the lowest cost or lowest-priced intervention, and then paying extra from there. If reference pricing gets tagged onto this work that we desperately need, that's going to be politically very difficult. So, I'm back to emphasizing expedience.

Steven Pearson: I want to posit that modeling itself is a very powerful part of deciding the risks, meaning the harms and benefits. There won't be a clean winner in many of these cases, for both reviews and head-to-head trials. Since we're not up front as much with cost effectiveness, our current model of systematic reviews and decision making is really focused on this idea of comparability, within a drug class, for example. If a drug has some potential marginal benefit, but a review says that the evidence is weak and we really can't say anything other than it's comparable, the drug is doomed. I'd rather have the drug be recognized as having a marginal benefit.

Roger A. Johns, M.D. (Professor of Anesthesiology, Johns Hopkins University): I'd like the panel to assume that Congress continues to move forward at the accelerated pace this effort has gone over the last 6 months. Let's assume that Congress calls all of you before a Finance Committee meeting, and asks you to assist them in deciding who should lead this organization, and also how to engage the various stakeholders. How would you reply?

Carolyn Clancy: One guiding principle is: build on what is already in place. So, I would say that we've begun a very important set of activities at AHRQ with very modest funding, and that that would be a place to start building from. In terms of setting priorities, I would hope that Congress would help us in terms of creating a separate advisory group for this enterprise.

Congress often stipulates the composition of such an advisory group. Hopefully the advisory group would include several different constituencies of consumer groups, because it is difficult to have an all-purpose consumer representative. Clearly, practitioners, leaders of health care organizations, even someone from research, payers and so forth, would need to be part of such a group as well.

I would hope Congress would ask us, “How would we assure continued transparency and input from a variety of organizations?” If I had one pitch to make, it would be to make sure that Medicaid is a strong part of this, which is also difficult because of variations in states‘ capacities.

Michael McGinnis: At a recent meeting of the IOM Roundtable on Evidence-Based Medicine, the analysis identified three general categories of models, all of which seemed to be perfectly reasonable approaches.

One model built on the infrastructure that AHRQ has in place, partnered with an independent external advisory capacity, and perhaps an entity akin to a Federally Funded Research and Development Center (FFRDC) to conduct the work. The second model, proposed by AHIP and others, is built on a Federal Reserve approach and would establish a new entity. The third model offered a hybrid activity building on some existing capacity with a totally new governance enterprise.

These models all entail an independent governing board comprised of stakeholders. Most of these models envision a public/private funding capacity, and clearly, if a sector is providing funding, it will want involvement in governance.

Now, the governance in all these scenarios would essentially focus on two areas: priority setting and financial distribution of funds. Presumably a firewall between the governance priority setting and the conduct of the science would maintain the entity's integrity.

David Helms: The AcademyHealth study (2005) lays out several options: (1) AHRQ does this with an external board, and a panel of experts; (2) AHRQ sponsors and conducts studies with oversight from an external board and creates some kind of intermediary like an FFRDC; and (3) AHRQ continues on its present course, creating a new separate quasi-governmental entity that would conduct this work. (4) A fourth option would be to reconstitute AHRQ as a new quasi-governmental entity retaining its existing functions and adding comparative effectiveness research.

While we are now moving to a debate about structure, and that is important, my hope would be that we don't get ourselves hung up on the “shape of the table,” and that we now use this opportunity to begin investing in the infrastructure that will be needed for a robust comparative effectiveness capability for this country.

AcademyHealth has proposed a panel of experts that would be empowered to look over the methods and to review the data that are used for particular studies. My concern is that there's no right answer to what should be the structure for such an entity, and in fact, AcademyHealth did not make a firm recommendation in this area. Instead, we offered our thoughts about pros and cons of the basic options. My concluding thought is that the entity should be close to AHRQ, because this function needs to be linked to the lead agency for health services research.

Ira B. Wilson, M.D., M.Sc. (Professor, Department of Medicine, Tufts University School of Medicine): I'm concerned that just putting more data points, more procedures, more comparisons on that cost-effectiveness list won't actually change the “pay line,” and that people will continue to pay for things that they shouldn't pay for. What's really going to solve the underlying problem of cost? Is there anything built into this that offers a more honest social dialogue about the difficult rationing issues that some of this boils down to?

Carolyn Clancy: I would say the answer is no. I mean, to be really honest, rationing would shut down that discussion. We hear rationing as rationalizing, and making smart, informed choices. Much of the public and much of the Congress, however, interpret rationing as “Somebody gets to say no on an arbitrary whim, and a green-eye-shade guy is standing between me and something that could help me.” On the other hand, I do think that what we've seen, over the past 20 years or more, is a growing number of consumers who actually want to make smart, informed decisions. I take your point about not just assembling more data points. You want to be able to give good information to people when they know they want it and might do something differently.

But this is not a guarantee that we're going to solve the cost problem. The cost problem alone, I think, is much harder because as Uwe Reinhardt reminds us, “One person's cost is another person's income.” Or, as one of my colleagues says, “There's no constituency for cost containment, really.”

Steven Pearson: Ira, I was struck by your comment. You've put your finger on a missing piece—the translation of these results into what's going to happen next. Congressional staff have asked us, “So, what can Medicare do with this to save costs?” And we get nervous, because we're afraid it's going to look like rationing. But let's imagine that we actually have a way for Medicare to use this information as part of determining its reimbursement. I hope that we can develop some way of translating comparative effectiveness information into either reimbursement or tiered benefit approaches. A value-based insurance design, I think, is where Medicare will be headed. We need better information and a little bit more experience to get us there.

Michael McGinnis: The assumption behind the question is that the output of this kind of entity would be limited to raw data tables. I think we agree that the entity needs a translation function that makes the results of the studies as publicly clear as possible.

Carolyn Clancy: The social dialogue you alluded to, Ira, is difficult and will start to happen. The flip side is growing evidence that if we really honored patients‘ wishes, we would probably be doing fewer procedures for some patients, if, in fact, they were given the information to make those decisions.

Ruth Lopert, M.D. (Visiting Associate Professor, George Washington University; formerly Principal Adviser and Director of the Pharmaceutical Policy Taskforce in the Commonwealth Department of Health and Aging in Canberra, Australia): Coming from an environment where we do this sort of thing all the time, I find this dialogue really interesting. I keep coming back to this idea that you've mentioned several times that you're going to have to show a return on this investment on this exercise. And if cost is off the table, and the social dialogue is a long way into the future, and the making of information available about which treatments or therapeutic options are effective, have an impact on the market and also on the price of those options, how are you going to demonstrate a return on this investment? How is this not going to be a recipe for unconstrained growth and expenditure?

Carolyn Clancy: We tend to frame return on investment in terms of what percentage of GDP is going into health care and, if we frame this as cutting that back, it's not going to happen. I think that creative minds are needed to find some new ways—and certainly AcademyHealth would be the place to go to find those creative minds. If we had a way and can bring in lots of people to frame return on investment so that the proportion of health care that we're spending—you know, that the value-to-cost ratio—in some way, was increased, then people would be making decisions that they valued in some fashion, or that we knew were consistent with better value. Because, you know, what I've heard Nancy-Ann Min DeParle say (which I thought was a great way to say it) was what she worries about, having been a previous administrator of what is now CMS, isn't what's the “right” percentage of GDP, but that we'll still continue to see this cost-quality disconnect at much, much higher percentages, and that is scary. So, I think we'd have to be very careful about how we frame a return on investment.

Ruth Lopert: Yes, but surely, if the marginal cost increases because you've identified a marginal benefit, then the value ratio is potentially going to get worse, not better.

Carolyn Clancy: Yes, I would agree.

Patrick O'Connor (family practitioner at an HMO in Minnesota): There's a lot of comparative effectiveness data already available. And, in the real world of managed care, these data are actually being used extensively on a daily basis to guide treatment policies and initiatives through instruments like clinical guidelines, co-pays on prescriptions, and financial incentives and disincentives to use more or less of a certain type of service.

It seems to me, when I'm seeing patients and writing guidelines out in Minnesota, that we've been doing this for about 10 years. So I would urge you to consider rounding up some people from that community of managed care—or whatever you want to call it—to be involved in this debate.

Having said that, I think that there's a huge missed opportunity to examine data available in evidence-based guidelines. Guidelines have been essentially devoid of comparative effectiveness data for various reasons. And yet we know, using diabetes as an example, comparing glucose control and blood pressure control in people who have type 2 diabetes, that blood pressure control is more effective than glucose control in preventing eye and kidney complications, and much more effective in preventing macrovascular complications. There's a lot that could be done just with the information we have today that would advance this agenda.

David Helms: This information needs to be integrated into the existing health system and used effectively, so that would be one response to your question.

Steve Pearson: Thank you for your comment about what practitioners and researchers and others out in the health system, including managed care systems, are doing. I think that's partly why health plans have been so supportive of this effort.

It's a long conversation about why guidelines are not at the center of what people are talking about. I think most people believe that the information gathered and translated through these processes would feed into guidelines. For a variety of reasons, however, to put guidelines up front as the poster child for this effort hasn't been seen as the right way to go.

Carolyn Clancy: First, the Robert Wood Johnson Foundation did a nice analysis of communities that might be receptive to aligning forces for quality initiatives. One way to look at the results of a very thoughtful analysis and distill it into a sound byte was to say, “There was Minnesota, and there was everybody else.” Minnesota is truly the land of the above average.

Second, one issue we clearly heard at a recent meeting of the evidence-based medicine roundtable coming from the Hill and from the democratic side was that managed care, for a lot of people, has a lot of “baggage.” Now, being very familiar with the work that both of you have been involved in with terrific organizations, I'm not including that. But I do remember a staffer from the Hill saying, “You know, most members of Congress wouldn't know what you're talking about, and maybe you ought to be grateful for that, because if they did, they wouldn't like it.” They hear evidence-based medicine as rationing. And the link to managed care was made quite directly.

We're at an earlier phase of this social dialogue that you referenced, Ira, or, as we should acknowledge, the political aspects of it will continue to be very interesting.

Sean Tunis: Let me thank our panelists for a really good discussion.