Recently, the American Journal of Tropical Medicine & Hygiene published a paper by Shepard et al. evaluating the impact of HIV/AIDS funding on Rwanda’s health system. The headline of the press release was catchy and assertive: “Six-year Study in Rwanda Finds Influx of HIV/AIDS Funding Does Not Undermine Health Care Services for Other Diseases. Study Addresses Long-standing Debate about Funding Imbalances for Global Diseases.”

But after reading the report, we quickly assessed that a more accurate and appropriate press release headline for this paper would be “Some Differences Observed in General Healthcare Delivery between Facilities with and without HIV/AIDS Services in Rural Rwanda.” The study has serious limitations associated with its design and its generalizability that aren’t reflected in its catchy press release, and thus have unfortunately gone unrecognized. And because there is, in fact, an important and “long-standing debate about funding imbalances for global diseases” that this study does not sufficiently address, it’s important to examination the shortcomings of the study’s results:.

1. Internal Validity: Does the study do what it claims to do?

No. Treatment was not randomly assigned, while matching and control strategies do not mitigate the effects generated by non-random assignment. As a result, the study’s current comparisons between the treatment and comparison groups are problematic in validly testing the proposed hypothesis.

The paper analyzes a “randomly selected” intervention group of 25 health centers that provided HIV/AIDS services, which is then “perfectly matched” to a control group of 25 health centers that did not offer HIV/AIDS services. But in reality, the intervention group was “randomly” selected only in the sense that the authors chose to study them, not that the health centers in the intervention group were randomly assigned for treatment.

Indeed, why were these health centers chosen to receive HIV/AIDS funding in the first place, back in 2002 or whenever? It’s quite possible that the centers were assigned to have HIV/AIDS funding because the centers were already more likely to have better outcomes. For example, centers that received funding may have had more and better (or better paid) doctors, or perhaps they were located in areas with higher population density, or with higher HIV/AIDS prevalence rates. Similarly, the authors note that, unlike the rural areas that were the subject of the study, all urban health centers in Rwanda provide HIV/AIDS services; this fact alone suggests that treatment (HIV/AIDS funding) was initially assigned based on facility characteristics rather than a random assignment in a representative list of centers.

The authors attempt to address this issue by matching the 25 intervention health centers to 25 control health centers. But the authors match on just three characteristics – (1) health center ownership, (2) performance-based financing, and (3) district income in 2002; however, it is unclear that these were the criteria for initial assignment to treatment.

Further, the authors do not provide any information to reassure us that the intervention group and control group were comparable on a range of relevant characteristics prior to treatment that might otherwise explain differential performance.

2. External Validity: How generalizable are study’s claims?

Beyond the internal validity constraints, the generalizability of the study’s findings is very limited.

The study—and particularly the press release—claims to measure the effects of HIV/AIDS funding on non-HIV/AIDS health services. Such a claim, however, ignores the numerous channels by which HIV/AIDS funding can affect a health system besides funding HIV/AIDS treatment in existing facilities; for example, HIV/AIDS funding can lead to technical assistance at the national level, newly built facilities operated by international NGOs or other foreign organizations, as well as health promotion and preventive care at the community level. But the authors’ indicator for HIV/AIDS funding is simply a binary categorization of whether a facility offered HIV/AIDS treatment or not. Moreover, the paper does not discuss the magnitude of funding, the funding source (PEPFAR or Global Fund vs. Ministry of Health disbursements), or whether the facility received an earmarked funding stream specifically for HIV/AIDS rather than general funds which it then elected to spend on HIV service provision. The narrowly focused study does not consider the wide array of other system level effects created by HIV/AIDS funding that have been raised in the previous literature.

In particular, the study does not tell us anything about the effects of parallel NGO service delivery or the impact of new or dedicated facilities exclusively for HIV/AIDS, both of which are hot topics in the HIV/AIDS health systems debate. Indeed, in 2008, less than 5% of Rwanda’s PEPFAR funding was channeled through national institutions; the rest was delivered via a range of contractors, most of which were American NGOs or universities (Table 1). The paper makes no effort to address the consequences this funding arrangement and the presence of the 44 PEPFAR prime partners in Rwanda.

Table 1: Top Planned Recipients of PEPFAR Funding for Rwanda (USD), FY2008

What’s more, this particular country (Rwanda) is likely to be an outlier among HIV/AIDS funding recipients due to its exceptional national healthcare system, high quality HIV/AIDS service delivery, and innovative health initiatives like community-based health insurance. According to the World Bank’s World Wide Governance Indicators for 2009, Rwanda ranked 7th out of 45 Sub-Saharan African countries for government effectiveness, scoring more than one standard deviation above the mean. Moreover, HIV/AIDS funding in Rwanda accounted for about a fifth of total health spending, a percentage higher than 30 other countries in sub-Saharan Africa.

We understand that the authors likely suffered from significant data constraints; likewise, we recognize the enormous empirical challenges in demonstrating system-wide effects at the national level. Still, it remains important to carefully state qualify results and recognize the limitations of one’s research.

Bottom line: The jury is still out on whether HIV/AIDS funding has displaced or improved efforts on other disease control priorities.  Let the debate about funding imbalances for global diseases continue…

But in the current U.S. political and fiscal environment, it’s becoming increasingly clear that this kind of support for AIDS funding may have also reached a “tipping point” and will be increasingly difficult to maintain in coming years. Congress and the Administration face a tough election in 2012, and there isn’t much appetite to put HIV/AIDS on the political agenda. As the budget debate slogs forward on Capitol Hill, funding for global health could face cuts up to 9% in FY2012 and much more in 2013 if sequestration is triggered. It’s unclear what percentage of those cuts will come directly from PEPFAR’s budget but it would be naive to assume AIDS funding will remain untouched regardless of its necessity and merit.

Unfortunately, capitalizing on the AIDS tipping point is precariously dependent on continued U.S. support. In 2010, 54% of publically donated funds for HIV/AIDS worldwide came from the United States, and continued support is necessary to sustain the progress that has been made thus far. If political and financial support tips away from HIV/AIDs right now, it could offset the gains that have been made and once again place an AIDS free generation out of reach.

This dynamic is setting the scene for a showdown next summer when the International AIDS Conference (IAC) returns to the United Stated for the first time in 22 years. For one week in July, AIDS advocates, researchers and practitioners from around the world will convene in Washington, DC to discuss how to tip the balance of the AIDS epidemic in their favor. Meanwhile, 15 blocks away, U.S. policy makers will be discussing how to tip the balance of the budget back towards the green, and HIV/AIDS funding (dollars that sustain many of those researchers and practitioners up the road) will be in their crosshairs.

As a sheer numbers game, reconciling these two sides should not be difficult. A recent analysis from the American Foundation for AIDS Research on the human impact of sequestration in FY2013 shows that reductions in U.S. global health funding would have a minimal impact on the deficit, representing only 0.42 percent of the $1.2 trillion in mandated reductions. But the human impact of these cuts would be devastating: 403,000 people would not receive life-saving treatment, 92,000 more people will die from HIV/AIDs related illnesses and 181,000 more children will become orphans. The bottom line –global health spending is such a small percentage of the total U.S. budget that across the board cuts will not tip the balance of the deficit reduction, but they will cost thousands of lives, reverse years of gains against HIV/AIDS, and instantly thrust the AIDS tipping point in the wrong direction.

But the case for ending the AIDS pandemic is more complicated than a numbers game. It will require fresh approaches and bi-partisan political will in both the Administration and Congress – with the later likely to be even scarcer than usual in 2012.

So is there a middle ground? Maybe. Taking a cue from the new CGD/Center for American Progress working group, a period of austerity may present opportunities to improve the global response to AIDS and give rise to discussions on how to make structural and procedural changes to programming that would make it more effective and efficient moving forward.

In turn, sustained (if not expanded) financial support from the U.S. at this critical time could help turn the tide of the HIV/AIDS pandemic enough that the need for total investment in the future would drop significantly. This may necessitate realigning the donor/recipient relationship through innovative approaches in aid financing and delivery (cue Cash on Delivery Aid for HIV/AIDS) as my colleague Mead Over explains in his new book, Achieving an AIDS Transition. These changes would make funding more sustainable and predictable over the long-term, and help prevent tax payers from concluding that foreign assistance doesn’t deliver and never ends.

On this World AIDS Day, as we contemplate these two tipping points and look forward to a promising but uncertain 2012, I have two wishes. For Congress, as they continue to address the budget deficit, I hope they consider how close the AIDS pandemic is to a tipping point – one that the U.S. can be proud of helping achieve and one that is worth seeing through. Seeing that the Global Fund will not expand its treatment roll for the next two years, U.S. leadership is more important than ever. Neglecting to act would discount billions of dollars in investments already made, and leave millions of lives hanging in the balance.

For members of the AIDS community, I hope they can recognize that the days of perpetually increasing resources are gone and they may soon have to work towards an AIDS free generation with a smaller piece of the pie. Getting the most out of a shrinking pie means reducing the number of slices to go around. Likewise, turning the tide of the AIDS pandemic amidst constrained resources means getting serious about trying new strategies to incentivize effective prevention and reducing the number of new infections. Doing so will help mitigate the reality of today’s fiscal environment and lead us down a more sustainable path. And perhaps, come this time next year, everyone will find a way to tip the balance in their favor.

I present her comments here and invite readers to add their own thoughts in the comment section below (where I also reserve the right to rebut!)

Excerpt of e-mail from Laurie Garrett

Overall I would make a couple of suggestions:

1) Prevalence does NOT equal incidence. Here in the USA, with all the technology and money we have, the CDC officially declared the annual INCIDENCE of new HIV infections to be 40,000, every year from 1983-2007. This was ludicrous. In 1983 the CDC could be forgiven a back-of-the-envelope guess because we didn’t even have a valid HIV test. But by 2000 — long after the SF Dept of Health invented the “de-tuned assay” for incidence measurement, it was unforgiveable. Either the hundreds of millions of $$’s spent annually on HIV prevention were an utter failure, allowing 40,000 new cases annually, or nobody knew what they were doing. In 2008 the CDC finally started creating valid ways to measure incidence, and discovered their data was off by 40%: There were some 60,000 new cases/year in 2005-7. Moreover, the incidence was overwhelmingly African American and gay. For the first time in more than 2 decades the USA was applying a reasonable toolkit to prevention of HIV. Folks at Hopkins estimated the newly discovered incidence amounted to a $250 million/year increase in treatment costs, alone. That is why the Obama Admin has put so much prevention energy into DC and Baltimore, where things are exploding and incidence now surpasses Uganda.

2) If you are going to hold GOVERNMENTS accountable in a COD approach for HIV prevention, you had better have a toolkit better than the ones we have used in the USA. Even now, post-2008 I am not sure the USA could meet your COD standards.

3) Confusing prevalence and incidence is forgivable in general conversation, but not as a basis for cutting off funding to a nation. As the numbers of people surviving HIV thanks to ARVs grows worldwide prevalence is an increasingly irrelevant and misleading number. [Mead is proposing to estimate incidence by first subtracting prevalence in a baseline serosurvey from prevalence in a follow-up serosurvey and then using a model to correct for mortality and ARVs. However,] creating a valid serosurvey on a national basis for annual incidence assessment is a daunting, expensive, scientifically extremely difficult task. In our UNAIDS effort we determined that it would be impossible in a country without head-of-state and legislative full support: the logistic, political and ethical dilemmas are so large that only top-of-government mandates can make the effort doable. Do you think that is possible in Swaziland? Perhaps even more relevant, in Uganda where Museveni has a stake in downplaying incidence, due to his advocacy in ABCs?

4) The actual HIV Transition is a moving target. As mortality declines with ongoing roll-out of ARVs the magic point for incidence targets also gets harder to attain. Here in the USA our HIV-related mortality is now so low that corresponding incidence would probably have to drop from the current 60,000/year to less than 10,000 (maybe even 5,000) to come close to the transition. (At the mortality peak in the USA in 1995-6 we experienced about 17:100,000 deaths annually due to AIDS. For the last 8 years we’ve held at 8:100,000. In July a multi-country survey found that death rates in the US and Western Europe were rising among elderly HIV+ individuals, but had dropped significantly for most people on ARVs. A summary stated: “The authors calculated an 88 percent reduction in excess mortality in 2000-2001 compared with the period prior to 1996. This was very close to the 87 percent reduction seen in 1997-2001. In 2004-2006, the excess mortality was 94 percent lower than pre-1996 levels.”) Imagine what it would take for us to achieve that! For a generalized epidemic like Kwazulu-Natal the first few years of effort would require an incidence target FAR easier to attain, but if treatment rollout achieves Mead’s 80% uptake target mortality should plummet so dramatically that the incidence target gets down to the less than 3%/annual level — a very hard threshold to pass in such an epidemic.

Both sides argued vigorously and reasonably, but the underlying issue remained unresolved – how can countries and funding partners decide on the allocation of scarce resources amongst different prevention uses, particularly when “new” technologies arrive on the scene?

Well, how do we do it now? We don’t know. Choices on the allocation of prevention budgets –whether domestically or as part of PEPFAR- are opaque, driven –perhaps- by some combination of intuition, making-do, budgetary inertia, advocacy, built infrastructure, lack of data and vested interests, among others. So much so that UNAIDS just bid out a qualitative study to try and figure out what is driving resource allocation decisions (see here).  We do know that the result is less than optimal, as Forsythe et al suggested here in 2009.

While the debate itself focused mostly on the scientific merits or demerits of an early ART approach, the discussion suggested that we aspire to a different, better model of resource allocation, one that could incorporate health goals as well as economic, scientific and ethical considerations. But where are we with respect to that “better” priority-setting process?

To conduct economic evaluation, we need to know the relative effectiveness and costs of the existing arsenal of prevention interventions. Nancy Padian and co-authors’ recent systematic review of the effectiveness of HIV prevention interventions find only 9 of 45 trials with positive effects, with the majority (3) concentrated in male circumcision.  An earlier review in 2009 found that there are several types of interventions for which cost-effectiveness studies are still not available including surveillance, abstinence, school-based education, universal precautions, prevention for positives and most structural interventions. When the PRO side of the debate was asked what they would cut to make “space” for more spending on early ART, they advocated for the elimination of the unevaluated interventions in favor of an intervention that has one study under its belt (albeit with huge impact). On the cost side, the situation is bleaker. Secretary Clinton mentioned a $335/case/year number for ARV treatment in her speech this week; but this study hasn’t yet been published. The estimate by Menzies et al finds that PEPFAR’s median annual economic costs come to US$ 880 for ART patients, including medicines. The Global Fund so far only reports the costs of medicines – the antiretroviral regimens that they finance, which come to US$125/case/year. So we know something about what is effective, but less about the fully loaded costs of screening and treating.

A second piece of economic evaluation is to take the international data on efficacy and effectiveness, and combine it with local data on disease dynamics and costs. The quality of this local data is problematic, with implications for a country’s ability to assess the cost-effectiveness of a given combination of interventions. I can’t find a recent assessment of the quality of HIV surveillance systems by country, only this from 2004, that reported 58% of countries with generalized epidemics had fully implemented HIV surveillance systems. Local data on costs is largely absent, although tools to collect cost data are growing. Yet as Stefano Bertozzi pointed out today, and others have noted elsewhere, the mix of prevention interventions that will minimize new infections will depend greatly on the characteristics of the epidemic and the costs of providing different kinds of care in that particular country.

While recognizing the data limitations facing the economic evaluation of prevention interventions that could inform the question of how to allocate the prevention budget, let’s assume for a moment that countries –with support- can undertake these studies or apply tools to determine an “optimal” mix of prevention interventions that would lead to zero new infections, and that early ART comes out a winner. According to 2010 WHO guidelines, UNAIDS estimates that 65% of the eligible population is not currently receiving treatment.  Nancy Padian et al point out “an essential question is how a country’s health service could maintain antiretroviral therapy in legions of healthy patients with high CD4 cell counts mainly for prevention benefits to partners, when it is not able to initiate and maintain high retention of those with low CD4 cell counts who need ART for survival.” This choice will put a global –or a U.S.- goal in stark contrast, perhaps, to locally expressed demands or preferences for acute care. How will these choices be made? By PEPFAR’s Scientific Advisory Committee? I hope not.

Countries and their donors need better, more systematic ways to consider the scientific evidence, run the economic evaluations, understand the trade-offs between different uses of money, consider the ethical and other implications and take fully informed decisions about the uses of their budgets today and in the future. Current practice –from the Copenhagen Consensus to countries being asked to allocate in the absence of national evidence and fair process, to donors deciding directly what to provide to whom, under what conditions- is problematic ethically and likely counterproductive to an AIDS-free future.

At CGD, we are running a working group on priority-setting institutions, and how donors can better support and facilitate national processes that are evidence-based, ethical, reasonable and accountable. We’ll issue a report in 2012, and look forward to future debates on the issue.

The HIV/AIDS international subculture is big and impressive.

They enjoy for the moment a kind of dream world in which they are able to debate optimal use of a pie (public money to deal with AIDS prevention and treatment) which they assume is fixed or increasing.

It’s hard for an economist to buy into that world (as it is for doctors to conceive that it doesn’t exist).

The debate about allocation of this (apparently fixed) HIV/AIDS pie between prevention and treatment (see Mead Over’s new CGD book here – the book launch is next week) is newly raging because of new evidence from a randomized controlled trial: much earlier use of anti-AIDS drugs (“treatment”) is in a biological sense hugely effective in preventing transmission of HIV infected people to their partners. In her speech on HIV/AIDS this week, Hillary Clinton was upbeat about the new promise of this new evidence (for a comment on the speech go here), bringing new hope to the medical community and AIDS advocates that the world is now at the beginning of the end of the pandemic.

That takes me back to the pie. What about the pie? Even if it grows, there will be tradeoffs… No one mentioned discount rates (lives lost now vs. more lives saved later…); tradeoffs are still a taboo in AIDS-world if not in AIDS-policy world. No one mentioned other lower-cost (right now) ways to make life better for more people– reduce malarial deaths, combat sex trafficking, educate more girls.

I came away with a question, perhaps for social psychologists. What is the effect of big dreams (let’s go to the moon/we can eliminate AIDS altogether) on the psychology of policymakers and legislators? Does shooting for the moon generate ADDITIONAL money for the moon shot, eliminating tradeoffs? Could this new exciting evidence mean the AIDS pie, which has stopped growing in the last couple of years, will now grow again?

I think US’s $100 million spent on understanding combination HIV prevention– which is matched by almost as much investment from the Bill and Melinda Gates Foundation and other partners– is money well spent to show whether combination prevention can work. But here’s the rub, the prospect of eventually having an AIDS-free generation depends crucially on the outcome of these rigorous multi-year research projects that are just now being launched. And none of the current PEPFAR funded trials will be able to untangle the separate contributions of the several prevention interventions being combined (to do so would require more “arms” for each study, which would increase costs almost commensurately). The fact is, we don’t yet know whether any of them will succeed in lowering rates of new infection in the community as a whole. But we do know their costs will be exorbitant.

What is lacking in the Secretary’s vision of an AIDS free generation is the intermediate goal of an “AIDS transition” that I describe in my new book (which will be launched at CGD on November 16). The objective suggested by Secretary Clinton of reducing the number of new infections to match the number of new people placed on treatment is sadly not sufficient, as the people living on AIDS treatment are dependent on daily medication and require expensive lifetime support. Instead, the number of new infections must fall below the number of AIDS deaths, while ART is sustained and expanded. A country that accomplishes this objective will have achieved what I call the “AIDS transition”.

So how can the U.S. promote the AIDS transition? A solid, clearly defined multi-year US commitment is the fair and incentive-compatible framework upon which the US can plan its own future engagement and the recipient government can grapple with a well-defined unmet need. As Secretary Clinton pointed out, the US has for the first time made a multi-year commitment to the Global Fund for AIDS, TB and Malaria. This is commendable, but recipient African countries are extraordinarily vulnerable to the ups and downs of volatile foreign assistance disbursements and need reliable commitments even more than does the Global Fund. To each country where the US has already granted virtual “entitlements” to AIDS treatment, the administration should promise that it will sustain financing for those thousands of patients and expand treatment to X percent of those will need each year until the AIDS transition is reached. By specifying concretely the value of X, which is likely to be well below 100%, the US will be guaranteeing the minimum rate of treatment expansion, a minimum on which other donors and the recipient government can build.

How should the specific percentage for expansion of treatment be decided upon for a given country? The US should use the recent trends in new infections and the cost of treatment to estimate how much will be required to finance the given percentage of new patients until the AIDs transition is reached. Then with an eye on its overall budget, the US should only promise to treat the percentage of need that it is willing to finance from now until the projected AIDS transition, a decade or more in the future. The US and the recipient government will then share the savings if costs go down even faster. And the US can also promise that evidence of a faster-than-anticipated decline in new infections over the next few years will be rewarded with an increase in the US-funded rate of treatment expansion.

I’m glad that Secretary Clinton has taken the stage to proudly champion the US’ leadership role in AIDS. The fact that the US government has been able to continue to expand the numbers it supports on ART despite a flat budget for PEPFAR since 2008 is a tribute to the Obama administration’s leadership and to hard work by Ambassador Goosby and the PEPFAR staff. To effectively lead into the future, the US should clearly articulate its willingness to keep on expanding treatment, but not without limit. The promise of an AIDS free generation does indeed lie before us. In today’s budget environment, the only feasible path to that promise lies through the AIDS transition.

This seems a surprising list in several ways.  First, economists tend to give priority to government interventions which attempt to correct market failures, such as those caused by “externalities” (i.e. spillover effects) or “asymmetric information”.   Neither the authors nor the panelists analyze the two interventions which target populations where spillover effects and asymmetric information enhance the benefit cost ratio of interventions:  High risk groups and couple counseling.

Spillover effects: In situations where the behavior of identifiable subsets of the population make a disproportionate contribution to the HIV epidemic, a targeted intervention can have a much higher social benefit-cost ratio than an untargeted one.  While the heterogeneity of sexual behavior is evident both within and across African countries and has been known since the 1976 paper of Anderson and May to accelerate a sexually transmitted epidemic, the only assessment paper that recognized this heterogeneity is that by me and Geoff Garnett on ART.  In particular, the assessment paper on sexual transmission, authored by Jere Behrman and Hans-Peter Kohler, ignored the heterogeneity of behavior and therefore omits analysis of the potential benefit-cost ratio of interventions targeted at most at-risk populations (MARPs).
Asymmetric information:  As pointed out in Susan Allen’s comment on my Friday post, Behrman and Kohler’s analysis of “large scale testing and counseling” omits any consideration of targeting HIV testing to couples, an option that reduces the asymmetry of information regarding HIV status and therefore should have a higher benefit-cost ratio than would the individual testing on which the authors focus.

Another surprising feature of the Copenhagen Consensus list is its inclusion of an intervention which was never analyzed by any of the assessment papers and only mentioned as an additional or supplemental idea by one of the discussants: infant male circumcision.   It displaces the circumcision of adult men, which the panelists ranked in 7th place, and deprived of any of the $10 billion budget.   What happened?  How and why did the panel become so “creative”?

Here is the information that the assessment paper authors gave the panel on adult male circumcision (AMC) and on the intervention to which they allotted the largest budget share, ART.  At the discount rate of 3% and the value of a life-year of $5,000, these two interventions would have the following benefit-cost ratios:

So despite the fact that the authors considered adult male circumcision to yield from 12 and 20 times more benefit per dollar of investment cost, the panel of distinguished economists decided, on the basis of this analysis, that antiretroviral therapy ranked in fifth place among all the interventions, two places above the seventh place ranking of adult male circumcision.  Since the panelists allocated their entire hypothetical $10 billion to the first 5 interventions, they provided no incremental funding, even hypothetically, for adult male circumcision.  This decision is remarkably obtuse given the evidence on this intervention and the fact that this most promising intervention needs support to be scaled up in African countries today.

Part of the problem was that the authors of the assessment paper analyzing AMC, Jere Behrman and Hans-Peter Kohler, chose to use the 30% effectiveness assumption for their main results instead of the 60% result found in the randomized trials.  This is a very conservative assumption.  Given that male circumcision has been consistently and rigorously shown to prevent 60% of infections in the trials, field effectiveness could only be degraded down to 30% if either of two offsetting behavioral effects is extremely powerful:  (a) Selection Bias: willingness to accept circumcision is much higher among men who would have otherwise protected themselves by condom use or having fewer partners or (b) Disinhibition: willingness to accept circumcision is much higher among men who, once circumcised, engage in many times more risky sex than they would have without the circumcision.  The first effect posits that prudent well-informed men will disproportionately seek AMC, while the second posits that circumcision converts such prudent men into reckless thrill seekers.  If the sexual behavior of African men were this sensitive to the risk of HIV infection, they would have ceased their risky behavior long ago, when awareness of the danger of risky sex became widespread in African societies.

But the panel of distinguished economists apparently thought Behrman and Kohler were not being conservative enough.  At the meeting on Wednesday reported in USA Today, two of the panelists explained why they had downgraded AMC.  They justified their decision partly by the selection and disinhibition arguments, but also partly based on the undocumented assertion that adult African men would simply find male circumcision unacceptable.   Apparently in reaction to the cognitive dissonance produced by these unsupported beliefs and the authoritative estimate that the benefit-cost ratio of AMC is 60 to one, the panelists creatively introduced an 18th intervention to be added to the list they had been given: neonatal circumcision.  Arguing that neonatal circumcision would be immune to the selection and disinhibition problems and would be more acceptable to African populations, they made this one of their top five interventions.

Now neonatal circumcision is not a bad idea.  It can be expected to cost about the same as adult male circumcision and to eventually achieve, after a lag of 15 years, the optimistic benefits of adult male circumcision.  Since the costs are incurred today, but the benefits accrue 15 years later, the benefit cost ratio must be discounted by multiplying it by (1/(1.03)^15 = .64, which yields a benefit cost ratio of about 77 to one, down from 120 to one.  However, during the fifteen years we wait until the children reach maturity, the epidemic will continue to spread to a larger and larger proportion of the population.

So if neonatal circumcision crowds out adult circumcision before adult circumcision has been widely scaled-up, that would be a bad thing indeed.

Which brings me to a more general criticism of the Copenhagen Consensus’ application of benefit-cost analysis to HIV/AIDS.  Epidemics are highly non-linear.  They are characterized by a parameter called Ro, the “reproductive rate”, which is described so compellingly by Kate Winslet in the super new movie, Contagion.  If Ro is above unity, the epidemic grows until it saturates the population.  If it is below unity, the epidemic gradually disappears.  The challenge then is to find the combination of interventions, which, when working together, will bring the value of Ro down below unity.  That solution might well be a combination of ART, male circumcision, behavior change and an eventual vaccine.  The Copenhagen Consensus process, by requiring each author to analyze only one of the 17 interventions at a time, makes it difficult or impossible to find the combination of interventions which can reduce Ro below 1.0.  Thus, when applied to the control of an epidemic, the Copenhagen Consensus should modify its process to require each author to evaluate a different combination of interventions, instead of a single one.  With this approach, the Copenhagen Consensus process might have had a better chance to produce sensible results for HIV/AIDS.

(1) commissioning “Assessment Papers” on competing ways to spend a hypothetical additional $10 billion on HIV/AIDS in Africa over five years;

(2) commissioning “Perspective Papers” by discussants who critique the Assessment Papers and suggest alternatives;

(3) commissioning a “Nobel Laureate Expert Panel” to judge the competitors and rank the alternatives from most to least advantageous for the developing world populations they are intended to help.

Together with Geoff Garnett, formally of Imperial College, London and now at the Bill & Melinda Gates Foundation, I am the author of one of the six competing assessment papers in this competitive endeavor.  On Monday and Tuesday Bjorn Lomberg, Director of the Copenhagen Consensus Centre, and Marina Galanti, co-founder of the Rush Foundation will be chairing sessions at which the Nobel Laureate Expert Panel will hear all six of the authorial teams argue our analyses of our respective assigned interventions.

The six competing interventions are;

1. Prevention of sexual infections
2. Prevention of non-sexual transmission
3. Vaccine research
4. Social policy
5. Health systems strengthening
6. AIDS treatment

Geof Garnett and I are the authors of the Assessment Paper on AIDS treatment.  I am honored to be included with such a distinguished set of authors, whose names you can find on the project the RethinkHIV website.  The papers should be posted in draft form on Monday.

The rules of the exercise have been:

1. Compute the benefit-cost ratio of spending $10 billion over five years on our assigned intervention in sub-Saharan Africa
2. Show the sensitivity of the benefit-cost ratio to two alternative values of a life-year, $1,000 and $5,000, and two alternative discount rates, 3% and 5%.
3. Compare the increased $10 billion of spending to a plausible counterfactual, which might be a continuation of the current trend in program expansion.
4. Use the $10 billion five-year budget to select a trajectory for policy for coming decades from among the trajectories that would cost $10 billion in the first five years, and then evaluate the benefit-cost ratio of that trajectory.  (For a very long-period event, like the HIV epidemic or a climate intervention, it would be nonsensical to restrict a benefit-cost analysis to the benefits that occur within five years.  Most of the benefits of today’s AIDS treatment or HIV prevention will be reaped more than five years from now.)

Though I have given countless presentations in my life, I am somewhat intimidated about this one.  I’m used to the prospect that a discussant can sometimes ask pointed and challenging questions, but never since my thesis defense (an embarrassingly long time ago) have I been in the position of being grilled by a panel, and this panel consists of extremely distinguished members of my own discipline.

My discomfiture is partly that neither Geoff nor I, if we had to choose how to spend an additional $10 billion on AIDS in Africa, would spend every penny of it on AIDS treatment.  And we are pretty sure that the other five authorial teams would feel the same way about their interventions.  The stated goal of the Rush Foundation is to ”fund disruptive ideas in the fight against HIV in sub-Saharan Africa”.  Currently the most disruptive idea is “combination prevention,” an approach to combatting HIV/AIDS that combines in creative ways AIDS treatment, various medical and behavioral HIV prevention ideas and social policy in an attempt to benefit from synergy among them and create a winning mix.  (See the information here, here, here and here.)   By constraining the authors of the Assessment Papers to analyze interventions in isolation from one another, we may have been constrained from offering the most “disruptive” or beneficial ideas.

In defense of the Copenhagen Consensus approach, we are asked to analyze our interventions in a context where all the things currently funded continue to be scaled up at plausible current rates of expansion.  So when Geoff and I estimate a benefit-cost ratio for AIDS treatment, the rules permit us to suppose that the incremental $10 billion on treatment will be in addition to the trend levels of behavioral prevention and vaccine research, etc…   Furthermore, while the Assessment Paper authors were constrained, the Perspective Paper authors could propose more disruptive approaches, including combination prevention.  And of course the overarching panel of experts is free to be as creative and disruptive as they wish.

Watch this space for a report from inside the closed meetings.  In the meantime, please feel free to offer your own assessment.  How would you spend an additional $10 billion over five years on AIDS in sub-Saharan Africa?

Looming budget cuts for FY2012 and recent reports about the decline in AIDS funding from the USG in FY2010 relative to FY2009 have triggered the classic Washington, D.C. tug-of-war;  global health and development advocates are pushing to maintain funding levels, if not to increase them, and the U.S.  Congress is looking for ways to increase oversight and management of taxpayer dollars. Advocates are rightly pointing out what would happen if we don’t have the money and Congress is rightly signaling that the party is over.  What’s new? Nothing.

But lost in this old push and pull battle is a pragmatic and productive way forward, particularly for programs like PEPFAR.  Since its creation in 2003, the President’s Emergency Plan for AIDS Relief (PEPFAR) has received relatively unfettered political backing in the United States (by global health standards); enjoying bi-partisan support, full funding in annual appropriations and re-authorization in 2008.  It’s clear this kind of support will be increasingly difficult to maintain in the current political and fiscal climate which is why now, more than ever, PEPFAR needs to better demonstrate the effectiveness and value of its programs.  So here are my thoughts on why PEPFAR needs a Value for Money Plan and some steps towards creating that plan.

Why PEPFAR Needs a Value for Money Plan

PEPFAR’s efforts to combat the global HIV/AIDS epidemic have achieved impressive results.  But results data that are publically available are largely reported through program anecdotes and measures of people impacted by PEPFAR programs.  PEPFAR disaggregates results by country and program area (prevention, treatment and care), but does not tie data to specific activities.  From these data, we can make general observations about PEPFARs impact in FY2010, such as:

• PEPFAR directly supported antiretroviral treatment for more than 3.2 million men, women and children worldwide.
• PEPFAR directly supported antiretroviral prophylaxis to prevent mother-to-child HIV transmission for more than 600,000 HIV-positive pregnant women, and averted an estimated 114,000 infant HIV infections.
• PEPFAR directly supported 11 million people with care and support, including nearly 3.8 million orphans and vulnerable children.

These publically available data illustrate important gains PEPFAR is making in the global fight against HIV/AIDS, but they stop short of permitting assessment of outcomes and impact of PEPFAR programs, robust cost-effectiveness analyses and comparisons of investments with results.  After 8 years of an unprecedented multi-billion dollar foreign aid program, it’s time to know more.

What Should the U.S. Congress Do?

As congress considered measures for increased oversight of PEPFAR, they shouldn’t make the best (expenditure data) the enemy of the good (strengthening information systems for better monitoring and evaluation of programs).  While ideal, chasing expenditure data at the country level is impractical in a developing country context where strong fiduciary systems often don’t exist.  After interacting with PEPFAR recipients in Mozambique, Uganda, and Zambia for CGD’s HIV/AIDS Monitor initiative research, it is clear to me that information systems are dysfunctional and/or overwhelmed, whether for financial or program data reporting. This is not to say that expenditure data isn’t important, but good expenditure data may only be generated when strong financial management and information systems are in place.

The World Bank has spent years trying to get countries to develop better fiduciary systems and report expenditures but it’s still unclear if this has led to better development outcomes.  Continuing to invest increasingly scarce resources to produce bad or fake expenditure data won’t help Congress decide if PEPFAR dollars are having an impact.  Isn’t it time for us to develop information systems to learn how and which programs work to achieve a stated set of objectives instead of obsessively trying to capture elusive expenditure data? I think so.

What Should OGAC Do?

1.    Share Existing Data

PEPFAR implementers collect a wide range of valuable financial and related data on its programs but do not make this detailed information publically available.  Financial data that are publically available are largely planned and obligated funding, and only illustrate broadly where funds are being directed in terms of priority disease and program areas.   Data are also available on the cumulative obligations and outlays spread among the various implementing agencies and the three relevant appropriations bills (Foreign Ops, Labor-HHS-Education, and DoD).  From these data, we can infer that of the $6.9 billion appropriated for PEPFAR in FY 2011:

• $5.6 billion (81%) is for HIV, $243 million (4%) for TB, and $1.050 billion (15%) for the Global Fund.
• The largest share of planned funding in FY2010 is for treatment (37.6%), followed by prevention (35.9%) and care (26.5%). Funding for OVCs is part of care funding, and totaled 10.3% of approved funding in FY 2010.

But, that’s all we can really tell. A 2008 CGD HIV/AIDS Monitor Analysis and a memo to the then incoming President Obama recommended that PEPFAR make spending data publically available, specifically highlighting official data on obligations to prime- and sub-recipients. OGAC has since released data on obligations and outlays to each prime recipient, though data for sub-recipients is still not publically available.  In addition, PEPFAR recently reported they are strengthening their use of economic and financial data and moving towards routine implementation of expenditure tracking in some country programs and issued specific indicator guidance in 2009 that emphasizes a greater focus on outcome measures of program coverage and quality.

Providing access to a disaggregated country breakdown of PEPFAR funding activities and increasing investments in better reporting systems for outcomes measures would improve transparency and help to measure impact, optimize resource allocation, and clarify important policy debates.  The challenges of attributing results in the field directly to PEPFAR support, including weak fiduciary and information systems, should also be recognized and taken into account when considering ways to improve data collection and reporting. Making this data publically available will make funding decisions more transparent, and could reward effective programs and incentivize poor performers to improve (read more on how PEPFAR could better link funding decisions to performance here).

2.    Develop Meaningful Metrics of Success to prioritize data collection and reporting

PEPFAR should report results that go beyond  outputs (counts of people treated and cared for, and counts of HIV infections averted) and show us what has happened, for example, as a result of enrolling people into treatment programs. For example, of all the people who require ART in say, the catchment area of a PEPFAR supported program in Ethiopia, what proportion are actually receiving ARTs? Or of the people who are receiving ART, how many have been able to go back to work, or to school in the case of children?  Spending some time and money, after 8 years of learning from PEPFAR, to develop strong and meaningful outcome measures is long overdue. Knowing what one’s measures are BEFORE projects are funded may also assist in prioritizing data collection and could prove to be a better investment for the USG than trying to set up impossible oversight processes that yield little and low quality expenditure data.

Next month, Congress will begin making critical decisions about the allocation of future spending – including for PEPFAR – and unfortunately in this political and fiscally constrained climate, the current metrics aren’t going to cut it.   With 20 percent cuts already on the table for the international affairs budget, PEPFAR is in for a tough fight in the fall and it’s more critical than ever to demonstrate how effective its programs are.   What do you think PEPFAR’s leaders can do to successfully implement a value for money plan?

This dramatic evidence that ART lowers the infectivity of HIV supports the hypothesis of a beneficial biological external effect of ART.  But is the evidence surprising and powerful enough to be a “game changer” as Michel Sidibe, the director of UNAIDS declared, or “to end, or at least diminish, a bitter feud within the AIDS world over how much funding should go to treatment versus prevention,” as the Wall Street Journal has suggested?

It has long been known that ART can have both biological and behavioral effects on prevention.  A simple typology of these effects classifies them as either beneficial (because they slow transmission) or adverse (because they speed transmission).  (See the table from our 2004 book reproduced at the bottom of this blog.)

So the new experimental finding adds support to the upper left quadrant of this table positing a beneficial biological effect and thus to the proposition that I blogged in 2008 here and here that these beneficial externalities of treatment might justify a greater  allocation of resources towards treatment, as compared to a pure cost-effectiveness analysis that ignores such effects.  In the majority of African countries that have not yet managed to treat a large proportion of those with more advanced AIDS disease, the push to use ART for prevention increases the tension between the ethical mandate to treat the sickest patients first and the stronger efficiency argument for allocating limited treatment resources to those who are not yet sick.

But it takes more than this confirmatory experimental trial to argue that prevention resources should be re-allocated towards treatment.  Modeling shows that even if people start treatment when their CD4 count is 500, as they did in this trial, and then faithfully adhere to their daily dose for their entire lives, the future burden of the epidemic continues to rise until 2046.  The two panels of the following figure project the future of the epidemic in Sub-Saharan Africa through 2050, incorporating the trial result that the infectiousness of treated persons is reduced by 96 %.  These calculations also assume optimistically that financing will be found to enroll every HIV-infected person whose CD4 count is below 500, that circumcision roles out to 80 % of adult males by 2025 and that a 60% effective vaccine begins to roll out to all adults in 2025.  Panel a of the figure shows that without enhanced behavioral prevention the number of Africans living with HIV/AIDS continues to increase until 2046, reaching a peak of 38 million before the AIDS transition occurs.  And so does the cost of treatment, rising from its current level of around $6 billion to above $60 billion by 2050.

Figure.  Simulations of the future African HIV/AIDS epidemic if treatment reduces infectivity by 96% and reaches everyone with a CD4 count below 500.  Both scenarios assumed male circumcision reaches 80% by 2025 and a 60% effective vaccine begins to roll out in 2025.  Panel a) assumes risk behavior is unchanged from current levels, while panel b) assumes a 50 % reduction in risk behavior.

(Source: Projections from the AIDSCost model.   If you own a copy of Stata version 10 or later, you can produce the above projections by typing “findit AIDSCost” from inside Stata, installing the program, getting the data file and executing the following command for panel a:
“aidsproj using aidscgd2010.dta, uptake(.99) takeoff(2010) horizon(2050) cd4(500) maxep(0) gp(.96) weo regions(SSA) graph grcombine”   For panel b: Same command replacing maxep(0) with maxep(0.5).  )

These projections assume constant cost per patient-year of both 1^st and 2^nd-line treatment, which seems reasonable since current pressures on unit costs are in both directions.  Perhaps the unit cost of treatment will fall as pharmaceutical firms recognize the profit potential of 35 million customers at a low price as compared to 5 million at a high price.  But no matter how inexpensive the individual drugs, the number of people taking them daily would still rise by a factor of seven, creating an ever larger dependency of recipient countries on the donors and pharmaceutical firms providing the drugs.

In contrast panel b shows a scenario combining 99% uptake at a CD4 level of 500 with effective behavioral prevention which reduces infections from high risk behavior by 50 %.  Adding only this behavioral change moves forward the date of the AIDS transition by 16 years, to 2030, so that by 2050 the number of Africans living with HIV/AIDS has declined to 24 million and annual treatment expenditures will be about 25 % less than without this behavioral change.

With these considerations in mind, behavioral HIV prevention still deserves the highest priority.  The potential savings in AIDS treatment cost from even small behavioral changes accumulate to extraordinarily large benefits over the next 40 years and effective prevention shields the African countries and their citizens from an important part of the looming burden of this disease.

Source: Based on Table 3.3 on page 46 in Over et al., HIV/AIDS Treatment and Prevention in India: Modeling the Cost and Consequences (2004) available from Google Books here.