My challenges to this proposal are four:

1. Legality. Hugh argues that child labor is wrong because it is illegal in many countries with microfinance. Excellent point! In fact, most microfinance clients are engaged in illegality one way or another: squatting on city land to build houses the width of a queen-sized bed, failing to pay taxes on their meager earnings, failing to register their tiny businesses with the authorities… So to expunge microfinance of scofflawery, we need to shut it all down. Seriously, Hernando de Soto showed how, at least in Latin America, elites have purposely complicated the law in order to make formality—legality—a privilege rather than a right. Being poor means you are almost automatically illegal. Thus legality is a wobbly compass microfinance.
2. Ethics. We are all descendants of children who survived to adulthood only by laboring, whether as farmers or herders or gatherers. Only with their labor could the family subsist. I look forward to the day when there is no child on earth for whom this is the best choice. But we are not there yet. And we are not as close as you might think. Going by the numbers, the world has made great progress getting kids into school. However, a huge number of these children aren’t learning very much. So how quick should we be to tell parents struggling under circumstances far different from our own what the right choice is? Many of them agree with you on the value of education. Whether it is best to put their children in the schools they can afford today is another matter.
3. Evidence. The effect of microfinance on child labor is an empirical question, whose answer will probably vary by context. On the hand, microfinance sometimes stimulates at-home businesses, leading parents to pull kids out of school and employ them at home. On the other, it gives parents a new way to finance school fees, providing them the discipline to set aside money each week for this purpose.

   The evidence, like the ethics and the legal argument, is ambiguous. A good non-randomized study in Thailand found credit to increase child labor. One in Guatemala found the opposite. (Hat tip to Hugh for both.) The randomized studies, which I trust more, have mostly found little impact. In Hyderabad microcredit availability did not lift or lower the number of kids in school. In Manila, loans made no difference for the average response to “Any Household Member Helping in Family Business?” Ditto, essentially, in Mongolia. In Morocco, children worked 5.05 hours/week in areas with more microcredit versus 4.88 in areas without, a difference that is not statistically significant; meanwhile the number of children per family in school was slightly but statistically higher in the microcredit treatment areas, at 0.76 instead of 0.73.

   An exception appeared in Bosnia & Herzegovina. Among less-educated, and presumably poorer families, microcredit caused more 16–19-year-olds to work at home, where “home” often meant “farm.” As I blogged before, it is not easy to second-guess poor families in the midst of a major economic crisis if they use credit to invest in their farms, perhaps in more livestock, and put these near-adults to work.

4. Principals and agents. Now, one could retort that even if microfinance does not press children into labor on average, it still must do so sometimes. After all, how comforted would you be if I told you that microcredit does not increase slavery on average? And microfinance increases child labor in some cases, then one can argue that MFIs should vow never to finance such exploitation, and that microfinance investors must demand such vows in return for funds.

   I do hope that microfinance officers don’t leave their ethics at the home, that if they find a child laboring in great duress, and they know that a loan would make things worse, then they will not lend. But in general, microfinance, especially group microfinance for the lower-income clientele, has succeeded by not taking much interest in clients’ business. Monitoring clients takes time, time costs money, and higher costs lead to higher interest rates. Anyway, trying to determine what people do with their loans or savings withdrawals is often a fool’s errand because of fungibility. Moreover, as Hugh’s book dramatizes, reality tends to diverge from rhetoric as one moves along the microfinance investment chain—from individual investor, to investment fund, to MFI headquarters, to field practice. MFIs may say they have banned loans for child labor, and MFI investors may buy that reassurance rather easily—but should we believe them? It will be a great achievement if a program like the Smart Campaign can reliably monitor and certify microfinance field practice as being transparent and non-coercive. I think it is a goal too far to certify what is being done with microfinance in each household. Microfinance investment funds promising to rid their portfolios of child labor will be setting the stage for hypocrisy.

This issue exemplifies a larger problem in international aid and philanthropy. A donor that enters a country with plans to make loans or drill wells or build roads cannot understand, much less control, many of the consequences of those interventions. Often local political structures undermine intentions: bed nets meant to be donated to the poor are pilfered and sold to the highest bidder. Pouring microloans into an Indian slum will perturb the paths of thousands of families. In some, more children will walk out the door each day, headed for the local schoolhouse. In others, more will be enslaved at looms (though I do wonder how much of the most vile child labor occurs in household-level enterprises). In the face of diversity and uncertainty of the outcomes, donors can either proceed or not. I think they’ll do best to base their choices on the evidence—which looks pretty good in this case—and a general theory about how the intervention contributes to development. Roads, for example, do harm as well as good, but in many cases clearly more good. Similarly microfinance (if the credit is administered in moderation) is a generally useful service that can give people more control over their lives. Some will use that control for ill, but that doesn’t make giving the poor more options a bad idea.

Apologies to Jerry Mander, author of Four Arguments for the Elimination of Television.

I have to admire the pointed prose:

The Commission even records in their report that we do not participate in board discussions. They complain that even the learned representatives of the Governments did not participate in the discussions either. What a great board! None of the members participate in the discussion, but the bank still wins the Nobel Peace Prize! A non- speaking board has created the most admired bank in the world! Why does the Commission underestimate us? Could it be because we are poor and because we are women!

I was a bit disappointed that they prefer to play the gender card, pointing out that the commission is all male, to addressing the substance of the interim report. Truly the two sides are talking past each other.

But Asif Dowla’s recent comment helped me appreciate one thing the board members wrote:

Most of us never went to school, but that did not mean that we did not know how to run our business and run Grameen Bank. We managed the bank better than anyone else in the banking world. We did not let our bank become a corruption-ridden bank, like the government banks are.

Maybe they are not finance wizards, but they are sterling board members by historical Bangladeshi standards in one sense, as Asif explained:

One has to compare Grameen’s governance with governance in state owned and private banks in Bangladesh. The state owned banks board membership are handed out to people not because they are “experts in law or finance”, but because they are supporters of ruling political party and the membership is a form of patronage and payoff. Until recently, the board members of private banks were full of people who were defaulting on loans from state owned banks! In many cases, the directors of private banks were taking loans from their own banks!

Business training programs are a popular policy option to try to improve the performance of enterprises around the world. The last few years have seen rapid growth in the number of evaluations of these programs in developing countries. We undertake a critical review of these studies with the goal of synthesizing the emerging lessons and understanding the limitations of the existing research and the areas in which more work is needed. We find that there is substantial heterogeneity in the length, content, and types of firms participating in the training programs evaluated. Many evaluations suffer from low statistical power, measure impacts only within a year of training, and experience problems with survey attrition and measurement of firm profits and revenues. Over these short time horizons, there are relatively modest impacts of training on survivorship of existing firms, but stronger evidence that training programs help prospective owners launch new businesses more quickly. Most studies find that existing firm owners implement some of the practices taught in training, but the magnitudes of these improvements in practices are often relatively modest. Few studies find significant impacts on profits or sales, although a couple of the studies with more statistical power have done so. Some studies have also found benefits to microfinance organizations of offering training. To date there is little evidence to help guide policymakers as to whether any impacts found come from trained firms competing away sales from other businesses versus through productivity improvements, and little evidence to guide the development of the provision of training at market prices. We conclude by summarizing some directions and key questions for future studies.

That’s from What are we learning from business training and entrepreneurship evaluations around the developing world?, a new working paper by two leading development researchers, David McKenzie of the World Bank, and Christopher Woodruff of the University of Warwick.

In short, it is still unclear whether catastrophe—corrupt political meddling, sweeping loan forgiveness, or a bank run—will befall the Grameen Bank or whether it will thrive beyond its founder, as all involved profess to hope.

On February 9, the second commission appointed in recent years by the government to investigate the Grameen Bank released an interim report. Where the fist commission’s report focused on themes in the Tom Heinemann documentary—Did Grameen charge high interest? What happened to that Norwegian aid money?—this one sticks mostly fundamental to legal questions. Is the Grameen Bank a public institution or do its 5 million-plus shareholders, mostly poor women, own it? Was it legal for the Grameen Bank—or at least its top people—to have created a web of related companies? Did the governing board exercise appropriate oversight?

After a close reading of the ordinance that created the Bank, board minutes back to 1983, and other documents, the Commission arrives at controversial conclusions. The nine female member-borrowers on the Grameen Bank’s board should resign immediately. The Bank should un-invite 90% of its members by buying back their shares. The remaining 10% should elect new board members, with only those who have completed 7th grade eligible to run. The license to operate a digital mobile network held by Grameenphone, Bangladesh’s largest phone company, should immediately be suspended—but as a matter of practicality, the firm should keep operating. All these conclusions rest on what are, if not technicalities, then purely legal issues. For example, as the Grameen Bank grew, it needed authorization to sell shares to more members; the report says that share expansions in 1991 and 1994, unlike in 1986, were merely approved by the Ministry of Finance and not effected by amendment of the Ordinance. Therefore they were illegal and all shares sold in excess of of the 1986 limit—90%—should be bought back. Similarly, the Bank’s support for some other Grameen organizations, given through loan guarantees, was illegal because those organizations strayed from the Bank’s official mission to serve landless poor in rural areas. Are you indignant yet?

I learned a lot from the careful analysis in this interim report, and I think it contains valid criticisms and questions about how the Grameen Bank was run. In particular, and unsurprisingly for a successful founder-led institution, it appears that the Board was ineffectual and that Muhammad Yunus ran the bank with a free hand. The highly touted female borrowers who constitute the board’s majority could not be expected to understand the octopus-like complexity of the Grameen family of companies, assuming they were apprised of it. And they certainly could not be expected to perform appropriate oversight. In the event, they hardly ever said anything at the meetings beyond pleasantries.

But I think the best way to understand the report is to start with what it does not say and does not change:

• After this report, as before it, there is no suggestion that Yunus was corrupt, or that any of the alleged illegalities were perpetrated for purposes other than the social good. This in a country known for corruption petty and grand. (A possible exception has to do with the alleged failure of the Norwegian phone company Telenor to comply with an agreement to sell Grameen Telecom 16% of Grameenphone. But nothing in the report implicates people at the Grameen Bank in connection to this.)
• After this report, as before it, it was clear that the Grameen Bank suffered from ills typical of successful founder-led organizations, notably the inability of the board to impose accountability. And it was clear that the founder’s departure creates both challenges and opportunities.
• After this report, as before it, the government can do whatever it wants to the Grameen Bank, because Prime Minister Sheikh Hasina commands an overwhelming majority in Parliament. The report’s rationales for government intervention were hardly required. Hasina awaited no such investigative findings before changing the rules last summer to take control of the process for picking Yunus’s successor.
• And, on that point, after this report, as before it, the Grameen Bank is less independent of a historically corrupt and unstable government than it was for its first 29 years.

Here are some key points from the report, with commentary:

• The Grameen Bank is a public institution, created by a special law. Even if Bangladeshi women own almost all the shares, the Bank is not private property. In reasserting control, the government has not seized private assets. The shares entitle holders only to participate in elections of board members, and perhaps to dividends—and probably not even that. I find these arguments largely convincing. I do wonder whether they go too far in interpreting the Ordinance’s dispensation that the “annual profit…shall be utilised in such manner as the Board may determine” as barring dividend issuance. (“Utilise,” the report’s argument goes, doesn’t encompass giving the money to others.) I do buy that the Grameen Bank is a public institution whose governance happens to give clients a say in the selection of the majority of the board. That doesn’t change the fact the the government just, um, emasculated the board by taking away its most important function, picking the director.
• The Grameen Bank has illegally strayed beyond the ends and means set forth in the Ordinance. The Ordinance opens this way:
  

  Whereas it is expedient to establish a Grameen Bank to provide credit facilities and other services to landless persons in the rural areas and to provide for matters connected therewith or incidental thereto…

  Later, the text defines “landless” as owning less than half a hectare of land, or total assets worth less than a hectare. The commission’s lawyer-members read this to say that Grameen should not be engaged in, say, microcredit in the cities or solar energy in the countryside. Yet you see how the text fuzzes the boundaries of Grameen’s aims (“matters connected therewith or incidental thereto”). And even if those fuzzy words are ignored, it must be recognized that strict respect for the law’s limits is impractical and inefficient. If you want to make loans to poor people in Bangladesh, you should want to keep interest rates down, which means limiting costs, which means lending on a mass scale, to all people poor enough that they are willing to accept the inconvenience of group microcredit. It means not devoting staff labor to tracking clients’ land holdings in real time. This is why the Grameen Bank has long lent to people over half an acre, why that has never bothered me, and why, if the law literally means what the commission says, it is an ass.

  Similarly, the commission questions the legality of the support that the Grameen Bank has given to other organizations with “Grameen” in their names. I think the main objection is to loan guarantees. In these arrangements, a separate organization—imagine Grameen Healthcare Services—borrows money from some other bank. To reassure the other bank as to creditworthiness, the Grameen Bank provides a guarantee: if Grameen Healthcare Services defaults, the Grameen Bank will repay, and then can try to recover its money from Grameen Healthcare Services. Now, providing guarantees is something banks do. But can Grameen do them? Giving guarantees is not in section 19′s list of permitted functions, so maybe not. However, the list ends with this clause, which the Commission does not mention:

  generally the doing of all such acts and things as may be necessary, incidental or conducive to the attainment of the object of the Bank.

  Again, the ordinance seems to give the Bank a lot of discretion. Now, the commission may well be right that some of Grameen’s activities have strayed beyond the spirit of the law. But given the flexibility of the law and social purposes of the activities—Grameen Shakti has brought solar power to a million families—-I don’t see cause for indignant finger-wagging.

• The nine elected ladies are a sham. The commission argues that they are selected more than elected, with heavy involvement of Bank staff in each step. Qazi Nazrul Huque describes it nicely in this comment. In a hierarchical process, all the Center leaders, Grameen clients who each represent about 40 peers, who are served by a given branch convene to select one from among them to represent the branch. Branch representatives do the same at the area level, then the zone, then the region level. Each of the nine regions chooses one board member. In the Commission’s telling, Bank staff play major, and perhaps heavy-handed, role in the nominations. There is no direct, secret ballot of the 5.6 million shareholders, nor even of the the 540,000 whom the Commission says hold shares legally. Meanwhile, the board members clearly are unqualified to assess the prudence of the Grameen Bank’s complex operations. The report says the “election” of board members
  

  ….was a powerful marketing tool, which resonates throughout the world. It has been used to cloak the hollowness of an elective procedure, which has placed as many as four electoral stages between the borrower-shareholder and the post of a Director of the Board. Worse, the description of this process by an ‘elected’ Director as recorded in the Bank’s Board meeting exposes the exploitation of an emotive slogan.195 There is need to make the participation of the borrower-shareholder more direct and meaningful so that the needs of the landless and poor women is more honestly addressed.

  I imagine that the tension between democracy and competence is common in cooperative financial institutions. Probably the solution is to balance the board between elected representatives and selected, highly qualified ones, and to arrange for candidates to be more qualified than most who vote for them.

• Certain dealings between Grameen Telecom (the nonprofit Grameen company), Grameenphone (the commercial mobile operator controlled by Telenor), and the government look shady. You figure that any process in which the government of Bangladesh sells a private company a potentially valuable asset—in this case, a license—involves some kickbacks. Beyond that, I have not tried to understand the details, which are complex and incomplete. Maybe someone else can explain.

So overall, I find the report interesting and serious, if with occasional flares of bias, archness, and naivete about social science (there is no credible evidence that “but for this mismanagement within Grameen Bank…more people could have been helped”). Strengthening rule of law is essential to Bangladesh’s political and economic development. So I’m glad these lawyers advocate for it. But one must consider context before precisely equating any illegality with immorality. In 2011, the report says, a Bangladeshi court found that all laws passed under the martial law regime lasting from March 24, 1982, to November 11, 1986—including the Grameen Bank Ordinance and its first round of amendments—were null and void! All these years, the Grameen Bank was illegal, you see. When then argue about whether it was exactly within the law’s limits? And under martial law just a few years ago, the current Prime Minister languished in jail for corruption.

The poverty of most Bangladeshi’s being urgent, the eagerness of outsiders to work with the Grameen Bank being great, and the dangers of dependence on the government being obvious, it is understandable that Muhammad Yunus and others at the Grameen Bank pushed the limits of their independence in searching for ways to make a difference. I think that is the right way to view the situation even if, in retrospect, we judge that they sometimes strayed too far from the law.

The spark that sets off a large-scale delinquency crisis can be anything and could come at any time – a rapid drop in economic growth, a populist political movement, a religious decree, or a collections effort gone bad. One can’t control the spark, but one can control how much fuel that spark can ignite.

Eleven months later the government of Andhra Pradesh ambushed the microcredit industry within its borders. Rozas became the Roubini of microfinance.

Now Daniel is worried again, this time about Chiapas, in Mexico, or maybe Mongolia or Cambodia:

So is Chiapas the next Andhra Pradesh? At the surface it seems that the comparison doesn’t hold. Chiapas is not the crucible of Mexico’s microfinance market in the way that Andhra Pradesh was. A crisis in Chiapas would hurt the local lenders badly, but direct damage to the large lenders operating across the country would likely be limited.

However, Chiapas also differs from Andhra Pradesh in a way that is more concerning. India’s huge states (Andhra Pradesh alone is nearly the size of Mexico) are separated by language, caste, religion, as well as local media – differences that are especially strong among the poorer, less educated social segments that comprise microcredit customers. This regionalism played an important role in containing the Andhra Pradesh crisis within the state – a pattern also seen during an earlier repayment boycott in the Indian state of Karnataka.

But in a country like Mexico, with a strong national media, a common language, and fewer social barriers, it’s not hard to imagine a payment boycott in Chiapas easily spreading to other parts of the country, as was the case with the crisis in Nicaragua.

Of course there are no certainties in this business. But, as he did last time around, Daniel brings data to the table. So you can see what you think.

Central to that story would have been the work of Sam Daley-Harris. He was drawn into activism in the late 1970s by the, well, cult-like est movement and associated Hunger Project, which had the grand goal of eliminating world hunger by 1997 (and apparently didn’t do much practical to achieve the goal). In 1980, Sam founded the more practical, and at least as disciplined, grassroots lobbying group Results, which in the early 1980s persuaded the U.S. Congress to increase funding for Oral Rehydration Therapy and the UN’s International Fund for Agricultural Development (IFAD). These were remarkable feats for a mostly-volunteer, start-up nonprofit.

As it happened, IFAD was an early funder of the Grameen project in Bangladesh. An IFAD documentary on Grameen brought microcredit to Sam’s attention in 1985. He later introduced Muhammad Yunus to the American Congress and press. As I recall, a story in the Christian Science Monitor led to a segment on 60 Minutes in 1990. Yunus was catapulted into fame in the US and beyond. Sam matched that publicity coup by organizing the Microcredit Summit in Washington, DC, in 1997, whose headliners included First Lady Hillary Clinton and Bangladesh Prime Minister Sheikh Hasina. (I think FINCA founder John Hatch first suggested the Summit.) At the event, a grand goal was announced: bringing microcredit to 100 million by 2005.

The Summit and the ensuing, permanent Microcredit Summit Campaign have been a major force behind the global microfinance movement, combining savvy publicity with behind-the-scenes lobbying for funding. Each year the MCS has released a painstakingly collected tally of microloans worldwide. After the 100 million goal for 2005 was met, the campaign issued two goals for 2015: 175 million of the poorest with microcredit and 100 million people lifted out of extreme poverty (under $1.25/day).

Two days ago I participated in a public conversation for the launch of the 2013 report. Joining me were host Larry Reed, moderator Susy Cheston, Wendy Abt, and Alexia Latortue. Speaking at the event, I admit, forced me to read the report more closely than in the past. Herewith a few thoughts.

• I enjoyed the report. It is a short, clearly written sampler of current thinking in the microfinance world, covering such topics as economic psychology and “graduation programs,” which assist the poorest with a package of services and assets that will, it is hoped, reduce their poverty and prepare them for microfinance.
• The most striking finding is that in 2011, for the first time, the number of microloans fell—and sharply. The graph is below. Most of the decline happened in Andhra Pradesh, but also some in Bangladesh (which I don’t know how to reconcile with the Mix Market’s finding of little change in 2011).



• Yet the report’s cover and title de-emphasize this arresting drop. The title, “Vulnerability,” connects to a quote in the report from India microfinance veteran Vijayalakshmi Das. Identifying with microfinance institutions (MFIs), she said, “we are as vulnerable as our clients.” That is, MFIs will be fragile as long as prey upon the vulnerabilities of the poor. They lost sight of their clients’ interests and paid the price. It’s a compelling formulation, and resonates as we try to understand what went wrong in Andhra Pradesh, and why the global total fell. But I wonder if it is the whole truth. In the U.S., payday lenders seem to be making a steady profit off a clientele whose finances are far less steady. In Mexico, so is Compartamos.

  Broadly, the interests of merchant and customer conflict on some margin and coincide on others. Many MFIs could gain by raising interest rates, and hurts clients. Yet clearly MFIs will go bankrupt if all their clients do.

  None of that is new. MFIs have always said that they were succeeding by helping their clients succeed. What is new is that in India, it came to appear that were not, in some cases. So what we need to learn from India is not that MFIs should remember their clients’ interests, but how the MFIs came to collectively work against the interests of some of their clients.

• An implication of the vulnerability theme is that some of the loans in Andhra Pradesh, and by implication elsewhere, were harmful. With microcredit, more is not always better. That cuts against the historical messaging of the MCS, with its goals and tallies and its fusing of a goal for outreach with one for poverty reduction. The general impression has been that expansion is urgent and equatable with poverty reduction.
• The text thoughtfully covers both mobile money and graduation programs. I found the discussion subtly asymmetric, cautious about high tech and more hopeful that graduation programs represents the best path for MFIs wanting to reduce the vulnerability of clients and themselves. One need not choose sides, and the report properly captures many nuances in these topics. But I’d argue that if anything the report gets things backwards. Mobile money is delivering useful financial services to tens of millions of people in a businesslike way. In my view, that is true to the spirit of microfinance. As for graduation programs, they appear in randomized trials to be helping people too. This is great. But to my limited knowledge, it’s not clear that the financial services that are part of these packages or become available after are key to this impact. So is it perhaps provincial to call them “graduation” programs, if by that is meant graduation to microfinance? Maybe giving very poor people carefully chosen goods and services is a good way to help them, regardless of whether it leads to greater microfinance use. Would we describe the U.S. food stamp program as a graduation program meant to increase financial access?

  I think microfinance has succeeded mainly by streamlining, not adding expensive extras. If adding the extras helps people and donors are willing to fund it, that’s excellent. Whether it’s microfinance, I don’t know. So while I applaud the pragmatic creativity of BRAC in developing the approach and the rigor of Ford and CGAP in evaluating it, I fear that casting it as a species of microfinance at once belittles it and exaggerates its promise for mainstream MFIs.

As usual, it is the friction of disagreement that most excites me to write. Yet there is much I like in the report and I encourage you to peruse it.

Stata has worked over the years to reward such sharing. One step was the founding of the Stata Journal (and before that the Stata Technical Bulletin) to give academics a venue to leverage their coding labors into career-boosting publications. A more recent step was the institution of an annual award for the best contributions to that journal in the previous three years. The first award was announced today. The recipient is: me.

The prize is awarded to David Roodman specifically for two outstanding papers in this journal:

• How to do xtabond2: An introduction to difference and system GMM in Stata (Roodman 2009b)
• Fitting fully observed recursive mixed-process models with cmp (Roodman 2011)

The titles alone are exciting I know! Ungated CGD versions here and here. My two papers were fortuitously timed for the period of this first award.

Born in Indianapolis, Indiana, in 1968, he grew up in Hanover, New Hampshire, and Binghamton, New York. Roodman’s formal education ended in 1990 with a Bachelor’s degree in theoretical mathematics from Harvard College. After years at the Worldwatch Institute and on a Fulbright in Vietnam, he arrived at CGD in 2002 knowing little about econometrics. He discovered that a great way to learn econometrics is to code it. His contributions to the Stata community since then were motivated by a desire to replicate and scrutinize complex, influential studies in development economics, which led him to write xtabond2, cmp, and other packages; and motivated by a pedagogic bent, which led him to document the packages and their mathematics in the Stata Journal.

I am humbled and happy about the award.

I wrote the first program as part of my appraisal of the literature on whether foreign aid causes economic growth. (See the technical Anarchy of Numbers and this non-technical guide for the perplexed.) At the encouragement of David Drukker of StataCorp, I then wrote my paper about the program.

As I blogged in 2009, the second paper documents a program I wrote in order to replicate the Pitt & Khandker study of the impact of microcredit in Bangladesh. It’s the most beautiful program I’ve written. Philosophers argue about whether mathematical ideas are discovered or invented. The concept of this program, cmp, is so elegant that I feel like it was there waiting to be discovered.

I echo the end of the award announcement:

As editors, we are indebted to…a necessarily anonymous nominator for a singularly lucid and detailed précis of Roodman’s work.

I’ve followed an unusual track at CGD, teaching myself econometrics by coding it, doing so primarily in order to be an annoyingly demanding consumer of econometrics, trying to decide which studies to believe, abrading some people along the way. I feel fortunate to have had the opportunity to grow and contribute in this way, to realize my peculiar latent potential. But especially in the early years, I also felt bashful about this strange path—real economists are trained to produce research not just be annoying consumers of it—even as I felt compelled to cut that path. The validation is appreciated.

Of course, writing cool code is not the same as improving lives, which is CGD’s reason for being. I only hope that the tools I have made, through their use in the hands of others, have in some small way advanced social science, especially as it relates to helping the world’s poor.

As you would expect when the same people write about the same things, the new paper shares much with its predecessors. It is written emphatically, seeming to raise profound concerns; I don’t find it that persuasive; and yet it has taught me something. Perhaps the most succinct rebuttal is that the paper does not refute the fact that if you drop the 16 data points with the most extreme (highest) values on household spending, less than 0.5% of the sample, the finding that microcredit increases household spending completely goes away. The new paper spends much more space challenging our hypotheses (what PK call “claims”) about why this happens than whether it happens. But for real-world implications, the whether matters more than the why.

Here, I will explain my thinking in more detail. I’ll try to keep section openers jargon-free, but no promises otherwise. If this post bewilders you, then you will know how I felt when I decided years ago to understand the debate over PK, which Jonathan initiated.

Section 1 of the new reply is an introduction. Section 2 does not deal with substance.

Section 3 criticizes an alternative to PK’s main statistical method, which we use in part of our paper. The alternative was first proposed by PK in 1998 (footnote 16) and first applied to the PK data by Pitt in 1999 in his attempt to rebut Morduch (1998). PK now describe the method as “outlandish” and “extraordinarily artificial.” To me, the theoretical critique and the demonstration through simulations both appear flawed. And even if they are correct, they mainly go just to that “why” question, not the “whether.”

PK (1998) propose estimating impacts via two-stage IV in which the instruments are interactions between included controls and each of the two dummies for female and male credit availability. Pitt (1999) implements this as 2SLS. We do classical linear LIML instead, solely in order to stay conceptually closer to PK’s nonlinear LIML. But this distinction matters little. In fact the associated under- and weak identification tests, whose availability substantially motivates use of the estimators, are identical. Both also have the virtue of being known to be robust to non-normality in the errors. PK’s theoretical attack on the method they once used to defend themselves can I think be distilled to this:

Consider the system of structural equations:

y = x1 + x2 + x3 + e
x1 = z1 + u1
x2 = z2 + u2

where e, u1, u2 are potentially correlated error terms; x3 is an exogenous control; z1, z2 are uncorrelated with e; and z1, z2 are strong explanators for x1 and x2, making them strong instruments for x1 and x2 in the y equation. 2SLS is appropriate for estimating the coefficients on x1 and x2; it would put x3, z1, and z2 in the first-stage equations for x1 and x2. However, the exogenous control x3 is a weak instrument because its expected coefficients in the first-stage equations are 0. (Its first-stage coefficients correspond to π_fx and π_mx in the PK (2012) exposition.) And z1 is a weak instrument because its expected coefficient in one of the equations, the x2 equation, is zero; and vice versa for z2. (These correspond to π_fm and π_mf.)

The above argument is wrong because a) the exogenous control x3 cannot be a “weak instrument”; b) z1 and z2 are collectively strong for x1 and x2 even if z1 is weak for x2 and z2 is weak for x1.

PK attempt to illustrate their contention that the linear estimators are fatally flawed by weak instruments. However, these and subsequent simulations deviate from the PK estimation framework in two ways that greatly and unrealistically weaken instruments in themselves. First, treatment quantity (how much is borrowed) is simulated with a zero-centered distribution rather than being always positive as in the real data. As a result, treatment averages zero for treated and untreated, and the availability of credit by gender, the key instruments in PK, are perfectly weak. (In the data and code file, in Table1&9\groups3a_liml.do, lines 22–33 define the zero-centered female and male treatment quantities.) And second, also unlike in PK, these dummies enter as controls rather than instruments! In a classical impact assessment, this is equivalent to looking at the impact of treatment while controlling for rather than instrumenting with intent-to-treat. This too should weaken the remaining instruments. (In the same .do file, note the appearance of “treatm treatf” in the second stages of the specifications in lines 71 and 83.)

I think what this first simulation set actually demonstrates is a problem PK don’t emphasize, and which I of all people should have taken more on board. The linear approach generates a lot of instruments, which causes overfitting bias, toward OLS. I think when we revise, we should add exactly-identified regressions, instrumenting only with the credit availability dummies, not their interactions with the controls. Tests on a Pitt (1999) simulated data set demonstrate the minimal bias (but inefficiency) of this method (see Appendix of our first paper). I think it was a mistake not to apply it to the real data earlier.

But filling this gap doesn’t resuscitate PK’s results. Exactly identified LIML or 2SLS regressions (they coincide), which are relatively free of overfitting bias and robust to deviations from normality, show no impact of microcredit on household consumption. See 3rd and 5th columns, which are new:

Section 4 helpfully spots a bug in our code. It then devotes almost 3 pages to its potential implications—rather than fixing it to see if it makes a difference, and rather than acknowledging, as I informed Mark Pitt this summer, that it doesn’t.

I forgot to factor in the sampling weights in the lines that compute the skew and kurtosis of the second-stage errors and test whether they deviate from normality. Fixing that strengthens our findings of non-normality: skew in errors in the replication regression rises from 0.64 to 0.71 and kurtosis from 4.78 to 5.12. (Add “[aw=weightpk]” clauses to the “sum ey, detail” and “sktest ey” lines in this.) This is to be expected since PK undersampled ineligible (less poor) households and then overweight them in their regressions to compensate. Properly overweighting them in the skew and kurtosis tests too accentuates the long right tail in the household consumption data.

All code has bugs. And most bugs once found can be made to look really dumb, especially by a good lawyer. Imagine if in one place in the complicated computer program for your study of childhood obesity, you typed an H instead of a W: you used height instead of weight. Equating height and weight, a critique could argue, goes against millennia of medical science, not to mention common sense. One can demonstrate with theory and simulations how it can produce all sorts of wrong results. And no justification is provided for this strange theoretical construct!

Seemingly, it would be easier to just fix the problem and see if it matters. Seemingly this is also the thing to do if one’s priority is getting the science right.

Section 5 confronts our finding that the PK estimation method is bimodal, tending to generate two contradictory results—microcredit increases or reduces household spending. The section argues that such bimodality is neither unusual nor a problem. The argument here seems flawed in three ways, one minor and two major.

The minor flaw is that the computer simulation code that shows the normalcy of bimodality doesn’t actually demonstrate the presence of two local maxima the way ours does (see Figure1\conloop3negt.do in the code and data file). The code loops over various possible impact coefficients for female and male credit, equating the two, each time maximizing the likelihood while constraining the estimated impacts to these values. It is a graph of constrained likelihoods over a subset of possible values for the constrained parameters. But I fixed this is and found that the two peaks in PK’s graph indeed correspond to true local maxima when the likelihood search is unconstrained. So the double-peaked graph is conceptually flawed but meaningful.

The first major problem was already mentioned. The simulations unrealistically deviate from the PK set-up in ways that weaken the instruments. One deviation is in the hypothesized data-generating process: amount borrowed averages zero. The other is in the estimator: credit availability is a control rather than instrument. When these two problems are fixed, the bimodality goes away. Compare this to PK’s double-humped Figure 1:

So instead of challenging Jonathan and me by showing that bimodality is the norm, PK’s simulations corroborate us by associating bimodality with econometric degeneracy.

The other major problem is an elision of the distinction between bimodality in the likelihood and bimodality in the estimator. It is absolutely the case, as PK say, that as a matter of theory ML does not require unimodality of the likelihood for consistency. If a particular mode is highest with probability 1 as sample size goes to infinity and the ML search always detects this mode, the estimator will be consistent. However, multimodality in the estimator is inconsistency prima facie. It’s a matter of definition, not theory. (Some narrow counterexamples for completeness: the estimator could be asymptotically bimodal such that the mass of all but one mode goes to zero in probability, or such that the modes become infinitely close. But the data do not suggest such scenarios.)

And we do present evidence of bimodality in the estimator, via bootstrapping. Using the best method we’ve found for detecting multiple modes, we found 65% of the mass of the ML estimate of the impact of Grameen lending to women to be below zero. But I recently discovered a subtle bug in that code: the right number, I now believe, is 36%:

So a one-tailed test of whether the impact is positive yields significance only at p=0.36. PK address our bootstrapping only in footnote 26, where they say it “lacks any econometric justification.” For justification, they can refer to authoritative texts.

It’s possible that the estimator is asymptotically stable, contrary to our finite-sample evidence. Perhaps 5,218 observations on 1,798 households is not large enough for asymptotic behavior to kick in. That brings us to theory. Econometric theory tells us that the PK estimator is consistent when the assumptions implied in its likelihood are correct, notably normality of the errors. It may well be robust to certain violations of these assumptions, such as the non-normality we detected, but no one has proven as much. Thus PK are correct that “RM’s concern with bias [actually, inconsistency] arising from non-normality draws no support from econometric theory.” But the opposite holds: neither does theory reassure. And I thought that in econometrics, estimators were presumed inconsistent until proven consistent. It takes chutzpah to defend an estimator by saying no one has proved it doesn’t work.

PK offer some hand-waving about why their estimator is probably robust to non-normality. The arguments are reasonable, but illustrate the danger of hand-waving, for they are wrong. These simulations demonstrate. We are correct that the classical linear estimators are strictly more robust to non-normality than PK’s, and thus provide a useful check.

Section 6 returns to the linear estimators that PK first proposed and used (and which again relate primarily to why there is instability, not whether).

There is some semantic confusion here—linear LIML is not a way of doing 2SLS, and parameter and moment covariance matrices are not the same. But the main upshot is that our first linear regression, in which there are six instrumented variables, credit by gender and lender—is underidentified. It is “the equivalent of sirens blaring and red lights flashing to proclaim that something is terribly wrong with the estimation.” Actually, our paper notes the underidentification (see the under-ID test in the table above) and does not rely on that regression for inference. We fix it by pooling credit by lender, which eliminates the underidentification, as shown above (the p values on the test plunge to 0.000).

Moreover, as that table newly shows, and contrary to PK, instrument weakness is not an irreducible source of trouble. In the exactly identified estimates, the instruments appear strong; the impact of microcredit on poverty still does not. It appears now that the instruments are not weak in the overidentified regressions, at least those that pool across lenders. Rather, instrument proliferation is distorting the test of instrument weakness. Adding instruments should increase instrument strength even if the test of strength says opposite. This finding does contradict our earlier thinking a bit, and I’ll return to it.

The section also provides an unusual interpretation of the linear estimates, combining the point estimates from LIML with the “perfectly valid” standard errors from 2SLS. In fact, the LIML and 2SLS regressions return the exact same weak instrument diagnostics, so it’s not clear why one is more valid than another. At any rate, it is an unorthodox move and a thin reed on which to rest a defense of PK.

Section 7 is interesting. It strips away components of the PK estimator to isolate the source of identification. What it still does not do, however, is use the formal language of probability to state and defend the conditions needed for identification of causal effects. PK, for example, have never motivated the assumption that variation in the availability of credit by gender is exogenous. They also have not explained why credit availability is a good instrument in a nonlinear IV set-up despite our demonstration (Table 4) that credit availability is correlated with the second-stage error.

Summary
Contrary to appearance, the new PK paper confirms rather than refutes the conjecture that bimodality in the PK estimator is a sign of weak instrumentation. The paper does not overturn the finding that dropping a handful of systematically picked outliers collapses the two modes into one near zero. It does not change the fact that linear estimators that are robust to demonstrated deviations from the likelihood model produce estimates close to zero. It does not change the fact that the PK estimator is demonstrably inconsistent in the face of such deviations. It does not address the bootstrap evidence that the estimator is inconsistent on the real data.

But we have learned from this round. Most important is the discovery of a paradox: PK now provide a laboratory demonstration of how weak instruments make their estimator bimodal, confirming one of our hypotheses; yet the friction with them led me to run exactly identified linear regressions that revealed the instruments to be strong in that context, cutting against our hypothesis.

This forces us to revise whatever tentative insight into the nonlinear PK regressions that we derive from the linear analogs. I would not now hypothesize that the PK instruments are weak in the usual sense, across the full sample. Nevertheless, as Jonathan and I noted in 2011, the PK result disappears when dropping villages where both genders can borrow and, symmetrically, persists strongly when restricting to just those villages. So the PK result seems to emanate from this subsample, in which the female and male credit availability dummies are identical, making them weak for explaining distinctive variation in the endogenous variables, credit uptake by gender. It seems as if instruments being weak only within a subsample, while irrelevant for linear estimation, can distort a nonlinear one—at least when there are outliers. This is why I continue to conjecture that the outliers and instrument weakness are interacting: fix either and the bimodality goes away.

Perhaps someone else can formulate a sharper explanation for why the PK estimator is unstable; our conclusion that it is unstable remains regardless.