Squeezing development research juice out of the Millennium Village Project evaluation

Thirteen years after the project started in 2005, the Millennium Village Project (MVP) founded by Jeffrey Sachs, has reached ‘peak evaluation’ with an article by Shira Mitchell and co-authors, ‘personal perspective’ by Jeffrey Sachs and another comment on the ‘fog of development’ by Eran Bendavid in the recent issue of The Lancet (all open access).
Some commentators were quick to point out that the final-final word may be left to yet another evaluation with a DfID-IDS-3ie-connection, but right now, the discussion about the findings is in full swing.


As Mitchell et al. point out the biggest positive impact has been on health-related goals:
The achievements of the MVP in health suggest support for the project’s emphasis on strengthening the continuum of care from households, to primary care facilities, and to tertiary care facilities. In particular, we believe that the project’s cadres of paid, professionalized community health workers, empowered with smartphones to aid in service delivery and real-time disease monitoring, contributed to the positive results. The project was also an early adopter of interventions and technologies that have since been implemented by development organisations and governments, in part because of the MVP’s demonstration and advocacy.
At this stage I am more curious about how the MVP and the ways it has been implemented, monitored and evaluated over more than a decade remains an interesting case study about the power of development paradigms and how ‘discourses’ are formed and maintained.

Heroic efforts to squeeze out development research juice
The parameters around this latest evaluation read like who-is-who of the development knowledge industry: The Lancet, Open Society Foundation, Harvard, Princeton, Stanford, Columbia, Jeffrey Sachs, DfID, IDS, 3ie etc. etc. and I am sure economists from the World Bank and various RCT and evaluation experts from the global North will comment on the data and findings as well.
For that reason, the analysis by Mitchell and colleagues represents the culmination of heroic efforts to identify the treatment effect of the MVP against stacked methodological odds
Eran Bendavid congratulates the team in a very 21st century (‘data is the new oil’)-way: By pointing out that large-scale development efforts are essentially methodological challenges these days and that today’s development ‘heroes’ use statistical tools to capture impact.
The authors deserve to be congratulated for squeezing as much juice as possible from this academic third rail, and for transparently and courageously telling the world: this was it, this is the best we can say about the MVP.
The ownership of this evaluation endeavor quickly moves to the authors of the study-not the people how have been living with the MVP for more than a decade. I would have chose different wording when talking about the real lives of half a million people.

500,000 people as data points?
While I do not want to deny that communities improved through the project, much of the discussion is focused on top-down planning and traditional economic indicators.
13 years after the project started advocates for bottom-up participatory development or alternative approaches and measurements of development will be disappointed, but the 500,000 people covered by the MVP only appear as ‘data points’, not as agents or citizens.

At the same time, the ‘power-knowledge’ discourse is still firmly based within the global North, its elite universities, bilateral agencies and philanthropic organizations. Given the resources, time-span and group of experts involved, the study remains surprisingly un-reflexive about how ‘we’ learned and changed-not just ‘them’. I am sure that there will be books, qualitative research articles and more in the future, but as per usual they will mainly be ex-post reflections, not adaptive, agile inputs into an on-going situation. 

It is not surprising that Jeffrey Sachs’ ‘personal perspective’ is actually an impersonal, general policy statement-perhaps the opposite of ‘personal’. There is no ‘I’, no personal reflection on the process, people, locations etc., but a lot of development policy-jargon:
The lessons from the MVP are highly pertinent. Multisector planning, budgeting, implementation, monitoring, and evaluation are feasible and necessary. Information platforms can be created for multisector plans and programmes. Computer technologies, including artificial intelligence and big data (responsibly managed), offer new cutting-edge solutions.
As academia is increasingly discussing gender, feminism and ‘decolonization’ questions this study seems to be a reminder of how marginalized these debates really are.
If the MVP process is deemed as a near ‘perfect’ example of how to engage with large-scale development projects, indigenous, action research-based or otherwise non-mainstream approaches to planning, monitoring or evaluation still have a long way to go-and it is already 2018…

Given the current debates in international development research the pendulum seems to swing towards bigger, longer, more sophisticated RCTs rather than thinking about transformative alternatives.

Legitimizing development knowledge and the habitus of proper research
One of the political discussions that could follow from these insights (and unlikely will in the current political climate in the UK or USA) is that one of the key findings of MVP is that a steady investment in infrastructure and people will improve many people’s lives-so essentially the opposite of austerity and simple neoliberal ‘let the market figure out healthcare and education’ mantras.

There is no doubt that the MVP debate will find its way into textbooks, course syllabi and many more academic research and writing. I am also pretty sure there will be a celebratory conference in 2025 to mark the 20th anniversary of the project.
But 13 years after the project started we need to continue those debates, even if they will remain marginalized, on how to communicate development differently, with communities and real people, on smaller scales rather than simply relying on more technology, better tools and bigger RCTs.

The Lancet papers are a powerful reminder of how development knowledge is produced and legitimized and how the habitus of conducting research seems to have changed very little during the 13 years of one of global development’s most discussed projects in the 21st century.

Comments

Post a Comment

Popular posts from this blog

Should I consider a PhD in International Development Studies?

After the Economist’s piece on the (non-)value of doing a PhD and some comments later (e.g. Prometheus doesn't read the Economist (I like the slightly cynical dichotomy between ‘civilians’ and the ‘academic insiders) or the ' 100 Reasons NOT to Go to Graduate School ' (they are only halfway through so check it out regularly in the future...) I had an interesting conversation with a prospective PhD student a few days ago. This was not the first time that I had been approached about doing a PhD and I always try to be as frank as possible, even playing the ‘devil’s advocate’ when it comes to the complicated ‘should I or shouldn’t I?’ decision-making process. At the end of our conversation I sat down and tried to summarize a few important and generic points from the point of view of doing a PhD in Development Studies and in the UK . I understand that every case is different and involves a range of motives, options and rationales, but there a few important questions and t
Read more

John Marsh: Why Education Is Not an Economic Panacea-insights for development?

John Marsh just published an interesting article in the Chronicle of Higher Education on ' Why Education is not an Economic Panacea '. The article is long and a bit confusing to read. It is based on the author's experience in an education project for poor members of the community: The idea was simple. Faculty from the University of Illinois would offer night classes in their areas of expertise [...] for anyone in the community who was between the ages of 18 and 45 and lived at 150 percent of the poverty level of income or lower. [...] Students who completed the nine-month course would receive six hours of college credit, which they could then transfer to other institutions of higher learning. Everything would be free: tuition, books, even child care at a nearby community center. We named it after a similar program in Chicago, the Odyssey Project. In short, the programme was based on the well-known dicussion that  education pays, and pays more than ever. If so, it
Read more

What if MrBeast really is one of the futures of philanthropy-and what does that mean for communicating development?

Image
“ I feel good seeing myself dancing after receiving the money. It makes me appreciate the moment my life changed and reminds me of when I was able to send my son to school .” (Resident of Nakapiripirit, Uganda) “ It might seem strange to carp over a few good deeds filmed for YouTube clicks. After all, even if this Gen Z mode of philanthropy is brash and narcissistic, some lives are being changed for the better. Yet at the same time, Donaldson is exploiting less fortunate people’s problems to shape his image of decency, while reinforcing the aid illusion that has blown so much money and achieved so little over the past half century .” (Ian Birrell, UnHerd ) “ As people have noted, if you watch the video in total isolation, it's maybe not that bad? Mr Beast is good at this. And he's not obviously mocking or manipulating these people. Then again, the TV commentators in the Hunger Games fawned over Katniss. That wasn't the issue .” (Justin Sandefur, Center for Global Developmen
Read more

Dear white middle class British women: Please don't send used bras (or anything, really) to Africa

Image
I am supposed to be on my summer blogging break , but the world of bad development does not seem to take a vacation... The first time I came across the #SWEDOW (Stuff We Don't Want) debate about what to do with unwanted stuff and whether to send it to 'Africa' was in 2010; in fact, I wrote about a shoe donation project on the blog in 2012. Fast forward to July 2019 and a local British Women's Institute Facebook page with about 500 page likes is posting a picture of two innocent-looking parcels that has been shared very widely. The vast majority of the 500+ comments under the post are positive and I am sure that both charities will receive quite a few parcels in the next few days. It seems almost futile and pointing out bad examples of charity and development is not always the best way to communicate, but I anonymize the post as much as possible to avoid simply shaming a particular Women's Institute chapter. 'Rapey men here are the same as rapey men t
Read more

Handbook on Humanitarianism and Inequality - Chapter 04 - Humanitarianism and the new wars: humanitarianism, security, and securitisation

Image
Every two weeks I am going to feature one of the chapters of our Handbook on Humanitarianism and Inequality which was published in spring 2024. This week we are taking a closer look at Chapter 04 - Humanitarianism and the new wars: humanitarianism, security, and securitisation - contributed by Michael Magcamit and Anastassiya Mahon. From the introduction This chapter demonstrates and probes the idea of ‘humanitarian securitisation’ propelled by humanitarianism amidst the emergence of new wars linked to inequality. Humanitarian securitisation is defined here as the process of securitising the humanitarian crises brought about by the new wars – specifically, internal conflicts and terrorism – to justify the use and implementation of humanitarian intervention in the target states. As the foremost authority on ‘new wars’, Mary Kaldor ( 2016: 147 ), describes these phenomena as the ‘combination of political violence, organised crime and massive violations of human rights’ happening in di
Read more