Equity Makes Evaluation More Effective

Values matter, and detachment does not equal rigour when trying to figure out 'what works.'

If there was a society where equity was the driving goal citizens and the state prioritized it above other goals, perhaps every evaluation could increase equity. We could simply identify the 'best' programs, where the 'best' was defined as reducing inequities. Armed with that knowledge, implementers could design more of the 'best programs' and equity would be achieved.

But these are not our societies.

We live in societies characterized by models of development that perpetuate inequality; where equity is often knowingly sacrificed for other goals. For example, despite robust growth, India has seen large increases in income inequality since 1990. The Gini coefficient is a way to measure this. A score of zero means you have perfect equality and wealth is shared equally among all citizens. A score of 100 means you have perfect inequality - or all the wealth is the hands of just one person. A recent IMF report shows the Gini coefficient of India rose to 51 by 2013, from 45 in 1990. India is now one of the most unequal countries in Asia.

This growing inequality between rich and poor is compounded by inequities by gender, social group, and location. Inequities shape all aspects of life.

For example, if we take health, while on average health has improved, these average improvements disguise gaps by caste, gender, class, religion and geography. Take three examples using readily available government data:

A woman in Uttar Pradesh is almost five times more likely to die during childbirth than a woman in Kerala

A child in Bihar is twice as likely to die before turning five than a child in Tamil Nadu

A child from a scheduled caste is twice as likely to be severely stunted and underweight, than a child who is not

Does this surprise or shock you? That the average reader may not be particularly surprised or dismayed by the above says something about the way we expect inequities. Inequities are taken as given - a necessary if unfortunate part of development. The inevitability of inequity in our current development pathway is largely unchallenged, and thus, reinforced.

So how can evaluation increase equity here? Can evaluations of interventions in systems that are expected to generate inequities lead to equity? Quite the opposite. Evaluation here can reinforce inequities by only looking at average improvements or otherwise underplaying growing inequity.

What differentiates evaluation from research is that evaluation attempts to make a judgement on something - to use evidence to determine whether something is good, or has value. While everyone makes judgement on policies and programs all of the time (and should), evaluators are meant to do this with the tools of social science. Given that the fundamental purpose of evaluation is valuing - part of the role of the evaluator is to ask 'whose values,' and 'what is being valued?' and to bring equity, into the equation if it is missing.

Evaluation is often presented as being external to the systems it evaluates, with evaluators as 'detached, unbiased, observers.' But the same biases or values that underpin interventions are often implicitly replicated in evaluations. Evaluation here, can become a technocratic exercise of propping up the status quo. For example, if you expect development to lead to growing inequities - why would you question this?

Evaluation here needs a fundamental reorientation from external observation to being an internal disrupter. A state or society that wants change should demand this. Young people looking for careers that can make a difference, should look to evaluation and evaluation research as one way of affecting meaningful change.

What might this look like?

Another example from health. In 2012, in partnership with the Bihar government, a team of evaluators collected data from over 13,000 women who had a child in the previous year. The data showed gaps in maternal services for all women. But further analysis revealed large differences among women as shown on the chart. For example, while 72% of women on average were not getting adequate antenatal care, only 16% of the most marginalized women were receiving at least 3 or more antenatal checkups compare to 54% of the less marginalized. Two years later a series, through a series of efforts, public health services for marginalized women improved as did life-saving behaviors. While large gaps remain, a focus on inequities in how the program was evaluated encouraged a focus on inequities in programming.

Inequalities at baseline among women in Bihar who delivered a child the previous year



This may sound like just doing good evaluation. It is. But it doesn't just happen. We all make choices about what to value all the time. We all need to ask, not only "if things are working," but, also working 'for whom?' If we don't, we're simply reinforcing existing biases by not illuminating the values (and people) being left behind.