Professionals in development service can play an important role

Dr Martin Bruder has been head of the “Civil Society and Human Rights” department at the Deutsches Evaluierungsinstitut der Entwicklungszusammen-arbeit (DEval, German Institute for Development Evaluation) since 2015. He is a social scientist with many years of experience in academia and evaluation and, in addition to his work at DEval, acts as chairman of the programme advisory board at aid organization CARE Deutschland e.V. DEval was founded in 2012 and is funded by the German Federal Ministry for Economic Cooperation and Development (BMZ). DEval employs approximately 120 people and is located in Bonn. Managing director of AGdD Dr Gabi Waibel spoke with Dr Bruder about DEval’s role and how it operates; the potential of development service; and the results of the latest DEval Opinion Monitor.

Dr Bruder, the BMZ has entrusted a great deal of its evaluation tasks to DEval. Could you provide us with a quick overview of DEval’s role?

DEval is the leading institute in Germany for analysing and evaluating German development cooperation activities. We are also a research institution that scientifically develops and adapts evaluation methods and approaches. In doing so, we work closely together with both governmental and non-governmental development cooperation organisations which evaluate their own projects themselves. 

Our work is primarily carried out on the policy or strategy level – we do not evaluate individual projects. We evaluate the development work being carried out on a national level – for example in Afghanistan –, topics such as climate or sexual and reproductive health, programmes such as “weltwärts”, or instruments such as the secondment of development workers.

We also carry out meta-evaluations. This means that we investigate the quality of project evaluations carried out by development organisations and provide suggestions for improving them. We also carry out the Opinion Monitor for Development Policy, which tracks the attitudes of the German population regarding development policy.

Our scope of activity may be broad, but our overarching goal is always the same: we want to contribute to effective, sustainable, economically sound development policy.

BMZ refers to DEval as “political innovation”. What is meant by this?

It represents innovation in the sense that the BMZ has decided to surrender important evaluation tasks to an independent scientific institution – thus relinquishing control. This means that DEval has the final say when it comes to what it will evaluate and how. At the same time, evaluation is always aimed at bringing about change – i.e., beginning a process that usually requires a certain level of engagement and exchange. This is why we collaborate closely with the BMZ. Each DEval evaluation is followed closely by a reference group that consists of representatives from the stakeholders in the process and the BMZ.

The BMZ also publishes an official response to our recommendations and makes an implementation plan for putting them into place. After two years we look into how successful this implementation plan was.

The DEval’s evaluation programme is an in-house operation. There is also a DEval Advisory Board. Who sits on this board and what role does it play?

As I mentioned, we carry out evaluations independently; we are not given specific evaluation tasks from the BMZ. At the same time, we want to ensure that our work is relevant and that we are not only evaluating what we personally believe to be of interest. To this end, we work closely with various parties, and with our Advisory Board in particular. The board is comprised of approximately 20 members from the BMZ, civil society, academia, developmental cooperation organisations, and politicians from all parliamentary groups. These members can all submit suggestions for evaluation. These suggestions are then submitted for inclusion in our evaluation programme.

This process is very important for our independence. We frequently carry our evaluations that the BMZ would not themselves have prioritised.

What makes the evaluation process at DEval unique and what questions do you aim to answer in your evaluations?

Our questions are based on the Organisation for Economic Co-operation and Development (OECD)’s quality standards, which we actively helped to further develop. We aim to answer questions of how relevant, cost-efficient, sustainable, coherent and effective German development cooperation is.

We are continually looking to improve and refine our methodology. For example, with regard to the responsible use of geodata or AI that can be used to evaluate huge amounts of qualitative data more quickly. Or with regard to how we can improve our partnerships or more consistently integrate factors such as human rights and gender equity into our evaluations. Of course, the specific methodology that we use depends on the topic at hand. Let’s take the evaluation of the German government’s “Action Plan for the Inclusion of People with Disabilities” as an example. In this case, it was very important to speak to the target groups and organisations representing people with disabilities. Here we deliberately allocated resources to the evaluation that enabled us to carry out focus group discussions with hundreds of people on-site. A profit-oriented consulting company probably would not have been able to justify the costs this entailed. However, this kind of work was necessary for gauging and understanding the complex, specific challenges facing people with disabilities.

DEval also evaluates the instruments used to achieve project or programme goals. How important is this aspect in your work?

This is a very important aspect of my work. However, it comes with one fundamental challenge. Our work is retrospective – we can look into what happened, or what instrument was used, and can then evaluate the efficiency and effectiveness of this instrument. However, we cannot use the data to prove whether another instrument may have been better suited to achieving the goals of the project or programme. In order to be able to make recommendations in favour of certain instruments, it would be ideal if numerous instruments were first tested in pilot projects before the project or programme is set in motion. This would provide us with the data we need to compare their effectiveness. Without this comparison it is very difficult to argue in favour or against the use of a certain instrument in a specific context.

Two of DEval’s evaluations deal specifically with development service: “Secondment of Development Workers” (2015) and “Structural Reform of Technical Cooperation” (2016). Do these prove the effectiveness of deploying development workers to other countries?

The Secondment of Development Workers evaluation was our first strategic evaluation of the instrument for the secondment of development workers. Here, we primarily looked at the impact and mechanisms of German development service as it had been practised for more than 50 years and investigated the instruments that were now being called into question as a result of the structural reform that was to combine three key development organisations into the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ). The evaluation showed that the deployment of development workers continues to be an effective and appropriate instrument and that building up shared experience between development workers and local organisations remains an extremely relevant and sensible mechanism. The Structural Reform of Technical Cooperation then built upon these findings to fuel structural reform.

This year, we are planning to begin an evaluation of the Civil Peace Service. The last evaluation was carried out in 2011 – when DEval had not yet been founded. We are currently considering what questions we want to ask and how we wish to design the evaluation.

AGdD is working on a new program for international development service that focuses on climate change and protecting the environment. In doing so, we are taking inspiration from the ZFD, i.e., we have a common topical focus and strategy. For this reason, we are very interested in what DEval has to say on the effectiveness of certain measures for adapting to climate change. The DEval synthesis report evaluating interventions for climate change adaptation (2023) recommends “more consistently demonstrat[ing] principles such as partnerships for development and target group orientation”. Can you explain and expand upon this?

The recommendation you are referring to is our recommendation to German development organisations to “create innovation spaces for transformative adaptation interventions and provide financing”. This is something that should happen in a partner-like manner that is target group oriented. To put it in simpler terms, it is a call to try out new approaches and experiment with the effectiveness of different instruments.

“Transformative climate policy” is something the German government is calling for in many areas. However, we have yet to recognise any kind of fundamental change. Nor has the concept of partnership in governmental development cooperation changed considerably in recent years.

The strategies that have been applied to date are insufficient when it comes to climate policy on an international scale. Real transformation requires novel approaches and innovation based on research and development. It also requires rethinking how we deal with those supporting this transformation.

Partnership and target group orientation are key competences of development service and the ZFD. Do you see potential here?

Like I said, we need a diverse range of approaches and will have to experiment quickly. The potential development service has is well documented and has been positively evaluated. This potential is worth investing in – and when doing so we should look and learn where it is particularly effective and where it is not. When it comes to collaborating with partners on the ground and implementing target group-oriented measures, I think the advantages of this are obvious. The direct working relationships and the development cooperation/ZFD partnership model creates optimal conditions for this. However, any kind of climate change-based development service program should also include climate justice as an important part of its strategy.

How relevant do you consider climate change?

Even when the political discussion currently seems to be focussed on other areas, climate change is an issue that is not going away. We are already seeing the effects and the topic is extremely relevant in many areas.

Transformation policy must be considered not only on the German level, but on an international level. One key example of this is the United Nations’ 2030 Agenda for Sustainable Development, which bears the guiding principle “transforming our world”. The EU is currently developing into a driving force of a post-2030 agenda and is highlighting partnership and target group orientation as important cornerstones of any approach.

Our evaluation work is increasingly showing the importance of locally led development and approaches that take gender equity and human rights into account.

This year you will be publishing the fifth Opinion Monitor. What are the main focuses and who was involved in the study?

The Opinion Monitor is a survey of a sample of the German population that is as representative as possible in terms of participants’ age, gender, education and federal state in order to record their basic attitudes towards development cooperation. This year we focused on two topics: “development cooperation in the context of security challenges” – particularly against the background of the Russian war of aggression on Ukraine – and “feminist development policy”.

Can you summarise the most important results of the study?

Since early 2022, we have been observing a considerable decrease in public agreement with development cooperation spending – across the entire political spectrum. The number of people who thought that the amount of development cooperation spending should stay the same or be increased fell from 68% to 47%. A 21% decrease in two years is a substantial decline.

Relevant factors here were the tensions surrounding the federal government budget and the poor economic situation. However, a decrease on this scale is new – we did not record such a sway in opinion in other crises such as the 2015/2016 refugee crisis or the Covid-19 pandemic. The general feeling that we have the ability to make an impact when it comes to development policy is also in decline. Overall, this represents a challenge for governmental and non-governmental development cooperation. The global political landscape and Germany’s personal geopolitical interests also played a role in the countries that interviewees were in favour of supporting. For example, the interviewees were in favour of supporting particularly poor countries but were less in favour of providing support for countries with strong ties to Russia or China. The latter sentiment was particularly strong this year compared to previous years.

With regard to feminist development policy, we found that the interviewees were fundamentally in favour of specific matters such as gender equity. However, the terminology used triggered a negative response among some of those interviewed – especially those who identified politically as conservative or right-wing. It polarised the respondents. On one hand, this can be seen as an opportunity to inspire broader public discussion on developmental policy. On the other hand, this may also represent a danger to the generally widespread approval that development cooperation used to meet with in Germany in the past.

Government austerity measures and the competition for increasingly scarce financial resources means that everyone is feeling the pinch. And this is without mentioning the frequently negative press and critical questions surrounding individual projects and German development cooperation in general. What can evaluation do to counteract some of this negativity?

The Opinion Monitor aims to provide a visible overview of the population’s general perceptions and mood. This is helpful as a basis for well-informed discussion. At their heart, our evaluations all ask the same questions: what is going well, and what is going less well? In doing so, we provide a foundation for critically examining development cooperation.

At the same time, we perceive criticism as an opportunity for improvement. We need to have these discussions about what development cooperation is capable of, why we need it, and how we can best put it into practice. There are many good arguments in favour of global collaboration. And we have learned a great deal through our work. We know where development cooperation is particularly strong and we know what could be done better. As a taxpayer I think it would be wonderful if other policy areas were monitored and critically reflected upon to the same extent as development policy.

Do you have any recommendations for development cooperation organisations when it comes to PR work?

Of course we are not, nor do we pretend to be, communication experts. One thing I will say is that I believe it is important to be open to criticism and to see it as an opportunity for improvement. You should communicate good examples of how effective development cooperation can be – and be transparent as to how you are monitoring it. As evaluators we are always grateful for dates, numbers and facts – but many people respond best to first-hand reports from trustworthy individuals who can report on specific projects based on their own experience. This is where professionals who have spent time abroad in development service can play an important role.

Dr Bruder, thank you for your time.