A similar event was held at Cumberland Lodge in May 2007 which resulted in the first three phases of the CSC Evaluation Programme. As the CSC moves into a new stage of activity, the aim of the 2012 event was to strengthen its relationships with similar programmes and funding bodies, and also to generate mutually useful discussions on the best way to tackle impact evaluation in scholarship programmes.

The core objectives of the seminar were to:

• share knowledge and experience relating to existing impact evaluations of scholarship and fellowship programmes
• discuss issues arising from the challenges of evaluating scholarship programmes and similar capacity building programmes, and identify potential benchmark indicators for future comparisons
• learn more about the various evaluation methodologies that can be used by such programmes and identify those which are the most appropriate
• explore the possibility of establishing a forum of like-minded professionals that allows for future collaboration between different organisations, so that best practice and experiences can continue to be shared

Programme

Participant list

Many participants also attended a complementary event, Measuring Impact of Higher Education for Development – organised by the London International Development Centre (LIDC), in partnership with the Association of Commonwealth Universities (ACU), and held on 19-20 March 2012 at Birkbeck College, University of London. The conference looked at the broader picture of higher education and development impact.

For further information, please contact evaluation@cscuk.org.uk

Guests included current Commonwealth Scholars and Fellows, members of the CSC’s panel of Academic Advisers, supervisors from UK universities, and representatives from Professional Fellowship host organisations in the field. Dame Sally Davies, Director General for Research and Development at the Department of Health, also attended.

Photos of the event are available on Flickr.

We asked about:

• location, field, type and level of employment
• the level of skill and experience gained attributed to the award
• the extent to which alumni are able to use these skills
• the level of contact maintained with organisations in the United Kingdom
• the extent to which alumni have been involved with projects, policymaking and wider socioeconomic impact

The main results of this survey were also published in a report that was launched and distributed at the 2008 Welcome Day for new Commonwealth Scholars and Fellows.

Entitled ‘Evaluating the impact of international scholarships’, the workshop was attended by representatives of the World Bank Institute, the German Academic Exchange Service (DAAD), the Netherlands Organisation for International Cooperation in Higher Education (NUFFIC).

Other attendees included representatives from Commonwealth Scholarships national nominating agencies in Canada, Ghana, Malawi, the Maldives, Malta, Mauritius, New Zealand, the UK and Zambia, as well as current Commonwealth Scholars and Fellows and other interested parties.

With such a variety of delegates, the event offered a forum for discussion and an opportunity to compare approaches to evaluation. Methodologies for evaluating capacity building for international development are well established, but it seems that reliable data for the impact of scholarships, in particular, are not so well documented.

Those involved in discussions were keen to establish a better view of the impact of scholarships and, where possible, to collaborate and share the data generated. Guidelines were drawn up to determine what information to collect, to design a framework for acquiring this information, and to develop best practice in evaluating the impact of scholarship funding in capacity building.

Speakers at the workshop included:

• Professor Trudy Harpham – Chair, Commonwealth Scholarship Commission in the UK: Scholarships and Capacity Building: An Overview
• Dr Andrew Sumner and Kattie Lussier – Institute of Development Studies, UK
• Dr Norm Geddes – Commonwealth Scholarship Commissioner: Defining the Purpose of Scholarships: How clear are our objectives?
• Dr Charles Tustin – Chair, Scholarships Committee of the New Zealand Vice-Chancellors’ Committee: Academic Objectives
• Ad Boeren – Senior Policy Officer, Netherlands Organisation for International Cooperation in Higher Education (NUFFIC): Development Objectives
• William Archer – Director, International Graduate Insight Group, UK: Quantifying the Benefits: Lessons from Previous Studies
• Kamel Braham – Scholarships Administrator, The World Bank Institute: Using the Alumni: Experiences of Past Alumni Studies
• Antje Schlamm – Director, London Office of the German Academic Exchange Service (DAAD)
• Professor Tim Unwin – Commonwealth Scholarship Commissioner: What are our Objectives?
• Dr Norm Geddes, Commonwealth Scholarship Commissioner: Taking Evaluation Further: Summary of key indicators and evaluation methodologies
• Dr John Kirkland, Executive Secretary, Commonwealth Scholarship Commission in the UK: The Way Forward: Proposals for Future Collaboration

Commonwealth Scholars and Fellows’ views

Melissa Felician (Commonwealth Professional Fellow, Institute of Policy Research in Engineering, Science and Technology):

Having experience in evaluation, including a recent ‘Evaluating Research’ module at my host institution, methodological issues were not new to me, but it was good to see attempts to integrate all relevant aspects. Points relating to quantifying benefits – which I initially saw as a major challenge – were well presented and provided food for thought.

The session on the recipient’s viewpoint was the most interesting part; I came away from that session acknowledging that the impact of the scholarships on the recipient (country and individual) goes beyond the measurable outcomes of the scholarship programme.

While much still needs to be done in building capacity at country level to improve the selection process, it was clear that the CSC is changing many lives by the opportunities that it has created over the years!

Imran Majid (Commonwealth Scholar, MPhil in Engineering, University of Surrey):

Highlights of the event for me included a cost-benefit analysis of the scholarship business and a review of academic and development approaches to scholarship evaluation. The World Bank Institute’s tracer study programme and the German Academic Exchange Service (DAAD)’s success in coordinating alumni activities inspired brainstorming sessions.

My contribution included ideas for prioritising regional cultures in impact assessment, while on the other hand applying the onion-shell concept (from a presentation at the UN Space Generation Forum, 1999) in scholarship evaluation; the individuals (inner core) through skill and experience affect institutions (middle layer) directly and through actions benefit the society (outer shell) indirectly.

Delegates at the evaluation workshop