Gender equality programme (GEP) evaluations: What are they? Why are they needed? How are they performed? How are multi-partner evaluations different?

Prof Mustafa Ozbilgin, Brunel University London
Dr Nur Gundogdu, Brunel University London

The evaluations of a gender equality program (GEP) typically involve a comprehensive analysis of the program’s goals, objectives, activities, and outcomes. Evaluations involve several stages: First, evaluators must plan and design the evaluation. This involves setting objectives, selecting appropriate methods and indicators, and developing a monitoring and evaluation plan. Second, a baseline assessment is typically conducted before the program begins. This assessment provides a benchmark against which program outcomes can be measured. This phase is often called an ex-ante evaluation. Third, there is a process evaluation. This evaluation assesses the program’s implementation, including the extent to which program activities are being carried out as planned. This phase could also be called interim or in-process evaluation. Fourth, there could be an ex-post impact evaluation. An impact evaluation assesses the program’s effectiveness in achieving its goals and objectives. It measures the extent to which program outcomes have been achieved and any unintended consequences of the program.

Evaluations of GEPs are essential for assessing the impact and effectiveness of gender equality plans of organisations: Evaluations help reveal if gender equality programmes are achieving their intended goals. They provide a framework for measuring progress, identifying areas of success, and determining areas that need improvement. Evaluations also help organisations allocate resources effectively. By evaluating programs, it becomes possible to hold stakeholders and implementers accountable for their actions. There is a significant learning dimension to evaluations. Evaluations provide learning opportunities through which partners and stakeholders can learn from successes and failures and use that knowledge to improve future programs.

Different methodologies and mixed methods can be used to evaluate gender equality programs (GEPs). The riches of the GEP evaluation methodologies have been their multi-faceted, multi-layered and multilevel expansion from international organisations to the organisations of science. GEP evaluation methodologies have been developed interrelatedly across macro, meso and micro levels (Ozbilgin and Gundogdu, 2020). International and supranational organisations such as the UN with  Sustainable Development Goals (Sach et al.,  2020), OECD with its report on gender and evaluation methods (OECD 2003), The World Economic Forum (World Economic Forum’s Gender Gap Index, 2020), European Institute for Gender Equality (European Institute for Gender Equality’s Index, 2020) and UNDP (UNDP Gender Inequality Index,2020) have generated considerable insight into how GEP could be evaluated from a comparative and international perspective, using both qualitative measures and quantitative metrics and with multiple dimensions. There are also national-level GEP evaluation methodologies that are developed. One national-level initiative in the UK is the Athena SWAN project, which has a gender equality charter for UK universities (Athena Swan 2020). At the organisational level, methodologies are developed to cater for specific institutional drivers for GEP evaluation.  Qualitative methods involve collecting and analyzing data through observations, interviews, focus group discussions, and case studies. These methods can provide detailed insights into the experiences of program participants, their attitudes and perceptions, and the factors that influence program outcomes. Quantitative methods involve collecting and analyzing numerical data through surveys, questionnaires, and statistical analysis. These methods can provide objective and standardized data that can be used to measure program outcomes and assess the program’s impact. A mixed-methods approach combines qualitative and quantitative methods to provide a comprehensive evaluation of the program. This approach can provide both detailed insights into program experiences and attitudes, as well as objective and standardized data to measure program outcomes. Outcome mapping focuses on identifying and assessing the changes that occur among the stakeholders of a program. It involves developing an “outcome map” that outlines the changes that the program seeks to achieve and tracking progress towards these changes. Combining different methods can help to provide a more comprehensive evaluation of the program.

In a multi-partner and diverse stakeholder setting across different institutional and national settings, GEP evaluations must attend to cross-institutional and national variations in regulatory, social, cultural, financial, political and historical considerations (Ovseiko et al., 2017; Ovseiko et al., 2020). As a result, in a multi-stakeholder setting, GEP evaluations cannot be easily conducted in a standardised way. It needs to be custom-made to recognise differences in priorities, plans, programmes, and processes and institutional arrangements. Such tailored design of GEP evaluations complicates cross-institutional and comparative evaluations. Although true comparisons are often difficult in GEP evaluations, GEP evaluations offer great potential for cross-institutional learning and transposing best practices across institutions.

References:

Athena Swan (2020) Athena Swan Charter and Principles, https://www.advance-he.ac.uk/equality-charters/athena-swan-charter#principles

European Institute for Gender Equality ( 2020). Gender Equality Index 2020. https://eige.europa.eu/gender-equality-index/2020

OECD (2003). Review on Gender and Evaluation https://www.oecd.org/dac/evaluation/dcdndep/31736413.pdf

Ovseiko, P. V., Chapple, A., Edmunds, L. D., & Ziebland, S. (2017). Advancing gender equality through the Athena SWAN Charter for Women in Science: an exploratory study of women’s and men’s perceptions. Health research policy and systems, 15(12), 1-13.

Ovseiko, P. V., Taylor, M., Gilligan, R. E., Birks, J., Elhussein, L., Rogers, M., … & Buchan, A. M. (2020). Effect of Athena SWAN funding incentives on women’s research leadership. bmj, 371.

Ozbilgin, M., Gundogdu, N. (2020). TARGETED-MPI Evaluation Plan. EU Horizon 2020 Project, Transparent And Resilient Gender Equality Through Integrated Monitoring Planning and Implementation (Grant Agreement No: 872260). https://targeted-mpi.eu/wp-content/uploads/2022/03/TARGETED-MPI_WP5-D5.1_TARGETED-MPI_Evaluation_Plan.pdf

Sachs, J., Schmidt-Traub, G., Kroll, C., Lafortune, G., Fuller, G., & Woelm, F. (2020). The Sustainable Development Goals and COVID-19. Sustainable Development Report, 2020. https://s3.amazonaws.com/sustainabledevelopment.report/2020/2020_sustainable_development_report.pdf

UNDP (2020). Gender Inequality Index http://hdr.undp.org/en/content/table-5-gender-inequality-index-gii

World Economic Forum. ( 2020). Global Gender Gap Report. http://www3.weforum.org/docs/WEF_GGGR_2020.pdf