Measuring Change in Education Systems: A Review

Published: 24 Nov 2021

Audience:: Academics; Education

Topics:: Te Ihuwaka | Education Evaluation Centre

Summary

This work explores global best practice for measuring education system change. It considers the variation found between different approaches to measuring education system change, looking particularly at how select individual countries and multilaterals each approach this. This work will also consider what can support countries to monitor education performance.

Whole article:

Measuring Change in Education Systems: A Review

List of abbreviations

CMEC: Council of Ministers of Education, Canada

DHS: Demographic and Health Surveys

DoSEL: Department of School Education and Literacy

DWMS: Development World Management Survey

EC: Children Under Five

ERO: Education Review Office

FCF: Child Functioning

FINEEC: Finnish Education Evaluation Centre

FL: Foundational Learning

GECDD: Global Early Child Development Database

GEM: Global Education Monitoring

GEPD: Global Education Policy Dashboard

GTP: Government Transformation Programme

ICILS: The International Computer and Information Literacy Study

ICT: Information and Communications Technology

INES: Indicators of Education Systems

MERL: Monitoring, Evaluation, Research and Learning

MFAT: Ministry of Foreign Affairs and Trade

MICS: Multiple Indicator Cluster Surveys

MICS-EAGLE: Multiple Indicator Cluster Surveys – Education Analysis for Global Learning and Equity

MITD: Mauritius Institute of Training & Development

MMT: Mass Media and ICT

OECD: Organisation for Economic Co-operation and Development

PCAP: The Pan-Canadian Assessment Program

PCEIP: Pan-Canadian Education Indicators Program

PEMANDU: Performance Management and Delivery Unit

PGI: Performance Grade Index

PIAAC: Programme for the International Assessment of Adult Competencies

PIRLS: The Progress in International Reading Literacy Study

PISA: Programme for International Student Assessment

PR: Parental Involvement Module

SABER: Systems Approach for Better Education Results

SALSUS: Scottish Schools Adolescent Lifestyle and Substance Use Survey

SDG: Sustainable Development Goal

SEN: Special Education Needs

SSLN: Scottish Survey of Literacy and Numeracy

TALIS: Teaching and Learning International Survey

TEDS-M: The Teacher Education and Development Study in Mathematics

TIMSS: The Trends in International Mathematics and Science Study

UIS: UNESCO Institute for Statistics

UNESCO: United Nations Educational, Scientific and Cultural Organization

UNICEF: United Nations International Children's Emergency Fund

USAID: United States Agency for International Development

Acknowledgements

This research was supported by the New Zealand Aid Programme, Ministry of Foreign Affairs and Trade, New Zealand.

The views expressed in this publication do not necessarily reflect those of MFAT or the New Zealand Government.

Executive summary

This review was undertaken by ‘Te Ihuwaka’, the Education Review Office’s (ERO) Education Evaluation Centre. It explores global best practice for measuring change, currently used by multilateral and government institutions outside the Pacific, to inform New Zealand Ministry of Foreign Affairs and Trade’s (MFAT) education focused work. This review used a solely desk-based research approach to draw on evidence about education measurement approach practices.

As would be expected, the review found substantial variations in the approaches. Across all multilaterals, there is no single conceptual model or approach that emerges as distinct and more apt for measuring change in educational outcomes at a system level. For example, the UNESCO and UNICEF frameworks have an emphasis on the Social Development Goals, the World Bank similarly has a development lens, and the OECD framework reflects the economic maturity of the advanced countries that it serves. While all approaches straddle key domains of the education system, they privilege some domains that are reflective of their specific mandates.

There are some broad differences between developed and developing jurisdictions in terms of data collection methods and sources. The differences, we suggest, probably reflect the stage of development of the education system in the country and the country’s internal capability and capacity. Typically, administrative data seems to be the mainstay for the developing countries in this study. Very few focussed on actual learning outcomes and mostly monitor proxies such as school leaving qualifications. The most common way that countries evaluated their education system was through the lens of tracking status and progress against specific goals or priorities.

Alongside status tracking and evaluations of thematic areas, advanced countries had extensive statistical indicator programmes and, in some instances like Canada, had a programme exclusively dedicated to a suite of learning assessments including the PISA. Scotland focused on what’s happening across the school system on six key improvement drivers so that it could direct improvement efforts accordingly.

Most developing countries had a simple monitoring dashboard of indicators on inputs-outputs and some outcomes in this review. India has, in recent years, developed a performance grading system (70 weighted indicators across five key domains) that groups various states that enable improvement focus. Most developing countries may be expected to align some level of reporting to meet multilateral commitments; however, we could not sight such reporting during this research on selected countries.

Overall, the review shows that:

there is variation in approaches and practices at both levels
there is more monitoring and less evaluation
emphasis appears to be placed on different aspects of the system: inputs, outputs and outcomes, with relatively less on learner outcomes
the purpose of the frameworks reflects the context of the multilateral organisation and the focus of the countries at the point in time
frameworks also reflect either an emphasis on tracking state-of-play/status information or an improvement focus emphasis
the underlying structures and incentives that prompt multilaterals and countries to focus on the things that they do is hard to pin down as context matters. At best, they appear to focus on what is fit for purpose in their contexts. This, in our experience, probably reflects a pragmatic approach that reflects the capability, capacity and resources available to the country at the time, along with the needs of the project or educational system.

One option for improving our understanding is to undertake further research to obtain insights about the gaps in knowledge, and to understand the appetite for a more cohesive approach to monitoring education system performance by funding recipient countries that MFAT operates with.

Overall, there are several useful things to consider based on this review. The review provides an overview of frameworks or principles for assessing education system measurement at a country-level. This could be further developed into a generic framework, or detailed at the level of inputs-outputs-outcomes as necessary for a given country’s context and development need. This can help with mapping education measurement systems and gap analysis in the first instance.

A checklist could be developed to assess what specific funding or recipient countries are currently doing or aspiring to do in terms of system level measurement. Such an approach could serve to identify areas for measuring system capability and where support is required or could be provided.

Using a mapping or checklist approach with a select list of funding countries will also provide insights into which areas of monitoring and evaluation they prioritise and why (for example, national interests, international commitments, resources).

Partner agency supported systems such as SABER and the UNESCO Institute of Statistics can only help collate the data that is available [1]. However, these systems can be used to encourage national governments to invest in more robust national level educational management information systems.

Part 1: Introduction

Background

The New Zealand Aid Programme is managed by the Pacific and Development Group of MFAT. The aid investments on education total about NZ$30-40 million per year, with nearly 60 percent going towards the Pacific region. The investments are aligned to MFAT’s Strategic Frameworks and Strategic Results Frameworks [2] and usually spread over three to five years. These investments directly relate to SDG 4: ‘ensure inclusive and equitable quality education and promote lifelong learning opportunities for all’. The monitoring and evaluation to date tends to focus mostly on outputs (such as the number of teachers who received training) and is mostly undertaken by implementing partners.

The purpose of this review is to understand the various measurement approaches used across developed and developing jurisdictions. MFAT is committed to understanding the impact of its education investments in the Pacific. This review undertaken by Te Ihuwaka, ERO’s Education Evaluation Centre, explores global best practice approaches for measuring change, currently used by multilateral and government institutions outside the Pacific, to inform MFAT’s education focused work.

How to use this document

This report is laid out in two parts and is supported by two detailed appendices.

The first part outlines the objectives for this research and the methodological approach.

The second part outlines the key findings from this review of the approaches of a set of multilaterals and selected countries on assessing educational outcomes and system performance. We pose pointers for future consideration by MFAT in the use of the frameworks. The analysis is based on the material presented in the two substantive appendices.

Appendix 1 presents a detailed description and brief commentary on the frameworks developed by multilateral organisations.

Appendix 2 focuses on frameworks used by selected countries identified as of interest to MFAT. Similarly, a description and brief commentary is presented for each country.

Objectives for this review

The project focuses on global best practice for measuring change in education system outcomes. The review covers:

education outcomes measurement systems rather than programmes or specific education investments or policy choices
frameworks and approaches for measuring change in school education systems among global agencies that have an education focused mandate (for example, OECD, UNESCO)
frameworks and approaches that help monitor equity of education outcomes
practices and approaches among selected countries for measuring changes in school education systems
innovative approaches to systems level monitoring and evaluations
analysis of the different systems and approaches reviewed.

The project scope excludes:

project, programme, and output focus
Monitoring, Evaluation, Research and Learning (MERL) tools and approaches for evaluating education investments
attribution and causality
Pacific and development context.

Methodology

In discussion with MFAT, Te Ihuwaka undertook a desk-based literature research review focused on identifying examples of education system performance and outcomes frameworks used by multilateral institutions and countries. A desk-based research approach is not without limitations. Limitations for desk-based research could include: not being able to connect frameworks with subsequent monitoring work, the rationale behind frameworks and alignment to indicators, prioritisation of indicators and subsequent changes to reporting, and the reasons why, as well as the quality of the data and the value of insights. Without communicating with in-country experts or key informants or access to a wide range of related data and documentary sources, the insights from a desk-based review can be limited.

Standard academic literature search databases were used to help identify key documents for review. These included the following databases:

ERIC – an excellent range of scholarly research relating to all areas of education. Includes surveys, descriptions and evaluations of programmes, curriculum and teaching guides, instructional materials, position papers and resource materials.
Education Research Complete - a database focused on all aspects and levels of education. This includes journal and magazine articles, book chapters, and conference papers. Geographic coverage is international, including New Zealand.

Websites searched included:

OECD - http://www.oecd.org/education/
UNESCO - https://en.unesco.org/themes/education
Education | UNICEF
World Bank - Education (worldbank.org) [3]

Key words such as education frameworks, outcomes, monitoring, were used to identify candidates. Other criteria were then developed to select countries to review, these included:

an existing education outcomes measurement system to review
material about the country’s education outcomes measurement system was available in English and could be accessed on the internet
a basic level of education system infrastructure in place and system maturity to assess education system activity and performance
a spread of geographic locations.

Institutions and countries

Using the criteria, the frameworks and approaches of the following institutions and countries were identified and included for review:

multilateral: UNESCO, UNICEF, World Bank and OECD Directorate of Education
countries: United Kingdom, Finland, Canada, Mauritius, Malaysia, Cambodia, Caribbean (Jamaica or Cuba) and India.

Approach to analysis of the frameworks

The review work was organised in three sequential parts comprising a scan of multilateral agency approaches, a country scan, and thematic analysis. A focus of the study was on understanding the different ways of thinking about measuring change in education system outcomes. We considered:

the context, type and focus of the theoretical or conceptual framework used, such as the OECD and UNICEF education frameworks. This analysis also looked at the extent to which measurement systems exist as a pillar of the wider education system and/or education policy system
the degree to which the framework or outcomes monitoring approach followed good outcomes measurement practice at a system level. For example, to what extent did the framework identify:
- core components
- add-on components
- standard tools and monitoring instruments to implement the framework
- innovative elements
- institutional arrangements to support the implementation of the framework (for example, systematic review points, benchmarking)
what the outputs of the frameworks looked like in practice.

Part 2: Key findings and discussion

What frameworks do multilaterals use?

Multilaterals use a range of conceptual models and approaches to describe and assess the performance of education systems. These are outlined in the following table and described in detail in Appendix 1– How multilateral institutions measure change in education systems.

Key findings from Te Ihuwaka analysis of the table and Appendix 1 are listed below.

Across all multilaterals, there is no single conceptual model or approach that emerges as distinct and more apt for measuring change in educational outcomes at a system level. Each has been developed in the context of their organisational focus and tends to emphasise different components. For example, the UNESCO and UNICEF frameworks have an emphasis towards the Social Development Goals, the World Bank similarly has a development lens, and the OECD framework reflects the economic maturity of the advanced countries that it serves.
The UNESCO SDG 4 Monitoring framework has a set of global indicators (outputs) as well as thematic indicators (inputs and outputs) representing context (for example, national context, policy priorities, technical capacity, and data availability).
The World Bank SABER system has a conceptual model of inputs linked to outcomes with a focus on outputs. This system looks at namely policies and institutions of education systems with emphasis for their quality.
The GEPD framework (adapted from the 2018 World Development Report conceptual framework) has learning outcome indicators at the centre, ringed by indicators for institutional practices and the wider political context and bureaucratic capacity of the system.
The OECD has the most elaborate and comprehensive inputs-activities-outputs-outcomes-impact framework bounded by contextual factors.

OECD Indicators of Education Systems (INES) Framework

Purpose

Performance of national systems as a whole; allows for benchmarks and country comparisons

Organising framework/ model

Human capital approach; encompasses multiple levels of inputs-outputs-outcomes and key actors; contextual factors (demographic, socio-economic and political)

Focus areas

Participation and learning outcomes; school settings; wider enabling conditions

Each element and their interplay; (e.g. quality of education outcomes, equality and equity; adequacy, efficiency of resources; relevance of policy measures)

Data sources

INES drawn from OECD and Eurostat databases, plus tailored surveys (e.g. PISA)

OECD Education Policy Outlook

Purpose

Assess education policy systems

Organising framework/ model

Focus on policy issues, strategy

Focus areas

Policy levers

Data sources

Country reviews, and OECD bespoke survey and data

Data sources

Surveys through a principal country investigator

World Bank - GEPD

Purpose

Speedily identify areas and gaps for improving learning outcomes

Organising framework/ model

Learning outcomes; enabled by practices and politics/bureaucracy capacity

Focus areas

Dynamic progress dashboard (across the three areas)

Data sources

Three surveys: schools; policy; and public officials

UNICEF: MICS-EAGLE

Purpose

Improving learning outcomes and equity through building national capacity and global data foundation

Organising framework/ model

Equity lens (seven areas): gender, socio-economic status, ethnicity etc

Focus areas

Development context: data analysis linked to further analysis of policy issues

Data sources

MICS6 Survey (sole data source)

Are there any key gaps?

The analysis showed that there were two key gaps.

Learner outcomes are focused on a range of inputs and outputs (for example, policy and sectoral areas of education, and school conditions). There is no framework that comprehensively focuses on and collects learning outcomes for learners.
The frequency of many of the monitoring and evaluation approaches appear to vary from annual to once every few years and some of the monitoring data is not current. In addition to this, for some frameworks and associated reports using the frameworks, there is little visibility on the quality of the data collated. This can occur for a wide range of reasons including pragmatic trade-offs around timeliness of reporting and cost, capability, and capacity to undertake monitoring and evaluation work, and participant sensitivities about the information.

Likely rationale for framework selection

The multilateral frameworks demonstrate several aspects as below.

A strong focus in tracking across the whole education system. There are cross-cutting domains (for example, policy levers) and themes (for example, sectors) central to all of them from an education systems perspective. The difference is in the inclusion/exclusion or level of emphasis placed on domains and themes and the emphasis on collecting data or information relating to each. For example, the UNCIEF and UNESCO approaches place an emphasis on Social Development Goals and a focus on educational infrastructure (i.e. ‘inputs’), while the OECD frameworks tend to focus on learning and educational outcomes of students alongside inputs-outputs and contextual factors.
The mixed approaches affirm the complexity inherent in education systems and that impacts result from the interplay of inputs-outputs-outcomes; hence they seek information at different levels from different perspectives and actors (for example, students, policy makers, and education stakeholders).
The use of a mix of tools and methods appears to provide triangulation of results through the broad monitoring and evaluation activities undertaken. These included bespoke (for example, PISA, MISC) and routine surveys (for example, EU HLFS), thematic deep dives and stakeholder feedback, and administrative data generated by institutions and government agencies. The use of a mixed methods approach enables different insights to be generated and for results from different sources to be checked. Tools such as PISA and MISC are established data collection tools and methods, and are internationally acknowledged as representative of best research practice.
The approaches reflect a general split between developed and developing/emerging countries in line with country development context.
- Within the UNESCO and the World Bank frameworks, the focus is on participation and enrolment in compulsory schooling, or proportion of learners assessed as meeting a specific grade level, or the number of qualified trained teachers.
- The GEPD framework includes five outcome measures, 11 indicators of practices (or service delivery), 18 policy levers, and five indicators for politics and bureaucratic capacity. There are, however, also enduring areas of education interest (for example, school governance and conditions) and signs of adaptation (for example, PISA-D survey for developing jurisdictions).
- The OECD-SEA’s use of a case study approach in Indonesia and Malaysia is a pragmatic approach that reflects the goals of the in-country review projects and the resources available.

The split between developed and developing/emerging country-focus reflects the context and framework the respective multilateral agencies are working within. Differences can also be seen at the project level, and the monitoring activities reflect the specific purpose of the review projects.
Theoretical frameworks such as Human Capital Approach are referred to by the OECD and World Bank-GEPD but are not necessarily the dominant theme for overall analyses. In this approach, labour market outcomes such as employment, earnings, and return on education investment are routinely monitored as part of the human capital approach. In contrast, Table 1 and the discussion in Appendix 1 show that the UNICEF and UNESCO frameworks are explicitly set within the Social Development Goal framework, which places a greater emphasis on establishing core building blocks for education delivery. These building blocks include, for example, rooms, teaching resources, developing appropriately trained teachers, and establishing a monitoring system.

What frameworks do individual countries use?

Key points

It looks across the whole system – a wide view. No evidence of this framework being used to evaluate education systems can be found

Long-term plan which touches on educational outcomes

Key findings from this analysis of the above table and Appendix 2 are listed below.

Across the countries reviewed, in general there are no universal coherent frameworks used by countries for measuring education outcomes at a system level that approach the comprehensive nature of the multilateral frameworks – with one exception, Finland.
The developed countries tend to have system-wide frameworks for assessing change and progress. However, national or annual level reporting is not consistently aligned to frameworks.
- For example, Finland in 1999 produced a map of the entire education system (1999) including lifelong learning and placed the map under evaluation criteria such as efficiency, effectiveness, and economy. However, the recent Finnish four-year national evaluation outputs plan does not appear to align with the comprehensive 1999 map.
- Similarly, Canada’s multi-dimensional framework covers not only the pillars of learning (early childhood, elementary-secondary, postsecondary, adult Learning) but also the information levels (learner, provider, jurisdiction) and the dynamic elements (inputs, processes, output and outcomes). However, it is unclear how the Canadian Federal Framework aligns with actual provincial review activity, and with established tools and instruments for generating information to populate the framework.

Scotland uses a National Improvement Framework (2016) with the child at the centre of this framework. As the framework suggests, it is guided by an improvement focus on a mix of priority goals and system improvement drivers at the level of schools and institutions. Improving labour market outcomes is one of the four priorities. The Scottish framework has similarities to the ERO ‘Learners’ model that informs ERO’s review services in their evaluation of school performance.
Of the developing countries, India and Malaysia have relatively more substantial frameworks. India’s PGI stems from the rationale that there must be constant monitoring of inputs, outputs, and outcomes, and fast course correction in order to create an efficient, inclusive, and equitable school education system. This is similar to the approach of some multilaterals. Malaysia has a framework subsumed within its education National Blueprint. The relevant section ‘Current Performance’ covers a mix of input-output priorities such as access to education, quality of education, equality in education, building unity through education, and maximising efficiency.
Mauritius, Cambodia, and Jamaica have no explicit or implicit frameworks.

Overall, what can we say about selected countries?

As expected, there are some broad differences between developed and developing or emerging jurisdictions in terms of data collection methods and sources. The differences, we suggest, probably reflect the stage of development of the education system in the country and the country’s internal capability and capacity.

Typically, administrative data seems to be the mainstay for the developing countries in this study. Very few focused on actual learning outcomes and most monitor proxies such as school leaving qualifications. In Te Ihuwaka’s experience, to make the move in monitoring activity requires a considerable increase in capability, capacity, and resourcing, which may not be available for some of the countries reviewed.
The most common way that countries evaluated their education system was through the lens of tracking status and progress against specific goals and priorities.
- Thematic areas were based on enduring areas of importance to the education system and the development of core education infrastructure, and then evaluated on a periodical basis. This was the case for a range of countries and tends to reflect political cycles or project development cycles.

- Alongside evaluations of thematic areas, advanced countries had extensive statistical indicator programmes and, in some instances like Canada, had a programme exclusively dedicated to a suite of learning assessments including the PISA.
- Scotland had the closest to providing an annual overview of the education system – with focus on what’s happening across the school system on six key improvement drivers but with no reference to the wider system settings (for example, workforce supply).
- Most developing countries had a simple monitoring dashboard of indicators on inputs-outputs and some outcomes in this review. Cambodia had a score card approach on key areas, for instance 10 core breakthrough indicators. These included inputs such as management education for school leaders and higher teacher qualifications. India has had a performance grading system (five domains and 70 indicators) since 2017-18 and Malaysia has a comprehensive baseline about key parts of the system including student outcomes, however, how this will be assessed periodically is unclear.
- Most developing countries may be expected to align some level of reporting to meet multilateral commitments. However, we could not sight such reporting during this research on selected countries.

Generally, of the countries reviewed, the type of evaluation and monitoring efforts undertaken only covered specific parts of the system and:
- these monitoring and evaluation goals can change over time
- these countries have some whole system information (inputs and outputs and outcomes)
- there are few countries that evaluated all parts of their education system year on year. India’s new PGI reporting is an attempt at systematic annual measurement of the education system. In Te Ihuwaka’s experience, this type of activity is very resource intensive. Its usefulness depends on what the national priorities are and how sensitive the priorities are to changes in the system
- countries are more likely to have monitoring and evaluative information on their priority areas at any given time.

Conclusions and next steps

Education systems are complex with elements of linearity and non-linearity [4]. The presence of multiple approaches to assessing education system performance reflects the reality of the complexity.

The purpose of the review was to understand the various approaches, across developed and developing jurisdictions, to measuring the performance of education systems.

Within the scope of the review, Te Ihuwaka has provided insights about the types of approaches and practices at multilateral and country levels to educational system performance.

Overall, the review shows that:

there is variation in approaches and practices at both levels
there is more monitoring and less evaluation
emphasis appears to be placed on different aspects of the system - inputs, outputs and outcomes with relatively less on learner outcomes
the purpose of the frameworks reflects the context of the multilateral organisation and the focus of the countries at the point in time
frameworks reflect either an emphasis on tracking state-of-play/status information or an improvement focus emphasis
the underlying structures and incentives that prompt multilaterals and countries to focus on the things that they do is hard to pin down as context matters. At best they appear to focus on what is fit for purpose in their contexts. This, in our experience, probably reflects a pragmatic approach that reflects the capability, capacity and resources available to the country at the time, along with the needs of the project/educational system.

There are some gaps that remain in our understanding of what countries do.

The links between strategy documents and annual or periodical reporting is often unclear. There are several possible reasons for this, which could be explored as part of another project.
Alignment between frameworks and various indicator dashboards and reporting schedules varies.
The presence or absence of institutional or system level accountabilities and capabilities for producing various indicator or evaluation outputs appears to be a common issue. There are several possible reasons for this, including a lack of resourcing.
How country efforts are linked to multilateral reporting obligations and the quality of such reporting is not always clear. For many countries, responding to different funding agency reporting requirements can be resource intensive. Adoption of a single multilateral reporting framework across countries by key funding agencies, could be of benefit as it would provide opportunities for comparative analysis and reduce the burden on recipient countries. However, this is not always straight forward to achieve because of the context that the funding agencies are operating within themselves.

Overall, there are several useful things to consider based on this review.

The review provides an overview of frameworks or principles for assessing education system measurement at country-level. This could be further developed into a generic framework or detailed at the level of inputs-outputs-outcomes as necessary for a given country’s context and need. This can help with mapping education measurement systems and gaps analysis in the first instance.
A checklist could be developed to assess what specific funding/recipient countries are currently doing or aspiring to do in terms of system level measurement. Such an approach could serve to identify areas for measuring system capability and where support is required or could be provided.
Using a mapping or checklist approach with a select list of funding countries will also provide insights into which areas of monitoring and evaluation they prioritise and why (for example, national interests, international commitments, and resources).
Partner agency supported systems such as SABER and the UNESCO Institute of Statistics can only help collate the data that is available [5]. However, these systems can be used to encourage national governments to invest in more robust national level educational management information systems.

Based on the review we have undertaken, Te Ihuwaka suggests that the OECD or UNICEF frameworks would make a good starting point for thinking about where a country is at in terms of education system design. These frameworks provide a systems-level view for thinking about the educational system as a whole and what appropriate monitoring and evaluation activities could look like for any given type of educational intervention in the system. They have components that align well with the Social Development Goals and have well established resources and instruments for data collection that could be flexibly used depending upon the focus of the education system development or intervention.

Appendix 1– How multilateral institutions measure change in education systems

OECD

OECD organising framework of education systems evaluation

The OECD approach to education systems evaluation is laid out in Education at a Glance 2020: OECD Indicators. According to the OECD, their framework offers a rich, comparable, and up-to-date array of indicators that reflect a consensus among professionals on how to measure the current state of education internationally. The indicators provide information on the human and financial resources invested in education, how education and learning systems operate and evolve, and the returns to investments in education.

They are organised thematically, each accompanied by information on the policy context and interpretation of the data. The indicators are organised within a framework that distinguishes between the actors in education systems, groups them according to the types of issues they address, and examines contextual factors that influence policy. In addition to these dimensions, the time perspective makes it possible to visualise dynamic aspects of the development of education systems.

Actors in education systems

The OECD Indicators of Education Systems (INES) programme seeks to gauge the performance of national education systems as a whole, rather than to compare individual institutional or other subnational entities. However, there is increasing recognition that many important features of the development, functioning and impact of education systems can only be assessed through an understanding of learning outcomes and their relationships to inputs and processes at the level of individuals and institutions.

To account for this, the first dimension of the organising framework distinguishes the three levels of actors in education systems. These are:

education systems as a whole
providers of educational services (institutions, schools), as well as the instructional setting within those institutions (classrooms, teachers)
individual participants in education and learning (students). These can be either children or young adults undergoing initial schooling and training, or adults pursuing lifelong learning programmes.

Indicator groups

The second dimension of the organising framework is the education indicators, which are grouped into three sections. The indicators are dynamic year-to-year and are tailored to fit an overarching focus. For 2020, the indicators focus on vocational education and the transition into work.

Indicators on the output, outcomes, and impact of education systems

Output indicators analyse the characteristics of those exiting the system, such as their educational attainment. Outcome indicators examine the direct effects of the output of education systems, such as the employment and earning benefits of pursuing higher education. Impact indicators analyse the long-term indirect effects of the outcomes, such as the knowledge and skills acquired, contributions to economic growth and societal wellbeing, and social cohesion and equity.

The following indicators on outputs, outcomes and impact are used by the OECD in 2020.

A1: To what level have adults studied?
A2: Transition from education to work: Where are today's youth?
A3: How does educational attainment affect participation in the labour market?
A4: What are the earnings advantages from education?
A5: What are the financial incentives to invest in education?
A6: How are social outcomes related to education?
A7: To what extent do adults participate equally in education and learning?

Indicators on the access, participation, and progression within education entities

These indicators assess the likelihood of students accessing, enrolling in, and completing different levels of education, as well as the various pathways followed between types of programmes and across education levels.

The following indicators on access, participation, and progress are used by the OECD.

B1: Who participates in education?
B2: How do early childhood education systems differ around the world?
B3: Who is expected to complete upper secondary education?
B4: Who is expected to enter tertiary education?
B5: Who is expected to graduate from tertiary education?
B6: What is the profile of internationally mobile students?
B7: How do vocational education systems differ around the world?

Indicators on the input into education systems or the learning environment

These indicators provide information on the policy levers that shape the participation, progression, outputs, and outcomes at each level. Such policy levers relate to the resources invested in education, including financial, human (such as teachers and other school staff), and physical resources (such as buildings and infrastructure). They also relate to policy choices regarding the instructional setting of classrooms, pedagogical content, and delivery of the curriculum. Finally, they analyse the organisation of schools and education systems. This includes governance, autonomy, and specific policies to regulate participation of students in certain programmes.

The OECD splits indicators on ‘inputs’ into two categories, consisting firstly of ‘financial resources invested in education’, and secondly of ‘teachers, the learning environment and the organisation of schools’. The following indicators are used when considering financial resources.

C1: How much is spent per student on educational institutions?
C2: What proportion of national wealth is spent on educational institutions?
C3: How much public and private investment in educational institutions is there?
C4: What is the total public spending on education?
C5: How much do tertiary students pay and what public support do they receive?
C6: On what resources and services is education funding spent?

The following indicators are used when considering teachers, the learning environment, and the organisation of schools.

D1: How does time spent by students in the classroom vary over the years?
D2: What is the student-teacher ratio and how big are classes?
D3: How much are teachers and school heads paid?
D4: How much time do teachers and school heads spend teaching and working?
D5: Who are the teachers?

Contextual factors that influence policy

Demographic, socio-economic, and political factors are all important national characteristics to take into account when interpreting indicators. For example, the 2008 financial crisis had a significant impact on public funds available to education. The characteristics of the students themselves, such as their gender, age, socio-economic status or cultural background, are also important contextual factors that influence the outcomes of education policy.

Indicator analysis using the framework

Analysis of each element of the framework and the interplay between them contributes to understanding a variety of policy perspectives, including:

quality of education outcomes and education opportunities
equality of education outcomes and equity in education opportunities
adequacy, effectiveness, and efficiency of resources invested in education
relevance of education policy measures to improve education outcomes.

Data sources

The OECD INES indicators and framework use a wide range of data sources. The main data source is the OECD and Eurostat databases which includes extensive information on OECD member states. They also use a range of tailored surveys and other data-collection tools, such as:

annual National Labour Force survey
annual INES Network Survey or Teachers and the Curriculum
EU statistics on income and living conditions
PISA (Programme for International Student Assessment)
PIAAC (Survey of Adult Skills)
TALIS (Teaching and Learning international survey)
Adult Education Survey for European OECD countries
many others.

In recent years, the INES indicators have only used limited evidence of student education outcomes provided through PISA (an international study of student education attainment) and TALIS (an international survey on teaching and learning).

Education Policy Outlook

In addition to INES indicators, the OECD also evaluates education system outcome changes through their Education Policy Outlook. Compared to INES, the Education Policy Outlook has a stronger focus on academic attainment (versus a human capital approach in INES).

The aim of the Education Policy Outlook is to provide an updated comparative perspective of policy continuity and policy change as part of education policy ecosystems, as well as provide the evidence available on their implementation outcomes. This overview of policy priorities and policies can serve as a source of inspiration for other education systems that share similar challenges and contextual characteristics

The Education Policy Outlook mainly utilises OECD and Eurostat data for statistics. The main sources of OECD data include Education at a Glance reports (discussed above), the Programme for International Student Assessment (PISA), and the OECD Teaching and Learning International Survey (TALIS). In some cases, where no OECD or Eurostat data is available, national data is consulted. For country-level policy information, the Education Policy Outlook National Survey for Comparative Policy Analysis is used, along with various OECD publications of country-based analysis.

The Education Policy Outlook uses an analytical framework to examine education policy ecosystems. Drawing on OECD work with countries on education policy, this framework serves as a lens through which readers can review education systems from the point of view of students, institutions, and systems. This framework is provided below and explores policy issues (and their respective levers and evidence) at three levels: students, institutions and systems.

UNESCO

Through its global network UNESCO supports education policy development, and in-country, regional, and global policy analyses, and national strategic planning processes. It does this all within the framework of the global education agenda, Sustainable Development Goal (SDG) 4 - Education 2030 and the targets of SDG4.

SGD 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.

A key role of UNESCO is monitoring the progress towards the education targets in the 2030 Sustainable Development Agenda. The Global Education Monitoring (GEM) report is the global mechanism for monitoring and reporting on SDG4 and on education in the other SDGs. One half of the GEM report is dedicated to this monitoring function.

The GEM report also reports on the progress of implementation of national, regional and international strategies to help hold all relevant partners accountable for their commitments as part of the overall SDG follow-up and review. This content is included in the ‘thematic’ half of the GEM report, with overarching themes changing annually. For 2020, the theme was inclusive education. The themes of previous reports have been migration, accountability, and people and planet. The thematic chapters are separate from the monitoring framework but can also be used to consider the broad components to include in evaluation of education systems. These are:

laws and policies
data collection
governance and finance (including education ministries, ministries/agencies in other related sectors, subnational education authorities, and NGOS)
curricula, textbooks, and assessments
teachers
schools
students, parents, and communities.

Indicator framework to monitor SDG4 targets

At the international level, UNESCO uses a set of indicators to monitor progress against achieving SDG4. These include 11 global indicators (blue rows in Table 3) and 43 thematic indicators (white rows in Table 3). These indicators were developed by the Inter-Agency and Expert Group on Sustainable Development Goal Indicators based on inputs from countries, international and regional organizations, civil society, academia, and the international community.

The 11 global indicators represent the minimum set of indicators proposed to countries for the global monitoring of the SDG4 targets. The broader set of 43 internationally comparable thematic indicators serve to chart global progress on education and to monitor the SDG4 education targets more comprehensively across countries, allowing the possibility to identify challenges regarding concepts of the targets that are not reflected well by the global indicators. The thematic indicators represent a recommended set of additional indicators that countries may use to monitor based on the national context, policy priorities, technical capacity, and data availability.

Target 4.1 - Primary and secondary education

By 2030, ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes

Proportion of children and young people (a) in Grade 2 or 3; (b) at the end of primary education; and (c) at the end of lower secondary education achieving at least a minimum proficiency level in (i) reading and (ii) mathematics, by sex
Administration of a nationally-representative learning assessment (a) in Grade 2 or 3; (b) at the end of primary education; and (c) at the end of lower secondary education
Gross intake ratio to the last grade (primary education, lower secondary education)
Completion rate (primary education, lower secondary education, upper secondary education)
Out-of-school rate (primary education, lower secondary education, upper secondary education)
Percentage of children overage for grade (primary education, lower secondary education)
Number of years of (a) free and (b) compulsory primary and secondary education guaranteed in legal frameworks

Target 4.2 - Early Childhood

By 2030, ensure that all girls and boys have access to quality early childhood development, care and pre-primary education so that they are ready for primary education

Proportion of children under 5 years of age who are developmentally on track in health, learning and psychosocial well-being, by sex
Participation rate in organized learning (one year before the official primary entry age), by sex
Percentage of children under 5 years experiencing positive and stimulating home learning environments
Gross early childhood education enrolment ratio in (a) pre-primary education and (b) and early childhood educational development
Number of years of (a) free and (b) compulsory pre-primary education guaranteed in legal frameworks

Target 4.3 - Technical, vocational, tertiary and adult education

By 2030, ensure equal access for all women and men to affordable quality technical, vocational and tertiary education, including university

Participation rate of youth and adults in formal and non-formal education and training in the previous 12 months, by sex
Gross enrolment ratio for tertiary education by sex
Participation rate in technical-vocational programmes (15 to 24-year-olds) by sex

Target 4.4 - Skills for work

By 2030, substantially increase the number of youth and adults who have relevant skills, including technical and vocational skills, for employment, decent jobs and entrepreneurship

Proportion of youth and adults with information and communications technology (ICT) skills, by type of skill
Percentage of youth/adults who have achieved at least a minimum level of proficiency in digital literacy skills
Youth/adult educational attainment rates by age group, economic activity status, levels of education and programme orientation

Target 4.5 – Equity

By 2030, eliminate gender disparities in education and ensure equal access to all levels of education and vocational training for the vulnerable, including persons with disabilities, indigenous peoples and children in vulnerable situations

Parity indices (female/male, rural/urban, bottom/top wealth quintile and others such as disability status, indigenous peoples and conflict-affected, as data become available) for all education indicators on this list that can be disaggregated
Percentage of students in primary education whose first or home language is the language of instruction
Extent to which explicit formula-based policies reallocate education resources to disadvantaged populations
Education expenditure per student by level of education and source of funding
Percentage of total aid to education allocated to least developed countries

Target 4.6 – Literacy

By 2030, ensure that all youth and a substantial proportion of adults, both men and women, achieve literacy and numeracy

Proportion of population in a given age group achieving at least a fixed level of proficiency in functional (a) literacy and (b) numeracy skills, by sex
Youth/adult literacy rate
Participation rate of illiterate youth/adults in literacy programmes

Target 4.7 - Sustainable development and global citizenship

By 2030, ensure all learners acquire knowledge and skills needed to promote sustainable development, including among others through education for sustainable development and sustainable lifestyles, human rights, gender equality, promotion of a culture of peace and non-violence, global citizenship, and appreciation of cultural diversity and of culture’s contribution to sustainable development

Extent to which (i) global citizenship education and (ii) education for sustainable development, including gender equality and human rights, are mainstreamed at all levels in: (a) national education policies, (b) curricula; (c) teacher education; and (d) student assessment
Percentage of schools that provide life skills-based HIV and sexuality education
Extent to which the framework on the World Programme on Human Rights Education is implemented nationally (as per the UNGA Resolution 59/113)
Percentage of students by age group (or education level) showing adequate understanding of issues relating to global citizenship and sustainability
Percentage of 15-year-old students showing proficiency in knowledge of environmental science and geoscience

Target 4.a – Education facilities and learning environments

Build and upgrade education facilities that are child, disability and gender sensitive and provide safe, non-violent, inclusive and effective learning environments for all

Proportion of schools with access to: (a) electricity; (b) internet for pedagogical purposes; (c) computers for pedagogical purposes; (d) adapted infrastructure and materials for students with disabilities; (e) basic drinking water; (f) single-sex basic sanitation facilities; and (g) basic handwashing facilities (as per the WASH indicator definitions)
Percentage of students experiencing bullying, corporal punishment, harassment, violence, sexual discrimination and abuse
Number of attacks on students, personnel and institutions

Target 4.b - Scholarships

By 2020, substantially expand globally the number of scholarships available to developing countries, in particular least developed countries, small island developing States and African countries, for enrolment in higher education, including vocational training, information and communications technology, technical, engineering and scientific programmes in developed countries and other developing countries

Volume of official development assistance flows for scholarships by sector and type of study
Number of higher education scholarships awarded, by beneficiary country

Target 4.c – Teachers

By 2030, substantially increase the supply of qualified teachers, including through international cooperation for teacher training in developing countries, especially least developed countries and small island developing States

Proportion of teachers in: (a) pre-primary education; (b) primary education; (c) lower secondary education; and (d) upper secondary education who have received at least the minimum organized teacher training (e.g. pedagogical training) pre-service or in-service required for teaching at the relevant level in a given country, by sex
Pupil-trained teacher ratio by education level
Proportion of teachers qualified according to national standards by education level and type of institution
Pupil-qualified teacher ratio by education level
Average teacher salary relative to other professions requiring a comparable level of qualification
Teacher attrition rate by education level
Percentage of teachers who received in-service training in the last 12 months by type of training

UNESCO data sources

As the primary responsibility for monitoring SDG4 lies at the national level, UNESCO expects countries to establish effective monitoring and accountability mechanisms which are adapted to national contexts and priorities, in consultation with all stakeholders. At the global level, the UNESCO Institute for Statistics (UIS) remains the official source of cross-nationally comparable data on education and supports countries to strengthen national education data systems.

Three data sources are key for monitoring progress on SDG 4 indicators: household (and other) surveys, learning assessments, and administrative data.

Household and other surveys are the foundation for disaggregating global education indicators by individual characteristics. Examples include completion (4.1), early childhood education participation (4.2), adult education participation (4.3), youth and adult information and communication technology skills (4.4), and adult literacy (4.6). Such surveys are also the basis for calculating global indicator 4.5.1, the parity index, by gender, location and wealth. The main international household survey programmes are the Demographic and Health Surveys (DHS) [USAID] and the Multiple Indicator Cluster Surveys (MICS) [run by UNICEF and discussed in more detail below].

Surveys should be frequent, their questions comparable and their data publicly available to allow open discussion. These conditions are met for 59 percent of countries, corresponding to 87 percent of the population. Northern Africa and Western Asia has the lowest coverage in population terms (46 percent). Oceania has the lowest coverage in country terms (29 percent). There are a range of projects that aim to harmonise international data, as well as international organisations who support statistical capacity development programmes, in the aim of having high-quality and comparable survey data across the world.

Learning assessments are the source of information on global indicator 4.1.1., but also a potential source of information on selected thematic indicators, including knowledge of environmental science (4.7.4) and bullying (4.a.2). While many countries opt to report results from their participation in cross-national assessments, national assessments are also used, for instance, for data on reading skills in countries such as China (lower secondary education) and India (primary education). The UIS database shows 26 percent of countries in Africa have reported reading skills data for the early grades of primary education since 2014, corresponding to 28 percent of the population. Considerable capacity and financial constraints need to be overcome to ensure African countries carry out nationally representative, sample-based national or cross-national assessments every three to five years that meet quality standards.

Administrative data provide information on teacher-related indicators, for example, global indicator 4.c.1 - the percentage of trained teachers. Administrative data is somewhat patchy. About 58 percent of countries in sub-Saharan Africa have reported data on primary education since 2016 but only 25 percent on upper secondary education. Data interpretation suffers from lack of clarity in the definition of trained teachers and in the distinction between trained and qualified teachers. For example, teachers can be qualified, trained, both or neither. In practice, this distinction may not be straightforward depending on country context, or may clash with established terminology whereby a qualified teacher is one who has been trained. This is a challenge the UIS will tackle with a new international standard classification for teachers.

The 2020 GEM report notes that progress has been made in formulating, endorsing, and refining an expanded SDG monitoring framework, but much more effort is needed to ensure that countries report on the global indicators across the SDGs. Custodian agencies need to communicate indicators’ meaning, significance, and methodologies to national authorities. National authorities need to collect data and build their capacity to analyse, report, and use them. Funders need to coordinate their data collection and capacity development programmes.

World Bank

The World Bank utilises two projects for measuring education system performance. These are SABER (Systems Approach for Better Education Results) and the GEPD (Global Education Policy Dashboard).

SABER: Systems Approach for Better Education Results

The World Bank Group has focused its efforts in education in two strategic directions: reforming education systems at the country level and building a high-quality knowledge base for education reforms at the global level.

SABER is the primary tool used by the World Bank to achieve these goals. At the country level, it is intended to provide education systems analyses, assessments, diagnosis, and “opportunities for dialogue”. At the global level, it is intended to improve the education systems knowledge base and use this information to implement effective reforms.

Using diagnostic tools and detailed policy information, SABER produces comparative data and knowledge about education system policies and institutions. It evaluates the quality of those education policies against evidence-based global standards, with the aim of helping countries systematically strengthen their education systems.

When it was established, SABER focused exclusively on the quality of policies and institutions. It has now grown to include the quality of policy implementation (in some policy areas). SABER does not focus on the quality and quantity of education delivered, this is covered in other World Bank projects (such as the GEPD, discussed later).

Methodology

The SABER programme collects comparable data on the policies and institutions of education systems around the world and benchmarks them against good practice. The World Bank states that SABER’s aim is “to give all parties with a stake in educational results a detailed, objective, up-to-date, easy-to-understand snapshot of how well their country's education system is oriented toward delivering learning”. This is based on measures that can be easily compared across education systems around the world.

For each policy area, the World Bank has undertaken substantial research to develop a “What Matters Framework Paper”. From the evidence gathered in these papers, rubrics were developed (for every policy area) to assess the performance of education systems (as well as individual schools). A rubric is a framework that sets out criteria and standards for different levels of performance and describes what performance would look like at each level.

The rubrics consist of various indicators alongside a scoring table. For each indicator, an education system (or a school) can receive one of the following scores: 1) Latent, 2) Emerging, 3) Established and 4) Advanced.

High-level indicators in each SABER policy area (adapted from World Bank)

1. Levels of education

Early childhood development

Establishing an enabling environment
- Legal Framework
- Inter-sectoral coordination
- Finance
Implementing Widely
- Scope of programs
- Coverage
- Equity
Monitoring and assuring quality
- Quality standards
- Compliance with standards

Workforce development

Strategic framework
- Articulating a strategic direction
- Fostering a demand-driven approach to workforce development
- Strengthening critical coordination for implementation
System oversight
- Ensuring efficiency and equity in funding
- Assuring relevant and reliable standards
- Diversifying pathways for skills acquisition
Service delivery
- Enabling diversity and excellence in training provision
- Fostering relevance in training programmes
- Enhancing evidence-based accountability

Tertiary

Vision
Regulatory environment
Governance
Finance
Quality assurance
Relevance

2. Resources

Finance

Ensuring the basic conditions for learning
Monitoring learning conditions and outcomes
Overseeing service delivery
Budgeting with adequate and transparent information
Providing more resources to students who need them
Managing resources efficiently

Teachers

Setting clear expectations for teachers
Attracting the best into teaching
Preparing teachers with useful training and experience
Matching teachers’ skills with students’ needs
Leading teachers with strong principals
Monitoring teaching and learning
Supporting teachers to improve instruction
Motivating teachers to perform

School health and school feeding

School health

Health-related school policies
Safe school environment
School-based health and nutrition services
Health education

School feeding

Policy framework
Financial capacity
Institutional capacity and coordination
Design and implementation
Community roles – reaching beyond the school

3. Governance

School autonomy and accountability

The level of autonomy in the planning and management of the school budget
The level of autonomy in personnel management
Role of the school council in school governance
School and student assessment
School accountability

Engaging the private sector

Encouraging innovation by providers
Holding schools accountable
Empowering all parents, students, and communities
Promoting diversity of supply

4. Information

Assessment (classroom assessment, examinations, and system-level assessment)

Enabling context
System alignment
Assessment quality

Education management information systems

Enabling environment
System soundness
Quality data
Utilisation for decision making

5. Complementary inputs and cross-cutting themes

Information and communication technologies

Vision and planning
ICT Infrastructure
Teachers
Skills and competencies
Learning resources
EMIS (Education Management Information Systems)
Monitoring and evaluation, assessment, research and innovation
Equity, inclusion and safety

Equity and inclusion

Establishing an enabling environment and providing adequate resources
Ensuring that all children and ready to learn and in school
Ensuring all children and especially vulnerable groups learn in school

Resilience

Education in adversity
Assets and engagement
Relevant school and community support
Aligned education system support

Evidence sources

SABER primarily uses bespoke data collection instruments for gathering the required information to score an education system against each of the indicators. These are generally surveys intended for a single respondent — an experienced principal investigator in the country — to fill out using information from key informants, documents, and other sources.

In the typical model, an experienced principal investigator will collect the policy information and data necessary to fill out the data-collection instrument by drawing on his or her knowledge of the system and on government contacts. Data collection can usually be completed within a few weeks in this way. An alternative approach, used in some domains, is to convene a workshop of experts, including government officials, and use that group process to collect the evidence and code data. In either case, data sources are clearly identified and made public when the data are posted.

Analysis

SABER domain teams use the gathered data to analyse “how developed the country’s education policies and institutions are”, from the perspective of achieving key education goals. In the process, it will also generate benchmarks of progress in those specific areas against other countries or provinces. In some domains, these evaluations will be embedded in a more in-depth report discussing policy options and relevant experiences from other countries.

GEPD: Global Education Policy Dashboard

The second World Bank project is the GEPD. According to the World Bank, the GEPD collects and presents data on the specific areas where countries need to act to improve learning outcomes, using indicators that can show progress relatively quickly.

The dashboard tracks progress in three areas – practices (or service delivery), policies and politics. The World Bank states that the GEPD indicators are comprehensive (in that they holistically cover the most important drivers of learning at scale) but also focused (so that they can focus stakeholders’ attention on what is most important). With these indicators, the dashboard 1) highlights gaps between what the evidence suggests is effective in promoting learning and what is happening in practice in each system; and 2) allows a way for governments to track progress as they act to close those gaps.

GEPD Framework

The structure/framework of the dashboard was adapted from the 2018 World Development Report conceptual framework (see Figure 6: World Bank GEPD framework).

At the centre are the outcome indicators capturing learning for all (meaning learning combined with access).

These are encircled by indicators representing the four main school-level service-delivery factors. These factors labelled as “practices” include: prepared learners, capable teaching, appropriate inputs and infrastructure, and capable school management. All these factors bring the other factors together to produce learning. The next set of indicators are markers for the policies that affect each of these areas, and the final set captures the political context and bureaucratic capacity of the system. Sustained system-wide improvement in learning will likely depend on better performance in these policy and politics domains. In summary, this framework considers the politics, policies and practices which all have an impact on learning outcomes.

In selecting and developing the indicators for the dashboard, the World Bank applied three main criteria, listed below.

First, each indicator should predict better learning and access outcomes (based on sound empirical evidence or a strong conceptual presumption to support the relationship).
Second, with concerted effort, it should be possible to improve the indicator over a relatively brief period — one to two years, say —so that the indicator can serve as a marker of progress for a government committed to the longer-term challenge of improving learning.
And third, it should be possible to generate the data for the indicator every two years at a reasonable cost.

The list includes 39 indicators distributed among the four levels symbolized by Figure 6. These include five outcome measures, 11 indicators of practices (or service delivery), 18 policy levers, and five indicators for politics and bureaucratic capacity. Figure 7 provides a summary of all the indicators that are included. Underneath each indicator sit a range of sub-indicators. For example, for the indicator “basic infrastructure”, one sub indicator is the availability of internet connectivity.

Methodology

The dashboard project collects new data in each country using three new instruments: a School Survey, a Policy Survey, and a Survey of Public Officials. Data collection involves school visits, classroom observations, legislative reviews, teacher and student assessments, and interviews with teachers, principals, and public officials. In addition, the project draws on some existing data sources to complement the new data it collects.

School Survey: The School Survey collects data primarily on practices (the quality of service delivery in schools), but also on some de facto policy indicators. It consists of streamlined versions of existing instruments—including Service Delivery Surveys on teachers and inputs/infrastructure, Teach on pedagogical practice, Global Early Child Development Database (GECDD) on school readiness of young children, and the Development World Management Survey (DWMS) on management quality—together with new questions to fill gaps in those instruments.

Policy Survey: The Policy Survey collects information to feed into the policy indicators. This survey is filled out by key informants in each country, drawing on their knowledge to identify key elements of the policy framework (as in the SABER approach to policy-data collection that the Bank has used over the past seven years). The survey includes questions on policies related to teachers, school management, inputs and infrastructure, and learners.

Survey of Public Officials: The Survey of Public Officials collects information about the capacity and orientation of the bureaucracy, as well as political factors affecting education outcomes. The survey includes questions about technical and leadership skills, work environment, stakeholder engagement, impartial decision making, and attitudes and behaviours.

While most GEPD indicators are derived from data collected using the three instruments described above, the team also draws on existing data for a small number of indicators. This is particularly key for outcome data e.g. school participation and learning. Similarly, because factors outside the education system also affect education outcomes, the dashboard also includes a few indicators based on existing data from other sectors. For example, many factors that affect whether children are in school and ready to learn lie outside the education system. Thus, policy levers for this practice area include indicators like the rate of children that are well-nourished and the share of children that are fully immunised, among others. These indicators draw on non-dashboard data sources.

UNICEF: MICS-EAGLE (Multiple Indicator Cluster Surveys – Education Analysis for Global Learning and Equity)

UNICEF launched Multiple Indicator Cluster Surveys (MICS) in 1995 to monitor the status of children around the world. Over the past 25 years, this household survey has become the largest source of statistically sound and internationally comparable data on women and children worldwide. MICS was a major data source for the Millennium Development Goals indicators and continues to inform more than 150 Sustainable Development Goals (SDG) indicators in support of the 2030 Sustainable Development Agenda.

UNICEF launched the MICS-EAGLE (Education Analysis for Global Learning and Equity) Initiative in 2018 with the objective of improving learning outcomes and equity issues in education by addressing two critical education data problems – gaps in key education indicators, as well as lack of effective data utilization by governments and education stakeholders.

MICS-EAGLE is designed to:

build national capacity for education sector situation analysis and sector plan development, and leverage the vast wealth of disaggregated education data collected by MICS6 (or future iterations)
build on the global data foundation provided by MICS6 to conduct analysis of disaggregated data at the national, regional, and global level that yields insights as to how barriers to education opportunities can be reduced so each child can reach his or her full potential.

MICS-EAGLE will analyse the following areas through an equity lens (gender, socio-economic status, ethnicity, etc.).

The MICS-EAGLE initiative also offers activities at the national, regional, and global levels. National level activities seek to identify specific education issues using MICS data linking to policy discussions, and key education data gaps that require additional studies or data sources. Global and regional level activities are geared towards capacity building through increased foundational knowledge regarding the use of household data analysis to inform education sector policy discussions.

Methodology

The MICS6 survey is the source of evidence for MICS-EAGLE. MICS surveys are conducted by trained fieldworkers who perform face-to-face interviews with household members on a variety of topics.

MICS6 was launched in 2017 with data innovations to better reflect measurement of the SDGs and to address data availability issues. Three of the key education innovations involve assessment of early grades learning that will greatly enhance global understanding of educational outcomes and the factors contributing to learning outcomes, collection of data on parental involvement, and collection of data on ICT skills among youth and adults.

The Foundational Learning Skills (FL) module assesses learning outcomes in reading and numeracy skills expected for Grade 2 or 3 levels, covering both in-school and out-of-school children aged between 7-14 years old, which provides data for SDG4.1.1.a. The Parental Involvement (PR) module collects data about the learning environment at home and the extent to which parents are involved in the child’s education at school. The Mass Media and ICT module collects data on ICT skills covering nine ICT activities among youth and adults who are older than 15 years of age. These new modules help shed light on foundational learning and parental participation in education through a global household survey programme for the first time.

Furthermore, MICS6 offers modules on child disability (Child Functioning) and other critical information which helps assess children’s skills and identify education issues around learning and equity for the most marginalized and vulnerable individuals. These changes reinforce the value of MICS6 as a leading household survey in the field of education globally.

These and other education data collected by MICS6 play a critical role in calculating SDG indicators, including 4.1.1.a (learning), 4.2.1 (early childhood development), 4.2.2 (access to pre-primary education), 4.4.1 (ICT skills) and 4.5.1 (equity with a focus on disability).

SDG4 Indicator	Indicator description	Corresponding MICS module
SDG 4.1.1.a	Proportion of children and young people in Grade 2 or 3 in (i) reading and (ii) mathematics, by sex	Foundational Learning Skills (FL) – new module in MICS6
SDG 4.2.1	Proportion of children under 5 years of age who are developmentally on track in health, learning and psychosocial wellbeing, by sex	Children Under Five (EC)
SDG 4.2.2	Participation rate in organized learning (one year before the official primary entry age), by sex	Children Under Five (EC)
SDG 4.4.4	Proportion of youth and adults with information and communications technology (ICT) skills, by type of skill	Mass Media and ICT (MMT) – new module in MICS6
SDG 4.5.1	Parity indices (female/male, rural/urban, bottom/top wealth quintile and others such as disability status, indigenous peoples and conflict-affected, as data become available) for all education indicators on this list that can be disaggregated	Child Functioning (FCF) – new module in MICS6
Various	Disaggregation and regression analysis of the above mentioned SDG4 indicators with parent’s participation in education Data available in the PR module	Parental Involvement Module (PR) – new module in MICS6

Commentary on MICS-EAGLE

Unlike other institutions, MICS-EAGLE only gathers data on outcomes for learners. The survey is firmly set within the development context and the survey is designed to specifically to answer indicators on SDG4 targets. However, UNICEF further analyses the survey findings to identify specific education issues and link this to policy discussions. Another potential limitation is that MICS-EAGLE is based on a single, self-contained survey, whereas other institutions utilise a range of data sources.

Appendix 2 – Practices in select countries

Finland

Education Evaluation Plan

In Finland, the Finnish Education Evaluation Centre (FINEEC) is responsible for the evaluation of education. The FINEEC, established in 2014, brought together the evaluation work of the Finnish Higher Education Evaluation Council, the Finnish Education Evaluation Council, and the Finnish National Agency for Education, to centralise and establish a coherent approach to education system evaluation.

Every four years Education Evaluation Plans are developed by the Evaluation Council of FINEEC and approved by the Ministry of Education and Culture. The Education Evaluation Plans are used to determine education objectives and priorities for the following four-year period. Finland’s Education Evaluation Plan for 2016-2019 details the evaluation projects and guides the development of the evaluation system in 2016–2019.

The projects include audits of quality systems in higher education institutions, assessments of learning outcomes in vocational education and training, and assessments of learning outcomes in basic education. National assessments are sample-based and carried out according to focus areas selected in the plan, generally covering a range of subjects. In addition, the plan sets out thematic and system evaluation projects, such as initial teacher education for certain subjects and the self-evaluation and quality management procedures for basic and general upper secondary education.

The individual evaluation projects appear to be the outputs of this plan and the education system performance seems to be measured at this level. There are some details about each evaluation project in the four year plan, but details such as indicators or how evaluations will be conducted are not included. The evaluation plan notes that halfway through the plan validity period, FINEEC and the Ministry meet to discuss the completed evaluations and their results as well as any revisions, additions, and specifications to the evaluation plan.

A framework for evaluating education outcomes in Finland

In addition to this, there appears to be a comprehensive framework designed for evaluating educational outcomes in Finland. This was created in 1999 by the National Board of Education. This framework does not appear to be used or referred to in the National Education Evaluation Plan 2016-2019 in relation to any of the evaluation projects. The framework states it is designed primarily for national-level evaluations but suggests evaluations need to take place at every level of the education system. The same concepts and approaches can be applied to evaluations at all levels and there is also a section about conducting school-based self-evaluations.

No evidence of this framework being used for evaluating educational outcomes at the national or local level could be found during this research. National evaluation indicators are also mentioned as a work in progress within the framework but no further evidence of these being published could be found.

Evaluating the state of the Finnish education system: Results of the Finnish education evaluation

In addition to these documents there is a further report: Evaluating the state of the Finnish education system: Results of the Finnish education evaluation. It is a summary and synthesis of evaluations conducted between 2014-2018. It highlights four key themes: 1) Trust, 2) Equity, 3) Inclusion and Wellbeing and 4) Society renewed by competence. These are different to the themes identified in the evaluation plan. This summary report makes no reference to the education outcomes framework or evaluation plan for the same time period and those documents also make no reference to this report. We assume that this summary uses information from the evaluation projects planned and conducted in accordance with the evaluation plans for the corresponding time.

In summary Finland has a few different ways in which the education system could be monitored and evaluated. The comprehensive outcomes framework created 20 years ago doesn’t seem to have been used in any official documents. The main way the education system appears to be evaluated is through the evaluation projects set out in the Education Evaluation Plans. It appears most likely that these projects are them synthesized into the summary report about the education system set under four key themes.

Scotland

National Improvement Framework for Scottish Education

In 2016 the Scottish government published a National Improvement Framework for Scottish Education identifying four key priorities and six key drivers for improvement.

The four key priorities are:

improvement in attainment, particularly in literacy and numeracy
closing the attainment gap between the most and least disadvantaged children
improvement in children and young people’s health and wellbeing
improvement in employability skills and sustained, positive school leaver destinations for all young people.

The six key drivers for improvement are:

school leadership
teacher professionalism
parental engagement
assessment of children’s progress
school improvement
performance information.

In 2016 the government produced an evidence report in relation to the National Improvement Framework. It is an overview of what is known about the Scottish education system and provides information about the context in which the children and young people learn. It synthesises evidence from a wide range of sources with the goal to learn from good practice and produce plans for improvement where needed. Figure 14 shows the way they visualize the drivers of improvement within the framework.

This report brings together information on pupils’ achievement, attainment, and health and wellbeing from a range of existing data sources, such as the new Achievement of Curriculum for Excellence Levels return (based on teacher professional judgements), Scottish Survey of Literacy and Numeracy (SSLN), Pupil Census, Teacher Census, Attendance, Absence and Exclusions, Growing Up in Scotland, 27-30 month Child Health Reviews, the OECD’s Programme for International Student Assessment (PISA), Scottish Health Survey, the Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) and Her Majesty's Inspectorate of Education inspections.

The Education Act 2016 requires Ministers to review the National Improvement Framework and publish a plan on an annual basis. These National Improvement Framework plans appear to have been created every year since 2016. They give an update on education but also continue to focus on the drivers of improvement, updating the available evidence and providing detailed improvement activities under each driver. There is a summary of the ongoing and completed activities from the previous year’s plan.

The 2021 plan states certain sources of information were unavailable due to disruptions caused by Covid-19. It suggests the national plan is, therefore, a summation of what schools across Scotland have told us they want to focus on, informed by local consultation and evidence. The plan has also been informed by the data in the NIF Interactive Evidence Report, evidence from school inspections, and local authority self-evaluation reports, and recommendations and advice from the International Council of Education Advisers.

In addition to these annual plans which report on progress, the National Improvement Framework is referenced in the Education Scotland Annual Report, giving an update on performance in many areas including National Improvement.

In summary, Scotland has a clear framework for evaluating and monitoring their education system. They use a range of sources to inform their reports. Their system seems a lot more linked up than other systems and various different reports reference the existence of each other, showing they are used and functioning as intended.

Canada

A Framework for Statistics on Learning and Education in Canada

In 2008, there was a joint declaration between provincial and territorial Ministers of education to use the Learn Canada 2020 Framework to enhance Canada’s education systems, learning opportunities, and overall education outcomes. This framework identified four pillars of lifelong learning:

early childhood learning and development
elementary to high school systems
postsecondary education
adult learning and skills development.

The framework also listed the following key priorities and activity areas.

Literacy: Raise the literacy levels of Canadians.
Aboriginal Education: Eliminate the gap in academic achievement and graduation rates between Aboriginal and non-Aboriginal students.
Postsecondary Capacity: Enhance and stabilize the long-term capacity of postsecondary systems to meet the training and learning needs of all Canadians seeking higher education learning opportunities.
Education for Sustainable Development: Raise students’ awareness and encourage them to become actively engaged in working for a sustainable society.
International and National Representation: Speak effectively and consistently for education and learning in Canada in both pan-Canadian and international settings.
Official Languages: Promote and implement support programs for minority-language education and second-language programs that are among the most comprehensive in the world.
Learning Assessment Programs and Performance Indicators: Support the implementation of national and international learning assessment programs and performance indicators for education systems.
Education Data and Research Strategy: Create comprehensive, long-term strategies to collect, analyze, and disseminate nationally and internationally comparable data and research.

The Canadian Education Statistics Council created a strategic plan for 2010-2020 to assist them in meeting the goals of Learn Canada 2020. To help achieve the last key activity area, Education Data and Research Strategy, the Canadian Education Statistics Council created A Framework for Statistics on Learning and Education in Canada.

The Framework was designed as a tool to assist with deciding the information priorities of the Canadian Education Statistics Program. It is a way of organising the information within the learning and education environment in Canada. It highlights the relevance of information along with the relationships between information. In addition, the framework addresses reporting to the Organization for Economic Cooperation and Development (OECD) on education and learning. It covers not only the pillars of learning (early childhood, elementary-secondary, postsecondary, adult learning) but also the information levels (learner, provider, jurisdiction), and the dynamic elements (inputs, processes, outputs, and outcomes).

As an appendix to the general framework there is a comprehensively populated framework specific to the Canadian context at the time. It is made up of four matrices, one for each pillar of learning. This is a summary of the information about the education system at the time. The matrices identify the existing sources of information for each dynamic element, as well as providing limitations of the sources, but do not provide details on the technical or procedural aspects of collecting or reporting information. These are all details to be worked out in the plan to implement the framework. There are a wide range of sources suggested here for ways to gather this information.

There is no apparent evidence of this framework being used in any annual education report produced by the provinces or anyone using it for evaluating the education system at any level. The Learn Canada 2020 framework also doesn’t appear to be referred to in the annual reports either.

Council of Ministers of Education, Canada learning assessment programs

The aim of national and international large-scale assessments is to make information about academic achievement available to policy makers, administrators, teachers, and researchers so they can gain insight about how the education system functions and identify areas for improvement. Such assessment is not designed to provide academic achievement information at the student, school, or school-board level.

The Council of Ministers of Education, Canada (CMEC) list the learning assessment programs they are part of the:

Pan-Canadian Assessment Program (PCAP)
Programme for International Student Assessment (PISA)
Programme for the International Assessment of Adult Competencies (PIAAC)
International Computer and Information Literacy Study (ICILS)
Teacher Education and Development Study in Mathematics (TEDS-M)
Progress in International Reading Literacy Study (PIRLS)
Trends in International Mathematics and Science Study (TIMSS).

Looking at the PCAP more closely, it was developed in 2007 by the provinces and territories, coordinated by the CMEC and built on its predecessor, the School Achievement Indicators Program. It assesses the skills and knowledge of Grade 8 (Secondary II in Quebec) students in reading, mathematics, and science on a three-year cycle. It is to be used primarily by provincial/territorial ministries of education to assess their education systems. From 2019 the PCAP was conducted online, being responsive to both technological developments and special education needs students.

Education data and research

The Council of Ministers of Education, Canada along with Statistics Canada produce yearly reports called Education Indicators in Canada: An International Perspective. These reports are produced as part of the Pan-Canadian Education Indicators Program (PCEIP). They present data for the provinces and territories allowing comparisons between them, and with data for OECD countries.

The main goal of the PCEIP is to provide statistics that give insight about education and learning in Canada with the aim of supporting evidence-based policy making. PCEIP produced its first set of education indicators in 1996 and has been publishing every year since.

There are several ways in which the Canadian education system can be evaluated but there doesn’t seem to be any coherent links between these. None of the reports above make references to any of the other reports mentioned and make it hard to piece together how they come together in informing the overall education system. Both the PCAP and the PCEIP appear to provide data on how provinces perform compared to each other but there is no cross-reference between these reports.

India

PGI report

The Indian education system is one of the largest in the world. The Department of School Education and Literacy (DoSEL), Ministry of Education is currently focused on improving the quality of education across the system. In order to achieve this, DoSEL has designed a tool called the Performance Grading Index (PGI) to incentivise transformational change in school education.

The PGI score is derived using databases on 70 parameters across five domains. These are access, equity, governance processes, infrastructure and facilities, and learning outcomes. These data are mostly self-reported by the States and Union territories but vetted by the Centre. The total weighted score is a maximum of 1000. The data for 54 parameters come from a range of administrative data and for the remaining 16, scores from National Achievement Survey (NAS) 2017 conducted by the National Council of Educational Research and Training (NCERT) have been used. Find below an example of how the indicators are assembled for Domain 1 – Learning Outcomes and Quality.

The first PGI report was published in 2019 for the reference year 2017-18. The goal of the PGI is to motivate States and Union Territories to undertake multi-pronged interventions that improve education outcomes. The PGI can also help States and Union Territories to identify the gaps in outcomes and prioritize areas for intervention. The PGI is also a good source of best practice information for States and Union Territories to follow.

The PGI provides grades to the States and Union Territories, as opposed to ranking. This system allows multiple States and Union Territories to sit at the same level. It removes the requirement for one State and Union Territory to fall in position before another can improve. The PGI stems from the rationale that there must be constant monitoring of inputs, outputs and outcomes, and fast course correction in order to create an efficient, inclusive, and equitable school education system.

It was reported that the PGI system has been effective since it was put in place. Many States and Union Territories have made substantial improvements to their scores and grades for many of the outcome parameters in 2019-20.

Mauritius

Education Card

The Ministry of Education, Tertiary Education, Science and Technology create an ‘Education Card’ every year that presents the key figures around their education system. They collect and present data on pre-primary, primary, secondary (general and pre-vocational), technical and vocational, education in specialized institutions as well as tertiary level education.

Below are the areas they present statistics and graphics on.

Brief country profile
Educational indicators
Public expenditure on education
Government expenditure on education by sector
Number of schools and enrolment - pre-primary
Number of schools and Enrolment - primary
Number of schools and enrolment - secondary general
Number of schools and enrolment - secondary pre-vocational
Number of centres and enrolment in Mauritius Institute of Training & Development (MITD)
Number of schools and enrolment in Special Education Needs (SEN)
Enrolment in tertiary institutions by gender & type of providers
Enrolment in tertiary institutions by level and field of study
Examination results - Primary School Achievement Certificate (school candidates only)
Examination results - School Certificate (school candidates only)
Examination results - Higher School Certificate (school candidates only)
ICT in pre-primary, primary and secondary Schools

The 2019 Education Card provides information on 2018 and 2019, allowing the reader to see any changes across this time. Figure 17 gives an example of the type of statistics presented. This report provides very basic information on the inputs and outputs in the system. It doesn’t provide any deeper evaluation or analysis. These are similar to the broad UNESCO indicators.

Malaysia

National Education Blueprint 2013-2025

In 2011, the Malaysian Ministry of Education conducted a review of the education system in Malaysia to improve educational outcomes. The government were responding to parental and public concerns about the Malaysian education system’s ability to prepare students for the challenges they would face in the 21^st century, and keep up with rising international education standards. The National Education Blueprint is the document capturing the extensive research and public engagement undertaken to evaluate the performance of Malaysia’s current education system along with a vision of where they want it to go. The report includes 11 shifts that would be required to achieve their vision. They collected information from many sources.

The objectives of the review were to:

understand the current performance and challenges of the Malaysian education system, with a focus on improving access to education, raising standards (quality), closing achievement gaps (equity), fostering unity amongst students, and maximising system efficiency
establish a clear vision and aspirations for individual students and the education system over the next 13 years
outline a comprehensive transformation programme for the system, including key changes to the Ministry which will allow it to meet new demands and rising expectations, and to ignite and support overall civil service transformation.

The National Blueprint is a long document covering:

context and approach
vision and aspirations
current performance
student learning
teachers and school leaders
Ministry transformation
system structure
delivering the road map.

The part that is evaluating the education system is the Current Performance section and it covers the following areas:

access to education
quality of education
equality in education
building unity through education
maximising efficiency.

This section is an analysis of how students are performing over time and in comparison to other countries. It is designed to be an objective fact base to establish a baseline of how the Malaysian education system is performing. They comment that access, quality, equality, unity, and efficiency have been the priority outcomes as far back as 1956 but acknowledge that only by conducting an examination of where they currently sit in term of achieving those outcomes, can they truly move towards improvement.

Multiple perspectives were gathered from various experts and international agencies to evaluate and assess the performance of Malaysia’s education system. These included the World Bank, the United Nations Educational, Scientific, and Cultural Organisation (UNESCO), the Organisation for Economic Co-operation and Development (OECD), and six local universities. The Ministry also worked with other government agencies to ensure alignment with other public policies related to education. For example, the Ministry has worked closely with the Performance Management and Delivery Unit (PEMANDU) to develop the Government Transformation Programme (GTP)2.0 initiatives on education so they reflect the priority reforms in the Blueprint from 2013 to 2015.

In addition, over 55,000 Ministry officials, teachers, school leaders, parents, students, and members of the public across Malaysia, were engaged via interviews, focus groups, surveys, national dialogue townhalls, open days and roundtable discussions. More than 200 memorandums were submitted to the Ministry and over 3000 articles and blog posts were written on the issues raised in the Blueprint. The Ministry also appointed a 12-member Malaysian panel of experts, and a four-member international panel of experts to provide independent input into the review findings.

They have three waves to their education transformation plan over the 13 years with key outcomes noted for each wave. There doesn’t appear to be a system noted in this document about monitoring of these outcomes, however, on their website there is an annual reports section. The reports that this research could find in English appear to give updates on the progress towards the five outcome goals. Administrative data appears to be the main source of information for these annual reports but it’s not totally clear.

In summary, a wide range of sources were used to gain a baseline picture of what the Malaysia education system performance looked like across five key areas. They report on the progress on these areas each year in their annual reports. Monitoring data is main source rather than results of individual thematic evaluations that have been carried out.

Cambodia

Education Strategic Plan 2019-2023

In Cambodia, the education system sits under the authority of the Ministry of Education Youth and Sport MoEYS. Most of the website is not in English so it’s possible there is something else more appropriate on their website, but this is what this research could find that appears to be most relevant.

The 2019- 2023 report begins with a section on the performance of the education sector from 2014-2018, which is the timeframe for the previous Education Strategic Plan. It comments on the progress made during the plan’s implementation period. A midterm review had been conducted in 2016 (mid-way between 2014 and 2018) and the report notes the changes this review resulted in. There were two medium term education policies identified (these are in line with SGD 4).

Policy 1: Ensure inclusive and equitable quality education and promote life-long learning opportunities for all.
Policy 2: Ensure effective leadership and management of education officials at all levels.

There are 10 core breakthrough indicators that have been identified in relation to these policies and the progress on them is presented. This section of the report also goes through how both policy objectives are being reached in each subsector (early learning, primary etc). The report identified eight indicators in relation to the policies for the plan implementation period. The report also sets out a plan for how these will be achieved in each of the sub-sectors.

Education Road Map 2030

In 2019, the Education Road Map 2030 was produced as a strategy to improve education. Its aim is to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all and defined five policy priorities, listed below.

All girls and boys have access to quality early childhood care and education and pre-primary education, and complete free, equitable and quality basic education (primary and lower-secondary) with relevant and effective learning outcomes.
All girls and boys complete upper-secondary education with relevant learning outcomes, and a substantial number of youth have increased access to affordable and quality technical and vocational education.
Ensure equal access for all women and men to affordable and quality technical, vocational and tertiary education, including university.
All youth and adults achieve literacy and numeracy, and learners in all age groups have increased life-long learning opportunities.
Governance and management of education improves at all levels.

There is a reporting framework that organizes the policy priorities with expected results and sector indicators sitting beneath each. This framework was created to help monitor the progress against these priorities. The framework displays the 2018 baseline along with the targets for the plans covering the period 2019-23 and 2024-28 and the target for 2030.

Jamaica

Vision 2030 Jamaica and three-yearly Medium-Term Socio-economic Policy Frameworks

Jamaica has a document called Vision 2030 Jamaica which is a 21-year plan about how Jamaica will progress to become developed. This long-term plan:

identifies long-term goals and outcomes
presents national and sector strategies for achieving the outcomes
proposes outcome indicators to measure performance over the long term
identifies key strategies and actions for years one to three
presents the framework for its implementation, monitoring, and evaluation.

It encompasses all aspects of becoming a developed country including a focus on education. There are four key national goals and 15 national outcomes that sit under these goals. The Jamaican education system doesn’t appear to have a specific framework for evaluating education outcomes, but it can be evaluated through the national level tracking and evaluation of the goal related to education. The following aspects focus on education.

Goal 1: Jamaicans are empowered to achieve their fullest potential

Below this goal there is a National Outcome 2: World class education and training.

Next are the National Strategies linked to the National Outcome above:

Ensure that children zero to eight years old have access to adequate early childhood education and development programmes.
Enable a satisfactory learning environment at the primary level.
Ensure that the secondary school system equips school leavers to access further education, training, and/or decent work.
Accelerate the process of creating and implementing a standards-driven and outcomes-based education system.
Develop and establish financing and management mechanisms for schools.
Ensure a physical environment in all schools that is safe and conducive to learning at all levels of the school system.
Ensure that adequate and high-quality tertiary education is provided with an emphasis on the interface with work and school.
Expand mechanisms to provide access to education and training for all, including unattached youth.
Promote a culture of learning among the general populace.
Establish a National Qualification Framework.
Strengthen mechanisms to align training with demands of the labour market.

Within the Vision 2030 Jamaica document National Outcome Indicators are proposed for each National Outcome, including Outcome #2: World class education and training.

As part of the implementation plan there are three-yearly medium-term socio-economic policy frameworks produced. These report on the specific indicators and targets which can be used for results-based monitoring and evaluation. The 2018-2021 plan reports on each of the national strategies identifying the sector strategies that fall under each national strategy. It highlights the actions to be taken under the sector strategy and the main implementing entity responsible or the action.

First Quarter Performance report

In addition to this, the Jamaican Ministry of Education, Youth and Information produce quarterly performance reports (only available report was April-June 2020). There is however very little overlap between this document and the Vision or the Medium-Term goal report.

In summary, Jamaica appear to have a clear plan for evaluating and monitoring their education system progress towards identified priorities. However, it is it mostly reliant on administrative data to assess performance. They have a plan taking them through to 2030 which outlines most of their goals and in addition they have three-yearly reporting frameworks that specify more details.

References

Council of Ministers of Education. (2008). Learn Canada 2020: Joint Declaration Provincial And Territorial Ministers Of Education. https://www.cmec.ca/Publications/Lists/Publications/Attachments/187/CMEC-2020-DECLARATION.en.pdf

Council of Ministers of Education. (n.d). Learning Assessment Programs. https://www.cmec.ca/131/Learning_Assessment_Programs.html

Council of Ministers of Education. (n.d). The Pan-Canadian Assessment Program. https://www.cmec.ca/240/Pan-Canadian_Assessment_Program_(PCAP).html

Council of Ministers of Education. (n.d). The Pan-Canadian Education Indicators Program (PCEIP). https://www.cmec.ca/259/Programs-and-Initiatives/Education-Data--Research/Indicators/index.html

Council of Ministers of Education & Statistics Canada. (2010). A framework for statistics on learning and education in Canada. http://www.cmec.ca/Publications/Lists/Publications/Attachments/257/cesc-data-framework-sept2010.pdf

Education Scotland. (2019). Annual Report and Accounts 2018-19. https://education.gov.scot/media/jpran3nk/es-annual-accounts2018-19.pdf

Finnish Education Evaluation Centre. (2016). National Plan For Education Evaluation 2016–2019. https://karvi.fi/app/uploads/2016/06/National-Plan-for-Education-Evaluations-2016-2019.pdf

Finnish Education Evaluation Centre. (2019). Evaluating the state of the Finnish education system: Results of the Finnish education evaluation. https://karvi.fi/app/uploads/2019/10/KARVI_T1419.pdf

Government of India. (2021). Performance Grading Index (PGI) 2019-20, For States and Union Territories, Department of School Education and Literacy, MOE pgi_2019_20_en.pdf (education.gov.in)

Ministry of Education Malaysia. (2013). Malaysia Education Blueprint 2013 - 2025 (Preschool To Post-Secondary Education). https://www.ilo.org/dyn/youthpol/en/equest.fileutils.dochandle?p_uploaded_file_id=406

Ministry of Education Malaysia. (2018). 2018 Annual Report: Malaysia Education Blueprint 2013 – 2025. https://www.padu.edu.my/wp-content/uploads/2019/07/AR2018-Eng.pdf

Ministry of Education, Tertiary Education, Science and Technology. (2019). Education Card 2019. https://education.govmu.org/Documents/downloads/Documents/Statistics/Education%20%20Card%20%202019.pdf

The Ministry of Education, Youth and Sport. (2014). Education Strategic Plan 2014-2018. https://www.globalpartnership.org/sites/default/files/2015_02_cambodia_education_sector_plan.pdf

The Ministry of Education, Youth and Sport. (2016). Mid-Term Review Report in 2016 of the Education Strategic Plan 2014-2018 and Projection to 2020. https://www.globalpartnership.org/sites/default/files/2016-11-cambodia-mid-term-review-education-sector-plan.pdf

The Ministry of Education, Youth and Sport. (2019). Cambodia’s Education 2030 Roadmap Sustainable Development. https://www.moeys.gov.kh/index.php/en/planning/2901.html#.YK8TQ4fiuUk

The Ministry of Education, Youth and Sport. (2019). Education Strategic Plan 2019-2023. https://www.globalpartnership.org/sites/default/files/2019-10-education_sector_plan-cambodia.pdf

Ministry of Education, Youth & Information. (2020). First Quarter Performance Report April -June 2020. https://drive.google.com/drive/folders/1yQ05bLGe5euMyBNF0x1ZTClrbuI_kuSi

National Board of Education. (1999). A Framework for Evaluating Educational Outcomes in Finland. https://karvi.fi/app/uploads/2014/09/OPH_0499.pdf

OECD. (2019). Education Policy Outlook 2019. https://www.oecd-ilibrary.org/content/publication/2b8ad56e-en

OECD. (2020). Education at a Glance 2020: OECD Indicators. https://www.oecd-ilibrary.org/content/publication/69096873-en

Planning Institute of Jamaica. (2018). Vision 2030 Jamaica – National development plan: medium term socio-economic policy framework 2018 – 2021 http://www.vision2030.gov.jm/Portals/0/MTF/MTF%202018%20-%202021%20-%20%20March%202019.pdf

Planning Institute of Jamaica. (2009). Vision 2030 Jamaica: National Development Plan. http://www.vision2030.gov.jm/Portals/0/NDP/Vision%202030%20Jamaica%20NDP%20Full%20No%20Cover%20(web).pdf

The Scottish Government. (2016). National Improvement Framework For Scottish Education 2016 Evidence Report. https://www.gov.scot/publications/national-improvement-framework-scottish-education-2016-evidence-report/

The Scottish Government. (2020). Achieving Excellence and Equity 2021 National Improvement Framework and Improvement Plan https://www.gov.scot/publications/2021-national-improvement-framework-improvement-plan/

UNESCO (2015). Incheon Declaration and SDG4 – Education 2030 Framework for Action. https://iite.unesco.org/publications/education-2030-incheon-declaration-framework-action-towards-inclusive-equitable-quality-education-lifelong-learning/

UNESCO (2020). Global Education Monitoring Report 2020: Inclusion and education: All means all. https://gem-report-2020.unesco.org/

UNICEF. (2020). MISC-EAGLE: Implementation Manual 2020: Leveraging Data and Building Capacity. https://data.unicef.org/wp-content/uploads/2019/07/MICS-EAGLE-manual-FINAL-10March.pdf

UNICEF. (n.d). The MICS-EAGLE initiative: Leveraging data and building capacity. https://data.unicef.org/wp-content/uploads/2019/07/MICS-EAGLE-Initiative-Brochure_Revised_20210118.pdf

World Bank. (2013). The What, Why and How of the Systems Approach for Better Education Results (SABER). http://wbgfiles.worldbank.org/documents/hdn/ed/saber/supporting_doc/Background/SABER_Overview_Paper.pdf

World Bank (2021). DRAFT Global Education Policy Dashboard (Provided in confidence by World Bank).

World Bank. (n.d.). About SABER and Methodology. Systems Approach for Better Education Results (SABER) (worldbank.org)