Further Learning

Monitoring and evaluation (M&E) are integral components of health sector reforms, ensuring that activities are implemented as planned and achieving their intended outcomes. Monitoring is a continuous process intended to ensure that an activity occurs as planned. So in the context of health sector reforms, monitoring is a continuous assessment of the level of inputs (financial, human and logistics), dynamics, processes and the outputs in implementing the reform. Evaluation assesses the extent to which an activity has achieved its intended outcome and impact. On the other hand, evaluation determines whether these activities have achieved their desired impact, focusing on improvements in equity, effectiveness, quality, and sustainability (WHO, 2017b).

​

​

Components of Monitoring

Monitoring activities consist of various key indicators that can be categorised into the following components:

​

• Input Indicators: These measure the resources required to drive change, such as finances, staff, and logistics.

• Process Indicators: These track the implementation steps of an activity to assess progress.

• Outcome Indicators: These measure the degree to which the processes have achieved their objectives.

• Dynamics: This involves analysing the organisation and context of the health system in relation to inputs, processes, and outcomes (WHO, 2017b).

A strong and interoperable health management information system is essential for effective M&E of National Health Insurance (NHI) implementation. Measurable standards can be generated and compared to the information obtained to form an M&E Framework that can be used for surveillance, monitoring and evaluation of policies and health services and programmes (Peter & Gershim, 2020). An M&E framework supporting progress towards the achieving the NHI should also reflect the countries unique epidemiological and demographic characteristics, its economic development status and should be guided by the principles of financial protection and coverage of health services (Peter & Gershim, 2020). Some primary outcomes suggested by Peter & Gershim (2020) include “effectiveness, equity, and quality of healthcare at all levels of the health system” for all citizens.

​

​

Monitoring Universal Health Coverage

In 2017, the World Health Organisation released their UHC service coverage index (SCI) (Figure 3) which is a monitoring tool used to measure the progress towards attaining universal health coverage and target 3.8.1 of Social Development Goal 3 (WHO, 2019). The UHC SCI measures the average coverage of 4 crucial health service areas (including their sub-components):

​

1. Reproductive, maternal, newborn, and child health.

2. Infectious diseases.

3. Noncommunicable diseases.

4. Service capacity and access.

​​

​

​

​

​

​

​​

​

​

​

​

​

​

​

​

​

​

​

​

To capture the coverage, 14 tracer indicators categorised across these four areas are used to calculate the respective overall performance of the specific service area. Each of the category-specific means are aggregated to derive a final summary index, which represents the overall service coverage. This index was intended to be transparent and fairly simple for countries to adopt and amend to their local conditions and data availability. The indicators should emphasise effective service coverage – understood as the proportion of people in need of services who receive services of adequate quality to realise potential health gains – and cover the range of service types (i.e. prevention, treatment, rehabilitation and palliation) (Day & Zondi, 2019). Day and Zondi point out that the service capacity and access domain is challenging to interpret because it represents the availability or rate of use rather than the proportion of people who need those services, thus the optimum levels are unknown. 

​

​

​

​

​

​

​

​

​

​

​​​​​

Additionally, the UHC SCI aims to reflect services received rather than entitlement to services. This is particularly relevant to South Africa where the entire population would be entitled to freely access a basket of health services yet in practice there are a range of barriers accessing quality healthcare. The UHC SCI also includes key measurement concepts to guide adoption and global trends since 2000, enabling countries to compare their coverage and analyse factors for success or impediments (WHO, 2019).

​

Despite its utility, interpreting the service capacity and access domain remains challenging, as it measures availability or utilisation rates rather than the proportion of people needing services. . They are also not meant to be understood as an exhaustive list of health services covered in a country and can be replaced with proxy indicators to address any data gaps. To disaggregate the index through inequality dimensions, Wagstaff, Adam, and Neelsen (2020) further developed it by using eight service coverage indicators and one financial protection indicator. Service coverage was divided into prevention and treatment domains which were also individually calculated from geometric average of four indicators. The financial protection indicator was (100 minus) the proportion of the population burdened by catastrophic out-of-pocket expenses (OOP expenses in excess of 10%) (Wagstaff & Neelsen, 2020).

​

​

Budget Accounting and Reporting for Transparency

Effective budget accounting and reporting practices are critical for improving transparency, accountability, and performance in health sector financing. The World Health Organisation (2017a) recommends that an Analytical Team documents the accountability mechanisms that are in place to monitor the budget and assess whether funds have been used appropriately according to their intended purposes. This is done by linking intended results with health sector objectives and commitments related to priority populations, programmes and services. This team should also be responsible for identifying potential obstacles to accountability for expenditures on priority populations, programmes and services. The information obtained in this process can be completed or supplemented through other assessments conducted in the country (e.g. MTEF, resources tracking studies, or funds flow studies) (WHO, 2017a). This can be split up into three tasks.

​

Key tasks for the Analytical Team include:

​

1.  Documenting (current situation and key challenges) the processes or mechanisms used to monitor the budget execution (responsible entity & information flows) – including programme budgets, the specific indicators used to monitor budget execution, the budget audit process and the evaluation of actual spending. The budget monitoring task also includes identifying and assessing the means used to identify the consequences for underspending or overspending budgets as well as the relationship between targets and policy objectives.

2. Assessing how spending agencies are held accountable for meeting objectives through identifying the processes and indicators tracked to measure the effective use of budget, programme, earmarked and donor funds, how monitoring outputs are used and the types of information used to hold spending agencies accountable.

3. Assessing the accountability mechanisms used to assess the efficiency, effectiveness, quality or equity of budget use. This is done by identifying the extent to which funds can be traced and linked to expenditure on populations, programmes and services through accountability measures.
   The information regarding a country’s budget accounting and reporting practices can be placed along a continuum from the least alignment between budget monitoring and health financing objectives (Figure 1) (WHO, 2017a).

​

​​

​​

​

​

​

​

​

​

​

​

​

​

Conclusion

Monitoring and evaluation are indispensable tools for health sector reforms, offering a framework to assess progress, identify gaps, and ensure accountability. From using indices like the UHC SCI to improve service coverage to establishing robust budget monitoring mechanisms, these practices create a foundation for achieving equitable and sustainable health systems. As South Africa moves toward implementing NHI, adopting and refining these tools will be critical for success.