Multi-Criteria Decision Analysis: Principles and Methods

Multi-Criteria Decision Analysis (MCDA) offers a structured way to evaluate and prioritise health interventions by considering multiple factors simultaneously. This approach is particularly useful in addressing the complexities of healthcare decision-making, where different criteria such as cost, severity of disease, and societal preferences need to be balanced. By combining both qualitative and quantitative methods, MCDA provides a transparent framework for comparing options and guiding decisions.

The Performance Matrix

At the heart of MCDA lies the performance matrix, a tool that organises information about various interventions (referred to as "options") and their performance across multiple criteria. Each row in the matrix represents an intervention, while each column corresponds to a specific criterion used to evaluate the interventions. Criteria are carefully chosen to avoid redundancy and ensure relevance, feasibility, and independence from one another. They might include factors like cost-effectiveness, the severity of the disease, and the prevalence of the disease among disadvantaged populations.

The performance of each intervention is recorded in the matrix using different types of measurements. For instance, some criteria might be binary (e.g., whether a disease is more common among the poor), others may use nominal scales (e.g., age groups), ordinal scales (e.g., disease severity), or ratio scales (e.g., cost-effectiveness scores). The matrix serves as the foundation for both qualitative and quantitative analyses.

Qualitative Analysis of the Performance Matrix

In a qualitative analysis, decision-makers can use the performance matrix to make intuitive comparisons between interventions. This might involve visually inspecting the data to identify patterns or ranking options based on their performance. A key concept in this process is dominance, where one option clearly outperforms another across all criteria and is strictly better on at least one. However, dominance is rare in practice, as interventions often perform well on some criteria but not others.

While qualitative analysis is quick and may help decision-makers form initial impressions, it has significant limitations. Subjective interpretation can lead to biased decisions, particularly if all criteria are assumed to be equally important without proper justification. This method also risks oversimplifying complex data, potentially resulting in unjustified rankings.

Quantitative Analysis of the Performance Matrix

Quantitative analysis builds on the performance matrix by converting the raw data into numerical values that reflect preferences for each intervention across the criteria. These values are then combined using mathematical techniques to produce an overall assessment for each option.

The process typically involves two steps:

Scoring: Interventions are assigned numerical scores for each criterion, reflecting their relative desirability. These scores may be derived from value functions that translate achievement levels into preference scores or from expert judgments through direct rating or pairwise comparisons.

Weighting: Each criterion is assigned a numerical weight based on its relative importance. For example, criteria like cost-effectiveness or addressing diseases prevalent among the poor might be given higher weights. Weights are often determined through discussions or comparisons and are normalised so that they sum to 100.

The scores and weights are then combined mathematically to calculate a weighted total score for each intervention. This approach allows for trade-offs, where strong performance on one criterion can compensate for weaker performance on another. However, in some cases, such trade-offs may be ethically or practically unacceptable, requiring non-compensatory methods that avoid aggregation across criteria.

Techniques for Aggregating Data in MCDA

Several methods can be used to combine scores and weights into an overall assessment:

Simple Linear Additive Model: This model assumes that the criteria are independent and combines the scores by multiplying each criterion’s value by its weight and summing the results. For example, if an intervention scores 50 for cost-effectiveness with a weight of 40%, its contribution to the total score is 50 × 0.4 = 20. Adding contributions from all criteria produces a total score for comparison.

Analytic Hierarchy Process (AHP): AHP also uses a linear additive model but derives weights and scores through pairwise comparisons. Decision-makers compare criteria and options in pairs, assessing their relative importance or performance. This method helps capture subjective judgments systematically.

Outranking Methods: These approaches do not rely solely on numerical aggregation. Instead, they assess whether one option significantly outperforms another across enough criteria to "outrank" it. This technique considers the influence of individual criteria more explicitly and may better reflect political and practical realities where poor performance on a single criterion can disqualify an intervention.

Limitations and Applications

Each method has strengths and weaknesses. Simple linear models are intuitive and widely applicable but assume independence between criteria, which may not always hold. Outranking methods, while useful in capturing political sensitivities, can lack discriminatory power when applied to large datasets with many criteria.

Despite its potential, MCDA is underutilised in health policy. Few applications exist to guide decisions on resource allocation, despite growing interest in its ability to handle complex, multifaceted problems. By integrating multiple criteria into a single, coherent framework, MCDA provides a way to make more rational, transparent, and systematic decisions in healthcare. It holds particular promise for addressing the complexities of priority setting in resource-constrained systems, where trade-offs are unavoidable, and the stakes are high.