Monitoring and evaluation

A well-designed project performance management system (PPMS) from the outset can enhance the quality of economic analyses. This system facilitates the systematic collection of data on project benefits and pertinent metrics, enabling the monitoring of project performance until completion. Specifically, monitoring the project's progress, identifying areas for improvement, and collecting socioeconomic data through a PPMS, could all help establish a comprehensive assessment of a project’s development results and improve the quality of economic analysis.

The timing of scope changes need to be carefully planned, especially taking account of the possible implementation disruptions these can cause. Adjustments to the design and monitoring framework need to be made to reflect scope changes and other design modifications undertaken during implementation.

Using readily available published data enables proper assessment of a project’s development impact. The installation and use of a robust energy resource planning system in a newly formed utility enables a comprehensive management information system; transparency and good governance in operations; and timely reporting.

Continuous monitoring and fluent communication with the consulting team coordinator can ensure on-time delivery, particularly when using teams of individual consultants.

Current TA rating methodologies, which require reaching certain budget expenditure levels, can disincentivize appropriate cost management and encourage prioritizing budget use over project objectives.

Inaccurate quantitative targets of DMF indicators can be misleading and penalize performance unfairly.

The formulation of a coherent results framework that logically links the inputs, activities, and outputs with the envisaged outcome facilitates better monitoring and measurement of project results. The quality of a design and monitoring framework is enhanced by outcome targets that are realistic and appropriately reflect the immediate changes of the outputs.

While customized, survey-based performance targets can be useful, they should not account for the bulk of the targets such that the overall outcome of the program depends simply on one’s ability to replicate the survey. In this case, 4 of 6 outcome targets were generated by custom surveys which, for reasons unknown, could not be replicated. Also, proxy data could not be located for three of the targets. Thus, the program’s effectiveness, despite evidence to the contrary, was compromised simply by a lack of data and an inability to confirm progress.

More formal and comprehensive systems, supported by the TA, are needed to track and validate compliance with gender-based performance targets. The first two subprograms were classified effective gender mainstreaming, and subprogram 3 was classified gender equity as a theme. Explicit targets were set to prioritize women in multiple areas, including entry into labor activation programs and training, and covering both job seekers and government staff. Yet, tracking was less than robust and required significant follow-up to evaluate program performance.

For the gender action plan, indicators need to be designed to require reporting of baseline and latest data disaggregated by sex for evidence that the practice exists. In this project, indicators such as “MOH public information on drug safety including rapid alert system improved by 2014 (data on persons affected by unsafe drugs are sex-disaggregated)” and “minutes of community meetings and attendance lists were disaggregated by sex” were reported without evidence of sex-disaggregated data.