Monitoring and evaluation

Projections about the impact of road projects should be realistic and adequately consider the accessibility of the roads to be built to communities. The impact statement in the design and monitoring framework about increased access to markets, jobs, and social services in the 4 project provinces should have been restricted to the 10 districts along the project roads, as the project roads could not generate impacts in farther areas. Performance Indicators and targets should also consider data availability and baselines that proved to be limited in at least one of the project provinces.

The lack of available data across government departments can be challenging especially during times of crisis. The government encountered difficulties in collecting sex-disaggregated data in some instances; making it difficult to understand how women and girls were affected by the COVID-19 pandemic. Disaggregation by employment and disability status was challenging as well. The sex-segregated data on testing capacity and incentive for health workers included in the completion report were proxies based on total beneficiaries and the male and female ratio among the healthcare work force.

Indicators and targets relying on surveys that may not be available at the time of measurement could benefit from the identification of alternative measures during program design. One of the outcome Indicators in the CARES design and monitoring framework is measured by the WHO only every 5 years and information about its most recent assessment is due for release in mid to late 2022; after the end of the CARES program. Uncertainties and disruptions created by emergencies need to be considered when designing Indicators to ensure that targets are achievable and realistic.

Lessons from impact assessments undertaken by the government need to be explored for possible replication or application in future crisis response operations. In addition to monitoring of program expenditures; the government conducted its own assessment of program results; such as the social impact of cash transfer programs on affected people.

The COVID-19 pandemic experience provided the government an opportunity to develop its approach to monitoring and reporting. During the loan period; ADB supported the executing agency in the quarterly monitoring of expenditures under the government’s emergency loans. This was appreciated by the government and because it provided the only English-language summary of government expenditure under the two emergency decrees, the quarterly monitoring was continued at government’s request even after loan closure.

It is important to target monitoring at end-state outcomes. Due to the lockdowns and mobility restrictions; early difficulty in the speed of PEN disbursements was unavoidable. Although the government recognized the need for urgent rollout; poor infrastructure raised the risk of leakage through corruption and led to a slower and more cautious approach than would otherwise have been optimal. A lesson to be drawn from the experience involves the need to target monitoring activities at end-state outcomes.

Difficulties with data accentuated problems that included health targets, such as the vaccine rollout. Although testing rates had reached World Health Organization targets by August 2021, it was not clear that the national average testing rate applied uniformly at the provincial level.

Gender considerations need to be fully integrated in the design of projects and be reflected in the design and monitoring framework (DMF) to ensure that these are effectively addressed during project implementation. The executing and implementing agencies had limited commitment and involvement in the implementation of the Gender action plan (GAP), reflecting the view that the Gender dimension is separate from the main project activities.

The results chains of original and additional financing projects (AFPs) need to be aligned to ensure consistency in performance evaluation. Given that the scope of the AFP was different from that of the original project, it should have been processed as a new project. Because of the difference in scope, the impact and outcome statements as well as performance Indicators of the two projects did not align, making it challenging to reconcile and evaluate the overall project achievements.

The design and monitoring framework (DMF) requires careful consideration of inputs and a strong results chain to ensure that the project outcome targets are directly addressed by the project outputs and are not likely to be influenced by other factors. Last-mile connectivity solutions, improved signs around stations, and TOD measures could be considered as separate outputs in future metro projects. Gender aspects should be clearly stated in the DMF to highlight ADB’s value addition.