Monitoring and evaluation

Budget approval and procurement of new technologies being piloted could take time and cause implementation delays; when installed, rigorous monitoring of the performance of the new technologies is required to ascertain benefits and draw lessons from the pilot experience. In implementing the four pilot smart grid projects, substantial delays occurred due to the lengthy budget approval process and the procurement requirement of open competitive bidding while allowing international bidder participation.

Adequate safeguard reporting arrangements with frequency and report templates included in the construction contracts and technical guidance on the implementation of safeguard actions can strengthen project compliance to ADB safeguard requirements. The introduction of program actions through the RBL programs has improved PLN staff's awareness and capacity concerning ADB safeguard requirements. The grievance redress mechanism was found to be an area for further improvement during the evaluation of the previous RBL in Sumatra.

Insufficient up-to-date macroeconomic data on government websites, which could hinder timely and effective planning especially during crisis periods, needs to be rectified. The State Statistical Committee (SSC) has numerous macroeconomic indicators on its website. However, in most cases, very recent data are missing. International organizations such as ADB, the IMF and the World Bank release forecasts of macroeconomic indicators for upcoming years, and SSC should consider following the same approach.

Accessibility of healthcare data in Azerbaijan needs to be improved. Reports from the Ministry of Health Care are not publicly available on its website. In addition, the State Agency for Mandatory Health Insurance considers healthcare information to be highly sensitive. Accordingly, this information is not contained on its website. Information related to daily coronavirus infections and vaccinations was released publicly. It would be useful for the agency to disseminate gender, age, and region-disaggregated information to the public.

Sex-disaggregated data need to be accurate and usable for in-depth analyses to support policy development and program planning and implementation. The government has increased its focus on gender equality and has made significant progress on gender data tracking and transparency. Although there are a few publicly available national resources, they lack qualitative information.

Understanding the Private Sector Small Medium Enterprises (PSME) context is essential both at the design stage and during implementation. A detailed assessment of SME policy and institutional, political, and social context is required, with ADB engagement as part of a coordinated program of support by development partners for SME reform. As achieved under the PSME2, it is important that the supported policy actions focus on a limited number of sequenced, high-impact reforms that lead to development outcomes that are clearly defined and reflected in the DMF.

Ensuring the direct link between supported activities, outputs, and outcome, and relevance of their indicators is critical. For example, this program failed to achieve or achieved only in part some outcome indicators even though all planned outputs were delivered.

The development partner assistance was ambitious. Its timeliness and the volume of ADB's Comprehensive Response to the COVID-19 Pandemic (CPRO) financial assistance were well-received by the government. However, this led to a rushed design and inadequate attention to monitoring of implementation, particularly regarding existing capacities to collect gender-disaggregated data in the absence of any existing digital social security systems.

Monitoring and continuous technical support is needed for effective continuation of reform because program impacts will be felt over the long term. The government has agreed to a postprogram engagement framework whose broad purpose is to ensure that policy reforms are not reversed and that Joint-Stock Company Georgian State Electrosystem stays on the path towards financial sustainability.

Without a comprehensive and robust design and monitoring framework (DMF) and without undertaking the necessary updates, the level of project achievements against targets cannot be ascertained. In this project, the scale and character of planned physical works in one scheme were substantially altered during implementation, but the impact of this change on project effectiveness cannot be properly captured by the indicators in the DMF. Similarly, without conducting baseline and impact assessments, the economic viability of project investments cannot be reliably established.