Gender considerations need to be fully integrated in the design of projects and be reflected in the design and monitoring framework (DMF) to ensure that these are effectively addressed during project implementation. The executing and implementing agencies had limited commitment and involvement in the implementation of the Gender action plan (GAP), reflecting the view that the Gender dimension is separate from the main project activities.
The results chains of original and additional financing projects (AFPs) need to be aligned to ensure consistency in performance evaluation. Given that the scope of the AFP was different from that of the original project, it should have been processed as a new project. Because of the difference in scope, the impact and outcome statements as well as performance Indicators of the two projects did not align, making it challenging to reconcile and evaluate the overall project achievements.
The design and monitoring framework (DMF) requires careful consideration of inputs and a strong results chain to ensure that the project outcome targets are directly addressed by the project outputs and are not likely to be influenced by other factors. Last-mile connectivity solutions, improved signs around stations, and TOD measures could be considered as separate outputs in future metro projects. Gender aspects should be clearly stated in the DMF to highlight ADB’s value addition.
In preparing the design and monitoring framework, careful consideration should be given to the relevance of the performance Indicators against the output and outcome targets. Greater cohesion between Indicators and targets will lead to a proper evaluation of project achievements. The effectiveness of the capacity building could have been identified through appropriate Indicators to allow closer monitoring and timely intervention by ADB
Purposeful selection of outcome Indicators that are easily and regularly monitored is critical in crisis response projects. There is a high level of uncertainty surrounding the duration and impact of a crisis. Thus, choosing outcome Indicators that provide timely and crucial information on the progress of the crisis and the impact of policy measures can help manage a crisis, as it allows to quickly introduce new measures when the situation worsens.
Limited baseline data could lead to unrealistic performance Indicators and targets. Because this was ADB’s first TVET grant project in Timor-Leste, which introduced a new area of formalized skills development, baseline data for the targets were limited. This led to some unrealistic performance Indicators and targets. For example, the number of students and trainers to be trained appeared to be modest, whereas the target of 65% of graduates to find employment within 3 months of graduation was overambitious.
Design and monitoring frameworks (DMFs) need to have all the necessary Indicators as well as flexible timelines to effectively be used as planning, monitoring, and evaluation tools. Given the breadth of the program, the DMF should have included at least one indicator for each reform area to better monitor the outcome and long-term impact of key policy actions. Although three new outcome Indicators were included in subprogram 3, these mainly covered financial inclusion and nonperforming loans.
Ensuring that Indicators are specific, measurable, achievable, relevant and time-bound (SMART) is essential to tracking and measuring project results. One project target output was not realistic. The indicator used involves a complex bureaucratic process and requires allocation of revenue budget from the government, which was beyond the project’s scope and administrative boundary.
Weaknesses in the initial design and monitoring framework (DMF) led to difficulties in measuring project achievements. The initial DMF for the facility had to be revised. Some outcome targets were not updated even though there were changes in the associated investments under the different tranches. The DMF included output Indicators for activities not directly financed under the MFF.
A well-designed RBL can make program implementation more efficient and reduce transaction costs during program implementation. This came about in this program with the disbursement-linked Indicators (DLIs) focusing on the main elements of the results chain and framework in the process highlighting the importance of teachers’ and managers’ capacity building and competency-based curriculum (CBC) development, which built a solid foundation for TVET’s sustainable development in Guizhou.