Evaluation of Green Prosperity Facility

In 2011, the Millennium Challenge Corporation (MCC) entered into a $600 million, five-year Compact Agreement with the Republic of Indonesia, reflecting its focus on sustainable economic growth. The Compact’s largest component, the $332.5 million Green Prosperity (GP) Project, invested in Renewable Energy (RE) and Natural Resource Management (NRM) as part of the Indonesian government’s national development strategies to reduce greenhouse gas emissions. Much of this investment occurred through the centrepiece Green Prosperity Facility (GPF), which provided grant financing to mobilise greater private sector investment and community participation in RE and sustainable land use practices. The GPF investments were intended to enhance sustainable economic growth and social conditions while also reducing Indonesia’s carbon footprint. The GPF portfolio of grants was fully awarded by July 2017,1 each falling into one of five types of grants: Window 1 (Partnership Grants): These grants leveraged private sector or other outside funding to promote increased investment in sustainable NRM, renewable energy, and improved land-use practices. Window 2 (Community-based Natural Resources Management [CBNRM]): These grants funded smaller-scale, community-based projects. Window 3 (RE): These grants funded community-based off-grid (3A) and commercial-scale on-grid (3B) renewable energy projects. Technical Assistance and Project Preparation (TAPP): These grants funded studies (environmental, social, feasibility) and technical assistance to enhance the quality of the projects in Windows 1 and 3 in order to reach the quality required for grant approval. Green Knowledge (GK): These grants built local, provincial, and national capacity to drive forward Indonesia’s nationwide low-carbon development strategy within GP’s context. As implementation progressed, grants in Windows 1–3 were further organised into a range of portfolios, namely: Sustainable Agriculture, Peatland, Social Forestry, Women’s Economic Empowerment (WEE), Community/Off-grid RE, Commercial/On-grid RE, Cocoa, and Eco-tourism. The grants awarded under the GPF were implemented across 14 provinces in Indonesia. ​ The GPF evaluation was an ex-post performance evaluation, conducted in two parts. First, SI assessed implementation fidelity through a process study (Evaluation Question 1) to explore how and why the original design of the Facility changed. SI answered the remaining evaluation questions through qualitative interviews, an online survey, and review of administrative data, with both “Facility-level” findings and “Portfolio-level” findings, wherever it was reasonable to make such a distinction. This process and performance evaluation served two primary purposes: ​

• To inform the design of future grant facilities (by MCC) and/or trust fund facilities (by the Indonesian government), based on GPF learnings; and
• Some grants were terminated between July 2017 and the end of the Compact. 2 If GK grants were excluded, implementation occurred in 11 provinces.

The evaluation questions were as follows:

1. How and why did the design of the GPF evolve over time? Did the Participatory Land-use Planning (PLUP) activity feed into the work of the GPF? To what extent did Green Knowledge (GK) contribute to the GPF?
2. Was the GPF an effective model to achieve the objectives and/or delivery of grant funding? Why or why not? Which aspects of the GPF were particularly beneficial or detrimental to the achievement of the GP Project objectives?
3. Did the GPF approach result in a set of grants that aligned with the GP objectives? What key results did the GPF have with respect to processes, policy, or sustainability? Were the approved grants higher quality than they may have been through other processes?
4. Did the Facility catalyse government policy changes, lay groundwork for future investment, or leverage private sector funds using a new approach? Were there indications that investments would continue to have enduring benefits after the lifetime of the Compact? Was the GPF cost effective? How much did it cost to implement the GPF?
5. What did the Facility achieve in terms of grants awarded and outputs or outcomes? Were the benefit streams modelled in the cost-benefit analyses for the grants appropriate and/or realistic? What were the key successes, challenges, and lessons learnt with respect to operationalising the GPF at each stage of work?

To answer these questions, the evaluation team leveraged a mixed-methods design using document review, 82 key informant interviews, focus group discussions with 21 individuals, and an online survey with 92 responses. The team met with grantees, MCA-I staff, MCC staff, project management contractors for GP, Government of Indonesia (GoI) representatives, and selected donors that also ran facility projects in Indonesia. The team travelled to Jakarta, Yogyakarta, Bogor, Bali, Pontianak, Lombok, Jambi, Mamuju, and Makassar, as most implementers were based in these sites, allowing the team to reach a large number of grant implementers with the resources available.

The quantitative analysis was largely descriptive and comparative in nature. All qualitative data were entered into Dedoose and coded by themes related to the evaluation questions. The team also examined the ERR and feasibility studies to assess whether the items included in the benefit streams were appropriate and realistic. Though the team was able to address all questions, there were some limitations to data collection. First, because many of the grants had been awarded within 18 months prior to data collection, it was not possible for the team to measure long-term (or even short-term) outcomes. As such, the team could only comment on grantees’ perceived contributions to GP objectives. Second, the team was unable to identify suitable comparison facilities to the GP Facility that had sufficient available data (particularly related to cost) and was therefore unable to draw reliable comparisons related to effectiveness, cost-effectiveness, or efficiency.

At the Portfolio level, SI collected qualitative data from 40 of the 83 (48%) non-TAPP grants. All grantees were included in the sample for the online survey. Because of the small number of selected grants in some of the portfolios for qualitative data collection (particularly Window 2 grants), it was not possible to generalise the findings beyond the grantees interviewed to be representative of the entire portfolio in the country, though the team noted areas of similarity or difference across portfolios.