Pay for Success Memo, Montgomery County, MD

OLO Memorandum Report 2018-5 To: County Council From: Kristen Latham, Senior Legislative Analyst Office of Legislative Oversight Subject: Social Impact Bonds/Pay for Success: Primer and Case Studies Social impact bonds (SIB), also known as pay for success (PFS) projects, are a relatively new method for financing and contracting the delivery of social service programs by governments. Social impact bonds/pay for success projects are a type of impact investing.1 Impact investing includes any type of investment made with the intention to positively impact the public while also receiving financial returns.2 Social impact bonds/pay for success projects are based on two fundamental beliefs: (1) government should only pay for effective services and programs; and (2) the risk of providing social services can be transferred from local and state governments to the private sector in some instances. In the past few years, SIB/PFS projects have gained significant momentum around the world. As of December 2017, there were about 80 social impact bonds/pay for success projects under development in the United States, with approximately twenty having formally been launched.3 This OLO memorandum report provides the Council with a brief primer on social impact bonds/pay for success projects, including: A.      Part A, the definition and history of social impact bonds/pay for success projects; Part B, overview of the steps recommended for establishing a social impact bond; Part C, the potential benefits and concerns associated with social impact bonds; Part D, synopsis of resources available to jurisdictions considering social impact bonds; and Part E, summary of five social impact bond/pay for success project case studies. Definition of Social Impact Bond/Pay for Success Social impact bonds/pay for success projects use an approach to contracting that ties payment to the delivery of measurable outcomes, similar to a performance based contract. SIB/PFS projects have been used to expand effective programs and to test innovative models of service delivery. In a typical SIB/PFS, governments contract with a private sector intermediary to obtain social services. The intermediary acquires operating funds for the program by raising capital from private commercial or philanthropic investors. Investors provide upfront capital in exchange for government payments that 1 Other terms for impact investing include social innovation financing, social impact investing, socially responsible investing, social financing, and social enterprise. 2 https://pdfs.semanticscholar.org/7feb/4528069108e5402df36d2b79fc947da26f06.pdfAn Examination 3 http://pfs.urban.org/pay-success/pfs-perspectives/future-pfs-expanding-access-high-quality-preschool-tempe- arizona February 13, 2018 1 OLO Memorandum Report 2018-5 become available if the performance targets are met, based upon a determination from an independent evaluator. The amount of the total payout is often based on the government’s estimated savings from contracting out the service delivery. The intermediary will often contract with one or more service providers to deliver the interventions necessary to meet the performance targets. Payments are typically for performance that exceeds the minimum defined target, up to an agreed-upon maximum payment level. If the intermediary fails to achieve the minimum target, the government does not pay. 4 See the flowchart below for a basic overview of the SIB/PFS process. Source: https://www.forbes.com/sites/jonhartley/2014/09/15/social-impact-bonds-are-going- mainstream/#96369fa6306b Variations on the Social Impact Bond Model. Stakeholders often use the terms social impact bond and pay for success interchangeably. The variations in name often refer to specific elements of the contract, targeted recipient, or arenas of the service. Other variations of the SIB model include (1) development impact bonds, in which the entity that repays the investor is a third party, such as an aid agency or a foundation instead of the government, or (2) environmental impact bonds, which is like an SIB but is an actual bond, financing environmental outcomes instead of social outcomes, and structured to incentivize innovation by sharing risk.5 4 https://govlab.hks.harvard.edu/files/siblab/files/social-impact-bonds-a-guide-for-state-and-local- governments.pdf 5 https://www.brookings.edu/blog/education-plus-development/2015/02/13/identifying-education-outcomes-for- social-impact-bonds-for-early-childhood/ 2 OLO Memorandum Report 2018-5 History of Social Impact Bonds/Pay for Success. Economist Ronnie Horesh coined the term “social policy bonds” in 1988 to refer to a method for funding worthwhile preventive social welfare interventions while producing savings over time for governments. The first social impact bond was implemented in September 2010 in the United Kingdom. The Peterborough Social Impact Bond sought to reduce recidivism among 3,000 short-term incarcerated young men released from Peterborough Prison through comprehensive services and support. With an investment of almost $8 million dollars from 17 investors, the programs included housing, budgeting knowledge, family support, health services, and employment training. In August 2014, Social Finance U.K.’s (the organization administering the program) initial data indicated that there was a reduction in recidivism but not enough of a reduction to trigger an outcome payment to investors.6 The first social impact bond in the United States launched in 2012 - the Rikers Island Adolescent Behavioral Learning Experience. This recidivism prevention program will be discussed in more detail in the case study section of this report.7 Social Impact Bond/Pay for Success Legislation. Under the Obama Administration, the Federal government showed significant interest in expanding the use of social impact bonds through the following measures. The status of these initiatives is unclear under the Trump Administration.  Funding the Social Innovation Fund and similar initiatives across government, including $3 million in grants to eight government organizations for Preschool Pay for Success feasibility pilots;8  Creation of the Office of Social Innovation and Civic Participation (OSICP); and  Passage of the Every Student Succeeds Act of 2016, which created the Pay for Success Initiative. In addition, according to the National Conference on State Legislatures, at least 24 states and the District of Columbia have considered, are considering or are implementing SIB related projects as of September 2016. Eleven of these states have enacted legislation on social impact bonds, ranging from establishing study committees to creating funds and supporting pilot projects.9 Maryland does not currently have any laws or regulations related to pay for success or social impact bonds. During the 2013 General Assembly session, House Bill 517 was introduced, which required that the State Board of Education issue a request for proposal for social impact bonds to improve education, including minimum proposal guidelines and project selection guidelines. The bill received an unfavorable report by the Ways and Means Committee and was withdrawn.10 6https://www.nationalservice.gov/sites/default/files/documents/CNCS%20PFS%20State%20of%20the%20Field%2 0Document%20Final%204-17-15%20sxf.pdf 7 http://www.payforsuccess.org/learn/basics/ 8 https://www.ed.gov/news/press-releases/us-department-education-announces-3-million-pay-success-grants- preschool-programs 9 http://www.ncsl.org/research/labor-and-employment/social-impact-bonds.aspx 10 http://mgaleg.maryland.gov/2013RS/fnotes/bil_0007/hb0517.pdf 3 OLO Memorandum Report 2018-5 State of Maryland Social Impact Bond Feasibility Studies In the 2012 Maryland Legislative Session, the Maryland General Assembly required the Department of Public Safety and Correctional Services (DPSCS) submit a report on the feasibility of using social impact bonds for re-entry services. In the interim, the Department of Legislative Services conducted a similar review of the feasibility, potential benefits, and risks associated with financing reentry programs using SIBs. The following summarizes the findings of those studies.  DLS Study (January 2013). This study found that a pilot reentry program in Maryland cannot self-finance its operations, even when applying optimistic assumptions. The authors of the report found that “Given the difficulty of linking the evaluation of a social program to a highly complex contract centered on an outcome payment, the government may actually increase its operational risks in undertaking a SIB.” The report recommended that DPSCS should directly finance and operate reentry programs.  DPSCS Study (January 2013). The study completed by the Department of Public Safety and Correctional Services echoed the DLS report and found that the SIB model does not appear to be a good fit for reentry programs. However, the Department expressed an interest in continuing to pursue other unique financing arrangements with experienced investors. The Department also will continue to track and review the use of SIBs for reentry programs in other jurisdictions. At the end of the 2013 General Assembly Session, the General Assembly wanted further research into the use of SIBs. The General Assembly’s House Appropriations Committee requested that the Division of Early Childhood Development within the Department of Education (MSDE) look into the use and feasibility of social impact bonds in early childhood education. MSDE commissioned The Institute for Innovation & Implementation at the University of Maryland School of Social Work to facilitate a SIB Workgroup. The workgroup emphasized that two key elements were necessary for the state to pursue a social impact bond – the evaluation is critical and must have clear and measurable indicators and SIBs require negotiations and the government is not the driver of the negotiation process. Overall, the workgroup thought that SIBs could potentially be used in Maryland, as long as the government leadership is supportive and the target population and outcomes are clear. B. Creation of Social Impact Bonds/Pay for Success Projects Organizations and stakeholders that offer technical support for developing social impact bonds have identified several characteristics of feasible and impactful service program models:11 11 http://digitalcommons.usm.maine.edu/cgi/viewcontent.cgi?article=1001&context=cepare_funding 4  OLO Memorandum Report 2018-5  Program should align with other government policies;  Program should have prior evidence-based success using rigorous methods;  Treatment populations should be well-defined;  Performance outcomes must be clearly measured and monetized to determine primary investment return;  Service should focus on prevention or early intervention;  Program must be replicable and scalable;  All key players (government, intermediaries, investors, service providers, and evaluators) must maintain cooperation and commitment to implementation;12  Federal and state regulatory barriers must be removed and/or adjusted;13 and  Governments should be aware of the potential effect the project will have on future budgets.14 In recent years, several types of intervention programs have been the focus of social impact bonds across the country:  Reduction of recidivism;  Services for at-risk youth;  Pre-natal, early childhood, and preschool services;  Homelessness prevention; and  Employment/workforce development programs. Key Roles in Implementation. The most common model of social impact bond/pay for success requires the government to contract with private-sector or non-profit organizations, who administer the program or project. Stakeholders in a social impact bond project may vary in terms of roles and responsibilities; however, there are five main categories of stakeholders that commonly participate in a SIB/PFS project:  Government. The government (sometimes called the outcome payor15) originates the social impact bond contract for defined outcomes and pays the investors with agreed upon return if the program is successful.  Investors. Investors are commercial, community investment, or philanthropic organizations that fund the social impact bond program and potentially receive a return on the investment if the defined program outcomes are met.  Intermediary/Project Manager. The intermediary or project manager facilitates the social impact bond process. Intermediaries can take on a variety of roles in a project, including coordinating work between the government, investors, service provider and evaluator partners; providing technical support on determining the feasibility of a project; identifying the service provider; raising operating funds from investors; negotiating the terms of contracts or agreements; disbursing payments to service providers; and overseeing day-to-day performance 12https://www.nationalservice.gov/sites/default/files/documents/CNCS%20PFS%20State%20of%20the%20Field%2 0Document%20Final%204-17-15%20sxf.pdf 13 Ibid. 14 https://www.gao.gov/assets/680/672363.pdf 15 https://www2.ed.gov/about/inits/ed/pay-for-success/pay-for-success-toolkit.pdf 5  OLO Memorandum Report 2018-5 management of the project. Intermediaries typically enter into a government contract to provide these services and usually receive a fee for the services they provide. 16  Service Provider. The service provider is the organization or company is contracted to deliver the intervention to the target population. The service provider can be a public or a private entity. Service providers usually have proven success in service delivery, capacity to collect and analyze data metrics for evaluation, and ability to scale up.17  Independent Evaluator. An independent evaluator determines whether targeted outcome measures are achieved according to the terms and conditions of the agreement, determining whether investors are paid or not. In order to maintain independence, the evaluators are typically paid directly by the government, not by the intermediary or investors.18 Formation Process. Social impact bonds/pay for success projects are still a relatively new method for financing social programs. Jurisdictions have used a variety of model types including a standard model where the intermediary serves as the link between the government and investors/evaluators; an “investor-provider partnership” model with no intermediary; and a tiered system of SIBs structured around levels of investor risk.19 The Harvard Government Performance Lab’s Social Impact Bonds: A Guide for State and Local Governments20 provides a more detailed step-by-step manual for jurisdictions considering the pay-for- success/social impact bond approach. The following summarizes their identified steps. 16 https://www.gao.gov/assets/680/672363.pdf 17 http://digitalcommons.usm.maine.edu/cgi/viewcontent.cgi?article=1001&context=cepare_funding 18https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 19https://www.nationalservice.gov/sites/default/files/documents/CNCS%20PFS%20State%20of%20the%20Field%2 0Document%20Final%204-17-15%20sxf.pdf 20 https://govlab.hks.harvard.edu/files/siblab/files/social-impact-bonds-a-guide-for-state-and-local- governments.pdf 6 OLO Memorandum Report 2018-5 1. Determining whether SIBs are a Good Fit for Your Organization Governments should consider whether following conditions are present when deciding if a SIB is a good fit:  Enthusiasm and commitment among leadership;  Realistic ability to scale up;  Alignment with overall performance goals; and  Sufficient interest from investors. 2. Selecting a Suitable Policy Governments should use a variety of sources for social impact bond program ideas, including internal discussions, already successful programs, or a request for proposal. Ideas should be filtered out based on policy priorities, potential for net benefits, and feasibility. 3. Navigating the Development Process The development of the SIB includes three types of tasks for government:  Data Analysis: Match administrative data sets; analyze historical baselines; choose target population; analyze historical performance of specific providers; and conduct analysis necessary to develop evaluation methodology.  Financial Modeling: Conduct a benefit-cost analysis; build a model of payment schedule options; and develop a financial cash flow model for the project.  Structural: Begin initial engagement with potential partners; obtain authority from the legislature; undertake a procurement process to engage services of an intermediary; undertake a process to engage services of providers and hire an independent evaluator. 4. Putting All the Pieces Together During the development of the SIB contract, the government must negotiate payment terms; support the raising of capital, develop operating procedures, draft the contract, and establish plan for expansion (if applicable). 5. Administering the Program The daily operations of the program should be handled by the service provider. However, the government must monitor the program, provide field assistance, and support evaluation efforts. 6. Wrapping Up At the end of a project, a jurisdiction must: (1) determine outcomes and payments; (2) interpret the results; and (3) determine future of social impact bond use. 7 OLO Memorandum Report 2018-5 Financing. The financing structure of a social impact bond/pay for success project typically combines elements of “pay for performance” and municipal bonds – a private investor(s) make an initial investment and is repaid if the agreed upon performance metrics are met. In the current SIB examples in the U.S., the philanthropic organizations or private investors assume the guarantor role and absorb any losses. Social impact bond/pay for success projects usually offer less than a 10% return for investors; however, SIBs may offer opportunities for investors to meet federal portfolio requirements, such as those identified under the Community Reinvestment Act.21 Most service providers are not in a strong enough financial position to deliver services for several years with no dedicated revenue, especially with the risk of not being repaid if they do not achieve the pre- agreed upon outcomes. For this reason, SIB/PFS contracts are usually accompanied by financing agreements with investors that provide upfront capital to support service delivery throughout the project period.22 C. Social Impact Bonds/Pay for Success Benefits and Concerns Social impact bonds or pay for success projects have potential to increase funding and accountability for social service programs that have proven successful. However, critics express concerns that private investors should not be in the business of traditional public service delivery. This section summarizes the potential benefits and concerns on the use of social impact bonds. Social Impact Bond/Pay for Success Benefits. SIB/PFS projects can provide the government with a mechanism to fund social programs and spur innovation. This section summarizes the potential benefits of social impact bonds identified by stakeholders and social impact bond organizations. 21 From https://www.ffiec.gov/cra/ - “ 22 http://digitalcommons.usm.maine.edu/cgi/viewcontent.cgi?article=1001&context=cepare_funding 8 The Community Reinvestment Act (CRA), enacted by Congress in 1977 (12 U.S.C. 2901) and implemented by Regulations 12 CFR parts 25, 228, 345, and 195, is intended to encourage depository institutions to help meet the credit needs of the communities in which they operate. OLO Memorandum Report 2018-5 Risk Protection Social impact bonds shift financial risk away from government agencies to the private sector while retaining the potential social benefit for the public, enabling government officials to pursue addressing social issues without the risk of funding unsuccessful programs with taxpayer money.23 Market Incentives Social impact bonds create a market incentive for an investor to fund a social program, which may not be otherwise funded (particularly when there is little political will to fund the program). The public is less likely to oppose SIBs than other social programs because the government only pays for programs that successfully reduce a social problem. 24 Increased Accountability and Data Integration Because the pay out of a social impact bond is based on performance, data is necessary to identify target populations, track progress over time, and support payout for investors if the program is successful.25 Return on Financial and Social Investments In addition to the financial return on investments if performance metrics are met, investors often receive positive feedback/press for making investments in programs with social impact. Improved Effectiveness and Efficiency of Social Programs Typically, SIB/PFS projects are used on proven programs. The flexibility of social impact allows funding to shift toward effective approaches more easily than in traditional government contracts. Further, SIBs improve the effectiveness of social programs by establishing reliable metrics for measuring performance and allowing for creativity.26 Ability to Scale Up SIB/PFS projects increase funding for providers to expand their operations and allow investment in capital up-front. As a result, service providers can rapidly scale up practices that have proven successful.27 Increased Collaboration Traditionally, many government social services are “siloed,” not integrated across departments. SIB/PFS projects target outcomes for populations or individuals rather than the output of specific services, allowing for multiple government agencies and other organizations to work collaboratively to find solutions for difficult social problems.28 Increased Flexibility Compared to traditional government contracts, SIB/PFS can allow for more flexibility in service delivery. Contracts do not specify service providers’ day-to- day operations and funding, but rather the outcomes of the program, allowing for service providers to tailor their approaches to meet the needs of the target population.29 Government Savings Successful SIB programs may result in substantial government savings as intermediaries innovate to find more efficient ways to deliver services. Typically, the return to investors is based on the cost savings calculated by the government, based on costs that would have otherwise been required. 23https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 24 https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 25 https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 26 https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 27https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 28 https://www.gao.gov/assets/680/672363.pdf 29 https://www.gao.gov/assets/680/672363.pdf 9 OLO Memorandum Report 2018-5 Social Impact Bond Concerns. The following summarizes concerns about social impact bonds and other pay for success models used by governments across the country. Administrative Costs Negotiating and implementing a complicated SIB/PFS with other stakeholders can be time-consuming and, as a result, costlier.30 Some critics have argued that, given the extra costs associated with intermediaries and investors in addition to the administrative costs of the government, using a social impact bond to implement a program is inherently more expensive than providing the prevention program as a direct government service.31 Development Time SIB/PFS contracts are complicated and therefore, can take a significant amount of time to create. Some jurisdictions report that social impact bond contracts can take up to two years to complete.32 Impact on Service Because SIB/PFS focus on the delivery of defined performance metrics, critics believe that service providers may prioritize the delivery of those metrics over overall service quality.33 Organizational Capacity Typically, SIB/PFS projects are used for already established and successful programs. However, organizational capacity of a service provider that can be temporarily built up by a contract may not be sustainable for the organization over time.34 Profit Motive Critics argue that SIB/PFS is currently being used for program areas such as education, healthcare, workforce development, housing – where profits should not be the priority. These are areas where the government and charity organizations should provide services because of the social impact, not the bottom line. 35 Difficulty in Measuring Impact Critics express concerns that the performance metrics may be “gamed” or inflated to ensure that the payout is made to investors. 36 Performance The government does not pay investors unless specified performance metrics are met and therefore the external organization has strong incentives to do its best to achieve the outcomes. However, if the organization does not feel as if the metrics are going to be met, critics believe that service providers could just walk away from the project if the contract allows it. This would leave the target population who are actually benefiting from the program (even if not meeting performance metrics) disadvantaged with no access to the intervention.37 Service Delivery Post- Financing If a SIB/PFS project is successful, governments must decide whether and how to continue to fund the program in future year. The government must decide whether it may be more cost-effective for the government to take on the role of financing the project or continue with the SIB/PFS financing or other performance-based contract. 30 https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 31 https://www.gao.gov/assets/680/672363.pdf 32 https://epaa.asu.edu/ojs/article/view/2640/1918 33 http://www.thirdway.org/report/social-impact-bonds-a-new-model-for-investing-in-social-services 34 https://epaa.asu.edu/ojs/article/view/2640/1918 35 https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 36 https://wws.princeton.edu/sites/default/files/content/Social%20Impact%20Bonds%202014%20Final%20Report.pdf 37 https://www.gao.gov/assets/680/672363.pdf 10 OLO Memorandum Report 2018-5 D. Social Impact Bond/Pay for Success Tools and Resources Available for Governments This section highlights the resources and tools available to jurisdictions and other organization who are considering the use of SIB/PFS, both generally and specifically for early childhood education. Non Profit Finance Fund (NFF). The NFF is a nonprofit organization that consults and invests in pay for success projects around the country. The organization’s specific work includes: investments in PFS projects, consulting on and administering grant programs, and providing training and resources such as market reports/analysis and the Pay for Success Learning Hub.38 Social Innovation Fund (SIF). Launched in 2010, the Social Innovation Fund is a program of the Corporation for National and Community Service that provides grants to intermediaries who in turn identify and grow nonprofits in three focus areas— “youth development, economic opportunity, and healthy futures.”39 The SIF provides organizations and jurisdictions with resources and tools on innovative financing practices, including pay for success. One of SIF’s programs, the Pay for Success Program (launched in 2014), grants funds to experienced social finance institutions or government entities to run open competitions to select, fund, and support PFS projects that produce measurable outcomes in low-income communities across the country.40 Harvard Kennedy School Government Performance Lab. The Government Performance Lab provides technical support to jurisdictions implementing social impact bonds, including assistance with project development, results-driven contracting, and government performance improvement. The Lab also conducts independent review of Pay for Success projects.41 Social Impact Bond Intermediaries. As discussed earlier, social impact bond intermediaries play an essential role in managing social impact bond projects. Because of the proliferation of pay for success throughout the country, the number of intermediaries has increased in recent years. The most prominent organization, Social Finance, is the non-profit which structured and managed the first SIB project in the world. Other intermediary organizations include Third Sector Capital Partners, Private Capital for Public Good, and Finance for Good.42 Early Childhood Education Resources. As discussed earlier, early childhood education is one of the program areas in which stakeholders and jurisdictions believe social impact bonds could be beneficial. This section summarizes the efforts of two Washington, D.C. think tanks (Brookings Institute and Urban Institute) and the United States Department of Education (USDE) to support and guide the implementation of social impact bond/pay for success specifically for early childhood education. 38 https://www.nonprofitfinancefund.org/ 39 https://www.nationalservice.gov/programs/social-innovation-fund 40 https://www.nationalservice.gov/programs/social-innovation-fund/our-programs/pay-success 41 https://govlab.hks.harvard.edu/our-projects 42 https://www.forbes.com/sites/jonhartley/2014/09/15/social-impact-bonds-are-going- mainstream/#57093ccd6306 11 OLO Memorandum Report 2018-5 United States Department of Education (USDE). Under the Obama Administration and continued in the Trump Administration, the USDE awarded numerous grants to innovative education initiatives, including pay for success efforts. As part of its support, the USDE published Pay for Success Feasibility Tool Kit, Considerations for State and Local Leaders in October 2017 to provide general information on PFS, including the three typical phases of a PFS project. The Toolkit includes the following: (1) Pay for Success Phases Checklist; (2) Pay for Success Resources; (3) Support for Education-Focused Pay for Success Projects; (4) Outcome Measurement Questions Checklist; (5) Data and Evaluation Resources; and (6) Cost-Benefit Resources.43 Brookings Institute. In 2015, the Brookings Institute published Identifying Education Outcomes for Social Impact Bonds for Early Childhood,44 in collaboration with the Institute for Child Success and the Ounce of Prevention Fund. The report identified what types of performance metrics should be used for social impact bond contracts in early childhood education:  Meaningful outcome indicators should be empirically predictive of the life trajectory of a child and should include a longer timeframe;  Measurable outcomes should use simple, accurate, and consistent measurement tools; and  Timely outcomes should be achieved in a reasonable timeframe because investor interest is critical. However, short term outcomes should not be the end goal but rather proxies for longer term outcomes. In January 2016, the Brookings Institute followed up with Using Impact Bonds to Achieve Early Childhood Development Outcomes in Low- and Middle-Income Countries,45 that detailed steps on how to implement and evaluate early childhood education social impact bonds, particularly for low- and middle- income countries. Some of the specific performance metrics identified in the United States include: kindergarten readiness, primary school repetition rate, or special education avoidance. Urban Institute. In 2015, the Urban Institute launched its Pay for Success (PFS) Initiative to support, design, and evaluate PFS project efforts across the country. The Institute’s PFS Project Assessment Tool46 describes essential elements of a project, provides a scoring system to help jurisdictions evaluate a project, and makes recommendations for improvement. In addition, the Institute created the Early Childhood Education (ECE) Toolkit, which is designed to guide jurisdictions through a PFS project in early childhood education:  PFS + ECE: Evaluation Design;  PFS + ECE: Project and Performance Management;  PFS + ECE: Program Funding and Financing;  PFS + ECE: The State of the Science on Early Childhood Interventions;  PFS + ECE: Using Data to Inform Decision-making; and  PFS + ECE: Outcomes Measurement and Pricing. 43 https://www2.ed.gov/about/inits/ed/pay-for-success/pay-for-success-toolkit.pdf 44 https://www.brookings.edu/blog/education-plus-development/2015/02/13/identifying-education-outcomes- for-social-impact-bonds-for-early-childhood/ 45 https://www.brookings.edu/wp-content/uploads/2016/07/Impact-Bonds-for-ECDweb.pdf 46 http://pfs.urban.org/library/content/pay-success-project-assessment-tool 12 OLO Memorandum Report 2018-5 E. Social Impact Bond/Pay for Success Case Studies The implementation of social impact bonds/pay for success projects in the United States is relatively new. Because of this, the number of case studies with final results is limited. A preliminary review of SIB/PFS projects in the United States shows mixed results. OLO summarized five case studies of social impact bonds/pay for success projects, including two early childhood education projects. The following table is a summary of the five case studies OLO identified, followed by a more detailed description of each case study. Overall, while most SIB/PFS projects in the United States have not yet formally be evaluated yet, four of the five case studies that OLO identified have shown preliminary positive results. 13 OLO Memorandum Report 2018-5 Adolescent Behavioral Learning Experience, Rikers Island NYC (2013) Chicago Child-Parent Center Initiative (2014) Utah High Quality Preschool Program (2013) Partnering for Family Success, Cuyahoga County OH (2015) Massachusetts Juvenile Justice Pay for Success Initiative (2014) Policy Area Youth Recidivism Early Childhood Early Childhood Education Homelessness and Foster Care Youth Recidivism Target Population Estimated 3,000 16-18-year olds entering Rikers Island with a stay of more than four days 2,620 4-year old public school students eligible for free and reduced lunch 3,500 3-and 4-year olds eligible for free lunch 135 homeless parents with children in foster care 929 17-24-year-old men on probation/parole, incarcerated, or exiting the juvenile justice system Outcome Metrics Reduction of “reentry bed- days” Increase kindergarten readiness and literacy, decrease in special education Reduction in the use of special education services Reduction of length of stay in out-of-home foster care Reduction of incarcerated bed-days, increase in job readiness and employment Evaluation Design Quasi-experimental, comparison to cohort entering Rikers prior to start of ABLE program Quasi-experimental, comparison to children never enrolled in preschool Longitudinal study of children in program Comparison of eligible families which are randomly assigned to program or control group Randomized Controlled Trial Size of Investment $9.6 million $16.9 million $7 million $4 million $21.76 million Payout $11.7 million $34 million Not Available $5 million $27 million Current Status Program terminated in 2015 because did not meet performance goals Final evaluation not completed – initial results positive Initial evaluation showed performance met goals; initial payments have been made Final evaluation not completed – initial results positive No evaluation completed, internal reports show positive results 14 OLO Memorandum Report 2018-5 ABLE – Adolescent Behavioral Learning Experience New York City – Rikers Island Implementation Date. February 2013 Issue/Intervention. Program aimed at 16-18-year-old youth detained at Rikers Island. The program attempts to break the cycle of reincarnation using the Moral Reconation Therapy - an evidence-based intervention that focuses on improving social skills, personal responsibility, and decision making. Population. 3,000 Youth Stakeholders. Outcome Goals. Reduction of Teenage Recidivism by 10% Program Cost & Pay Out. Goldman Sachs was scheduled to grant $9.6 million to New York City (with Bloomberg Philanthropies guaranteeing $6 million or 83% of the loan). If the program is successful, the payout would be:  8.5% reduction in recidivism would repay Goldman Sachs its investment; and  Greater than 10% reduction would have resulted in profit for Goldman (between $500K and $2.1 million depending up rate of reduction). Result. The Vera Institute of Justice evaluation found that program did not reduce recidivism and therefore did not meet the pre-defined threshold of success. While 87% of youth entered jail during the program and were held for seven or more days were able to attend at least one class, the change in recidivism for the eligible 16- to 18-year-olds, adjusted for external factors, was not statistically significant when compared to the matched historical comparison group. Further, youth in the program showed similar trends in rates of recidivism over time compared to 19-year-olds who were not in the program, indicating that any shifts were the result of factors other than the ABLE program. The program was terminated in July 2015, with Goldman Sachs reporting a $1.2 million-dollar loss and Bloomberg Philanthropies not receiving any success payments (after investing $6 million). Sources  https://ssir.org/up_for_debate/article/the_payoff_of_pay_for_success  https://nonprofitquarterly.org/2015/08/07/what-we-learned-from-the-failure-of-the-rikers-island-social- impact-bond/  https://www.mdrc.org/project/social-impact-bond-project-rikers-island#overview  http://www.nyc.gov/html/om/pdf/2012/sib_fact_sheet.pdf Government New York City Intermediary MDRC Service Provider Osborne Association, Friends of Island Academy Investors Goldman Sachs, Bloomberg Philanthropies Evaluator Vera Institute of Justice 15 OLO Memorandum Report 2018-5 Expansion of Child-Parent Center (CPC) Early Childhood Program Chicago, Illinois Implementation Date. December 2014 Issue/Intervention. The Child-Parent Center (CPC) model is one of the most successful and longest- running early childhood interventions in the country, serving low-income families living in communities withashortageofpubliclyfunded,high-qualitypre-Keducation. TheCPCmodelprovideshalf-day preschool and parent support, including a team of professionals that align and coordinate curriculum, intensive family supports, parent involvement and engagement, effective learning experiences through the Creative Curriculum, and professional development for teachers. Chicago’s first pay for success project, the CPC expanded this program to six city public schools over the course of four years beginning in the 2014-2015 school year. In 2015-2016, two additional sites were added to the project. Population. 2,620 public school children over 4 years - the first cohort (2014/2015 school year) will include 374 students, second and third years will add 782 children each year, and another 680 children in the 4th year. Stakeholders. Outcome Goals. A 50% increase in kindergarten readiness, a 50% increase in 3rd grade literacy, and 44.5% reduction in the usage of special education services among program participants. Program Cost & Pay Out. The total investment for this pay for success project was $16.9 million, with the following breakdown:  Goldman Sachs Social Impact Fund - $7.4 million;  Northern Trust Corp - $5.4 million;  JB & MK Pritzker Family Foundation - $4 million;  City of Chicago & Chicago Public Schools - $9.4 million; and  State of Illinois - $4.5 million If the program is successful, the payouts will include $2,900 for each child who is school-ready for kindergarten, $750 for each child who is literacy-proficient in grade 3, and $9,100 for each year a CPC participant avoids special education as compared to a matched comparison group without preschool services. The investors anticipate a total payout about approximately $25 million, according to Chicago Government City of Chicago Intermediary IFF (formerly Illinois Facilities Fund) and Metropolitan Family Services Service Provider Chicago Public Schools Investors Goldman Sachs Social Impact Fund, Northern Trust, J.B. and M.K. Pritzker Family Foundation, Finnegan Family Foundation Evaluator SRI International 16 OLO Memorandum Report 2018-5 Reporter review of the loan documents. However, if maximum payout performance was achieved, CPS would be responsible for about $30 million while City Hall would add in the other $4.4 million, for a total of over $34 million. Results. The first evaluation of the program, released in the Spring 2015, found that 61% of the 310 children who participated in CPC preschool during 2014-15 had kindergarten readiness ratings that met or exceeded national averages, which was almost 20% more than their peers in other early childhood programs at CPS. The special education rate was slightly lower in the intervention group than the comparison group (4.38% in the CPS group vs. 4.94% for children in the comparison group). The second evaluation of the program occurred in April 2017 and found that: 42% of the 586 children in Cohort 2 were considered ready for kindergarten. In addition, children who attended full-day CPC preschool had higher rates of kindergarten readiness (50%) compared to children who attended half-day CPC preschool (36%). Because of the initial positive results, the first SIB repayment occurred in March 2017, which was $500,000 for investors. Sources  https://ssir.org/up_for_debate/article/the_payoff_of_pay_for_success  http://www.chicagoreporter.com/wp-content/uploads/2016/05/SIB-CPC_Report_FINAL_041916.pdf  http://www.goldmansachs.com/what-we-do/investing-and-lending/impact-investing/case- studies/chicago-social-impact-bond-multimedia/press-release.pdf  http://www.payforsuccess.org/project/chicago-child-parent-center-pay-success-initiative  http://www.iff.org/wp-content/uploads/2017/03/SIB-CPC_Report_YEAR2_v4_2017-04- 27_SRI_FINAL2.pdf  http://learntogethermn.org/wp-content/uploads/2015/04/SIB-Chicago-Summary-HCRC-November-7- 20141.pdf  http://www.chicagoreporter.com/record-paying-preschool-social-impact-bonds/ “Success” Measurement Expected Amount Maximum Allowed Kindergarten readiness tests $3,530,373 $3,530,373 Reductions in special ed $21,481,790 $30,000,000 Third-grade literacy tests $839,397 $839,397 Total $25,851,560 $34,369,770 17 OLO Memorandum Report 2018-5 The Utah High Quality Preschool Program Implementation Date. September 2013 Issue/Intervention. The Utah High Quality Preschool Program was the country’s first pay for success project in early education. In order to establish the program, the Utah State Legislature had to pass House Bill 96, which allocated funds to support grants to local education agencies and private providers to increase the quality of early childhood programming throughout Utah, including allowing the School Readiness Board to enter into Pay for Success financing contracts. This Preschool Program is an expansion of already existing school-readiness curriculum to two school districts in the state. The program provides half-day classes for two days a week for 3-year-olds and four days a week for 4-year-olds. Population. 3,500 children in five cohorts. Stakeholders. Outcome Goals. At least 50% of children in the program to avoid special education. Program Cost & Pay Out. The total investment for this project was approximately $7 million including $4.6 million from Goldman Sachs and $2.4 million from J.B. Pritzer. Payouts for success in the program will be made based on the avoidance of special education. In order to determine this, participating children were given the Peabody Picture Vocabulary Test – if a student tested below the average, he/she was determined to be a likely user of special education. These students were tracked and for every year the student did not use special education services through sixth grade, investors were paid $2,470, or 95% of the avoided costs of special education ($2,600 in Utah). After sixth grade, investors were paid $1,040 per child not using special education services (40% of the avoided costs). Result. Initial evaluation findings of the project show positive results. For the first cohort of participants, 110 low-income three- and four-year old’s were identified at highest risk for school failure. Of these participants, only one used special education services in kindergarten. A follow-up longitudinal study of the first three cohorts of participants found similar results – of the 238 children to be at highest risk for school failure in the three cohorts, only 11 (5%) had used special education services through third grade. As a result, the first payments were made to investors were made in 2015 and amounted to approximately $267,000 and will continue through June 2021 if the program remains successful. Concerns about the Evaluation. It is important to note that there has been some criticism among education professionals about the Utah pay for success preschool project. Nine early-education experts Government Utah Intermediary United Way of Utah Service Provider Granite School District, Park City School District, Guadalupe School, YMCA of Northern Utah, Children’s Express, Lit’l Scholars Investors Goldman Sachs, JB Pritzker Evaluator Utah State University’s Early Intervention Research Institute 18 OLO Memorandum Report 2018-5 who reviewed the program for The New York Times identified two concerns about how the program’s success was measured:  Experts had concerns about the cost of preschools in the project. They claim that preschool programs that have been shown to be effective in reducing special education needs typically cost four to five times what the pay for success project cost. In addition, even well-funded programs have been found to only reduce special education need by 50%, but typically only by 10-20%. The experts expressed concern that extremely high success rate was based on initial misidentification of high risk students.  The test used to select program participants is not a test usually used to determine risk of school failure or to identify potential special education use. It is especially problematic for students in which English is not a first language; non-English-speaking students have been known to score poorly on the test, which may not be a sign of special education needs but rather English instruction. Therefore, any improvements in performance may not be a result of the program but rather a larger number of students in the baseline than should be. Sources  https://www.nytimes.com/2015/11/04/business/dealbook/did-goldman-make-the-grade.html  Initial Results Released for First Social Impact Bond for Early Childhood Education Show Success  https://www.centreforpublicimpact.org/case-study/social-impact-bonds-early-childhood-education-utah/  First Social Impact Bond for Early Childhood Education (Revised on November 17, 2015) The Utah High- Quality Preschool Initiative – Pilot and Longitudinal Research Project  https://ssir.org/articles/entry/pay_for_success_is_working_in_utah 19 Implementation Date. January 2015 OLO Memorandum Report 2018-5 Partnering for Family Success Cuyahoga County, Ohio Issue/Intervention. The country’s first county-level pay for success project, Partnering for Family Success, addresses the interrelated issues of homeless and foster care. The aim of the program is to reduce out-of-home foster care placements for children of homeless mothers in the county through a mix of housing assistance and behavioral health services. The program will provide 135 eligible homeless parents with children in foster care a combination of housing assistance and evidence-based behavioral health services, called Critical Time Intervention (CTI). In order to legally establish the project, Cuyahoga County had to pass legislation to create a new Social Impact Financing Fund to cover program costs and to authorize the county executive to negotiate the associated multi-year contract. Population. 135 homeless parents with children in foster care. Stakeholders. Government Cuyahoga County, Ohio Intermediary Enterprise Community Partners with support from Third Sector Capital Partners Service Provider FrontLine Service, Cuyahoga Metropolitan Housing Authority, Emerald Development and Economic Network Inc., Famicos Foundation Investors The Reinvestment Fund, George Gund Fund, Nonprofit Finance Fund, Cleveland Foundation, Sisters of Charity Foundation of Cleveland Evaluator Center on Urban Poverty and Community Development at Case Western Reserve University Outcome Goals. Payments will be made to investors if children in the program have a reduction in out- of-home placements compared to the County’s current publicly provided services. Program Cost & Pay Out. The first pay for success project without private investment capital, the total investment for this project is $4 million dollars, including $1.575 million from The Reinvestment Fund (a community development financial institution), $325,000 from the Nonprofit Finance Fund, and the remaining $2.1 million is spread among the George Gund Foundation, the Cleveland Foundation, and the Sisters of Charity Foundation of Cleveland. The funding was provided through a combination of recoverable grants and loans, with interest rates ranging from 2-5%. The funders of the project will be repaid $75 per reduced day of foster care, which is the estimated cost to the county of providing such care, up to a maximum of $5 million. If the reduction is 25% the number of days that children of homeless mothers spend in foster care, the Cuyahoga County government will return investors the entire $4 million investment, plus a nominal interest payment. At a 50% reduction, success payments will be capped at $5.5 million (with an estimated $3.5 million savings to the County). 20 OLO Memorandum Report 2018-5 Results. The final evaluation of this project will be conducted at the end of the project’s fifth year (2019) and its results will determine success payments. The evaluation also includes an interim two year implementation study to determine which aspects of the program (such as housing stability, home visitation, or family meetings) are contributing to fewer out-of-home placement days. In July 2017, the interim report was released and found positive results that indicated increased family stability in the treatment group.:  Treatment clients were less likely than control clients to receive rapid re-housing, to enter emergency shelter, and to receive other types of homeless services as compared to treatment group clients;  Over half of the clients in both groups received SNAP assistance prior to enrollment, while after entry, nearly all clients in the treatment group (92%) received SNAP assistance (compared to 80% of the control group);  After involvement with PFS, the proportion of clients in the treatment group receiving TANF increased slightly, while the control group remained about the same;  After entry, no clients in the treatment group had substantiated maltreatment cases with DCFS, compared to 6% in the control group; and  Within the six months after entering PFS, the treatment group had less maltreatment reported (19%) compared to the control group (23%). Sources  https://www.thirdsectorcap.org/wp-content/uploads/2016/02/Final-Cuyahoga-Partnering-for-Family- Success-Program-Lessons-Learned-Report.pdf  http://msass.case.edu/new-report-on-cuyahoga-partnering-for-family-success-process-evaluation/  https://www.thirdsectorcap.org/wp-content/uploads/2015/03/Pay_for_Success_in_Child-Welfare_- _A_Case_Study.pdf  https://ssir.org/up_for_debate/article/the_payoff_of_pay_for_success  https://rockefeller.dartmouth.edu/sites/rockefeller.drupalmulti- prod.dartmouth.edu/files/sibsprsfinal.pdf  http://povertycenter.case.edu/wp-content/uploads/2017/07/Briefly_Stated_No_17- 01_Cuyahoga_Partnering_for_Family_Success.pdf 21 OLO Memorandum Report 2018-5 The Massachusetts Juvenile Justice Pay For Success Initiative Implementation Date. October 2014 Issue/Intervention. Program aimed at reduced recidivism among very high-risk youth aged 17-23 who are in the probation system or exiting the juvenile justice system. Delivered over a four-year period, the program includes intensive street outreach and targeted life skills, education, and employment programming. The program is administered by the nonprofit Roca, whose intervention model was built on seven years of evidence-based practices. In order to create the pay for success project, the Massachusetts Legislature authorized the Secretary of Administration and Finance to enter into PFS contracts, with up to $50 million in success payments backed by the full faith and credit of the state government. Population. 929 Youth. Stakeholders. Outcome Goals. Reduction of recidivism by 40% and increases in job readiness and employment. Program Cost & Pay Out. The total investment for the program is $21.76 million. Goldman Sachs is the profit-seeking investor, investing $9 million. The other investors include: The Kresge Foundation ($1.5 million junior loan), Living Cities ($1.5 million junior loan), the Laura and John Arnold Foundation ($3.7 million grant), New Profit ($2 million grant), and The Boston Foundation ($300,000 grant). In order to receive any payment, the program must reduce recidivism by 5.2%. If the project achieves its target impact (40%), funders will be repaid using success payments from the Commonwealth’s Social Innovation Financing Trust Fund as follows:  The Goldman Sachs Social Impact Fund will be repaid its principal funding and a base annual interest rate of 5.0%;  The Kresge Foundation and Living Cities will be repaid their principal funding and a base annual interest rate of 2.0%; and  Roca and Third Sector will be paid their deferred service fees. At higher levels of impact, Roca will receive additional payments up to $1 million, Goldman Sachs will be paid up to an additional $1 million, and the Kresge Foundation and Living Cities will each receive up to an additional $300,000. If Roca achieves a 70% reduction in recidivism, the payout will be capped at $27 million. The other payments for the program include: Government Massachusetts Intermediary Third Sector Capital Partners, Inc. through Youth Services, Inc. Service Provider Roca Inc. Investors Goldman Sachs, Kresge Foundation, Living Cities, Laura and John Arnold Foundation, New Profit, Inc., Boston Foundation Evaluator Sibalytics LLC and the Urban Institute 22 OLO Memorandum Report 2018-5  Increases in job readiness - $789 for every participant who engages a specified number of times with a Roca youth worker; and  Increases in employment - $750 for every participant in each quarter that a Roca participant is employed compared to similar youth who are not in the program. Result. The program model is four years long, so first outcome payments are not expected until 2018. However, Roca reports that internal tracking shows that performance is meeting projections for participation at the two year mark – overall retention is at 73% (projected 70%), attrition to incarceration is almost at projection (19 vs. 17), and overall attrition was lower than expected (74 vs. 78). Sources  http://www.goldmansachs.com/our-thinking/trends-in-our-business/massachusetts-social-impact- bond/MA-juvenile-justice-pay-for-success-initiative.pdf  https://ssir.org/up_for_debate/article/the_payoff_of_pay_for_success  http://rocainc.org/work/pay-for-success/  https://www.instituteforchildsuccess.org/publication/pay-success-us-summaries-financed-projects-april- 2016/ 23