A Focus On Results - Accenture on Pay For Success

Delivering Public Service for the Future A focus on results “Pay for Success” (PFS) deals provide a new route for state and local governments to tackle entrenched societal problems and achieve results. But to be successful transformational tools, they require government entities to reframe their thinking to focus on three priority areas: valuing outcomes and budgeting for results, procuring for results, and rigorously measuring performance and outcomes. Pay for Success projects require a fundamental shift in the partnership between government and the private sector. As transformational tools, they require governments to reset their thinking toward paying for results, while allowing the innovation and flexibility needed to achieve those results. Contracts between the government and service providers need to acknowledge this new flexibility, but must also guard against potential unintended consequences—explaining why, in most PFS agreements, a rigorous independent evaluation plays an essential role in determining if and when desired outcomes have been achieved. Pay for success (PFS) financing, also known as Social Impact Bonds (SIBs), is a new concept in which private investment supports the delivery of preventative or rehabilitative services that are more effective and save the government money, using those savings to repay the investment. PFS is attractive because it has the potential to finance innovative evidence-based services and provide that taxpayer dollars will only be spent once the desired outcomes have been achieved. In this article, we draw an important distinction between financing (raising capital) and funding (budgeting for results). In the roughly four years since “Pay for Success” (PFS) was introduced to the domestic US market, PFS deals worth over $70 million have been launched. States, counties, and municipalities across the country are actively exploring these deals as a new way of merging evidence and capital to address critical social needs. While funding PFS initiatives raises both challenges and opportunities, the extraordinary growth of PFS reflects increasing realization of its underlying promise: A promise rooted in the focus on government paying for the results and addressing critical social problems in innovative ways. Any governmental organization considering a PFS project must answer the related questions of how to define savings and how to capture them. This requires governments to clearly identify, and assign a value to, the outcomes sought—with all parties involved in the deal embracing the requirements for rigorous measurement of those outcomes. Governments must also understand and assess the risks associated with these new investments in order to attract philanthropic and other impact investors to this market. Three significant challenges must be overcome before state and local governments can realize the promise of PFS deals. First is attracting investment through the valuation of outcomes, and budgeting for results. PFS is predicated on the government’s ability to make payments at some future date once an agreed-upon set of outcomes has been achieved. In many cases, the amount that the government is willing (or able) to pay must be grounded in savings derived from avoided costs. This calculation becomes much more complex and will typically mean leveraging funds across multiple program silos and, often, multiple levels of government. The second challenge is that government procurement practices are not generally designed to facilitate this innovative approach in which the government is contracting for an outcome, not a prescribed service. Service providers and investors need to be given a fair amount of latitude with how to achieve the outcomes while ensuring adequate safeguards to protect vulnerable populations. Traditional procurements are designed to minimize risk and are often geared toward administrative compliance over innovation. The third challenge is how the government thinks about and supports an outcomes-based model that requires rigorous measurement to validate results. While the government agency does not need to conduct the measurement directly, it must be able to articulate the outcomes it seeks to achieve, and recognize how this model differs from the ways in which other programs are measured. This insight should not be restricted to the specific department or agency servicing the target population, but also understood in the budget office and the procurement office, as well as at the highest executive levels. 1 Any governmental organization considering a PFS project must answer the related questions of how to define savings and how to capture them. Challenge 1: Valuation of outcomes & budgeting for results PFS financing offers the government a way to attract external capital to implement new approaches to address critical issues, while generating savings and transferring risk in the process. In this model, external investors provide working capital to finance service delivery, with their returns contingent upon specific outcomes being achieved. While the government is offloading some performance risk to the investors, it must still set aside the funds that, in the future, will have to be used to pay back the investors. Identifying where this money will come from is a critical step in evaluating the feasibility of a PFS contract. Up to now, in exchange for public recognition and to establish early footholds in a potentially large new market, the small number of commercial investors in PFS projects has been willing to accept higher levels of risk than would typically be accepted in commercial markets. However, as more states and other jurisdictions enter the market, competition for these risk-tolerant dollars is likely to increase. By moving now to address and mitigate investor risk, governments will enhance their attractiveness for new and expanded investment. Appropriation and execution risk are, by far, the two greatest sources of concern for commercial investors in the PFS market. The former can be addressed through budget, contracts, and appropriations language to ensure the availability of funds; the latter through identification and selection of evidence-based practices backed by strong service delivery and effective use of data and analytics. More broadly, meeting the complex needs of low-income populations requires new ways of thinking about case management, rapid-cycle evaluation and performance optimization. Traditional methods of isolating interventions and measuring impact are not likely to achieve the high levels of performance required in many PFS initiatives. Accenture envisions an environment in which data is used dynamically to accomplish three key goals: 1. To enable case management designed for the individual and family 2. To facilitate low-cost and timely independent evaluations using administrative data, and 3. To optimize performance within and across providers. In the first case, dynamic case management can help target the most appropriate suite of services to the needs of the individual and family at the time they are needed. Too often in today’s environment, services are delivered based upon silo-based decision-making that can be uncoordinated or leave critical gaps in service. The second recognizes that while independent evaluation is a key component of any PFS project, independent evaluation can be expensive and is often not designed to inform ongoing operations. Using administrative data and rapid-cycle evaluation techniques can help to achieve both goals and inform all parties to the deal. Finally, performance within and across providers tends to follow a bell-curve with a small number at either end. Having the ability to harness and use granular data can help managers identify best practices and techniques at the high end and push those practices into the middle, while simultaneously culling those at the low end. Optimizing performance drives down the cost of delivery while delivering better results. 2 By moving now to address and mitigate investor risk, governments will enhance their attractiveness for new and expanded investment. Recognizing the benefits— and the challenges they can create PFS can only work when a program generates “monetizable” benefits that can be set aside for payment at a later date. These savings and how they are defined are critical to understand whether a PFS deal structure is warranted. There are generally three ways to think about savings in the context of PFS: • Budgetary savings: A reduction from costs that would have been incurred in the absence of the program. These savings typically stem from reductions in anticipated spending from uncapped program accounts (often referred to as mandatory or entitlement programs). • Productivity savings: A reduction in the costs of capped programs in which there may be a waiting list or insufficient funds to serve the entire population. In this case, reducing the cost per outcome allows more people to be served using the same level of funding. • Social or ancillary benefits: Benefits created from a re-oriented system, which are either attenuated or difficult to otherwise quantify and may occur over extended time-periods. Increasing the complexity of the calculation is the fact that savings may accrue to agencies and/or programs within or across levels of government. Shaun Donovan, the Director of the Federal Office of Management and Budget and former Secretary of the Department of Housing and Urban Development, referred to this as the “wrong pockets” problem. An investment made by one agency can yield savings in another. Consider a county trying to reduce homelessness to reduce emergency room visits. The costs are borne by the county. The savings accrue to the Medicaid program at the state and federal level or perhaps to a third-party provider. Although there are several potential solutions to this situation, most of them depend on accessing savings from the entity to which they accrue—and sometimes the ability to do that is constrained by regulation or law. Recognizing the problem, in 2013, the US Department of the Treasury proposed setting up a “Pay for Success Incentive Fund.” One of its primary purposes would be to enable states and local governments to access federal funds when there are federal savings. A similar fund, which has now been introduced through bi-partisan legislation in both chambers of Congress, will make US$300 million available to states and local governments for PFS projects that target savings across multiple jurisdictions. In a related initiative, the US Department of Health and Human Services recently launched a project to study potential applications of PFS within the Medicaid program. While significant efforts are being made to expand the use of evidence-based practices in the healthcare arena, accessing funds within the Medicaid program to pay for preventative services outside of direct care delivery remains a challenge. The results of this study (expected during the coming year) may help clarify this issue. How to value the outcomes? One of the most challenging questions in PFS contracts is how to establish a monetary value for an outcome. PFS is based on the concept in which private investment supports the delivery of preventative or rehabilitative services that save the government money, using those savings in the future to repay the investment. However, as discussed (see “wrong pockets” problem above), savings often accrue over long time-periods and may accrue to multiple jurisdictions and program areas. In addition to identifying how to capture savings from multiple agencies, a critical part of defining the value of outcomes is determining whether to count other, often less direct, social savings or longerterm savings that result from the positive outcome. For example, provision of maternal and child health services can often have direct positive outcomes, such as cutting healthcare costs by reducing the incidence of low birthweight children. These services can also have longer-term impacts on child learning and achievement which could translate to lower needs for special education classes in K-12, improved graduation rates and, later on, higher earnings. But attribution of these longerterm impacts to the maternal and child health services can be difficult over time, and the time periods for achieving these savings may be beyond the patience of most investors. To consider another example, reducing recidivism can generate direct cost savings for the prison system. However, if this outcome has been achieved by increasing community involvement and employment, there will likely be other positive effects such as decreased dependence on social benefit programs, reduced criminal justice costs and, potentially, increased revenue from income tax—all alongside the broader societal benefits. In most PFS deals, proximity, attribution and quantification are the three key elements in valuation of outcomes. Given this, as part of its valuation government may seek to identify “indicators” of positive outcomes that can be used as a proxy for these longer-term savings. 3 PFS can only work when a program generates “monetizable” benefits that can be set aside for payment at a later date. These savings and how they are defined are critical to understand whether a PFS deal structure is warranted. Risk and reward Having determined the amount saved by the government (either actual or predicted), the next step is to decide what portion of the anticipated savings the service provider or investor should receive. There are three primary benefits to engaging private entities and promising future returns based on the outcomes achieved through a PFS contract: 1. There are a very limited number of “proven practices.” Evidence-based practices are those that have been tested and rigorously evaluated over a period of time and within varied environments. These evaluations typically show that the program works under some conditions, but fidelity to the model is critical to achieving future success. PFS offers a method to further test these models as well as to help build the evidence base of other innovative models. 2. Basing payments made to outside investors on service providers achieving designated outcomes, rather than on activities and outputs, creates accountability for results, which can introduce new incentives for higher levels of performance. 3. Basing payments on outcomes, and reducing prescriptive requirements often found in service provider agreements, should encourage program and management innovation. Along with these benefits, there is a need to balance the return on the investment with the risk taken and savings achieved. If the risk of failure is small, the government may not be willing to pay a premium for the program goals being successfully achieved. If the risk of failure is great, a non-governmental entity may not be willing to assume the risk without the promise of a large reward. By way of comparison, venture capital firms invest in multiple companies because they expect no more than 20 percent to be successful, and those that are must pay back many times the original investment to make the entire investment cycle worthwhile. In most PFS projects, philanthropic capital has played an important role to limit other investor risk by taking a first loss or subordinated position. Accompanying any of the above strategies may be PFS legislation clearly delineating the level of protection investors can expect from the government where these types of contracts are concerned, and the mechanism the government will use to set aside and hold the benefits. “Set aside and hold” problem PFS contracts are usually multi-year agreements in which agencies commit to making a payment at a future date when certain outcomes have been achieved. However, most state and local governments are unable to commit future resources without specific budgetary authority. This situation raises questions around how sufficient funds can be set aside to pay for the results when they occur—without impacting current obligations. Investors may well be unwilling to come to the table if there is any risk of funds not being available. Or, at a minimum, they will expect to be compensated for this risk through an increased interest rate. Some states have addressed appropriation risk by setting aside funds on an annual basis; others have passed legislation to either establish a fund, or the availability of funds, once targeted results have been achieved. By setting aside a pro-rated portion of the expected payments on an annual basis, pressure on current budgets can be eased, but this approach may increase investor risk. Some contractual language can also contribute to appropriation risk. Wellintentioned clauses in traditional government contracts are designed to protect the government against unforeseen circumstances or poor performance by the contractor. See “Attention to terms and conditions” below for more information on standard government contract clauses that can pose challenges in a PFS deal. Appropriation risk is one of the two key risks associated with any PFS deal. As noted earlier, execution risk is the second. Both the government and investors can guard against execution risk by employing a number of tools, including: • A thorough review of the evidence supporting provider claims and due diligence over past performance • Identification and implementation of monitoring tools that allow the provider, as well as the government and investors, to see progress against defined milestones, and • Risk mitigation and remediation plans. An active flow of data between providers, project managers, the government, and the independent evaluator can help investors gain confidence that results are being achieved. 4 An active flow of data between providers, project managers, the government, and the independent evaluator can help investors gain confidence that results are being achieved Where is the money to pay for outcomes? In this complex environment, funding for PFS programs can come from three areas: 1. Out of current revenues. Because one legislature usually cannot bind the budgetary actions of another, investors would have to rely on the commitment of successive legislatures throughout the term of the deal. This approach bears the highest risk for investors. 2. Out of the costs avoided as a result of the program. The government would need to set up a mechanism to capture the avoided costs and make them available to pay the investors. Although this approach would give rise to the “wrong pockets” problem, as well as facing the risk of a future legislature changing its mind, at minimum, there is a dedicated source of funding. 3. Through a dedicated fund. This could be funded from a) bond proceeds, b) deposits of captured avoided costs, or c) annual deposits from general revenues. Any such dedicated fund would be able to make commitments that span legislatures; the risk to investors would equal the degree to which the fund had sufficient resources to pay current and likely future obligations. Many PFS projects are designed to capture future savings in the form of cost avoidance, and payments are predicated on this assumption. In these cases, the government may be able to create new budget authority based on the assumption that these future savings will be used to reimburse the new authority. However, due to the multi-year nature of PFS contracts, this approach can present challenges since the budget authority used to enter into the contract is a current obligation while the savings will occur in a future budget. Dedicated funds refer to a new appropriation or budget authority that can be accessed for PFS projects. These funds can include flow-down funds from other levels of government (e.g., Federal and/or state funds) or new budgetary authority. Most state or local governments require funds to be available before entering into a contract—whether PFS or some other program. Some states cannot enter into a contract to pay out of future funds, but may be able to enter into a contract that is contingent upon future appropriations. By increasing investor risk, this type of contingency may also increase the cost of funding, or deter some investors from participating altogether. Public-private partnership authority or bonding authorities are potential models that can be used to help inform this discussion. 5 Challenge 2: Procuring for results PFS transactions require new ways of thinking about procurement. In these transactions, the government is laying out little or no money upfront, with private capital being used to finance start-up activities and operations. The government commits to pay if and when certain outcomes are achieved. In this sense, PFS transactions are more akin to the structure of public-private partnerships (P3), typically associated with infrastructure projects rather than traditional government procurements. Over 30 states have passed P3 legislation, generally granting increased procurement flexibility that provides opportunities for private sector innovation, while enabling the government to secure funding for future payments. Several states have either introduced or enacted PFS legislation providing similar assurances. Also central to any PFS transaction is the fact that the government retains responsibility for protecting the population that is receiving the service, as well as for ensuring that perverse incentives are not introduced. For example, simply setting an outcome of increased employment can incentivize service providers to focus on those individuals that are easiest to serve, but are in fact in least need of service. Such situations can be avoided through clear definition of the populations to be served, the outcomes to be achieved, carefully matching incentives to desired outcomes, and the measurement methodology that will be used. 6 Attention to terms & conditions Traditional government procurement is a prescriptive process that outlines exactly what the government is buying and is precise about the price that it is willing to pay for the service. The terms and conditions of the contract reflect that level of specificity and are designed to minimize the risk to the government. By contrast, PFS is based on a more “handsoff” approach in which the government is contracting for an outcome, not a prescribed service. How that outcome is achieved is predominantly left to the innovation and diligence of the providers and investors. In procuring a PFS project, procurement and legal departments must understand that the nature of the risks shifted to the service providers and outside investors necessitate some modifications to terms and conditions typically found in government contracts. For example: • Termination for convenience: In a PFS project, the government will not incur payment obligation without demonstrable success by the service providers. This means there could be no protection for invested costs if the program were terminated early and there was no chance to prove success. Termination-for-convenience clauses are usually accompanied by the ability of the provider to claim costs that have been incurred to date. However, this would mean that the investors, services providers, intermediaries and other participants would need to account for and justify any costs claimed under this provision which could reduce flexibility and add administrative burdens to these contracts. • Unlimited liability: The service providers and investors take on delivery risk in PFS contracts. Since the government does not pay unless and until specific outcomes are achieved, the service providers cannot be held liable for whether their services worked or not—that is the risk they are taking. However, the state still retains the responsibility to protect the population being served and should include higher standards of care to prevent willful misconduct, gross negligence and fraud. • Termination for cause: In a PFS project, there is no breach of contract if the services fail to deliver the desired results. However, there may be other causes—such as compliance with laws or security breaches—that could trigger termination and which can be addressed in the contract terms and conditions. Should a project need to be stopped prior to the end of the project term, it is critical to include a wind-down clause and process through which the people receiving the services are supported while the services cease. Governments have taken different approaches to procuring these kinds of services. In Massachusetts, the government separately procured the intermediary and the service provider and brought them together for the project. While other states, like Michigan, have set out the outcomes they seek to achieve and have asked the market to put together their own team to deliver on the PFS contract. Because they rely on the efforts of private or non-government entities, PFS contracts can be seen as part of an evolving and growing group of performance-oriented forms of public/private cooperation (including performance contracting and P3). 7 Because they rely on the efforts of private or non-government entities, PFS contracts can be seen as part of an evolving and growing group of performance-oriented forms of public/private cooperation. Challenge 3: Validating results through rigorous measurement In Pay for Success, we move from measurement of inputs and outputs to measurement of outcomes. Measurement of outputs does not require a true counterfactual (a scientifically rigorous determination of what would have happened in the absence of the program), although comparative cost and quality are separately important. However, with the exception of certain narrow circumstances, measurement of program results— outcomes—does call for an estimate of counterfactuals. How can a counterfactual be estimated? Since PFS transactions came into the market three years ago, randomized control trials (RCTs) have been considered the “gold standard” by which all deals should be evaluated. All other things being equal, RCTs are the most likely evaluation methodology to achieve strong causal validity, that is, they are best at determining the extent to which causality can be established between the intervention and the outcome (and/ or impact) of interest. Historically, RCTs have been time consuming and costly due to the need to collect supporting data, often through extensive surveys. However, an emerging trend is to use large administrative data sets—data that is collected for program administration or other operational purposes—to significantly lower the cost and improve the timeliness of low-cost RCTs. These low-cost RCTs provide a strong method for evaluation, while allowing for more widespread use, more transparency, and the introduction of rapid cycle evaluations that can facilitate feedback for improved performance management. Low-cost RCTs address the cost barrier of traditional randomized trials by performing the assessment on existing programs, and evaluating outcomes with administrative data already in collection for the intervention. Whether an individual (“lottery”) or a site randomization (when there is a concern that having program and non-program groups in close proximity, perhaps in the same building such as a school, may result in treatment spillover), common randomization designs include: • Single program group randomizations— the most basic and common, randomly assign to either receive, or not receive, a single treatment. • Program implementation randomizations—used to avoid the problem of denying services to the control group by taking advantage of planned variations in when a treatment is implemented among different participants or in different places. They include holdback, rolling (“repeated randomizations”), phase-in, and rotation designs. • Different treatment randomizations— used to avoid the problem of denying services to those eligible by randomly assigning to one of multiple program groups. They include dose-response (or “behavioral response function”), factorial, two-stage, and randomization across treatment designs. • Different provider randomizations—seek to assess the comparative effectiveness of a particular program across different providers by randomizing program recipients to providers who may vary from each other in how they manage and implement the program. They include randomizations across different agencies and to different offices or individual staff members in the same agency. 8 The strength of an RCT in causal validity, however, usually comes at a cost of “generalizability”—the extent to which the evaluation findings can be applied beyond the specific program sites studied. Given this and the other trade-offs involved in running randomized experiments, alternate approaches, when carefully and appropriately applied, can also be used to determine if the program seems to be working. In order of their utility for these purposes, these approaches include: • Ipsative (before and after) designs that compare participating individuals before the intervention with themselves after the intervention. This approach includes simple pre/post comparison studies that are one-time comparisons, and interrupted time-series studies that make repeated comparisons over an extended period of time. • Instrumental variable designs that employ a factor outside of the causal chain to create program and nonprogram groups. For example, regression discontinuity designs (RDDs) take advantage of eligibility thresholds (cutoff scores) that assign individuals to program and non-program groups outside the control of the participants. • Comparison group studies that compare the program group to an “equivalent” non-program group selected on the basis of its at least 9 putative, pre-intervention similarity to the program group. They include generic control, matched comparison group, propensity score matching (PSM), and difference-in-differences studies. The key to deciding the most appropriate method for evaluation depends upon several factors including the size of the treatment population and ability to randomize within the environment, the availability of a comparable control group, the availability of historic data and other factors. Ultimately, the decision rests with the investors and the government to agree on a set of measures and a methodology that can be confidently relied upon to trigger payments when results are achieved. Ultimately, the decision rests with the investors and the government to agree on a set of measures and a methodology that can be confidently relied upon to trigger payments when results are achieved. Looking ahead—the future for PFS? For more information, please contact: As Judith Rodin, president of the Rockefeller Foundation, has pointed out1 , PFS contracts have real potential “…to substantially transform the social sector, support poor and vulnerable communities, and create new financial flows for human service delivery by offering an innovative way to scale what works and break the cyclical need for crisis-driven services.” Gary Glickman Managing Director, Pay for Success Practice gary.glickman@accenture.com Douglas Besharov Norman and Florence Brody Professor School of Public Policy University of Maryland besharov@umd.edu As such, these deals represent an exciting field of innovative finance. However, they need to be addressed thoughtfully and in a structured way. As more state and local governments across the US move to explore what PFS contracts have to offer, there is still much work to be done. As discussed in this paper, real thought needs to be given to how best to address the three principal challenges arising from this new method of collaboration between state and private sectors: valuing outcomes and ensuring future payment for investors, adapting government procurement practice, and establishing rigorous processes for measuring and validating outcomes. The introduction of consistent PFS legislation nationwide will be a vital step toward realization of the full potential of PFS, with practical guidance being issued on how to draft and implement PFS contracts. 10 1. http://www.rockefellerfoundation.org/uploads/files/655fab01-83b9-49eb-b856-a1f61bc9e6ca-small.pdf About Accenture Accenture is a global management consulting, technology services and outsourcing company, with more than 323,000 people serving clients in more than 120 countries. Combining unparalleled experience, comprehensive capabilities across all industries and business functions, and extensive research on the world’s most successful companies, Accenture collaborates with clients to help them become high-performance businesses and governments. The company generated net revenues of US$30.0 billion for the fiscal year ended Aug. 31, 2014. Its home page is www.accenture.com. Copyright © 2015 Accenture All rights reserved. Accenture, its logo, and High Performance Delivered are trademarks of Accenture.