| Notes |
Delivering Public Service for the Future
A focus on results
“Pay for Success” (PFS) deals provide
a new route for state and local
governments to tackle entrenched
societal problems and achieve results.
But to be successful transformational
tools, they require government entities
to reframe their thinking to focus on
three priority areas: valuing outcomes
and budgeting for results, procuring
for results, and rigorously measuring
performance and outcomes.
Pay for Success projects require a fundamental
shift in the partnership between government and
the private sector. As transformational tools, they
require governments to reset their thinking toward
paying for results, while allowing the innovation
and flexibility needed to achieve those results.
Contracts between the government and service
providers need to acknowledge this new flexibility,
but must also guard against potential unintended
consequences—explaining why, in most PFS
agreements, a rigorous independent evaluation
plays an essential role in determining if and when
desired outcomes have been achieved.
Pay for success (PFS) financing, also
known as Social Impact Bonds (SIBs),
is a new concept in which private
investment supports the delivery of
preventative or rehabilitative services
that are more effective and save
the government money, using those
savings to repay the investment.
PFS is attractive because it has
the potential to finance innovative
evidence-based services and provide
that taxpayer dollars will only be
spent once the desired outcomes
have been achieved. In this article,
we draw an important distinction
between financing (raising capital)
and funding (budgeting for results).
In the roughly four years since “Pay
for Success” (PFS) was introduced
to the domestic US market, PFS
deals worth over $70 million have
been launched. States, counties, and
municipalities across the country
are actively exploring these deals
as a new way of merging evidence
and capital to address critical social
needs.
While funding PFS initiatives raises
both challenges and opportunities,
the extraordinary growth of PFS
reflects increasing realization of its
underlying promise: A promise rooted
in the focus on government paying
for the results and addressing critical
social problems in innovative ways.
Any governmental organization
considering a PFS project must
answer the related questions of
how to define savings and how
to capture them. This requires
governments to clearly identify, and
assign a value to, the outcomes
sought—with all parties involved in
the deal embracing the requirements
for rigorous measurement of those
outcomes. Governments must
also understand and assess the
risks associated with these new
investments in order to attract
philanthropic and other impact
investors to this market.
Three significant challenges must
be overcome before state and local
governments can realize the promise
of PFS deals. First is attracting
investment through the valuation of
outcomes, and budgeting for results.
PFS is predicated on the government’s
ability to make payments at some
future date once an agreed-upon
set of outcomes has been achieved.
In many cases, the amount that the
government is willing (or able) to pay
must be grounded in savings derived
from avoided costs. This calculation
becomes much more complex and
will typically mean leveraging funds
across multiple program silos and,
often, multiple levels of government.
The second challenge is that
government procurement practices
are not generally designed to
facilitate this innovative approach in
which the government is contracting
for an outcome, not a prescribed
service. Service providers and
investors need to be given a fair
amount of latitude with how to
achieve the outcomes while ensuring
adequate safeguards to protect
vulnerable populations. Traditional
procurements are designed to
minimize risk and are often geared
toward administrative compliance
over innovation.
The third challenge is how the
government thinks about and
supports an outcomes-based model
that requires rigorous measurement
to validate results. While the
government agency does not need
to conduct the measurement
directly, it must be able to articulate
the outcomes it seeks to achieve,
and recognize how this model
differs from the ways in which
other programs are measured. This
insight should not be restricted to
the specific department or agency
servicing the target population, but
also understood in the budget office
and the procurement office, as well
as at the highest executive levels.
1
Any governmental organization considering a PFS
project must answer the related questions of how to
define savings and how to capture them.
Challenge 1: Valuation of outcomes & budgeting for results
PFS financing offers the government a way
to attract external capital to implement new
approaches to address critical issues, while
generating savings and transferring risk in
the process. In this model, external investors
provide working capital to finance service
delivery, with their returns contingent upon
specific outcomes being achieved. While the
government is offloading some performance
risk to the investors, it must still set aside
the funds that, in the future, will have to be
used to pay back the investors. Identifying
where this money will come from is a critical
step in evaluating the feasibility of a PFS
contract.
Up to now, in exchange for public
recognition and to establish early footholds
in a potentially large new market, the
small number of commercial investors in
PFS projects has been willing to accept
higher levels of risk than would typically be
accepted in commercial markets. However,
as more states and other jurisdictions
enter the market, competition for these
risk-tolerant dollars is likely to increase.
By moving now to address and mitigate
investor risk, governments will enhance
their attractiveness for new and expanded
investment.
Appropriation and execution risk are, by
far, the two greatest sources of concern for
commercial investors in the PFS market. The
former can be addressed through budget,
contracts, and appropriations language to
ensure the availability of funds; the latter
through identification and selection of
evidence-based practices backed by strong
service delivery and effective use of data
and analytics. More broadly, meeting the
complex needs of low-income populations
requires new ways of thinking about case
management, rapid-cycle evaluation and
performance optimization. Traditional
methods of isolating interventions and
measuring impact are not likely to achieve
the high levels of performance required in
many PFS initiatives.
Accenture envisions an environment
in which data is used dynamically to
accomplish three key goals:
1. To enable case management designed
for the individual and family
2. To facilitate low-cost and timely
independent evaluations using
administrative data, and
3. To optimize performance within and
across providers.
In the first case, dynamic case management
can help target the most appropriate suite
of services to the needs of the individual
and family at the time they are needed.
Too often in today’s environment, services
are delivered based upon silo-based
decision-making that can be uncoordinated
or leave critical gaps in service. The
second recognizes that while independent
evaluation is a key component of any
PFS project, independent evaluation can
be expensive and is often not designed
to inform ongoing operations. Using
administrative data and rapid-cycle
evaluation techniques can help to achieve
both goals and inform all parties to the
deal. Finally, performance within and across
providers tends to follow a bell-curve with
a small number at either end. Having the
ability to harness and use granular data
can help managers identify best practices
and techniques at the high end and push
those practices into the middle, while
simultaneously culling those at the low
end. Optimizing performance drives down
the cost of delivery while delivering better
results.
2
By moving now to address
and mitigate investor risk,
governments will enhance their
attractiveness for new and
expanded investment.
Recognizing the benefits—
and the challenges they can
create
PFS can only work when a program
generates “monetizable” benefits that can
be set aside for payment at a later date.
These savings and how they are defined are
critical to understand whether a PFS deal
structure is warranted.
There are generally three ways to think
about savings in the context of PFS:
• Budgetary savings: A reduction from
costs that would have been incurred
in the absence of the program. These
savings typically stem from reductions
in anticipated spending from uncapped
program accounts (often referred to as
mandatory or entitlement programs).
• Productivity savings: A reduction in
the costs of capped programs in which
there may be a waiting list or insufficient
funds to serve the entire population. In
this case, reducing the cost per outcome
allows more people to be served using
the same level of funding.
• Social or ancillary benefits: Benefits
created from a re-oriented system,
which are either attenuated or difficult
to otherwise quantify and may occur
over extended time-periods.
Increasing the complexity of the calculation
is the fact that savings may accrue to
agencies and/or programs within or across
levels of government. Shaun Donovan,
the Director of the Federal Office of
Management and Budget and former
Secretary of the Department of Housing
and Urban Development, referred to this as
the “wrong pockets” problem.
An investment made by one agency can
yield savings in another. Consider a county
trying to reduce homelessness to reduce
emergency room visits. The costs are borne
by the county. The savings accrue to the
Medicaid program at the state and federal
level or perhaps to a third-party provider.
Although there are several potential
solutions to this situation, most of
them depend on accessing savings from
the entity to which they accrue—and
sometimes the ability to do that is
constrained by regulation or law.
Recognizing the problem, in 2013, the
US Department of the Treasury proposed
setting up a “Pay for Success Incentive
Fund.” One of its primary purposes would
be to enable states and local governments
to access federal funds when there are
federal savings. A similar fund, which has
now been introduced through bi-partisan
legislation in both chambers of Congress,
will make US$300 million available to
states and local governments for PFS
projects that target savings across multiple
jurisdictions.
In a related initiative, the US Department
of Health and Human Services recently
launched a project to study potential
applications of PFS within the Medicaid
program. While significant efforts are being
made to expand the use of evidence-based
practices in the healthcare arena, accessing
funds within the Medicaid program to pay
for preventative services outside of direct
care delivery remains a challenge. The
results of this study (expected during the
coming year) may help clarify this issue.
How to value the outcomes?
One of the most challenging questions
in PFS contracts is how to establish a
monetary value for an outcome. PFS is
based on the concept in which private
investment supports the delivery of
preventative or rehabilitative services
that save the government money, using
those savings in the future to repay the
investment. However, as discussed (see
“wrong pockets” problem above), savings
often accrue over long time-periods and
may accrue to multiple jurisdictions and
program areas.
In addition to identifying how to capture
savings from multiple agencies, a critical
part of defining the value of outcomes
is determining whether to count other,
often less direct, social savings or longerterm savings that result from the positive
outcome. For example, provision of
maternal and child health services can
often have direct positive outcomes, such
as cutting healthcare costs by reducing
the incidence of low birthweight children.
These services can also have longer-term
impacts on child learning and achievement
which could translate to lower needs for
special education classes in K-12, improved
graduation rates and, later on, higher
earnings. But attribution of these longerterm impacts to the maternal and child
health services can be difficult over time,
and the time periods for achieving these
savings may be beyond the patience of
most investors.
To consider another example, reducing
recidivism can generate direct cost savings
for the prison system. However, if this
outcome has been achieved by increasing
community involvement and employment,
there will likely be other positive effects
such as decreased dependence on social
benefit programs, reduced criminal justice
costs and, potentially, increased revenue
from income tax—all alongside the broader
societal benefits.
In most PFS deals, proximity, attribution
and quantification are the three key
elements in valuation of outcomes. Given
this, as part of its valuation government
may seek to identify “indicators” of positive
outcomes that can be used as a proxy for
these longer-term savings.
3
PFS can only work when a program generates “monetizable”
benefits that can be set aside for payment at a later date. These
savings and how they are defined are critical to understand
whether a PFS deal structure is warranted.
Risk and reward
Having determined the amount saved by
the government (either actual or predicted),
the next step is to decide what portion of
the anticipated savings the service provider
or investor should receive.
There are three primary benefits to
engaging private entities and promising
future returns based on the outcomes
achieved through a PFS contract:
1. There are a very limited number of
“proven practices.” Evidence-based
practices are those that have been
tested and rigorously evaluated over
a period of time and within varied
environments. These evaluations
typically show that the program works
under some conditions, but fidelity
to the model is critical to achieving
future success. PFS offers a method to
further test these models as well as to
help build the evidence base of other
innovative models.
2. Basing payments made to outside
investors on service providers achieving
designated outcomes, rather than
on activities and outputs, creates
accountability for results, which can
introduce new incentives for higher
levels of performance.
3. Basing payments on outcomes, and
reducing prescriptive requirements often
found in service provider agreements,
should encourage program and
management innovation.
Along with these benefits, there is a need
to balance the return on the investment
with the risk taken and savings achieved. If
the risk of failure is small, the government
may not be willing to pay a premium for
the program goals being successfully
achieved. If the risk of failure is great, a
non-governmental entity may not be willing
to assume the risk without the promise
of a large reward. By way of comparison,
venture capital firms invest in multiple
companies because they expect no more
than 20 percent to be successful, and
those that are must pay back many times
the original investment to make the entire
investment cycle worthwhile. In most PFS
projects, philanthropic capital has played an
important role to limit other investor risk by
taking a first loss or subordinated position.
Accompanying any of the above strategies
may be PFS legislation clearly delineating
the level of protection investors can
expect from the government where these
types of contracts are concerned, and the
mechanism the government will use to set
aside and hold the benefits.
“Set aside and hold” problem
PFS contracts are usually multi-year
agreements in which agencies commit to
making a payment at a future date when
certain outcomes have been achieved.
However, most state and local governments
are unable to commit future resources
without specific budgetary authority.
This situation raises questions around how
sufficient funds can be set aside to pay
for the results when they occur—without
impacting current obligations. Investors
may well be unwilling to come to the
table if there is any risk of funds not being
available. Or, at a minimum, they will
expect to be compensated for this risk
through an increased interest rate.
Some states have addressed appropriation
risk by setting aside funds on an annual
basis; others have passed legislation to
either establish a fund, or the availability
of funds, once targeted results have been
achieved. By setting aside a pro-rated
portion of the expected payments on an
annual basis, pressure on current budgets
can be eased, but this approach may
increase investor risk.
Some contractual language can also
contribute to appropriation risk. Wellintentioned clauses in traditional
government contracts are designed to
protect the government against unforeseen
circumstances or poor performance by the
contractor. See “Attention to terms and
conditions” below for more information on
standard government contract clauses that
can pose challenges in a PFS deal.
Appropriation risk is one of the two key
risks associated with any PFS deal. As noted
earlier, execution risk is the second. Both
the government and investors can guard
against execution risk by employing a
number of tools, including:
• A thorough review of the evidence
supporting provider claims and due
diligence over past performance
• Identification and implementation of
monitoring tools that allow the provider,
as well as the government and investors,
to see progress against defined
milestones, and
• Risk mitigation and remediation plans.
An active flow of data between providers,
project managers, the government, and the
independent evaluator can help investors
gain confidence that results are being
achieved.
4
An active flow of data between providers, project managers, the
government, and the independent evaluator can help investors gain
confidence that results are being achieved
Where is the money to pay
for outcomes?
In this complex environment, funding for
PFS programs can come from three areas:
1. Out of current revenues. Because
one legislature usually cannot bind the
budgetary actions of another, investors
would have to rely on the commitment
of successive legislatures throughout
the term of the deal. This approach
bears the highest risk for investors.
2. Out of the costs avoided as a result
of the program. The government
would need to set up a mechanism to
capture the avoided costs and make
them available to pay the investors.
Although this approach would give rise
to the “wrong pockets” problem, as well
as facing the risk of a future legislature
changing its mind, at minimum, there is
a dedicated source of funding.
3. Through a dedicated fund. This could
be funded from a) bond proceeds, b)
deposits of captured avoided costs, or c)
annual deposits from general revenues.
Any such dedicated fund would be
able to make commitments that span
legislatures; the risk to investors would
equal the degree to which the fund had
sufficient resources to pay current and
likely future obligations.
Many PFS projects are designed to
capture future savings in the form of cost
avoidance, and payments are predicated
on this assumption. In these cases, the
government may be able to create new
budget authority based on the assumption
that these future savings will be used to
reimburse the new authority. However, due
to the multi-year nature of PFS contracts,
this approach can present challenges since
the budget authority used to enter into the
contract is a current obligation while the
savings will occur in a future budget.
Dedicated funds refer to a new
appropriation or budget authority that can
be accessed for PFS projects. These funds
can include flow-down funds from other
levels of government (e.g., Federal and/or
state funds) or new budgetary authority.
Most state or local governments require
funds to be available before entering into
a contract—whether PFS or some other
program. Some states cannot enter into
a contract to pay out of future funds, but
may be able to enter into a contract that
is contingent upon future appropriations.
By increasing investor risk, this type of
contingency may also increase the cost
of funding, or deter some investors from
participating altogether.
Public-private partnership authority or
bonding authorities are potential models
that can be used to help inform this
discussion.
5
Challenge 2: Procuring for results
PFS transactions require new ways of
thinking about procurement. In these
transactions, the government is laying out
little or no money upfront, with private
capital being used to finance start-up
activities and operations. The government
commits to pay if and when certain
outcomes are achieved.
In this sense, PFS transactions are more
akin to the structure of public-private
partnerships (P3), typically associated
with infrastructure projects rather than
traditional government procurements.
Over 30 states have passed P3 legislation,
generally granting increased procurement
flexibility that provides opportunities for
private sector innovation, while enabling
the government to secure funding for
future payments. Several states have either
introduced or enacted PFS legislation
providing similar assurances.
Also central to any PFS transaction is
the fact that the government retains
responsibility for protecting the population
that is receiving the service, as well as for
ensuring that perverse incentives are not
introduced. For example, simply setting
an outcome of increased employment can
incentivize service providers to focus on
those individuals that are easiest to serve,
but are in fact in least need of service. Such
situations can be avoided through clear
definition of the populations to be served,
the outcomes to be achieved, carefully
matching incentives to desired outcomes,
and the measurement methodology that
will be used.
6
Attention to terms &
conditions
Traditional government procurement is a
prescriptive process that outlines exactly
what the government is buying and is
precise about the price that it is willing
to pay for the service. The terms and
conditions of the contract reflect that
level of specificity and are designed to
minimize the risk to the government. By
contrast, PFS is based on a more “handsoff” approach in which the government
is contracting for an outcome, not a
prescribed service. How that outcome
is achieved is predominantly left to the
innovation and diligence of the providers
and investors.
In procuring a PFS project, procurement
and legal departments must understand
that the nature of the risks shifted to the
service providers and outside investors
necessitate some modifications to
terms and conditions typically found in
government contracts. For example:
• Termination for convenience: In a
PFS project, the government will not
incur payment obligation without
demonstrable success by the service
providers. This means there could be
no protection for invested costs if the
program were terminated early and
there was no chance to prove success.
Termination-for-convenience clauses
are usually accompanied by the ability
of the provider to claim costs that have
been incurred to date. However, this
would mean that the investors, services
providers, intermediaries and other
participants would need to account for
and justify any costs claimed under this
provision which could reduce flexibility
and add administrative burdens to these
contracts.
• Unlimited liability: The service
providers and investors take on
delivery risk in PFS contracts. Since the
government does not pay unless and
until specific outcomes are achieved,
the service providers cannot be held
liable for whether their services worked
or not—that is the risk they are taking.
However, the state still retains the
responsibility to protect the population
being served and should include higher
standards of care to prevent willful
misconduct, gross negligence and fraud.
• Termination for cause: In a PFS
project, there is no breach of contract
if the services fail to deliver the desired
results. However, there may be other
causes—such as compliance with laws
or security breaches—that could trigger
termination and which can be addressed
in the contract terms and conditions.
Should a project need to be stopped
prior to the end of the project term, it is
critical to include a wind-down clause
and process through which the people
receiving the services are supported
while the services cease.
Governments have taken different
approaches to procuring these kinds of
services. In Massachusetts, the government
separately procured the intermediary and
the service provider and brought them
together for the project. While other states,
like Michigan, have set out the outcomes
they seek to achieve and have asked the
market to put together their own team to
deliver on the PFS contract.
Because they rely on the efforts of private
or non-government entities, PFS contracts
can be seen as part of an evolving and
growing group of performance-oriented
forms of public/private cooperation
(including performance contracting and P3).
7
Because they rely on the efforts of private or non-government
entities, PFS contracts can be seen as part of an evolving and
growing group of performance-oriented forms of public/private
cooperation.
Challenge 3: Validating results through rigorous measurement
In Pay for Success, we move from
measurement of inputs and outputs to
measurement of outcomes. Measurement
of outputs does not require a true
counterfactual (a scientifically rigorous
determination of what would have
happened in the absence of the program),
although comparative cost and quality are
separately important. However, with the
exception of certain narrow circumstances,
measurement of program results—
outcomes—does call for an estimate of
counterfactuals.
How can a counterfactual
be estimated?
Since PFS transactions came into the
market three years ago, randomized
control trials (RCTs) have been considered
the “gold standard” by which all deals
should be evaluated. All other things
being equal, RCTs are the most likely
evaluation methodology to achieve
strong causal validity, that is, they are
best at determining the extent to which
causality can be established between
the intervention and the outcome (and/
or impact) of interest. Historically, RCTs
have been time consuming and costly
due to the need to collect supporting
data, often through extensive surveys.
However, an emerging trend is to use
large administrative data sets—data that
is collected for program administration or
other operational purposes—to significantly
lower the cost and improve the timeliness
of low-cost RCTs. These low-cost RCTs
provide a strong method for evaluation,
while allowing for more widespread use,
more transparency, and the introduction of
rapid cycle evaluations that can facilitate
feedback for improved performance
management. Low-cost RCTs address the
cost barrier of traditional randomized trials
by performing the assessment on existing
programs, and evaluating outcomes with
administrative data already in collection for
the intervention.
Whether an individual (“lottery”) or a site
randomization (when there is a concern
that having program and non-program
groups in close proximity, perhaps in
the same building such as a school, may
result in treatment spillover), common
randomization designs include:
• Single program group randomizations—
the most basic and common, randomly
assign to either receive, or not receive, a
single treatment.
• Program implementation
randomizations—used to avoid the
problem of denying services to the
control group by taking advantage of
planned variations in when a treatment is
implemented among different participants
or in different places. They include holdback, rolling (“repeated randomizations”),
phase-in, and rotation designs.
• Different treatment randomizations—
used to avoid the problem of denying
services to those eligible by randomly
assigning to one of multiple program
groups. They include dose-response
(or “behavioral response function”),
factorial, two-stage, and randomization
across treatment designs.
• Different provider randomizations—seek
to assess the comparative effectiveness
of a particular program across different
providers by randomizing program
recipients to providers who may vary
from each other in how they manage and
implement the program. They include
randomizations across different agencies
and to different offices or individual staff
members in the same agency.
8
The strength of an RCT in causal validity,
however, usually comes at a cost of
“generalizability”—the extent to which the
evaluation findings can be applied beyond
the specific program sites studied. Given
this and the other trade-offs involved
in running randomized experiments,
alternate approaches, when carefully and
appropriately applied, can also be used
to determine if the program seems to be
working. In order of their utility for these
purposes, these approaches include:
• Ipsative (before and after) designs
that compare participating individuals
before the intervention with themselves
after the intervention. This approach
includes simple pre/post comparison
studies that are one-time comparisons,
and interrupted time-series studies that
make repeated comparisons over an
extended period of time.
• Instrumental variable designs that
employ a factor outside of the causal
chain to create program and nonprogram groups. For example, regression
discontinuity designs (RDDs) take
advantage of eligibility thresholds
(cutoff scores) that assign individuals
to program and non-program groups
outside the control of the participants.
• Comparison group studies that
compare the program group to an
“equivalent” non-program group
selected on the basis of its at least
9
putative, pre-intervention similarity to
the program group. They include generic
control, matched comparison group,
propensity score matching (PSM), and
difference-in-differences studies.
The key to deciding the most appropriate
method for evaluation depends upon
several factors including the size of
the treatment population and ability to
randomize within the environment, the
availability of a comparable control group,
the availability of historic data and other
factors. Ultimately, the decision rests with
the investors and the government to agree
on a set of measures and a methodology
that can be confidently relied upon to
trigger payments when results are achieved.
Ultimately, the decision rests with the investors and the
government to agree on a set of measures and a methodology
that can be confidently relied upon to trigger payments when
results are achieved.
Looking ahead—the future for PFS?
For more information,
please contact:
As Judith Rodin, president of
the Rockefeller Foundation, has
pointed out1
, PFS contracts have
real potential “…to substantially
transform the social sector, support
poor and vulnerable communities,
and create new financial flows for
human service delivery by offering
an innovative way to scale what
works and break the cyclical need
for crisis-driven services.”
Gary Glickman
Managing Director, Pay for Success Practice
gary.glickman@accenture.com
Douglas Besharov
Norman and Florence Brody Professor
School of Public Policy
University of Maryland
besharov@umd.edu
As such, these deals represent an
exciting field of innovative finance.
However, they need to be addressed
thoughtfully and in a structured
way. As more state and local
governments across the US move to
explore what PFS contracts have to
offer, there is still much work to be
done. As discussed in this paper, real
thought needs to be given to how
best to address the three principal
challenges arising from this new
method of collaboration between
state and private sectors: valuing
outcomes and ensuring future
payment for investors, adapting
government procurement practice,
and establishing rigorous processes
for measuring and validating
outcomes. The introduction of
consistent PFS legislation nationwide
will be a vital step toward realization
of the full potential of PFS, with
practical guidance being issued on
how to draft and implement PFS
contracts.
10
1. http://www.rockefellerfoundation.org/uploads/files/655fab01-83b9-49eb-b856-a1f61bc9e6ca-small.pdf
About Accenture
Accenture is a global management consulting, technology services
and outsourcing company, with more than 323,000 people serving
clients in more than 120 countries. Combining unparalleled
experience, comprehensive capabilities across all industries and
business functions, and extensive research on the world’s most
successful companies, Accenture collaborates with clients to help
them become high-performance businesses and governments. The
company generated net revenues of US$30.0 billion for the fiscal
year ended Aug. 31, 2014. Its home page is www.accenture.com.
Copyright © 2015 Accenture
All rights reserved.
Accenture, its logo, and
High Performance Delivered
are trademarks of Accenture. |