Promoted by HR4174 Foundations for Evidence-Based Policymaking Act of 2017
Promoted Results For America
Start Date 2018-00-00
Notes EXECUTIVE SUMMARY This brief examines the main implications of the Foundations for EvidenceBased Policymaking Act of 2018 (“Evidence Act”), including: (1) enhancing the infrastructure necessary for federal agencies to generate and use evidence in policy development; (2) improving the process for researchers to access government data, making it more open, streamlined, and secure, while also providing stronger privacy standards and legal protections; and (3) helping federal agencies develop a common understanding of how to articulate important policy questions, obtain relevant data, and better utilize existing data. (You can find a summary of the Evidence Act and the July 2019 OMB guidance to federal agencies of this law here and a section-bysection summary of the major provisions of the Evidence Act here). This brief also describes a possible forward-looking federal agenda that could help federal agencies and officials further harness the power of evidence and data to make federal policies and programs more effective and efficient. More specifically, a “2.0” version of the Evidence Act should require federal agencies to: • Set aside 1 percent of their program funds for program evaluations, including rigorous and independent evaluations whenever feasible; and • Prioritize rigorous evidence of effectiveness when allocating funds in their federal discretionary competitive and noncompetitive grant programs. 1 THE PROMISE OF THE FOUNDATIONS FOR EVIDENCE-BASED POLICYMAKING ACT AND PROPOSED NEXT STEPS 1 Evidence Act Brief 2 When combined with the gains achieved by the Evidence Act, these two recommendations would substantially improve the ability of government to invest in what works and improve the lives of millions of Americans. I. The Promise of the Evidence Act The principle behind evidence-based policymaking is simple: government decisions should be based on rigorous evidence and data about what works. Policymakers should start each decision-making process by seeking the best evidence and data available, then use that information to adopt policies and direct funding in a way that achieves better results for the American people. Against the backdrop of a $4 trillion federal budget, the implications of evidence-based policy are massive. Directing federal dollars to the most effective and efficient policies and programs can dramatically improve the well-being of communities across the country. But, this kind of sea change requires a change not just in how federal agencies do their work, but also in their will to repurpose dollars toward programs and practices proven to work. The past few years have seen dramatic shifts towards better incorporation of data and evidence in policymaking processes. Since 2015, Congress has passed landmark bipartisan legislation that is helping to identify and invest in what works in K-12 education (Every Student Succeeds Act), foster care (Family First Prevention Services Act), juvenile justice (Juvenile Justice Reform Act), opioid prevention, treatment, and care (SUPPORT for Patients and Communities Act), and federal agency operations (Foundations for Evidence-Based Policymaking Act, featured in this brief). The Foundations of Evidence-Based Policymaking Act (“Evidence Act”) was based in large part on the recommendations of the Commission on Evidence-Based Policymaking (CEP), which was created by a federal law championed by former U.S. House Speaker Paul Ryan (R-WI) and U.S. Senator Patty Murray (D-WA) and enacted in 2016 to study how government could better use data and evidence to improve federal programs. The Commission released a report in September 2017 that advanced 22 recommendations to foster the building and use of data and evidence in federal policymaking, many of which Results for America and other organizations had been promoting for several years. These recommendations, coupled with the bipartisan leadership from Members of Congress and committed federal civil servants, and the advocacy work of evidence-based policy champions, including Results for America, from all across the country formed the contours of the Evidence Act, which was passed by the U.S. Senate and House with overwhelming support from both parties—including passage by unanimous consent in the Senate—and was signed into law in early 2019. This bipartisan law dramatically expanded the ability of the federal government to evaluate the impact of its programs, to make data, evidence and evaluation regular 3 components of the policy process, and to increase public access to federally held data. The Evidence Act is far from the last word in evidence-based policy, but it represents a major step forward in measuring and improving the impact of federal government programs. A rigorous evidence-based process not only helps determine whether policy goals are being achieved, but also restores faith in the policymaking process and government itself. In the absence of objective evidence, lobbyists, and others with access, can dominate the process either by presenting their case with data skewed to support their interests, or by exerting political pressure that ignores data all together. In fact, a recent poll commissioned by Results for America indicated that just 8 percent of respondents thought policy was primarily driven by evidence—with 34 percent believing that lobbyists’ influence drove policy and 42 percent thinking that policy was driven by politicians’ desire to get votes. And a recent Government Accountability Office (GAO) report noted that almost four-in-ten federal managers were unaware of any evaluations of their respective programs. The Evidence Act provides federal agencies with the tools to right this ship, making data and evidence use a regular part of how they do business and invest taxpayer dollars. The Evidence Act’s focus on improving access to high-quality data will enable researchers and policymakers to better explore multidimensional solutions to difficult social problems. For example, in a series of landmark studies, economist Raj Chetty used government data on income, education, and housing to highlight the relative lack of American economic mobility and how that interplays with inequality, education, health, and location. Additionally, economists Bruce Meyer and Nikolas Mittag were able to link administrative data from government welfare programs to official poverty statistics. Using the combined data, they found that official poverty statistics greatly underestimate the amount of public benefits received by lowincome families and thus may misstate their level of economic hardship. (1) Enhancing the infrastructure necessary for federal agencies to use evidence in policy development. For government to incorporate evidence in policymaking, new habits and capabilities must be formed and infrastructure created. Designated personnel within an agency must take the lead in developing the structures and policies that define the agency’s priority questions for which evidence is needed, the standards for generating that evidence and using its resulting data, and the process for regularly evaluating its work. Moreover, key personnel must ensure that evidence and evaluation are incorporated into the agency’s budget, policy, and management decisions. The Evidence Act facilitates the creation of this structure and new processes by: • Requiring Chief Data Officers within each agency to be responsible for data management, privacy and confidentiality and data access. The Chief Data Officer is also required to lead a Data Governance Body that will set the agency’s data policy. 4 • Requiring the White House Office of Management and Budget (OMB) to create a federal government-wide Chief Data Officers Council that will, among other things, identify ways to protect data privacy and promote appropriate data sharing. • Requiring Evaluation Officers within the 24 government agencies covered by the Chief Financial Officers Act (CFO Act) to assess methods and data use. Evaluation Officers lead the drafting and implementation of written evaluation plans and multi-year learning agendas that lay out the process for answering key policy questions. • Prioritizing Evidence-Building by directing agencies covered by the CFO Act to craft a multi-year Learning Agendas that detail how they will identify policy questions and gather relevant data. These agencies also are required to develop Evaluation Plans that describe key questions for each significant evaluation study planned for the next fiscal year, including the key information to be collected. The goal behind mandating these evidence-building plans is for each agency to incorporate an evaluative process into its program design and policymaking so that the appropriate data is collected and the right methods are used to analyze that data. • Requiring agencies to conduct periodic capacity assessments and report on their ability to perform statistical evaluation, research and analysis in their daily operations, identifying where key skills and competencies are falling short. The U.S. Department of Labor has had a Chief Evaluation Officer since 2010. The Chief Evaluation Officer coordinates evaluations Department-wide, including office staff and leadership to interpret research and evaluation findings and to identify their implications for programmatic and policy decisions. The Chief Evaluation Office, overseen by the Chief Evaluation Officer, has nine full-time staff, several contractors, and a couple of detailees at any given time. This staff is augmented by staff from research and evaluation units in other DOL divisions. The Office receives $8.04 million in dedicated funding and may receive up to an additional 0.75 percent from statutorilyspecified program accounts at the Secretary’s discretion. Evaluation Officers in Practice 5 • Fostering inter-agency coordination and “cross-pollination” of best practices by requiring OMB to consolidate each agencies’ input into a unified report. In addition, the Evidence Act cuts through significant governmental “red tape” by reducing statutory limits on agencies’ ability to share data, resulting in less duplicative data queries, while simultaneously strengthening privacy and confidentiality protections. • Establishing an Advisory Committee on Data for Evidence–Building to recommend how to expand access to and use of federal data in policy decisions. (2) Improving the process for researchers to access government data, making it more open, streamlined, and secure, while also providing stronger privacy standards and legal protections. Prior to the Evidence Act, researchers who sought to assess the effectiveness of government programs had to navigate a daunting bureaucratic process to access relevant data. The Evidence Act helps researchers locate and securely access the data they need, and encourages federal agencies to release non-sensitive data publicly. Specifically, the Evidence Act: • Directs agencies to provide public access to collected data whenever possible and provide it in an open and machine-readable format, which helps various datasets be compatible and user-friendly. • Creates a streamlined, transparent application process for accessing restricted data. Rather than undertake multiple and diverse application processes, researchers can apply to access certain restricted data using a single, secure, common application system. Researchers must still meet appropriate qualifications and restrictions that protect personal and confidential information. However, the Evidence Act creates full transparency about how the data will be used as all applications (even those rejected) are made public. Prior to the public release of data, agencies must review privacy and confidentiality risks that could arise (particularly data collected under a pledge of confidentiality). Data will be “de-identified,” or “masked,” so that all personal identifiers are removed. The risk assessment examines whether the de-identified information is restricted by written agreement; the cost/benefit to the public; and risks involving security, legal liability, intellectual property rights, and confidential business information. Critically, the assessment examines identification risks if the data are combined with other public information. Comprehensive Risk Assessments 6 • Requires agencies to maintain a comprehensive Federal Data Catalogue, including metadata on each data set. Collected data will be cataloged and searchable with descriptive details to improve researchers’ queries. This helps researchers connect various data sets that have information on the same people or communities. The Federal Data Catalogue will increase the effective use of data by allowing researchers to locate publicly available data in a single searchable location. • Directs the Chief Data Council to review best practices in data security, and assess the coordination and availability of data across agencies. The Council also solicits feedback from data users outside of government. (3) Helping federal agencies develop a common understanding of how to articulate important policy questions, obtain relevant data, and better utilize existing data. The increased use of data must not threaten individual privacy or diminish data security. Fortunately, advances in statistical methods and computing technology provide ways to protect data while extracting value. The Evidence Act provides strong controls to protect personal information while increasing transparency about what data government collects and how this data is used. Specifically, the Evidence Act: • Encourages the consistent application of rules by directing the Chief Data Officer Council to promote standardization of data and rules throughout government. • Strengthens privacy laws and policies by amending existing law to ensure confidentiality pledges cannot be violated and adds stringent privacy requirements to acquire and use data. Statistical agencies must categorize data based on its sensitivity level and undertake comprehensive risk assessments prior to public release. • Institutes transparency by requiring agencies to publicly release the criteria used in developing comprehensive risk assessments, which determine the degree of protection for data. • Fosters public trust by requiring government to maintain objectivity, independence, and confidentiality in evidence-building by requiring that the use of statistical information and the interpretation of data be “accurate,” “unbiased,” and “credible.” The Evidence Act also requires confidential data be protected and used for statistical purposes only. Specifically, the Evidence Act amends the Confidential Information Protection and Statistical Efficiency Act. 1 1 7 The Evidence Act also fosters continual improvement and feedback. Within two years of the law’ enactment, the GAO must issue a report that summarizes agency findings and trends in agencies’ use of data and evidence as stated in the agencies’ strategic plans. The GAO report will also recommend actions to improve data use and techniques in evaluating evidence for policy. Similarly, OMB will generate guidelines for best practices in program evaluation and will create a new career path for program evaluation within an agency. Finally, the Evidence Act requires periodic guidance as part of its implementation. In July 2019, OMB released guidance related to the first phase of Evidence Act implementation—Learning Agendas, Personnel, and Planning. (You can find a summary of the Evidence Act and the July 2019 OMB guidance to federal agencies of this law here.) The Evidence Act lays important groundwork for increasing the use of evidence-based policymaking within the federal government by creating or enhancing the necessary personnel and infrastructure. But, more action is required to ensure federal agencies use data and evidence to inform their fiscal and policy decisions. The next step is to encourage agencies to use the Evidence Act’s infrastructure in a way that ensures federal resources are spent in an evidencebased manner. II. PROPOSED NEXT STEPS While the Evidence Act created essential evidence-based tools and infrastructure within the federal government, additional legislation is needed to incorporate evidence into the daily operations, policymaking, and grant making decisions of federal agencies. In this section, we identify two key components to a follow-up bill, an Evidence Act 2.0, that would build on earlier legislative efforts: • Invest 1 percent of existing federal program funds in evaluations; and • Direct federal agencies to prioritize evidence of effectiveness—including goldstandard randomized controlled trials wherever feasible—during competitions for new and existing federal grant programs. Invest in evaluations of government programs An Evidence Act 2.0 should require federal agencies to set aside at least 1 percent of their program funds to strengthen their evaluation capacity and to conduct rigorous program evaluations. These funds would provide Evaluation Officers with dedicated and consistent resources to carry out the activities included 2 This approach is also generally consistent with the approach outlined in the Commission’s final report, although we note that RFA advocates for at least 1 percent for evaluation, while the Commission’s report advocates for up to 1 percent for evaluation. 2 8 in the agency evaluation plans and learning agendas. A new Evidence Act should also require new and existing federal grantees to conduct rigorous program evaluations as a condition of receiving funding. Congress and federal agencies spend billions of dollars every day trying to tackle big problems, but they rarely evaluate how well their efforts address those problems. In a 2017 GAO study, 39 percent of federal managers reported that they did not know if an evaluation of any program, operation, or project they were involved in had been completed within the past 5 years. Another 18 percent of federal managers reported having had no evaluations during this same time period. Moreover, a March 2019 poll conducted by NORC at the University of Chicago found that 84 percent of Americans support investing 1 percent of federal program funds in evaluations to determine if they are effective or not. Leaders and organizations from across the political spectrum have also expressed support including the 15 bipartisan experts from the Commission on Evidence-based Policymaking and senior policy advisors in the Bush and Obama administrations. Despite widespread support for the 1 percent set aside, few programs meet this threshold. For example Results for America’s 2018 Invest in What Works Federal Standard of Excellence found that six of the largest federal human services departments and agencies invested, on average, only 0.38 percent of their budgets on evaluation activities in FY18. Nonetheless, two of the largest federal foreign assistance agencies, the Millennium Challenge Corporation and U.S. Agency for International Development, invested 5.1 percent and 1.4 percent, respectively, that same year. Providing 1 percent for evaluation would address federal policymakers’ lack of basic information about the effectiveness of government programs and would allow management to make better spending decisions based on a program’s effectiveness, helping government spending get more “bang for the buck.” Examples: • Federal appropriations laws in Fiscal Years 2016 through 2019 provide the Secretary of the U.S. Department of Labor (DOL) with the authority to set aside up to 0.75 percent of workforce training funds for program evaluations. DOL also requires all of its federal competitive grant program grantees to participate in evaluations if asked. • The bipartisan Every Student Succeeds Act (ESSA) authorizes the Secretary of the U.S. Department of Education to set aside up to 0.5 percent of federal K-12 education funds (excluding ESSA Title I funds) for program evaluations. It also authorizes $710,000 each year for evaluations of the ESSA Title I program. 3 For example, in a 2017 discussion at the Aspen Institute, John Bridgeland (President Bush’s domestic policy adviser) and Peter Orszag (President Obama’s budget director) both advocated for the 1 percent set aside for evaluation. 3 9 Prioritize evidence in competitive and non-competitive grantmaking The twin goals of evidence-based policymaking are to improve outcomes effectively and efficiently. Directing federal grant funds toward evidence-based solutions is a way to accomplish both goals. As such, an Evidence Act 2.0 should direct all federal agencies to prioritize evidence of effectiveness—including goldstandard randomized controlled trials wherever feasible—during competitions for new and existing federal grant programs. In most cases, federal agencies currently have the administrative authority to elevate evidence of effectiveness as a priority during the competitive grant process. The next version of the Evidence Act should require agencies to use this authority to the fullest extent possible to ensure that taxpayer dollars are increasingly invested in what works. On the other hand, federal agencies are required to allocate funds from their noncompetitive grant programs (including the largest block grant programs which provide funding to cities, counties, and states) as outlined in the federal laws. For this reason, the next version of the Evidence Act should amend these laws to encourage and/or require recipients of both competitive and noncompetitive grants to invest a portion of their grants in evidence-based, results-driven solutions. These changes are absolutely necessary if the federal government is to play a significant role in improving the lives of individuals, families, and communities given that in FY2019, seven of the largest federal human services agencies provided $96.7 billion through their five largest noncompetitive grants. Examples: • Thirteen of the 31 largest federal competitive grant programs at seven of the largest federal human services agencies use evidence of effectiveness when allocating funds. While the rigor and application of these evidence standards vary by agency and program, some leading examples include: ◊ U.S. Department of Education (ED): » In FY18, ED prioritized evidence of effectiveness when allocating funds from its five largest competitive grant programs ($2.15 billion total appropriation in FY18). » For example, ED’s TRIO Student Support Services program ($300 million in FY18), TRIO Talent Search program (over $150 million in FY18), and TRIO Upward Bound Math and Science program (nearly $400 million in FY18) awarded competitive preference points to grant applicants proposing to invest awarded funds in strategies supported by at least moderate evidence of effectiveness. 10 ◊ Corporation for National and Community Service (CNCS): » In FY18, CNCS’s AmeriCorps State and National competitive grant program application allocated up to 12 points out of 100 to applicants proposing strategies supported by rigorous evidence of effectiveness. • Five of the 35 largest federal noncompetitive discretionary grant programs at seven of the largest federal domestic human services agencies use evidence of effectiveness when allocating funds, including: ◊ The U.S. Department of Labor’s 2019 guidance on Unemployment Insurance (UI) Reemployment Services and Eligibility Assessment (RESEA) Grants, includes a new requirement for evidence-based approaches, encouraging “the Department to fund and states to use evidence-based strategies where they exist and to conduct evaluations and build evidence in places where needed. The goal is to ensure that each state employs RESEA interventions and service delivery strategies that, based on rigorous evaluations, improve employment outcomes and reduce benefit duration…” (Section 306 of the Social Security Act). ◊ U.S. Department of Education: The Every Student Succeeds Act requires states to set aside at least 7 percent of their ESSA Title I, Part A funds (this 7 percent set aside represents $1.1 billion in FY19) for a range of activities to help school districts improve low-performing schools. School districts and individual schools are required to create action plans that focus on investments in “evidence-based” interventions that demonstrate strong, moderate, or promising levels of evidence. • Opioids: The bipartisan 2018 SUPPORT for Patients and Communities Act, authorizes approximately $195 million for evidence-based services and evaluations to help fight our nation’s opioid crisis including: $15 million for the U.S. Department of Health and Human Services to replicate and rigorously evaluate the impact of a family recovery and reunification program and $10 million per year to support evidence-based services to prevent and treat substance use disorders affecting children, adolescents, and young adults (more information on the evidence provisions is available here). • Juvenile Justice: The bipartisan 2018 Juvenile Juvenile Justice Reform Act defines what constitutes “evidence-based” and “promising” for programs funded through the Juvenile Justice and Delinquency Prevention Act. Under the competitive incentive grants for prison reduction program, states must invest federal funds in evidence-based or promising programs, and they receive a priority for developing data-driven prevention plans, employing evidence-based prevention strategies, and conducting program evaluations (more information on the evidence provisions of this act is available here). 11 IV. Conclusion The Evidence Act requires the White House Office of Management and Budget and federal agencies to build the foundational capacity they need to be able to use evidence and data in their decisions and policymaking. Future federal legislation, however, is needed to ensure that sufficient federal resources are available to build evidence of what works and that federal agencies are prioritizing evidence of effectiveness when allocating federal funds.