CEO Library

In 2021, the Chief Evaluation Office (CEO) partnered with the Ohio Department of Job and Family Services and 11 participating counties in Ohio, funding contractor American Institutes for Research to conduct the Using Behavioral Insights to Increase Youth Use of Workforce Services in Virtual Contexts. The impact evaluation aims to test how behavioral insights can enhance service engagement and completion among young adults participating in Ohio’s Comprehensive Case Management and Employment Progr

In 2017, the Chief Evaluation Office (CEO) funded contractor Mathematica and its partners, American Institutes for Research, the Urban Institute, the W.E. Upjohn Institute, and ideas42, to assist with CEO’s Behavioral Interventions (BI) work, including developing rigorous impact evaluation design options for studying innovative program improvement strategies. In 2019, in partnership with the Employment and Training Administration (ETA), the BI team began investigating behavioral barriers to successful online job search.

Appendices to the Using Behavioral Insights to Increase Youth Use of Workforce Services in Virtual Contexts Final Report: Appendix A: Resources for Learning More About Behavioral Insights and How to Use Them for Continuous Improvement, Appendix B: Supplementary Details on Study Context and Design, Appendix C: Technical Details on Impact Estimates, and References.

Improving career readiness and job access for youth and young adults is vital. Millions of low-income Americans need better access to high-quality career pathways to escape poverty. This is no easy task— economic opportunity has shrunk dramatically in the United States in the past half-century (Chetty et al. 2016). Meanwhile, employers face rising shortages in the supply of skilled workers, making it harder to compete on the global market (World Economic Forum 2021).

The U.S. Department of Labor (DOL), Employment and Training Administration (ETA), Office of Workforce Investment (OWI) supports state and local workforce agencies in providing information to help job seekers successfully search for work. As part of this support, OWI asked the DOL Chief Evaluation Office (CEO) and the DOL Behavioral Interventions (DOL BI) team to explore whether applications of behavioral science could improve the usefulness of the information that job seekers use when searching for positions, investing in training, and considering career options.

In 2017, the Chief Evaluation Office (CEO), in collaboration with the Employment and Training Administration (ETA), funded contractor Mathematica, with Social Policy Research Associates, to conduct the Reentry Project Grants Evaluation.

As part of its support of Foundations for Evidence-Based Policymaking Act, the U.S. Department of Labor (DOL) is committed to collecting and creating information in a way that supports data sharing and dissemination. Toward that end, DOL’s Chief Evaluation Office (CEO) hosted a series of three seminars during August 2022 on topics directly relevant to sharing data, protecting confidentiality, and building a culture to support sharing and responsible use. The one pager provides a summary of the seminars.

The report presents the findings from the Reentry Project (RP) Grants implementation study, which includes analysis of data from virtual sites visits with 27 sites that received 2018 or 2019 grant or subgrant awards, a grantee survey administered to all 2017, 2018, and 2019 grantees, and Workforce Integrated Performance System (WIPS) records dating from program year (PY)2018 Q1 to PY2021 Q2 or July 1, 2018 to December 31, 2021. This report focuses on grantee survey findings and analysis of data from WIPS for 2018 and 2019 grantees.

The Reentry Projects (RP) Grant Evaluation Design Pre-Specification Plan follows the template that evaluators must use to meet the pre-specification practices articulated in OMB Memo M-20-12 Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices.

The brief draws on a variety of sources, including survey responses from 89 Reentry Project (RP) grantees, interviews with RP program staff and partner organizations from 27 sites, interviews with 37 RP participants, interviews with 41 employers, and national workforce data from the Workforce Integrated Performance System (WIPS) all with the aim of describing the types of work-based learning (WBL) provided to RP participants, RP programs’ experiences in offering WBL opportunities to justice-involved individuals, and some lessons learned from offering WBL to the reentry population.

Using data collected as part of the Reentry Project (RP) Grants Evaluation, the brief describes the differences and similarities between adult and young adult grantees in terms of the services they offered and the implementation challenges they reported. The analysis draws on quantitative data from a survey of all 116 organizations that received RP grants. Data from the grantee survey were analyzed using descriptive statistics as well as chi-squared tests to determine whether differences across grant types were statistically significant.

The brief highlights the service needs that interviewed Reentry Project (RP) participants reported when leaving incarceration; their barriers to employment; their experiences during and after they participated in the programs; their perspectives on program elements such as vocational training, supportive services, and job placement; and their recommendations for improvement.

The brief describes the 116 Reentry Project (RP) grantees funded in 2017, 2018, and 2019 and aims to address the research question, “What are the variations in the model, structure, partnerships, and services of the grants?” To answer this question, descriptive statistics from survey data collected in all three grant years was used to summarize and highlight findings about RP program structures, partnerships, and services, in addition to chi-squared tests to determine whether any differences across grant years and grant types were statistically significant.

The brief draws on data collected from virtual site visits with 27 Reentry Project (RP) grantees to identify the industries grantees commonly focused on, describe industry-specific training they used, discuss the development of industry partnerships, and provide insights for connecting individuals with justice involvement to locally in-demand industries. Site visit data included interviews with 33 employers; together with grantee interviews, the visits highlighted successes and challenges grantees experienced when engaging and partnering with employers.

The brief draws on literature on risk/needs assessments in the criminal legal system and grantee survey data collected from 89 community-based organizations (CBOs) that were awarded U.S. Department of Labor (DOL) Reentry Project (RP) grants from 2017 to 2019.

To assess the effectiveness of criminal justice policies, programs, or reforms, researchers frequently examine recidivism, defined as the return to criminal activity after a prior sanction (Council of State Governments Justice Center 2014; Deady 2014; National Institute of Justice 2022; Pratt and Eriksson 2013).

In 2019, the Chief Evaluation Office (CEO) partnered with the Employment and Training Administration (ETA) to fund contractors Urban Institute, Mathematica Policy Research, and Capital Research Corporation to design and conduct analysis to build and expand the evidence portfolio on apprenticeships, including models, components, partnerships, and strategies that often include the work of community colleges. 

The environmental scan sets the stage for a series of briefs that will be published under the State Apprenticeship Systems Capacity Assessment study. The study will conduct a deep dive into state-level Registered Apprenticeship systems in the United States, exploring how the public sector and its partners are supporting and implementing Registered Apprenticeship programs, including the use of more inclusive and equitable strategies and models.

The brief develops a typology of five different models of youth apprenticeship expansion used by Youth Apprenticeship Readiness Grants (YARG) grantees across their registered apprenticeship programs. The typology is based on information collected from the YARG grantee applications, follow-up clarification calls with grantees, and the grantees’ quarterly narrative reports to the U.S. Department of Labor (DOL). The models are not proscriptive templates for individual registered apprenticeship program standards.

The report provides background on youth apprenticeship programs in the U.S. to provide context for finalizing the design of the implementation evaluation of the Youth Apprenticeship Readiness Grants (YARG). The report also summarizes the planned activities of the 14 grantees and the key features of each grantee’s youth apprenticeship model and discusses key issues to include in the implementation evaluation.

In 2023, the Chief Evaluation Office (CEO) funded contractor American Institutes for Research (AIR), and their partners Abt Associates and Social Policy Research Associates, to conduct the Evidence to Action (E2A) at DOL: Portfolio Project & Process Study. This portfolio project builds on existing studies, evaluation technical assistance, communications, capacity building, and related efforts to grow the culture of evidence at DOL.

The report provides a snapshot of how Senior Community Service Employment Program (SCSEP) grantees adapted or expanded strategies to serve older workers in response to challenges faced during the pandemic. Strategies include (1) adopting new recruitment outreach, intake, and engagement activities; (2) promoting digital access and technology loaner programs; (3) providing training remotely; (4) adopting new host agency and employer strategies; and (5) implementing staffing strategies to support SCSEP participants’ needs.

The toolkit represents successful practices in negotiating agreements to obtain and use administrative data in U.S. Department of Labor (DOL) program evaluations. It includes detailed instructions, recommendations, and lessons learned on how to obtain data from various data providers including state workforce agencies, community colleges, criminal justice agencies, and other DOL grantees, primarily by establishing data use agreements.

The America’s Promise Job Driven Grant Program Evaluation Design Pre-Specification Plan follows the template that evaluators must use to meet the pre-specification practices articulated in OMB Memo M-20-12 Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices.

The report provides findings from the implementation study for the Evaluation of the Homeless Veterans’ Reintegration Program (HVRP), which relied on data from (1) a survey of all program year 2020 HVRP grantees, (2) semi-structured interviews with key stakeholders from eight deliberately selected grantees and their community partners, and (3) in-depth interviews with veterans who received services from one of those eight grantees.