About the Study
In 2015, the Chief Evaluation Office partnered with the Employment and Training Administration (ETA) to fund contractors Mathematica, Social Policy Research Associates, and MDRC to design and conduct an evaluation to better understand the implementation and outcomes of the National Guard Youth Challenge (YC) and Job Challenge (JC) program. The implementation study and outcome evaluation of three Job Challenge grants includes an analysis of post-JC outcomes between 2016 and 2018 for justice-involved graduates. The analysis drew on quantitative and qualitative data including interviews, observations, program and survey data, focus groups, and administrative data from state education and criminal justice agencies.
The National Guard YC program is an initiative that aimed to assist high school dropouts aged 16-18 in obtaining their high school diploma or GED through a 20-week, community-based residential course followed by a year of mentoring. ETA’s Reentry Employment Opportunities program funded YC programs in Georgia, Michigan, and South Carolina to include more court-involved youth and add a follow-on 20-week residential occupational training program called Job Challenge (JC). JC grantee programs recruited and enrolled 905 youth from January 2016 to December 2018.
This Department of Labor-funded study includes a final report and an issue brief and was a result of the annual learning agenda process. It contributes to the growing labor evidence-base to inform employment and training programs and policies and addresses Department strategic goals and priorities.
- Evaluation of the National Guard Youth ChalleNGe /Job ChalleNGe Program (Final Report, February 2021)
- Building on an Evidence-Based Program Model to Serve Court-involved Youth (Issue Brief, February 2021)
Research Questions:
- How was the Youth Challenge program implemented under the Job Challenge grant?
- How was the Job Challenge program implemented?
- How did the programs recruit and select youth for Job Challenge?
- How did youth experience the post-residential phase?
- What were the outcomes of Youth Challenge and Job Challenge participants?
- What expectations did youth have for the future?
- What can we learn from these grants about possible program models to serve court-involved and other at-risk youth?
- What can we learn from this implementation and outcomes study about the conditions that would be necessary to have a future impact study evaluation of Job Challenge?
Key Components:
- An evaluability assessment to build knowledge about the program to be used to design an evaluation which includes understanding the added value of the DOL Job Challenge and the effectiveness of the program for court-involved youth
- Recommendations for evaluation design
- An implementation and outcome or impact evaluation
- While the JC grants aimed to provide more services to court-involved youth, grantees reported difficulty reconciling that goal with the existing National Guard criteria for the YC program. Without a clear definition from DOL and an agreement with the National Guard that allowed a shift from the existing criteria, programs defined court involvement as they saw fit.
- Grantees continued to serve court-involved youth and placed a higher priority on documenting court involvement, but they did not substantially alter YC recruitment practices to reach new groups of participants or adjust service delivery for court-involved youth.
- JC Grantees successfully established new occupationally-focused, residential programs, creating key partnerships with community colleges. The community colleges provided intensive, certificate-based vocational training and supplementary education.
- Programs reported a need for increased staff and greater supervision and/or discipline in the JC program.
- Across the three grantees, 86 percent of JC participants were involved in a productive activity (employment, education, or military enlistment) approximately 14 months after JC. At the time of the survey, 81 percent of JC participants were employed.
- Court-involved JC participants had similar rates of post-program employment to non-court-involved participants, with no measurable differences in weekly earnings, benefits, hours, and job tenure.
- Court-involved youth were less than half as likely to be enrolled in postsecondary education one year after the YC program, with only 36 percent of court-involved participants receiving a credential compared to 50 percent of non-court-involved participants.
- Court-involved JC participants had higher rates of post-program justice system involvement, with 14 percent arrested in the year following YC, relative to only 5 percent of non-court-involved participants.
- An impact study would be needed to understand whether these outcomes were influenced by the program or merely reflect individual differences among program participants.
- An impact study would be needed to understand whether these outcomes were influenced by the program or merely reflect individual differences among program participants.
Final Report
Berk, J., Kahn-Lang Spitzer, A., Stein, J., Needels, K., Geckeler, C., Paprocki, A., Gutierrez, I., Millenky, M. (2020). Mathematica. Evaluation of the National Guard Youth ChalleNGe/Job ChalleNGe Program. Chief Evaluation Office, U.S. Department of Labor.
Brief
Berk, J. (2020). Mathematica. Building on an Evidence-Based Program Model to Improve the Education and Employment Outcomes of Court-involved Youth. Chief Evaluation Office, U.S. Department of Labor.
The Department of Labor’s (DOL) Chief Evaluation Office (CEO) sponsors independent evaluations and research, primarily conducted by external, third-party contractors in accordance with the Department of Labor Evaluation Policy. CEO’s research development process includes extensive technical review at the design, data collection and analysis stage, including: external contractor review and OMB review and approval of data collection methods and instruments per the Paperwork Reduction Act (PRA), Institutional Review Board (IRB) review to ensure studies adhere to the highest ethical standards, review by academic peers (e.g., Technical Working Groups), and inputs from relevant DOL agency and program officials and CEO technical staff. Final reports undergo an additional independent expert technical review and a review for Section 508 compliance prior to publication. The resulting reports represent findings from this independent research and do not represent DOL positions or policies.