TEN_14-16_Acc.pdf

ETA Advisory File Text

TRAINING AND EMPLOYMENT NOTICE NO. 14-16 DATE November 8 2016 EMPLOYMENT AND TRAINING ADMINISTRATION U.S. DEPARTMENT OF LABOR WASHINGTON D.C. 20210 TO AFFILIATE AMERICAN JOB CENTER MANAGERS COMPREHENSIVE AMERICAN JOB CENTER MANAGERS DEPARTMENT OF LABOR EMPLOYMENT AND TRAINING ADMINISTRATION REGIONAL ADMINISTRATORS STATE GOVERNORS STATE LABOR COMMISSIONERS STATE WORKFORCE INNOVATION AND OPPORTUNITY ACT LIAISONS STATE WORKFORCE ADMINISTRATORS WORKFORCE INNOVATION AND OPPORTUNITY ACT WORKFORCE DEVELOPMENT BOARD LOCAL CHAIRS WORKFORCE DEVELOPMENT BOARD STATE CHAIRS WORKFORCE DEVELOPMENT BOARD STATE EXECUTIVE DIRECTORS WORKFORCE DEVELOPMENT BOARD LOCAL EXECUTIVE DIRECTORS F ROM PORTIA WU s Assistant Secretary S UBJECT Release and Availability of Eleven Briefing Papers and Two New Reports from the WIA Adult and Dislocated Worker Programs Gold Standard Evaluation Providing Public Workforce Services to Job Seekers Implementation Findings on the WIA Adult and Dislocated Worker Programs and Providing Public Workforce Services to Job Seekers 15-month Impact Findings on the WIA Adult and Dislocated Worker Programs 1. P urpose. Since 2008 the Employment and Training Administration ETA has overseen the development and execution of a rigorous random assignment evaluation of the Workforce Investment Act WIA Adult and Dislocated Worker Programs referred to as the WIA Gold Standard Evaluation. This Training and Employment Notice TEN announces the release and availability of 11 briefing papers a report from the evaluation s implementation analysis and the interim report from the evaluation s impact analysis. Providing Public Workforce Services to Job Seekers Implementation Findings on the WIA Adult and Dislocated Worker Programs and 11 related briefing papers describe program operations providing the context for understanding the evaluation s impact results. Providing Public Workforce Services to Job Seekers 15-month Impact Findings on the WIA Adult and Dislocated Worker Programs presents the interim impact findings from the evaluation 15 months after participants were randomly assigned into the study.2. Re ferences. TEN 37 -09 The Workforce Investment Act Gold Standard Evaluation released on April 6 2010 2 TEN 13 -15 Update on the Workforce Investment Act Adult and Dislocated Worker Programs Gold Standard Evaluation and Announcement o f the Availability of a New Research Report released on October 15 2015 The Workforce Innovation and Opportunity Act WIOA The Workforce Investment Act of 1998 Providing Services to Veterans Through the Public Workforce System Descriptive Findings fr om the WIA Gold Standard Evaluation released in October 2015 Evaluating National Ongoing Programs Implementing the WIA Adult and Dislocated Worker Programs Gold Standard Evaluation released in January 2016 3. Background . In June 2008 ETA awarded a co ntract to Mathematica Policy Research working with subcontractors Social Policy Research Associates MDRC and the Corporation for a Skilled Workforce to conduct a rigorous impact evaluation of the WIA -funded Adult and Dislocated Worker Programs. As exp lained in TEN 37 -09 the WIA Gold Standard Evaluation will inform the general public program administrators the Office of Management and Budget and Congress about the effectiveness of WIA -funded intensive and training services relative to more limited s ervices. Specifically the outcomes of study participants eligible to receive WIA -funded training services will be compared to the outcomes of those eligible to receive WIA -funded intensive and core services but not training while the outcomes of those e ligible to receive WIA -funded intensive services will be compared to the outcomes of those eligible to receive only WIA -funded core services. The evaluation also will provide information about the cost of the range of services provided at American Job C enters AJCs and an analysis of the cost of the programs relative to the positive impact if any of program services. Further the evaluation will provide substantial detail about what contributed to the observed employment and earnings outcomes and est imated impacts providing context and possible explanations for the evaluation results and the services cost effectiveness. WIOA supersedes WIA. Although WIOA makes some important changes to the public workforce development system WIOA leaves intact important elements of the service - delivery structure of the Adult and Dislocated Worker programs services will continue to be accessed at A JCs a similar basic set of services will be offered and customers eligible for training will continue to choose th e training they view as most appropriate with some parameters . Thus under WIOA it continues to be the case that this evaluation will provide details about what works for whom and under what conditions. 4. WIA Gold Standard Eva luation Description . The evaluation uses a random assignment design to examine the impact of Adult and Dislocated Worker Programs as implemented by 28 randomly selected local workforce investment areas LWIAs operating nationwide. The random selection of LWIAs ensures that the customers in the study were representative of all the workforce investment system customers across the continental United States and not just those in LWIAs that agreed to participate in the study. As a result of the LWIA s participation the evaluation s results will be generalizable to the workforce development system . 3 With few exceptions for example veterans all eligible adults and dislocated workers in the local areas were randomly assigned to one of three research groups defined by the WIA - fund ed services that they could receive 1 all WIA services including training 2 core and intensive services but not training and 3 core services but not intensive or training services. Throughout all individuals enrolled into the study were able to re ceive services including training from other sources in their communities. Random assignment took place just after customers were found eligible to receive WIA -funded intensive services. In total approximately 34 000 individuals were enrolled into the study between November 2011 and April 2013. Study participants eligible for intensive and training services received those services before the implementation of WIOA. The evaluation has t hree major components an implementation analysis an impact analy sis and a cost -benefit analysis . Detailed information on the implementation of the WIA -funded intensive and training services the subject of the implementation report and the briefing papers was collected through interviews with state and local staff observation of service provision and focus group discussions with program participants. Information about the impacts measured 15 months after random assignment into the study is the subject of the interim impact report. The outcome data was collected t hrough individual participant follow - up surveys. 5. Publication Description . The implementation report and the briefing papers discuss the services offered by the Adult and Dislocated Worker Programs and the contexts in which they operated highlighting ser vice -delivery features that might have a direct bearing on customers employment and other outcomes. The report provides lessons learned about the configuration and provision of core intensive training supportive and follow -up services. The report also presents implications for the workforce system under WIOA. The interim impact report presents estimated impacts of intensive and training services based on customers experiences during the 15 months after they were found eligible for intensive service s and randomly assigned into one of the study s three research groups. The study does not have a control group. Rather it is testing the impact of the availability of different service intensities relative to lower -tiered services also offered at partic ipating AJCs. Specifically the report presents interim impact estimates of 1 the availability of intensive services relative to the availability of core services 2 the availability of training services relative to the availability of intensive servi ces and 3 the availability of training services relative to the availability of core services. 6. Key WIOA -Related Implementation Analysis Findings . As a result of the study design the findings from the implementation analysis are representative of what would be observed in LWIAs across the country . Findings from the evaluation suggest that many of the changes made by WIOA add flexibility for local areas to continue in directions they were already heading. For example Blending core and intensive servi ces into one tier . WIOA combines core and intensive services into a single career services tier. The distinction between core and intensive 4 services was often not clear across study local areas an indication that many areas already combine d them when the data were collected in 2012 and 2013 or are well set to do so under WIOA . Eliminating the sequence of services . WIOA removes the requirement that customers receive core and intensive services now career services prior to receiving training service s. Many LWIAs in the study especially those that focused on providing services to customers already interested in training moved customers through core and intensive services quickly. For example some LWIAs counted the interactions with the AJC greete r and determination of training eligibility as the core and intensive services required for training eligibility. Emphasizing credentials . WIOA increases the emphasi s on the importance of customers obtaining employer -recognized credentials. It adds a per formance measure to account for customers who achieve a credential or make progress toward a credential. However even under WIA this was already considered important by some LWIAs that were either making training approval contingent on the possibility o f attaining a credential or providing a larger Individu al Training Account for programs that led to credentials. Collocating E mployment Ser vice E S and WIA programs . WIOA requires WIA and ES staff to be collocated at AJCs. In 2012 and 2013 a ll but one of the 19 states with LWIAs participating in the study already required that both WIA and ES staff be physically located at comprehensive c enters at least part time. In addition in most LWIAs the same resource room served both ES and WIA customers reflecting grassroots expectations that collocation of resources would improve services to clients . WIOA also responds to barriers that study sites were facing in offering work -based training. For example staff in the study sites noted that one reason the local areas did not fund more customer placements in on -the -job and customized training was that it was difficult to obtain businesses par ticipation. WIOA decreases the costs to businesses by allowing local areas to fund employer -specific training for employed workers incumbent training and authorizes increased wage reimbursements for on -the -job training. WIOA also allows local formula funds to be spent on transitional jobs another type of work -based training. These changes might lead to more opportunities for customers to participate in work -based training. 7. Key Impact Analysis Findings . As a result of the study design the findings f rom the impact analysis are representative of what would be observed in LWIAs across the country. The information presented in this report was collected only 15 months after random assignment into the study. As such these results are considered interim. A significant portion of those in the study group eligible to access all WIA -funded services was still participating in training activities at the time thus these early estimates are expected to differ from the impact estimates derived from the final o utcomes to be collected 30 months after random assignment. The interim findings include The availability of WIA funding for training increased the proportion of customers who enrolled in a training program. Full -WIA customers were 13 percentage points more likely to enroll in training during the 15 months after random assignment than core -and - 5 intensive customers and 16 percentage points more likely to enroll in training than core customers. Both core -and -intensive and core -only customers were able to a ccess training from other providers in their communities. Full -WIA customers received more supportive services either funded by WIA or from other sources in the community than core -and -intensive customers who in turn received more than core customers. Although 21 percent of the full -WIA group received supportive services only 12 percent of the core -and -intensive group and 5 percent of the core group received them. Full -WIA customers were more likely to participate in training than core -and -intensive cu stomers in the first four quarters after random assignment. Rates of training participation were highest and differences across study groups in these rates were largest in the first quarter after random assignment and then generally declined over time . Trainees in the full -WIA group were more likely than trainees in the core group to complete a training program and more likely to earn a credential. Seventy -one percent of full -WIA trainees completed a training program and 54 percent of full -WIA trainee s received a credential. In comparison only 56 percent of core -only trainees completed a training program and 41 percent of core -only trainees received a credential. The availability of WIA -funded intensive services increased earnings by 600 17 percen t and increased employment compared to those eligible to receive core services only. Customers receiving core and intensive services earned more than those receiving only core services in each of the five quarters. The availability of both WIA -funded training and intensive services increased the likelihood of being employed a six percentage point increase in the fifth quarter after random assignment although higher average earnings were not observed . The impact of the availability of training on Quarter 5 earnings was negative but not statistically significant. The lack of positive earnings outcomes in this quarter was expected since those in training have less time for employment and was likely impacted by a significant number of customers st ill in training approximately 17 percent or just recently completing it. It is too soon to judge the impact of training. The availability of training or intensive services did not affect total annual household income receipt of public assistance repor ted health health insurance coverage or criminal activity within the 15 -month period. The estimated impact of the availability of both WIA -funded intensive and training services was significantly positive for men customers who were white and non -Hispani c and customers who did not work in the year before random assignment. Estimates suggest a positive impact of intensive services relative to core services. The estimated earnings impacts were generally similar across local areas with high and low unem ployment rates. There is some suggestive evidence that intensive services might be more effective in high -unemployment local areas. Nearly one -half of full -WIA customers able to receive training did not participate in training in the 15 months after r andom assignment suggesting unmet demand. Earnings impacts patterns are broadly similar for dislocated workers and adults. 6 The evidence on intensive services is encouraging. Study findings suggest that intensive services when offered as a standalone service without training were effective at increasing earnings and employment. However i t is too soon to judge the effectiveness of the availability of training services. It is to be expected that participating in training would have a negative impact on employment and earnings in early quarters because customers enrolled in training have l ess time for employment. I n fact many of the study group participants eligible for training services were still in training in the fifth quarter or had only recentl y completed it. Likewise it is too soon to judge whether the availability of both WIA -funded intensive and training services was more effective than the availability of core only services in which case neither of these WIA -funded services was available. The final report will provide more definitive evidence. It will discuss the estimates of the effectiveness of the programs based on follow -up data for a total of 30 months after random assignment. In addition to estimating the impact of the availabil ity of WIA -funded intensive and training services relative to lower -tiered services it also will examine whether any positive benefits of the services on earnings are large enough to cover the costs of the services. 8. The Evaluat ion Report Timeline . The F inal I mpact Report i s scheduled to be released in early 2018 . 9. Inquiries . For more information about this study contact Eileen Pederson Contract Officer s Representative Office of Policy Development and Research ETA at 202 693 - 3647 or pederson.eileen dol.gov . To view an abstract of th ese report s and to download the briefing papers the implementation report the interim impact report the VSS report or the report on the implementation of the evaluati on visit http wdr.doleta.gov research keyword.cfm .