TEN_5_13_Acc.pdf

ETA Advisory File Text

EMPLOYMENT AND TRAINING ADMINISTRATION U.S. DEPARTMENT OF LABOR WASHINGTON D.C. 20210 TRAINING AND EMPLOYMENT NOTICE NO. 5-13 DATE September 4 2013 TO STATE WORKFORCE AGENCIES STATE WORKFORCE ADMINISTRATORS FROM GERRI FIALA s for ERIC M. SELEZNOW Acting Assistant Secretary SUBJECT Reporting Training-Related Employment under the Workforce Investment Act WIA Programs 1. Purpose. This Training and Employment Notice TEN is intended to improve reporting on training related employment for WIA programs. The TEN begins with general overview of the current state of the training-related employment and occupation information in the Workforce Investment Act Standardized Record Data WIASRD for individuals that obtained employment after program exit. Second this TEN presents the challenges associated with collecting this information. Lastly the TEN will present some state practices on effective ways to collect and report on training-related employment for WIA exiters who received training and obtained employment. 2. References. Training and Employment Guidance Letter TEGL No. 17-05 and TEGL No. 17-05 Change 2 Common Measures Policy for the Employment and Training Administration s ETA Performance Accountability System and Related Performance Issues. TEGL No. 17-09 Quarterly Submission of Workforce Investment Act Standardized Record Data WIASRD . TEN No. 34-09 Workforce Investment Act WIA Adult and Dislocated Worker Training. TEGL No. 28-11 Program Year 2011 Fiscal Year 2012 Performance Reporting and Data Validation Timelines. Office of Inspector General OIG Audit Report No. 03-11-003-03-390 Additional Information Needed to Measure the Effectiveness and Return on Investment of Training Services Funded Under the Adult and Dislocated Worker Programs. 3. Background. Office of Inspector General OIG Audit Report No. 03-11-003-03-390 recommended among other things that the Employment and Training Administration ETA provide guidance to State Workforce Agencies SWAs and Local Workforce Areas LWAs regarding best practices for more effectively collecting and reporting data on training related employment and the occupation of post-exit employment. ETA responded by conducting conference calls with states in August of 2012 to identify practices that may improve reporting on training-related employment and on the occupation of employment held in the quarter after exit as contained in the WIASRD. In addition ETA modified the content of TEGL 28-11 to include additional guidance on the importance of collecting and reporting this information. This TEN expands on the guidance in TEGL 28-11 using as its foundation the information that the Department gained from states in the aforementioned conference call. 2 Collecting and reporting information on whether individuals that received training ended up in a job that was substantially related to their training is an integral piece of information that ETA needs in order to evaluate the performance of Federally-funded training programs. States collect this information on all WIA Adult Dislocated Worker and Older Youth program exiters that received training and report it in field 1505 of the WIASRD. Additional information regarding the occupation of employment held in the quarter after exit is collected in WIASRD field 1502. Note that the WIASRD references in this TEN reflect the most recent version of the WIASRD approved on May 8 2013. WIASRD field 1505 is required for all WIA Adult Dislocated Worker and Older Youth program exiters that received training and entered employment in the quarter after program exit and can be based on any job held after exit. The WIASRD specifications require that this data element should be set to a value of 1 if the employment that an individual enters utilizes a substantial portion of the skills taught in the training received by the individual. This data element should be set to a value of 0 if the employment in which the individual enters does not utilize a substantial portion of the skills taught in the training received by the individual. Blanks in this field indicate that the individual did not receive training services while values of 9 denote that the information is not known. Note that states should not be defaulting this field to a value of 0 i.e. defaulting to no as 0 represents an affirmative no. WIASRD item 1502 is required for all WIA Adult Dislocated Worker and Youth program exiters received intensive or training services and that entered employment in the quarter after program exit. The WIASRD specifications require that this data element is recorded using the eight-digit occupational code that best describes the individual s employment using the O NET classification system. This information can be based on any job held after exit from the program. If all eight digits of the occupational skills code are not collected states should report as many digits as are available. 4. Current State of the Data. Table 1 presents the data on the total number of WIA Adult and Dislocated Worker program exiters that received training and entered employment the number that entered training-related employment the number that did not enter training-related employment and the percent of the information that has been reported by states for Program Year PY 2010 and PY 2011. Nationwide approximately 30 percent of all pertinent exiters had missing training-related employment information in PY 2011. Seventeen states report 100 percent of the training-related employment information while 11 states report on less than 50 percent of the pertinent individuals. Excluding the 8 states that failed to report this information in PY 2011 86 percent of the relevant exiters have this information reported. Table 2 contains data on the total number of WIA Adults and Dislocated Worker program exiters that entered employment in the quarter after exit the total number of reported occupations of the employment held in the quarter after exit and the percent of the information that has been reported by states for PY 2010 and PY 2011. Nationally approximately 85 percent of all pertinent exiters are missing occupation of post program employment information. In PY 2011 16 states reported less than 20 percent of the data on the occupation of the job held in the quarter after exit while 12 states reported on more than 80 percent of all the relevant exiters. 3 5. Challenges to Collecting and Reporting this Data. Reporting on training related employment and the occupation of the job held in the quarter after exit is challenging because this information has to be collected via manual follow up as there is currently no automated process for collecting occupation information for WIA program exiters. The primary source for collecting post program outcomes information is state wage record databases. While the availability of state wage record databases has made collecting outcome information substantially less staff intensive state wage records do not contain sufficient information on the occupation of an individual s employment. The only available information in these datasets pertains to the quarterly earnings and industry code of the employer of any given individual. Since both the training related employment and occupation of the job held in the quarter after exit involve occupation codes the automated wage record systems cannot provide this information. As a result states staff must manually follow up with each individual in order to collect this information. Nearly every state on the conference call indicated that the cost of conducting the necessary follow up for reporting on training related employment and on the occupation of the job held in the quarter after exit was considerable. A number of states reported that the cost associated with manual follow up was the primary reason that this information was not well reported. Figures 1 and 2 below provide cursory evidence that states with large numbers of individuals to follow up on had less complete reporting on these variables. Figure 1 is a scatter plot showing the percentage of pertinent individuals where training related employment information is reported y-axis vis- -vis the number of exiters that received training and entered employment in the quarter after exit x-axis . While the trend line is only slightly negative the removal of the seven small states that reported little training related employment information and the two large states that reported most of this information would lead to a much more negative correlation between the percent reported and the number of individuals on which to collect this information. Figure 2 is a scatter plot showing the percentage of pertinent individuals where the occupation of employment in the job held in the quarter after exit is reported y-axis vis- -vis the number of exiters that entered employment in the quarter after exit x-axis . The negative relationship between these two variables is much more negative than in Figure 1. This is likely a function of the fact that the number of people requiring follow up is nearly ten times larger than for the training related employment variable. Figure 2 shows that no states with more than 20 000 individuals to follow up on reported on more than 20 percent of the required individuals. As a result both the conference call with states and the data contained in WIASRD suggest that the costs associated with the follow up necessary for reporting on these individuals is the fundamental cause of the lackluster reporting on these two data fields. 4 Table 1. Reporting Prevalence Entered Training Related Employment For WIA Adults and Dislocated Workers That Received Training and Entering Employment State Received Training and Entered Employment Entered Training - Related Employment Yes Entered Training - Related Employment No Percent Reported PY 2011 PY 2010 PY 2011 PY 2010 PY 2011 PY 2010 PY 2011 PY 2010 Indiana 4 885 4 13 8 0 0 0 0 0.0 0.0 Montana 503 150 0 0 0 0 0.0 0.0 Nevada 1 597 1 138 0 0 0 0 0.0 0.0 New York 9 925 11 708 0 0 0 0 0.0 0.0 Ohio 7 425 7 397 0 1 398 0 3 687 0.0 68.7 Oregon 2 022 1 820 0 0 0 0 0.0 0.0 Vermont 164 229 0 0 0 0 0.0 0.0 Vir gin Islands 92 187 0 0 0 0 0.0 0.0 District of Columbia 303 309 4 12 15 3 6.3 4.9 West Virginia 1 042 989 226 258 270 258 47.6 52.2 Maryland 1 400 1 369 397 303 290 895 49.1 87.5 Virginia 2 903 2 041 925 785 791 760 59.1 75.7 Texas 7 378 6 6 14 3 154 839 1 467 504 62.6 20.3 Iowa 1 184 1 016 159 0 593 0 63.5 0.0 South Carolina 4 539 5 062 952 1 086 1 959 2 241 64.1 65.7 Wisconsin 3 107 2 265 592 425 1 429 1 686 65.0 93.2 Florida 8 806 5 680 3 107 1 940 2 974 2 000 69.1 69.4 New J ersey 5 640 5 203 1 691 1 107 2 209 2 311 69.1 65.7 Louisiana 1 409 1 902 631 729 361 374 70.4 58.0 Wyoming 233 216 130 108 49 45 76.8 70.8 New Mexico 497 464 253 217 136 99 78.3 68.1 Kentucky 3 015 3 107 816 866 1 586 1 339 79.7 71.0 North Dakota 211 226 128 139 42 61 80.6 88.5 Rhode Island 1 028 962 451 279 428 267 85.5 56.8 Tennessee 6 237 6 098 2 719 2 826 2 726 2 748 87.3 91.4 California 15 098 10 913 6 585 4 612 6 611 4 799 87.4 86.2 New Hampshire 505 670 353 441 98 156 89.3 89.1 Washington 3 302 2 861 2 096 1 888 859 852 89.5 95.8 Nebraska 545 503 254 292 243 190 91.2 95.8 Hawaii 183 241 85 129 83 83 91.8 88.0 Puerto Rico 2 894 2 643 534 973 2 123 1 586 91.8 96.8 Illinois 6 885 6 560 3 449 2 925 2 944 3 620 92.9 99.8 Arizona 2 350 2 072 609 0 1 581 0 93.2 0.0 Pennsylvania 5 465 4 329 2 957 2 321 2 411 1 950 98.2 98.7 Georgia 4 986 3 831 3 001 2 512 1 981 1 305 99.9 99.6 Connecticut 1 354 954 620 500 733 454 99.9 100.0 Alabama 2 410 3 185 1 521 1 591 889 973 100.0 80.5 Alaska 164 280 109 202 55 78 100.0 100.0 Arkansas 1 013 1 136 631 732 382 404 100.0 100.0 Colorado 1 510 1 231 653 539 857 692 100.0 100.0 Delaware 617 488 285 191 332 297 100.0 100.0 Idaho 1 171 981 768 555 403 426 100.0 100.0 Kansas 1 041 833 394 341 647 492 100.0 100.0 Maine 782 734 443 360 339 374 100.0 100.0 Massachusetts 4 167 4 013 2 640 2 478 1 527 1 535 100.0 100.0 Michigan 9 143 8 713 6 353 6 229 2 790 2 484 100.0 100.0 Minnesota 2 481 2 412 1 5 27 1 582 954 830 100.0 100.0 Mississippi 1 812 3 891 686 1 697 1 126 2 194 100.0 100.0 Missouri 3 548 2 506 1 419 1 917 2 129 589 100.0 100.0 North Carolina 5 591 6 064 2 881 2 951 2 710 3 113 100.0 100.0 Oklahoma 703 848 344 352 359 457 100.0 95.4 South Dakota 445 374 254 179 191 195 100.0 100.0 Utah 2 901 3 339 607 727 2 294 2 612 100.0 100.0 Total 158 611 146 895 58 393 52 533 50 477 52 018 68.6 71.2 Source PY 2011 Q4 WIASRD. Notes PY 2010 exiters are from 10 1 2009 9 30 2010 and PY 2011 exiters are from 10 1 2010 9 30 2011. 5 Table 2. Reporting Prevalence of Occupation of Job Held Quarter After Exit WIA Adults and Dislocated Workers that Received Training and Entering Employment in the Quarter After Exit State Entered E mployment Occupation of Employment Captured Percent Reported PY 2011 PY 2010 PY 2011 PY 2010 PY 2011 PY 2010 Iowa 8 064 9 577 0 38 0.0 0.4 Vermont 194 291 0 72 0.0 24.7 Oregon 84 102 67 921 154 65 0.2 0.1 Montana 934 536 5 23 0.5 4.3 New Yor k 242 760 246 875 1 517 847 0.6 0.3 West Virginia 1 618 1 685 25 0 1.5 0.0 Indiana 42 018 56 718 676 159 1.6 0.3 Utah 54 445 43 358 1 069 1 235 2.0 2.8 Oklahoma 24 172 24 732 541 570 2.2 2.3 Missouri 109 867 20 920 3 310 6 509 3.0 31.1 Ne vada 3 212 2 899 118 60 3.7 2.1 Virgin Islands 169 270 15 20 8.9 7.4 District of Columbia 795 655 102 128 12.8 19.5 Ohio 12 074 11 784 1 829 2 047 15.1 17.4 Alabama 2 730 3 751 430 523 15.8 13.9 Louisiana 37 994 61 352 7 277 6 974 19.2 11.4 California 51 374 59 685 10 610 8 256 20.7 13.8 Kansas 3 974 5 648 883 834 22.2 14.8 Mississippi 14 590 24 643 3 381 5 074 23.2 20.6 New Jersey 6 927 6 602 1 993 1 468 28.8 22.2 Tennessee 8 009 7 865 2 637 2 174 32.9 27.6 Texas 14 927 16 796 5 084 3 168 34.1 18.9 Wisconsin 5 882 4 754 2 484 2 131 42.2 44.8 Maryland 2 302 2 225 1 010 270 43.9 12.1 Delaware 767 834 354 320 46.2 38.4 Kentucky 4 815 4 373 2 315 1 890 48.1 43.2 Arizona 3 714 3 897 2 091 1 549 56.3 39.7 Pennsyl vania 10 932 9 629 6 474 0 59.2 0.0 South Carolina 7 969 9 960 4 904 6 087 61.5 61.1 North Dakota 348 483 215 272 61.8 56.3 Georgia 6 297 4 891 3 989 3 329 63.3 68.1 Virginia 5 147 3 958 3 433 2 940 66.7 74.3 Puerto Rico 7 062 6 421 4 761 5 848 67.4 91.1 Florida 13 253 9 545 9 065 6 981 68.4 73.1 Wyoming 265 279 187 199 70.6 71.3 Minnesota 4 808 5 488 3 574 4 502 74.3 82.0 New Mexico 613 673 460 332 75.0 49.3 Connecticut 2 412 1 490 1 811 1 176 75.1 78.9 Colorado 2 007 1 593 1 526 1 345 76.0 84.4 North Carolina 6 331 7 389 4 852 4 524 76.6 61.2 Rhode Island 1 629 1 585 1 277 735 78.4 46.4 Massachusetts 5 796 5 735 4 829 4 842 83.3 84.4 South Dakota 764 713 642 627 84.0 87.9 New Hampshire 847 1 055 718 864 84.8 81.9 Illinois 10 861 10 672 9 307 7 748 85.7 72.6 Alaska 214 379 185 334 86.4 88.1 Michigan 13 150 13 078 11 426 11 438 86.9 87.5 Maine 923 1 055 822 945 89.1 89.6 Hawaii 330 520 294 423 89.1 81.3 Nebraska 635 601 567 63 89.3 10.5 Idah o 1 299 1 183 1 162 965 89.5 81.6 Arkansas 1 180 1 403 1 079 1 241 91.4 88.5 Washington 4 792 4 589 4 492 4 386 93.7 95.6 Total 852 293 795 013 131 961 118 550 15.5 14.9 Source PY 2011 Q4 WIASRD. Notes PY 2010 exiters are from 10 1 2009 9 30 2010 and PY 2011 exiters are from 10 1 2010 9 30 2011. 6 Figure 1. Percent Reported vs. Number of Exiters that Received Training and Entered Employment in the Quarter after Exit Training Related Employment Figure 2. Percent Reported vs. Number of Exiters that Entered Employment in the Quarter after Exit Occupation of Job Held in Quarter After Exit 7 6. State Practices. Given the challenges discussed in section 5 there were a limited number of promising practices that ETA identified in the conference calls with the states. This is due to the fact that every state uses manual follow up to collect this information since occupation codes are not available in state wage record databases. While follow up was the only means used by states to collect this information the following three practices were the most common among states with more complete reporting on these variables. 1. Emphasizing the training-related employment and occupation of the job held in the quarter after exit in the state s training materials on reporting outcomes. 2. Allowing case managers to report this information if it is known in the state s management information system MIS before the wage records are available. 3. Instituting a system generated check on the training-related employment and post- program occupation of employment information in the state s MIS before the case can be closed out. While ETA is cognizant of the challenges associated with collecting this information states should make it a priority to improve their reporting on these fields. These variables are required information in WIASRD and are extremely important pieces of information for program analysis and evaluation efforts pertaining to value of investments in WIA training. 7. Action Required. Please make this information available to appropriate program staff. 8. Inquires. Please direct questions regarding this information to the appropriate ETA Regional Office.