TEGL13-09acc.pdf

ETA Advisory File Text

RESCISSIONS None EXPIRATION DATE Continuing CLASSIFICATION WIA Youth CORRESPONDENCE SYMBOL OWI-DYS EMPLOYMENT AND TRAINING ADMINISTRATION ADVISORY SYSTEM U.S. DEPARTMENT OF LABOR Washington D.C. 20210 DATE February 16 2010 ADVISORY TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 13-09 TO STATE WORKFORCE AGENCIES STATE WORKFORCE LIAISONS FROM JANE OATES s Assistant Secretary SUBJECT Contracting Strategies That Facilitate Serving The Youth Most In Need. 1. Purpose . The purpose of this Training and Employment Guidance Letter TEGL is to provide guidance to states workforce investment boards and Workforce Investment Act WIA youth service providers on contracting strategies that facilitate servic e providers to serve the youth most in need while still achieving performance goals. 2. References . U.S. Government Accountability Office Disconnected Youth Federal Action Could Address Some of the Challenges Faced by Local Programs That Reconnect Youth to Education and Employment February 2008 http www.gao.gov new.items d08313.pdf TEGL No. 03-04 The Employment and Training Administration s ETA s New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act http wdr.doleta.gov directives corr doc.cfm DOCN 1589 TEGL No. 09-08 Negotiating Performance Goals for the Workforce Investment Act Title IB Programs and Wagner-Peyser Act Funded Activities for Program Year 2009 http wdr.doleta.gov directives corr doc.cfm DOCN 2702 TEGL No. 09-08 Change 1 Revised National Performance Goal Targets for the Government Performance Results Act GPRA to support Negotiating State Performance Goals for the Workforce Investment Act Title IB Programs Wagner-Peyser Act Funded Activities for Program Year 2009 http wdr.doleta.gov directives corr doc.cfm DOCN 2770 3. Background . In a February 2008 report reviewing 39 local youth programs the Government Accountability Office GAO found that contracts between local workforce investment boards WIBs and youth service providers often require providers to meet performance goals within time frames that are too short to reasonably achieve successful outcomes with youth most in need page 4 . The hardest-to-serve youth like many dropouts and out-of-school youth require additional time and assistance to attain positive outcomes as measured by the WIA Youth common and statutory measures. However the findings revealed that workforce boards often procure one year contracts which may unintentionally discourage programs from working with 2 lower skilled youth who may be unlikely to achieve positive outcomes during the term of the contract. The GAO recommended that ETA work with states and WIBs to provide youth program operators with information and guidance needed to develop and implement contracts that facilitate local programs to serve the neediest youth while still achieving performance goals. ETA has emphasized service to the youth most in need particularly dropouts and out-of-school youth. ETA encourages longer-term services for these youth in the WIA Youth program to prepare them for post-secondary education and or the workforce. This guidance letter is intended to provide strategies and tools for states WIBS and local service providers to help them negotiate performance requirements and develop contracts that encourage the system to enroll youth most in need and deliver the long-term comprehensive services needed for successful outcomes. Ensuring accountability while still encouraging service to the hardest-to-serve participants has long been a challenge within the workforce development system. Performance measures play a crucial role in ensuring that the WIA Youth program fulfills its objective to prepare youth to compete in the 21 st century economy. However research on the workforce development system also suggests that performance goals can act as an unintended incentive to skim the cream from the applicant pool or serve only those eligible individuals most likely to achieve positive outcomes. A 2002 GAO report found that the need to meet performance levels may drive who receives WIA-funded services. Consequently it is crucial that contracts are structured to mitigate or at least not exacerbate disincentives to serve the youth most in need of services. In November and December 2008 ETA held a series of focus groups with WIBs local youth service providers and state youth staff to discuss the challenges local programs face in developing contracts that promote service to the neediest youth. The focus groups provided input around several areas of program contracting and management a local workforce area procurement guidelines and processes b innovative strategies and processes that have produced contracts that encourage service to the youth most in need and 3 procedures and practices that may inhibit program collaboration for serving these youth. The focus group participants echoed many of the challenges identified in the February 2008 GAO report however they placed less emphasis on the length of contracts many already employed multi-year or option-year contracts with an average contract length of 2.9 years and more on performance goal negotiations and enrollment requirements as barriers to serving the neediest youth. Several focus group participants shared that performance requirements have caused them to enroll participants based on their likelihood of achieving a successful outcome rather than their ability to benefit from WIA services. ETA has based the recommendations in this guidance letter on input from the focus groups and on responses from direct service providers WIBs and state agencies administering WIA Youth funds. 4. Strategies to Promote Service to the Youth Most in Need . Findings from the focus groups and discussions with other partners within the workforce development system revealed several contracting strategies that could promote service to the neediest youth. These strategies fall into six main categories Employing Multi-Year or Option-Year Contracting Basing Enrollment Goals on a Total Participant Count in a Performance Period Using Real-Time Indicators or Interim Progress Benchmarks to Assess Provider Performance in addition to WIA Youth Common or Statutory Measures Factoring Participant Characteristics and External Factors into Performance Goal 3 Negotiations Creating Positive Incentives for Youth Service Providers to Enroll Youth who are Most in Need of Services and to Provide Services that will help these Youth Achieve Successful Outcomes Encouraging the Use of Interagency Collaboration to Create a Seamless System in which funds from other sources provide Services to Stabilize Basic Shelter Health and Safety Needs and Meet the Broad Developmental Needs of Youth prior to their Enrollment in WIA for Services Targeted to Career Education and Employment 5. Employing Multi-Year or Option-Year Contracting . Most of the 24 local WIBs and service provider focus group participants indicated that they employ multi-year or option-year contracts. While nine of the participants indicated that they had one-year contracts the participants asserted that unless there were serious performance issues the renewal of their contract was virtually assured. However as the GAO report found short-term contracts between local service providers and WIBs can create a barrier to serving the youth most in need. Youth with many barriers to education and employment require intensive long-term services making it unrealistic for these youth to achieve successful outcomes on the WIA Youth common or statutory measures within the one-year timeframes written into many contracts between WIBs and service providers. Consequently service providers on one-year contracts may be reluctant to enroll youth who need intensive assistance. Local WIBs that employ one-year contracts should consider replacing them with multi-year and option-year contracts to foster extended service to youth most in need. Option-year contracts have the advantage of maintaining annual accountability goals by tying future years of the contract to performance in the base year. Additionally option-year and multi-year contracts help reduce procurement costs by limiting the frequency with which WIBs need to issue requests for proposals. WIBs and state agency staff reported that multi-year contracts provide stability to programs so that providers do not feel that funds will be taken away from them due to circumstances out of their control. Multi-year contracts allow providers more time to implement comprehensive program models. 6. Basing Enrollment Goals on a Total Participant Count in a Performance Period . Enrollment requirements built into contracts also can have an unintended detrimental impact on service to youth most in need. Several service providers stated that their contracts require them to enroll a certain number of new participants each month or each year. Other focus group participants indicated that their contracts mandated a certain number of exits 3 within a given performance period. Requiring new enrollees or mandating a certain number of exits may have the unintended effect of causing service providers to prematurely stop service to some youth or to only enroll youth who can achieve program goals in a short period of time. ETA encourages local WIBs and providers to track program participation in the same manner that ETA itself tracks participants. Each year ETA reports national participation in the program on the basis of the number of youth served not on the basis of new program enrollees. Accordingly a youth served in Program Year PY 2007 and PY 2008 would appear in the participant counts for both years. 4 Instead of requiring a specific number of new enrollees or exits local WIBs should consider requiring service providers to maintain a minimum or average enrollment level that ensures that service providers are operating at a defined capacity while allowing them to serve youth based on their needs including those who need assistance for an extended period of time . For example a WIB may set an enrollment goal of 100 participants in the first year of a contract with a service provider. In the second year of the contract the enrollment goal would then transition into a service goal of 100 participants which would include youth carried over from the first year and any new enrollees. This would allow providers to serve youth who need additional assistance for multiple years without fear of failing to meet contractual enrollment targets. 7. Using Real-Time Indicators or Interim Progress Benchmarks to Assess Provider Performance in addition to WIA Youth Common or Statutory Measures . The common and statutory measures with the exception of the Literacy Numeracy and the Skill Attainment measures track outcomes that cannot be measured until well after participants have exited from the program e.g. six months or more after exit . In fact these measures were never intended to be used to track the performance of service providers in real time and as such do not lend themselves to use as program management tools. Nevertheless many service providers are being measured using common or statutory measures and only a small fraction of service providers indicate that their contracts contain real-time performance measures. For better provider and WIB tracking of outcomes during and immediately after the end of the contract period ETA encourages local WIBs to adopt real-time or interim indicators. Short- term or real-time measures are changes in attitudes behaviors knowledge and skills that can be expected to occur during an individual s participation in a specific program or immediately upon completion of a program or service. Required short-term outcomes can be included in service provider Requests for Proposals RFPs to assist potential service providers in proposing services that will lead to the desired short-term outcomes. Short-term outcomes set the stage for monitoring contractor performance and establishing whether a contractor may deserve a bonus or incentive payments under performance-based or hybrid contract designs which provide some payments based on the delivery of the proposed services and reserve some incentive or bonus payments for the achievement of desired participant outcomes . Effective short-term or real time measures can serve as proxies for desired intermediate and long-term outcomes as they help predict whether the desired outcomes will be achieved. Since they measure the impact of specific programs or services upon individual participants short-term measures may need to be customized for each service design and provider. Three criteria that can be used to assess the appropriateness of short-term indicators are a whether the outcome measures can be attributed to the services provided b whether the outcomes can be achieved during the contract period and c whether the outcome measures provide useful data about program performance. The following are examples of short-term measures that might be relevant a substantial learning gain such as grade level improvement b completion of at least one major project demonstrating progress toward or acquisition of basic skills and work-readiness skills per program year c an increase in the retention rate for youth participating in alternative education activities d for youth without a General Equivalency Diploma GED or high school diploma completion of an official GED practice test with a minimum score of 45 e for youth with a credit deficiency relative to high school graduation requirements completion of a specified number of credits toward earning a high school diploma f high school grade-level promotion 5 g competencies in specific subject areas h 30-day retention or other interval in positive exit status i improved attitudes toward reading and j increased school attendance. A. Basic Principles to Consider When Picking Interim Indicators. Developing interim measures that relate to common or statutory measures. Common or statutory measures represent milestones that have been achieved within two to five years of program commencement. One of the most important tasks of any interim measure is to provide some real-time indication of progress toward common or statutory measures. Thus if the common measure tracks a given outcome only 18 months after the exit of a participant then the interim measure might track at the point of exit whether the participant has achieved a milestone on their way to meeting the measure. For instance to demonstrate progress toward degree or certificate attainment interim indicators might track satisfactory in-program progress completion of classes or programs leading to a degree or certificate or the achievement of program milestones that occur prior to degree or certificate attainment. Certificate or degree attainment may also be used as an interim measure. This may be particularly useful for youth enrolled in degree or certificate programs that require more than one year for completion. To demonstrate progress toward placement in employment or education service providers could track such attainment in real time and thereby avoid the tracking delays that are associated with the post-exit tracking that is defined under common and statutory measures. For more comprehensive program management service providers may also track work readiness attainment completions of employment applications or job placement interviews. To provide real-time management information in support of the attainment of literacy numeracy skills service providers can track training and test progress between the pre-and post-tests or even specific assignment or project completion. For example the State of Delaware s WIB asks respondents to its Youth RFP to track WIA performance outcomes at specific dates in time that are earlier than they would normally be measured. To assist in monitoring the retention of youth in employment and other outcomes providers are responsible for reporting on participant retention in employment and other outcomes at the following points of time including Day 1 of employment Day 30 of employment continuous from day 1 Day 60 of employment continuous from day 1- Collecting this measure is optional Day 90 of employment continuous from day 1 with a maximum 15-day gap being allowable if the Day 30 goal was met Another example is the San Diego Workforce Partnership which negotiates performance standards and outcomes with each of its youth service providers at the beginning of every program year. Included in each contractor s Program Operating Plan are performance requirements for Real-Time Common Measures measured at program exit including placement in employment or education attainment of degree or certificate and literacy numeracy gains measured separately for in-school and out-of-school youth. Key to implementing the Real-Time Common Measures is the Customer Information Services Reporting System CISRS that produces monthly management reports on each contractor s annual performance to date. Contractors are held accountable for their performance on these measures at the end of the 2nd and 4th quarters of each contract year. 6 Develop interim indicators that are achievable in the time frame of the contract. Interim indicators should be applied in a way so that performance goals can be achieved within one contract year. When developing measures that are time frame appropriate WIBs should consider the types of goals or outputs that relate to the common measures that could be reasonably achieved by service providers within one contract year. Interim indicators may be useful for tracking daily weekly monthly or quarterly progress. Interim measures can also be tailored to specific contractors. Keep data collection simple. The development of interim measures should not produce significant additional data collection burdens for service providers. This can be achieved if indicators are based on data elements that are already part of the Workforce Investment Act Standardized Record Data WIASRD reporting information data set or based on data elements that service providers routinely record as part of their normal business processes. In addition for improved data accuracy data collection and data reporting procedures should include frontline staff when possible. If interim measures do require additional data collection be sure to provide user-friendly data collection tools where possible and inform service providers and contractors of new data collection requirements in the RFP. Consider the program management benefits of interim measures. Interim indicators are also a useful way for WIBs to manage oversight of service providers. While interim indicators associated with achieving the common or statutory measures provide useful insights into the effectiveness of a particular service or program they may not be useful for managing your program or tracking the management of your service providers. Developing enrollment attendance or participation measures or indicators to measure caseload size client demographics or client-staff ratios may provide important program management information. For instance service providers will likely differ in the number of hard-to-serve youth enrolled in their programs or the ratio of staff to hard-to-serve clients. WIBs may wish to set different benchmarks for contractors serving primarily hard-to-serve youth to reflect population-specific challenges. Developing interim measures that take this into account may provide information to help with future RFPs help you develop more individualized contracts or at the very least explain lower than expected outcomes. Use interim measures to reflect achievements or outcomes not captured by common or statutory measures. Interim measures offer WIBs and service providers the opportunity to demonstrate their strengths in areas not reflected by the statutory and common measures. For instance a WIB may have a particular interest in tracking skill attainment or job placement in a specific trade. WIBs may wish to apply different indicators for contractors serving youth with particular challenges. A service provider may have a particular strength in reducing gang-affiliation among previously gang-affiliated youth. A local area may have a large number of homeless youth who can benefit from education and job placement but first need access to drug rehabilitation programs or appropriate health care. When developing interim measures consider the specific needs of your local area the specific strengths of your service providers in meeting those needs and the measures that would capture the alleviation of these needs. While these measures may not directly relate to the statutory or common measures they will provide critical information about service provision to specific populations or in targeted trades. For example the Fresno County WIB in California reviews and reports on real time measures for youth. Among these measures are the number of youth in the active caseload 7 and the number pending high school graduation. The WIB categorizes all youth participants as being in one of two tracks- an academic upgrade track and a career track. For youth in the academic upgrade track the short-term outcome of interest is whether the youth is on track to achieve a two-grade increase or 10 th grade in reading and math. Once youth have achieved this outcome they enter the career track which begins with the assessment of employability skills and work experience. Youth who have not yet graduated from high school are tracked according to whether they are proceeding toward high school graduation. Inform contractors about interim measures. As with any new measure or data collection requirement make sure that you clearly communicate your expectations of your contractors during the RFP and contract development period. Provide a detailed explanation of data collection and reporting procedures for each interim measure collected through the contract. Include sample report forms within the RFP and contract. Outline a list of acceptable documentation of measure attainment. Supply contractors with easy-to-use data collection tools or systems to facilitate the collection of interim measure data. 8. Factoring Participant Characteristics and External Factors into Performance Goal Negotiations . Many focus group participants agreed that annually renegotiated performance goals with steady increases in required performance outcomes a process that one respondent called performance creep makes it more difficult to serve the hardest-to-serve youth effectively and succeed in meeting performance outcomes especially over time as WIA has experienced several successive increases in performance expectations. Furthermore most respondents at the state and local level reported that annual performance goal negotiations do not take into account the risk factors of the participant population or external factors that may affect performance outcomes such as unemployment rates. In recent TEGLs the Department of Labor describes several approaches to negotiating performance goals so as not to discourage services to youth most in need particularly in local labor markets with increasing levels of unemployment. These approaches include negotiating adjusted performance goals based on the level of services to youth most in need based on the higher level of risk involved in achieving successful outcomes for these youth and negotiating adjusted performance goals based on the level of unemployment in the local economy for jobs likely to be targeted by youth most in need. A. Negotiating adjusted performance goals based on the level of services to youth most in need. When negotiating the Youth goals states should be aware that ETA s strategic vision for youth services includes a focus on serving the youth most in need especially out-of school youth youth in foster care youth in the juvenile justice system children of incarcerated parents and migrant youth as well as youth with disabilities and Native American youth. In negotiating with ETA for appropriate Youth performance goals states that currently serve a high level of the neediest youth or that want to transition to serving a higher percentage of youth who are most in need are encouraged to provide data that shows how performance outcomes are impacted by serving this population. States are encouraged to bring appropriate information to the negotiation process that demonstrates either their past performance in serving these populations and or their current strategy for serving these populations. Relevant information as described in TEGL 09-08 may include historical data on performance between PY 2005 and PY 2008 national performance 8 goals set under the Government Performance Results Act GPRA data on the national distribution of WIA outcomes by region and national regional or local data on how the characteristics of participants influences outcomes. In Attachment 7 to TEGL 09-08 ETA provided the results of a number of different bivariate regression analyses to support state-by-state adjustments based on the characteristics of youth served. These adjustment factors were based on national-level WIASRD data on outcomes for exiters in the most recent year available for each performance measure. The adjustment factors summarize how outcomes are influenced by marginal changes in the proportions of youth participants who are female belong to different age groupings are high school drop-outs are basic skills deficient or have a disability. In TEGL 09-08 Change 1 ETA announced some important refinements to the statistical models that are available to support adjustments to state and local performance targets for PY 2009. In addition to modeling the effects of state and local unemployment rates on performance outcomes new multivariate models will provide estimates of how changes over time in participant characteristics influence national state-by-state and WIB-by-WIB performance outcomes. The estimates provide weights that will be used to reflect the effects of individual participant characteristics such as work history educational attainment limited English skills and disability variables on expected performance outcomes. B. Negotiating adjusted performance goals based on the level of unemployment in the local economy. In the current recession adjustments to performance goals based on the characteristics of the local labor market e.g. unemployment rate job growth rate are perhaps even more important than adjustments based on the characteristics of participants enrolled. TEGL 09-08 Change 1 announced the lowering of national WIA performance goals under the Government Performance Results Act GPRA in response to increased unemployment levels throughout the nation. This reduction in overall Federal performance expectations under WIA acknowledges that current economic conditions will make it more difficult to achieve successful outcomes for both adult and youth participants in WIA. Attachment 7 to TEGL 09-08 includes estimates of how unemployment rates affect performance outcomes based on bivariate regressions of national-level WIASRD data on outcomes for exiters in the most recent year available for each performance measure. As reported in TEGL 09-08 Change 1 refined multivariate models have recently been developed to provide estimates of how changes in unemployment rates over time influence national state-by-state and WIB-by-WIB performance outcomes. As states negotiate adjusted performance outcomes in response to economic pressures and planned characteristics of WIA Youth participants ETA encourages WIBs to make parallel adjustments to the expected performance outcomes they establish for youth providers taking into account both local economic conditions and the characteristics of the youth served by each provider. Thus providers serving a higher percentage of homeless or foster youth could have their performance goals adjusted downwards to account for the higher level of service to youth most in need and providers serving fewer youth in need could have their performance goals adjusted upwards. 9. Creating Positive Incentives for Youth Service Providers to Enroll Youth who are Most in Need of Services and to Provide Services that will help these Youth achieve Successful 9 Outcomes . Avoiding contracting practices that create disincentives to serve the neediest youth is important but equally important for states and local WIBs is to emphasize their policy goals of serving the youth most in need and to create positive incentives to encourage service providers to further these goals. Among the positive strategies that WIBs can use to promote services to youth most in need are the following Including clear statements of the WIBs desire to serve youth from the targeted groups in its RFPs and other communications with prospective service providers. Using the extent of services proposed to youth most in need as an announced criterion in rating provider proposals. Using state-level set-aside funds under WIA or the Recovery Act to issue RFPs for special projects to serve youth most in need. In contracts designing hybrid cost reimbursement and fixed price contracts with payment points to provide special bonus payments or higher rates of reimbursement for serving youth most in need. 10. Encouraging the Use of Interagency Collaboration to Create a Seamless System in which Funds from other sources Provide Services to Stabilize Basic Shelter Health and Safety Needs and Meet the Broad Developmental Needs of Youth Prior to their enrollment in WIA for Services Targeted to Career Education and Employment. Another strategy that is important in furthering the goals of serving youth most in need and achieving successful youth outcomes is the development of broad youth partnerships at the Federal state and local levels so that a variety of funding sources can be used to create a seamless array of services to meet the needs of participants for comprehensive intensive and long-term services. ETA encourages states and local areas to develop formal partnerships that will allow multiple agencies and organizations to fund different aspects of the youth service system. Each local system of services can be coordinated through a combination of contracts and memoranda of understanding between the participating entities. This strategy has the potential to reduce the performance pressure on WIA service providers by sharing the investment in service reforms with other agencies thus making sure the full range of needed services is available to youth using carefully constructed assessment and referral linkages between partners. As a result of such a strategy funds from other agencies like the Department of Justice under programs for incarcerated youth or youth on probation or the Department of Health and Human Services under programs for homeless or foster youth could provide front-end services for pre- employment preparation and or meeting broad developmental needs of youth and making sure that their immediate housing health and safety needs are stabilized before they are enrolled in WIA. Another example of collaborating across agencies is WIA youth programs partnering with WIA Title II Adult Education programs in order to access the basic skills remediation services many of the out of school youth in WIA programs need in order to be successful in education and the workforce. 11. Inquiries . Questions regarding this guidance should be directed to the appropriate ETA Regional Office.