TEGL15-03.pdf

ETA Advisory File Text

1 TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 15-03 TO ALL STATE WORKFORCE LIAISONS ALL STATE WORKFORCE AGENCIES s FROM EMILY STOVER DeROCCO Assistant Secretary SUBJECT Common Measures Policy 1. Purpose . To inform states and grantees of the Employment and Training Administration s ETA policy on common performance measures for Federal job training and employment programs including guidelines on program participation and exit. The common measures represent one component in ETA s development of a single comprehensive system to collect record and report program performance for workforce programs. 2. References . President s Management Agenda http www.whitehouse.gov omb budintegration pma index.html OMB Director s Memorandum M-02-06 http www.whitehouse.gov omb budintegration common.html Training and Employment Notice No. 8-02 Implementation of Common Performance Measures for Job Training and Employment Programs. 3. Background . ETA s statutory and regulatory authority to administer job training and employment programs includes provisions allowing for the requirement of performance reporting from states and grantees. ETA is in the process of developing and implementing a single comprehensive system to collect record and report program performance for workforce programs. This system will include a new individual record for participants a new consolidated report of workforce activities financial reports that relate costs to activities common performance measures data validation and information on services provided to employers through the One-Stop system. In 2001 the President announced a Management Agenda to improve the management and performance of the Federal government. One of the five government-wide goals budget and performance integration emphasizes program effectiveness. As part of the President s budget and performance integration initiative the Office of Management and Budget OMB and other Federal agencies developed performance measures for certain programs with similar goals. OMB Director s Memorandum M-02-06 and subsequent documents from OMB specify indicators of performance for job training and employment programs. CLASSIFICATION WIA Performance Reporting CORRESPONDENCE SYMBOL PRO EMPLOYMENT AND TRAINING ADMINISTRATION ADVISORY SYSTEM U.S. DEPARTMENT OF LABOR Washington D.C. 20210 DATE December 10 2003 RESCISSIONS EXPIRATION DATE Continuing 2 The common measures are an integral part of ETA s performance accountability system however these measures provide only part of the information necessary to effectively oversee the workforce investment system. ETA will continue to collect from states and grantees all the data on program activities participants and outcomes that is necessary for program management and to convey full and accurate information on the performance of workforce programs to policymakers and stakeholders. This data collection will continue to include information on individuals facing significant barriers to employment such as individuals who have disabilities have Limited English Proficiency are age 55 or older are homeless and are migrant and seasonal farmworkers. Four common measures apply to programs serving adults and four common measures apply to youth programs Adult Measures Entered Employment Retention Earnings Increase Efficiency Youth Measures Placement in Employment or Education Attainment of a Degree or Certificate Literacy and Numeracy Gains Efficiency Programs administered by six Federal agencies the Departments of Labor Education Health and Human Services Veterans Affairs Interior and Housing and Urban Development are subject to the common measures. Attachment A contains a list of Federal programs outside of ETA impacted by the common measures. The implementation of common performance measures across Federal job training and employment programs will enhance the ability to assess the effectiveness and impact of the workforce investment system including the performance of the system in serving individuals facing significant barriers to employment. Multiple sets of performance measures have burdened states and grantees as they are required to report performance outcomes based on varying definitions and methodologies. By minimizing the different reporting and performance requirements common performance measures can facilitate the integration of service delivery and break down barriers to cooperation among programs. This guidance includes the following sections Section 4 Program Participation and Exit Page 3 Section 5 Employment and Training Administration Common Measures Policy Page 5 Section 6 Data Sources Page 15 Attachment A Federal Job Training and Employment Programs Impacted by Common Measures Page 18 Attachment B Common Measures At-A-Glance Page 19 Attachment C Definitions of Key Terms Page 20 Attachment D Educational Functioning Level Descriptors Page 23 3 4. Program Participation and Exit . A. Point of Program Participation for Reporting on Common Measures When does program participation occur Individuals who are determined eligible and receive any service funded by the program in a physical location e.g. a One-Stop career center are participants. The criteria that are used to determine whether an individual is eligible to participate will be based on the guidelines for the program. Note Individuals who only receive a determination of eligibility but do not receive an additional service are not participants. Individuals who visit a physical location for reasons other than its intended purpose e.g. use of restrooms or ask staff for directions are not participants. States and grantees may choose but are not required to consider as participants those individuals who receive services that are available through the Internet and are not accessed through a physical location e.g. a One-Stop career center . What is the date of participation The date of participation is the date of the first service received after the individual is determined eligible. When do individuals count in the common performance measures All participants will be taken into account when measuring performance using the common measures. There may be some circumstances in which certain participants counted in the common measures may not be included in other measures of performance due to statutory or other reasons. For example under WIA Section 136 b 2 A and WIA Regulations 666.140 the core indicators of performance apply to all individuals registered for the Adult and Dislocated Worker programs except for those individuals who participate exclusively in self-service or informational activities. Any such circumstances will be addressed in future program guidance. B. Point of Exit for Reporting on Common Measures What is the purpose of exit For performance accountability and reporting purposes the term exit is used in most of the common measures except the efficiency as well as the literacy and numeracy gains measures to identify which participants are to be included in the computation of the measures. Comparability of performance information across states and other ETA grantees is possible only if a single point-in- time is used to begin measurement. Any participant who exits from a program becomes part of an exit cohort a group of individuals who are determined to be exiters and used collectively for performance measurement purposes. 4 When does exit from the program occur Exit occurs when a participant does not receive a service funded by the program or funded by a partner program for 90 consecutive calendar days. The term service does not include activities that are open to non-participants services and activities specifically provided as follow-up services or regular contact with the participant or employer to only obtain information regarding his her employment status educational progress or need for additional services. What is the date of exit The date of exit is the date on which the last service funded by the program or a partner program is received by the participant. Are there any exceptions to the definition of exit Participants who have a planned gap in service of greater than 90 days should not be considered as exited if the gap in service is due to one of the following Delay before the beginning of training Health medical condition Temporary move from the area that prevents the individual from participating in services Service providers grantees must document any gap in service that occurs and indicate the reason for the gap in service. How are youth who also receive adult services counted in the measures In the case of a youth participant who becomes a participant receiving adult services i.e. receiving services funded with both youth and adult funds the individual will be counted in both the youth and adult common measures. On the other hand neither set of exit-based measures will be applied until the individual has exited the adult program services and any other partner services . C. Excluding Participants from the Common Measures What is the purpose of excluding certain participants from the measures Occasionally circumstances arise that are beyond the control of both the participant and the program. The intent here is to identify a common list of specific reasons as to why a participant can be excluded from common performance measure calculations. Service providers and grantees must document the reason that excludes the participant from performance calculations. What are the specific exclusions Participants in the following categories either at the time of exit or during the three-quarter measurement period following the exit quarter will be excluded from the measures 5 Institutionalized - The participant is residing in an institution or facility providing 24-hour support such as a prison or hospital and is expected to remain in that institution for at least 90 days. This reason does not apply to the Responsible Reintegration of Youthful Offenders program. For purposes of exclusion from the common measures individuals with disabilities as defined in 29 CFR 37.4 residing in institutions nursing homes or other residential environments are not considered to be institutionalized. Health Medical or Family Care - The participant is receiving medical treatment or providing care for a family member that precludes entry into unsubsidized employment or continued participation in the program. This does not include temporary conditions or situations expected to last for less than 90 days. Deceased - Self-explanatory. Reservists Called to Active Duty - The participant is a reservist who is called to active duty for at least 90 days. 5. Employment and Training Administration Common Measures Policy . All programs administered by ETA will be subject to the common measures policy unless explicitly excluded. The ETA programs required to implement the adult and or youth common measures include the following Program Common Measures H-1B Technical Skills Training Adult Job Corps Youth Labor Exchange 1 Adult Migrant and Seasonal Farmworkers Adult Native American Employment and Training Adult and Youth Responsible Reintegration of Youthful Offenders 2 Youth Senior Community Service Employment Program Adult Trade Adjustment Assistance Adult WIA Adult Adult WIA Dislocated Workers including National Emergency Grants Adult WIA Youth Youth 1 ETA and the Veterans Employment and Training Service VETS share a data collection and reporting system. While the VETS-funded grant programs are not included in the above list of ETA programs decisions impacting the application of the common measures to the labor exchange will extend to those activities reported via the Labor Exchange Reporting System. ETA and VETS will provide additional guidance as necessary. 2 Future grantees of the Responsible Reintegration of Youthful Offenders program will be subject to the common measures. Grantees receiving funding under this program through the Department of Justice will not be subject to this Department of Labor policy document. 6 Designation that a program is subject to the adult and or youth measures in no way impacts the existing age eligibility requirements of the program. ETA s common measures policy will be implemented during Calendar Year CY 2004. Guidelines on specific implementation timelines in CY 2004 for each program and any accommodations needed to implement the common measures including those that might be required to effectively assess performance in serving people with disabilities will be issued separately for each program. A notice on new reporting instructions which will include information on the common measures will be placed in the Federal Register for public comment in early 2004. Final reporting guidelines will be issued in spring 2004. The remainder of this section describes the methodologies of the measures. Operational parameters and definitions of key terms for each measure are also provided in this section to further clarify details needed to implement the measures. The adult common measures are discussed in Part B of this section the youth measures are addressed in Part C and the efficiency measure which applies to both adult and youth programs is discussed in Part D. A. Introduction to Common Measure Methodologies The methodologies of the common measures are written as an equation clearly identifying who is in the numerator and who is in the denominator. In cases where there are conditions that apply to both the numerator and denominator the condition is represented in italics at the beginning of the measure. For example the adult entered employment rate is defined as Of those who are not employed at the date of participation The number of adults who are employed in the first quarter after exit divided by the number of adults who exit during the quarter. The condition of those not employed at the date of participation applies to both the numerator and denominator as follows the number of adults who are not employed at the date of participation and are employed in the first quarter after exit divided by the number of adults who are not employed at the date of participation and exit during the quarter. In addition the phrase who exit during the quarter is used frequently in the denominators of the measures. This phrase refers to the exit quarter. B. Adult Measures Entered Employment Methodology Of those who are not employed at the date of participation The number of participants who are employed in the first quarter after exit divided by the number of participants who exit during the quarter. Operational Parameters Individuals who are employed at the date of participation are excluded from this measure i.e. programs will not be held accountable for these individuals under this measure . 7 Employment at the date of participation is based on information collected from the individual not from wage records. Definitions Employed at the Date of Participation - An individual employed at the date of participation is one who during the seven consecutive days prior to the date of participation Did any work at all as a paid employee except the individual is NOT considered employed if a he she has received a notice of termination of employment or the employer has issued a Worker Adjustment and Retraining Notification WARN or other notice that the facility or enterprise will close or b he she is currently on active military duty and has been provided with a firm date of separation from military service Did any work at all in his her own business profession or farm Worked 15 hours or more as an unpaid worker in an enterprise operated by a member of the family or Was not working but has a job or business from which he she was temporarily absent because of illness bad weather vacation labor-management dispute or personal reasons whether or not paid by the employer for time off and whether or not seeking another job. Employed in First Quarter After Exit Quarter - The individual is considered employed if wage records for the first quarter after exit show earnings greater than zero. Discussion ETA recognizes concerns that have been raised in regards to excluding individuals who are employed at the date of participation from this measure. However including individuals with jobs at program entry does not provide an assessment of program impact in increasing employment and is not fully consistent with the concept of an entered employment indicator. The positive impact of services provided to incumbent workers and the underemployed can be demonstrated in both the retention and earnings measures. To ensure a comprehensive understanding of job attainment ETA will also collect information on the post-program employment status of individuals who are employed at program entry. Retention Methodology Of those who are employed in the first quarter after exit The number of participants who are employed in both the second and third quarters after exit divided by the number of participants who exit during the quarter. Operational Parameters Individuals who are not employed in the first quarter after exit are excluded from this measure i.e. programs will not be held accountable for these individuals under this measure . Employment in the first second and third quarters following exit does not have to be with the same employer. 8 Definitions Employed in First Second and Third Quarters After Exit Quarter - The individual is considered employed if wage records for these quarters show earnings greater than zero. Discussion A positive outcome on the retention measure does not necessarily indicate continuous employment. However by defining a positive outcome as employment in the first second and third quarters after exit the measure approximates retention for six months following entry into employment. Earnings Increase Methodology - Earnings Increase 1 Of those who are employed in the first quarter after exit Earnings in the first quarter after exit minus earnings in the quarter prior to the date of participation divided by earnings in the quarter prior to the date of participation. Methodology - Earnings Increase 2 Of those who are employed in the first quarter after exit Earnings in the third quarter after exit minus earnings in the first quarter after exit divided by earnings in the first quarter after exit. Operational Parameters Individuals who are not employed in the first quarter after exit are excluded from both earnings increase measures i.e. programs will not be held accountable for these individuals under this measure . Definition Employed in First Quarter After Exit Quarter - The individual is considered employed if wage records for the first quarter after exit show earnings greater than zero. Discussion The earnings increase measure has two parts. Earnings increase 1 is intended to evaluate program impact on earnings by comparing pre-program to post-program earnings. Earnings increase 2 is designed to capture earnings increases that occur as a result of raises promotions or other advancements following program exit. For purposes of the common measures the pre-program earnings of dislocated workers will be determined as described above one quarter prior to the date of participation regardless of the date of dislocation. However ETA will continue to collect the data necessary to determine a wage replacement rate for dislocated workers that is based on wages at the point of dislocation. 9 C. Youth Measures Placement in Employment or Education Methodology Of those who are not in post-secondary education employment or the military at the date of participation The number of participants who are in employment or the military or enrolled in post-secondary education and or advanced training occupational skills training in the first quarter after exit divided by the number of participants who exit during the quarter. Operational Parameters Individuals who are in post-secondary education employment or the military at the date of participation are excluded from this measure i.e. programs will not be held accountable for these individuals under this measure . Employment military and education status at the date of participation are based on information collected from the individual. Individuals in secondary school at the date of participation regardless of employment status will be included in this measure. Individuals in secondary school at exit are excluded from this measure i.e. programs will not be held accountable for these individuals under this measure . Definitions Advanced Training Occupational Skills Training - An organized program of study that provides specific vocational skills that lead to proficiency in performing actual tasks and technical functions required by certain occupational fields at entry intermediate or advanced levels. Such training should 1 be outcome-oriented and focused on a long-term goal as specified in the Individual Service Strategy ISS 2 coincide with exit rather than short-term training that is part of services received while enrolled in ETA-funded youth programs and 3 result in attainment of a certificate as defined on pages 10 and 11. Employed at the Date of Participation - An individual employed at the date of participation is one who during the seven consecutive days prior to the date of participation Did any work at all as a paid employee except the individual is NOT considered employed if a he she has received a notice of termination of employment or the employer has issued a WARN or other notice that the facility or enterprise will close or b he she is currently on active military duty and has been provided with a firm date of separation from military service Did any work at all in his her own business profession or farm Worked 15 hours or more as an unpaid worker in an enterprise operated by a member of the family or Was not working but has a job or business from which he she was temporarily absent because of illness bad weather vacation labor-management dispute or personal reasons whether or not paid by the employer for time off and whether or not seeking another job. Military Status at the Date of Participation - An individual is considered to be in the military at the date of participation if a he she currently is serving on active military duty and has NOT yet been provided with a firm date of separation from military service or b he she is a member of the 10 National Guard or one of the Military Reserves AND currently is serving in a mobilized i.e. active military duty status. Employed or in the Military in First Quarter After Exit Quarter - The individual is considered employed or in the military if wage records for the first quarter after exit show earnings greater than zero. Post-Secondary Education - A program at an accredited degree-granting institution that leads to an academic degree e.g. A.A. A.S. B.A. B.S. . Programs offered by degree-granting institutions that do not lead to an academic degree e.g. certificate programs do not count as a placement in post-secondary education for this measure but may count as advanced training occupational skills training. Discussion Participants who enter ETA-funded youth programs while already in post-secondary education employment or the military have achieved the desired outcome under this measure. Thus these individuals are excluded from this measure. Successfully returning young people to school or alternative education is desirable but secondary school enrollment does not qualify as a placement under this measure. To avoid penalizing programs for serving these youth this measure excludes youth participants who are in secondary school at exit and treats secondary school enrollment as a neutral outcome. Attainment of a Degree or Certificate Methodology Of those enrolled in education at the date of participation or at any point during the program The number of participants who attain a diploma GED or certificate by the end of the third quarter after exit divided by the number of participants who exit during the quarter. Operational Parameters Education refers to participation in secondary school post-secondary school adult education programs or any other organized program of study that leads to a GED General Educational Development High School Diploma equivalency test diploma or certificate. Individuals in secondary school at exit are excluded from this measure i.e. programs will not be held accountable for these individuals under this measure . Diplomas GEDs or certificates can be obtained while a person is still receiving services or at any point prior to the end of the third quarter after exit. Definitions Certificate - A certificate is awarded in recognition of an individual s attainment of measurable technical or occupational skills necessary to gain employment or advance within an occupation. These technical or occupational skills are based on standards developed or endorsed by employers. Certificates awarded in recognition of the attainment of only generic pre-employment or work readiness skills are not included in this definition. A certificate is awarded in recognition of an individual s attainment of technical or occupational skills by 11 A state educational agency or a state agency responsible for administering vocational and technical education within a state. An institution of higher education described in Section 102 of the Higher Education Act 20 USC 1002 that is qualified to participate in the student financial assistance programs authorized by Title IV of that Act. This includes community colleges proprietary schools and all other institutions of higher education that are eligible to participate in Federal student financial aid programs. A professional industry or employer organization e.g. National Institute for Automotive Service Excellence certification National Institute for Metalworking Skills Inc. Machining Level I credential or a product manufacturer or developer e.g. Microsoft Certified Database Administrator Certified Novell Engineer Sun Certified Java Programmer using a valid and reliable assessment of an individual s knowledge skills and abilities. A registered apprenticeship program. A public regulatory agency upon an individual s fulfillment of educational work experience or skill requirements that are legally necessary for an individual to use an occupational or professional title or to practice an occupation or profession e.g. FAA aviation mechanic certification state certified asbestos inspector . A program that has been approved by the Department of Veterans Affairs to offer education and training to veterans and other eligible persons under provisions of the Montgomery GI Bill. Office of Job Corps. Discussion Youth who exit ETA-funded programs while still in secondary school have not yet had the opportunity to attain a degree or certificate and cannot demonstrate a positive outcome under this measure. Exclusion of youth participants in secondary school at exit treats secondary school enrollment as a neutral outcome and avoids penalizing programs for serving these youth. Literacy and Numeracy Gains The literacy and numeracy gains indicator measures the increase in the skills of participants through a common assessment tool administered at program entry and regular intervals thereafter. Methodology Of those who are basic skills deficient The number of participants who increase one or more educational functioning levels divided by the number of participants who have completed a year in the program i.e. one year from the date of program participation plus the number of participants who exit before completing a year in the program. Operational Parameters Pre-test must occur within 60 days of the date of participation. Individuals who are determined not to be basic skills deficient at pre-test are excluded from this measure i.e. programs will not be held accountable for these individuals under this measure and post-testing of these individuals is not required. When administering assessment tools individuals with disabilities as defined in 29 CFR 37.4 should be accommodated according to 1 Section 188 of WIA 2 guidelines associated with the assessment tool used to determine functioning levels and 3 state law or policies. 12 The measure includes individuals who are given an initial assessment but either 1 do not post- test before exiting the program or 2 exit before completing a year in the program i.e. one year from the date of program participation . To be included in the numerator a participant must demonstrate through post-test that he she has advanced one or more educational functioning levels beyond the level in which he she was initially placed at pre-test. National Reporting System To maintain consistency with the implementation of the common measures by the Department of Education ED ETA is adopting policies for the ED outcome measure of educational gain as outlined in the National Reporting System NRS . The NRS was developed by ED s Division of Adult Education and Literacy for implementation of an accountability system for federally-funded adult education programs under WIA. Educational Functioning Levels Programs assess participants at intake or within 60 days of the date of participation to determine their educational functioning level. As outlined in the NRS there are two sets of educational functioning levels six levels for Adult Basic Education ABE and six levels for English-as-a- second language ESL students. The levels are outlined in Attachment D. The ABE levels roughly equate to two grade levels. Each ABE and ESL level describes a set of skills and competencies that students entering at that level demonstrate in the areas of reading writing numeracy speaking listening functional and workplace skills. These descriptors provide guidelines for placing participants in educational functioning levels based on performance on standardized tests. After a participant has completed a uniform standardized assessment procedure programs use these descriptors to determine the appropriate initial ABE or ESL level in which to place students. If a participant is functioning at different levels in reading writing numeracy speaking listening functional and workplace areas the individual is placed in different ABE and ESL levels across the functioning areas. In the post-test assessment if the participant demonstrates an increase to the next functioning level in any of the areas reading writing numeracy speaking listening functional and workplace areas he she has made an educational gain and should be included in the numerator. For example an individual is placed in the Beginning Basic Education level the second ABE level in math and the Low Intermediate Basic Education level the third ABE level in reading at pre-testing. The individual achieves an educational gain if he she places in either the third ABE level in math or the fourth ABE level in reading at post-testing. Assessment Tool To measure the increase in skills of individuals programs must use an assessment procedure comprised of a standardized test or a performance assessment with standardized scoring protocols. States grantees or contractors are not required to use the same assessment tool throughout their jurisdictions. However programs must adhere to the following in choosing an assessment tool 1 The same assessment tool is administered to the participant for pre-testing and post-testing 2 The assessment tool and its scores must crosswalk directly to the educational functioning levels so that educational gains can be reported in terms of increase in one or more ABE or ESL levels and 13 3 Tests must be administered in a standardized manner throughout the jurisdiction i.e. used consistently and reliably across programs and produces observable results . Tests must be administered to individuals with disabilities as defined in 29 CFR 37.4 with reasonable accommodations as appropriate. The NRS provides test benchmarks for educational functioning levels See Attachment D . The tests which have been benchmarked include the following o Comprehensive Adult Student Assessment System CASAS Life Skills o Test of Adult Basic Education TABE Forms 5-6 and 7-8 o Adult Basic Learning Examination ABLE Forms E-F o Adult Measure of Educational Skills AMES Forms 1 and 2 o WorkKeys for the following ABE levels High Intermediate Basic Education Low Adult Secondary Education and High Adult Secondary Education o Student Performance Levels SPL for ESL o Basic English Skills Test BEST for ESL The benchmarks are provided as examples of how students functioning at each level would perform on the tests. Inclusion of these tests in this guidance in no way implies that they are equivalent or that they should be used as the basis for assessment. In addition these tests do not necessarily measure the same skills. Testing Interval Participants identified as basic skills deficient should be post-tested at least once by the end of one year following the individual s participation date. For participants who have completed one year following the participation date if more than one assessment is administered after the initial test the latest assessment within one year of participation should be used to determine if the participant has demonstrated an increase in at least one educational functioning level. Individuals who remain basic skills deficient and continue to participate after completing a full year in the program including pre-test and post-test should continue to receive basic skills remediation services. These participants should be included in the measure for the first year of participation. They are not included again in the measure until they have completed a second full year in the program. At the completion of the second year these participants should be post-tested and included in the measure. To determine an increase of one or more levels the participant s post-test scores from the second year in the program should be compared to the scores from the test that was administered at the latest point during the first year. Definitions Basic Skills Deficient - The individual who computes or solves problems reads writes or speaks English at or below the eighth grade level or is unable to compute or solve problems read write or speak English at a level necessary to function on the job in the individual s family or in society. In addition states and grantees have the option of establishing their own definition which must include the above language. In cases where states or grantees establish such a definition that definition will be used for basic literacy skills determination. Pre-test - A test administered to a participant within 60 days of the date of participation. Post-test - A test administered to a participant at regular intervals during the program. 14 Educational Gain - At post-test participant completes or advances one or more educational functioning levels from the starting level measured on entry into the program pre-test . Discussion Youth participants for whom the goal of literacy and or numeracy gains is most appropriate are those with basic skills deficiencies. Participants who are not basic skills deficient cannot achieve a positive outcome because they have attained basic literacy and numeracy. Exclusion of individuals who are not basic skills deficient focuses the measure on the participants with the greatest need for remediation and minimizes the burden of testing individuals who by virtue of their existing skill level will not achieve a positive outcome under this measure. Under a normal distribution of pre-test scores most participants scores will place the individuals in a range where they have completed some of the skills in that particular educational functioning level. Therefore for a majority of participants a positive outcome for this measure i.e. completion of one educational functioning level and an increase to the next level is not likely to require the equivalent of completing two grade levels but will average to the equivalent of one grade level. D. Efficiency Measure The efficiency measure applies to both adult and youth programs. Methodology Program appropriation level divided by the number of program participants. Definitions Appropriation Level - The amount of Federal funding authorized by Congress for a program. Program Participant - Individuals who are determined eligible and receive any service funded by the program in a physical location e.g. a One-Stop career center are participants. The criteria that are used to determine whether an individual is eligible to participate will be based on the guidelines for the program. Program participation is discussed in more detail in Section 4 . Discussion The efficiency measure will allow Congress and other Federal policymakers to make high-level assessments of the utilization of funds by programs based upon the number of individuals receiving program services. While cost is an important part of an assessment of program effectiveness this element is only a starting point for analysis. Information on cost must be examined in the context of outcomes achieved characteristics of individual participants and the types of services provided. The efficiency measure is not intended to place greater value on low-cost services or to discourage the provision of services to individuals facing significant barriers to employment such as individuals who have disabilities have Limited English Proficiency are age 55 or older are homeless or are migrant and seasonal farmworkers. The efficiency measure will be calculated at the Federal level to determine an efficiency outcome for programs as a whole. In addition ETA may review efficiency levels as applied at the state and grantee levels based upon the allotment level of program funds. 15 6. Data Sources . This section describes data sources and methods to collect data for the common measures. The data source s applicable to each measure are as follows PERFORMANCE MEASURE DATA SOURCE Adult Measures Entered Employment Wage records Retention Wage records Earnings Increase Wage records Efficiency Administrative records Youth Measures Placement in Employment or Education Wage records for placement in employment and military Administrative records for placement in education or training Attainment of a Degree or Certificate Administrative records Literacy and Numeracy Gains Assessment instrument Efficiency Administrative records A. Wage Records To ensure comparability of the common measures on a national level wage records are the data source for the employment-related measures except as noted in this section . Unemployment Insurance Wage Records To the extent it is consistent with state law Unemployment Insurance UI wage records will be the primary data source for tracking the adult entered employment retention and earnings increase measures and the employment portion of the youth placement in employment or education measure. Additional Wage Records While most forms of employment in a state s workforce are covered and will be in the UI wage records certain types of employers and employees are excluded by Federal unemployment law standards or are not covered under states UI laws. States may use record sharing and or automated record matching with other employment and administrative data sources to determine and document employment and earnings for uncovered workers. These data sources include Wage Record Interchange System WRIS U.S. Office of Personnel Management U.S. Postal Service U.S. Department of Defense Railroad Retirement System State government employment records 16 Local government employment records Judicial employment records Public school employment records State New Hires Registry State Department of Revenue or Tax for individuals who are self-employed information must be obtained through record sharing or automated matching of state tax records ETA is currently working with the Office of Personnel Management U.S. Postal Service and the Department of Defense to create a data exchange system to provide access for all states to wage record information on Federal and military employment. B. Supplemental Sources of Data Currently mechanisms are not in place to provide access to UI wage records for grantees operating the following programs H-1B Technical Skills Training Job Corps Migrant and Seasonal Farmworkers Native American Employment and Training Responsible Reintegration of Youthful Offenders and Senior Community Service Employment Program. For these grantees supplemental sources of data such as participant surveys or contacts with employers will be permitted as an interim means of reporting on the common measures until all grantees in a program have access to wage records. Any adjustments necessary for the implementation of employment-related measures using supplemental information as the data source will be addressed in separate program guidance. Allowable sources of supplemental information for tracking employment and earnings in this interim period include case management follow-up services and surveys of participants and employers. All supplemental data and methods must be documented and are subject to audit. Supplemental sources of information will continue to be collected and reported by states and grantees. Supplemental data will be used for program management purposes and to gain a full understanding of program performance and activities. However supplemental sources of information will not be used to calculate outcomes on the employment-related common measures. C. Administrative Records Administrative records will be the data source for the education and training portion of the placement in employment or education measure and the attainment of a degree or certificate measure. All data and methods used to determine placement in education and training or achievement of a degree or certificate must be documented and are subject to audit. Placement in Post-Secondary Education or Advanced Training Occupational Skills Training The following data sources can be used to determine whether participants in youth programs are placed in post-secondary education and or advanced training occupational skills training 1. Case management follow-up services and surveys of participants to determine if the individual has been placed in post-secondary education and or advanced training occupational skills training or 2. Record-sharing agreements and or automated record matching with administrative other data sources to determine and document that the participant has been placed in post- 17 secondary education and or advanced training occupational skills training. These data sources may include State Board governing community colleges State Board governing universities State Education Associations Integrated post-secondary or higher education reporting unit Training institutions providers Degree or Certificate The following data sources can be used to determine whether participants in youth programs attain degrees or certificates 1. Case management follow-up services and surveys of participants to determine that the individual has received a degree or certificate or 2. Record sharing agreements and or automated record matching with administrative other data sources to determine and document that the participant has received a degree or certificate. These data sources may include State Board of Education State Board governing community colleges State Board governing universities State licensing boards for private schools State Education Associations Integrated post-secondary or higher education reporting unit State Department of Professional or Occupational Regulation possibly other units such as health care administration or specific boards like the Board of Nursing Professional industry or employer organization or a product manufacturer or developer Training institutions providers Adult Basic Education providers GED equivalent testing agencies 7. Action Required . Please make this information available to appropriate program staff. 8. Attachments . Attachment A Federal Job Training and Employment Programs Impacted by Common Measures Attachment B Common Measures At-A-Glance Attachment C Definitions of Key Terms Attachment D Educational Functioning Level Descriptors 18 Attachment A Federal Job Training and Employment Programs Impacted by Common Measures In addition to programs administered by ETA the following Federal programs are subject to the common measures Department of Labor Programs for Veterans Veterans Workforce Investment Program Disabled Veterans Outreach Program Local Veterans Employment Representatives Homeless Veterans Reintegration Program Department of Education Adult Education Rehabilitation Services Vocational Rehabilitation Grants to States American Indian Vocational Rehabilitation Services Supported Employment State Grants Projects with Industry Migrant and Seasonal Farmworkers State Grants for Incarcerated Youth Offenders Vocational Education Carl D. Perkins Vocational and Technical Education Act - State Grants Carl D. Perkins Vocational and Technical Education Act - Tech Prep State Grants Carl D. Perkins Vocational and Technical Education Act - Tribally-Controlled Postsecondary Vocational Institutions Department of Health and Human Services Temporary Assistance to Needy Families Department of Veterans Affairs Vocational Rehabilitation and Employment Services and Benefits Department of the Interior Job Placement and Training Department of Housing and Urban Development Youthbuild 19 Attachment B Common Measures At-A-Glance ADULT MEASURES Entered Employment Of those who are not employed at the date of participation of participants who are employed in the first quarter qtr after exit of participants who exit during the qtr Retention Of those who are employed in the first qtr after exit of participants who are employed in both the second and third qtrs after exit of participants who exit during the qtr Earnings Increase Of those who are employed in the first qtr after exit Earnings in the first qtr after exit minus earnings in the qtr prior to the date of participation Earnings in the qtr prior to the date of participation - and - Of those who are employed in the first qtr after exit Earnings in the third qtr after exit minus earnings in the first qtr after exit Earnings in the first qtr after exit Efficiency Program appropriation level of program participants YOUTH MEASURES Placement in Employment or Education Of those who are not in post-secondary education employment or the military at the date of participation of participants who are in employment or the military or enrolled in post-secondary education and or advanced training occupational skills training in the first qtr after exit of participants who exit during the qtr Attainment of a Degree or Certificate Of those enrolled in education at the date of participation or at any point during the program of participants who attain a diploma GED or certificate by the end of the third qtr after exit of participants who exit during the quarter Literacy and Numeracy Gains Of those who are basic skills deficient of participants who increase one or more educational functioning levels of participants who have completed a year in the program i.e. one year from the date of program participation plus the of participants who exit before completing a year in the program Efficiency Program appropriation level of program participants 20 Attachment C Definitions of Key Terms Advanced Training Occupational Skills Training - An organized program of study that provides specific vocational skills that lead to proficiency in performing actual tasks and technical functions required by certain occupational fields at entry intermediate or advanced levels. Such training should 1 be outcome-oriented and focused on a long-term goal as specified in the Individual Service Strategy 2 coincide with exit rather than short-term training that is part of services received while enrolled in ETA-funded youth programs and 3 result in attainment of a certificate as defined on pages 10 and 11 and in this attachment . Appropriation Level - The amount of Federal funding authorized by Congress for a program. Basic Skills Deficient - The individual computes or solves problems reads writes or speaks English at or below the 8 th grade level or is unable to compute or solve problems read write or speak English at a level necessary to function on the job in the individual s family or in society. In addition states and grantees have the option of establishing their own definition which must include the above language. In cases where states or grantees establish such a definition that definition will be used for basic literacy skills determination. Certificate - A certificate is awarded in recognition of an individual s attainment of measurable technical or occupational skills necessary to gain employment or advance within an occupation. These technical or occupational skills are based on standards developed or endorsed by employers. Certificates awarded in recognition of the attainment of only generic pre-employment or work readiness skills are not included in this definition. A certificate is awarded in recognition of an individual s attainment of technical or occupational skills by A state educational agency or a state agency responsible for administering vocational and technical education within a state. An institution of higher education described in Section 102 of the Higher Education Act 20 USC 1002 that is qualified to participate in the student financial assistance programs authorized by Title IV of that Act. This includes community colleges proprietary schools and all other institutions of higher education that are eligible to participate in Federal student financial aid programs. A professional industry or employer organization e.g. National Institute for Automotive Service Excellence certification National Institute for Metalworking Skills Inc. Machining Level I credential or a product manufacturer or developer e.g. Microsoft Certified Database Administrator Certified Novell Engineer Sun Certified Java Programmer using a valid and reliable assessment of an individual s knowledge skills and abilities. A registered apprenticeship program. A public regulatory agency upon an individual s fulfillment of educational work experience or skill requirements that are legally necessary for an individual to use an occupational or professional title or to practice an occupation or profession e.g. FAA aviation mechanic certification state certified asbestos inspector . A program that has been approved by the Department of Veterans Affairs to offer education and training to veterans and other eligible persons under provisions of the Montgomery GI Bill. Office of Job Corps. 21 Educational Gain - At post-test participant completes or advances one or more educational functioning levels from the starting level measured on entry into the program pre-test . Employed at the Date of Participation - An individual employed at the date of participation is one who during the seven consecutive days prior to the date of participation Did any work at all as a paid employee except the individual is NOT considered employed if a he she has received a notice of termination of employment or the employer has issued a WARN or other notice that the facility or enterprise will close or b he she is currently on active military duty and has been provided with a firm date of separation from military service Did any work at all in his her own business profession or farm Worked 15 hours or more as un unpaid worker in an enterprise operated by a member of the family or Was not working but has a job or business from which he she was temporarily absent because of illness bad weather vacation labor-management dispute or personal reasons whether or not paid by the employer for time off and whether or not seeking another job. Employed in Any Quarter After the Exit Quarter - The individual is considered employed in a quarter after the exit quarter if wage records for that quarter show earnings greater than zero. Exit - Exit occurs when a participant does not receive a service funded by the program or funded by a partner program for 90 consecutive calendar days. The term service does not include activities that are open to non-participants services and activities specifically provided as follow-up services or regular contact with the participant or employer to only obtain information regarding his her employment status or educational progress or need for additional services. Exit Date - The date of exit is the date on which the last service funded by the program or a partner program is received by the participant. Military Status at the Date of Participation - An individual is considered to be in the military at the date of participation if a he she currently is serving on active military duty and has NOT yet been provided with a firm date of separation from military service or b he she is a member of the National Guard or one of the Military Reserves AND currently is serving in a mobilized i.e. active military duty status. Military Status in First Quarter After Exit Quarter - The individual is considered in the military if U.S. Department of Defense wage records for the first quarter after exit show earnings greater than zero. Participant - Individuals who are determined eligible and receive any service funded by the program in a physical location e.g. a One-Stop career center are participants. The criteria that are used to determine whether an individual is eligible to participate will be based on the guidelines for the program. Note Individuals who only receive a determination of eligibility but do not receive an additional service are not participants. Individuals who visit a physical location for reasons other than its intended purpose e.g. use of restrooms or ask staff for directions are not participants. States and grantees may choose but are not required to consider as participants those 22 individuals who receive services that are available through the Internet and are not accessed through a physical location e.g. a One-Stop career center. Participation Date - The date of participation is the date of the first service received after the individual is determined eligible. Post-Secondary Education - A program at an accredited degree-granting institution that leads to an academic degree e.g. A.A. A.S. B.A. B.S. . Programs offered by degree-granting institutions that do not lead to an academic degree e.g. certificate programs do not count as a placement in post-secondary education but may count as a placement in advanced training occupational skills training. Post-test - A test administered to a participant at regular intervals during the program. Pre-test - A test administered to a participant within 60 days of the date of participation. 23 Attachment D Educational Functioning Level Descriptors EDUCATIONAL FUNCTIONING LEVEL DESCRIPTORS A DULT BASIC EDUCATION LEVELS Literacy Level Basic Reading and Writing Numeracy Skills Functional and Workplace Skills BEGINNING ABE L ITERACY Test Benchmark TABE 5-6 scale scores grade level 0-1.9 Total reading 529 and below Total math 540 and below Total language 599 and below TABE 7-8 scale scores grade level 0-1.9 Reading 367 and below Total Math 313 and below Language 391 and below CASAS Life Skills 200 and below AMES B ABE scale scores grade level 0-1.9 Reading 500 and below Total Math 476 and below Communication 496 and below ABLE scale scores grade level 0-1.9 Reading 523 and below Math 521 and below Individual has no or minimal reading and writing skills. May have little or no comprehension of how print corresponds to spoken language and may have difficulty using a writing instrument. At the upper range of this level individual can recognize read and write letters and numbers but has a limited understanding of connected prose and may need frequent re-reading. Can write a limited number of basic sight words and familiar words and phrases may also be able to write simple sentences or phrases including very simple messages. Can write basic personal information. Narrative writing is disorganized and unclear inconsistently uses simple punctuation e.g. periods commas question marks contains frequent errors in spelling. Individual has little or no recognition of numbers or simple counting skills or may have only minimal skills such as the ability to add or subtract single-digit numbers. Individual has little or no ability to read basic signs or maps can provide limited personal information on simple forms. The individual can handle routine entry level jobs that require little or no basic written communication or computational skills and no knowledge of computers or other technology. BEGINNING BASIC EDUCATION Test Benchmark TABE 5-6 scale scores grade level 2-3.9 Total reading 530-679 Total math 541-677 Total language 600-677 TABE 7-8 scale scores grade level 2-3.9 Reading 368-460 Total Math 314-441 Language 392-490 CASAS Life Skills 201-210 AMES B ABE scale scores grade level 2-3.9 Reading 503-510 Total Math 477-492 Communication 498-506 ABLE scale scores grade level 2-3.9 Reading 525-612 Math 530-591 Individual can read simple material on familiar subjects and comprehend simple and compound sentences in single or linked paragraphs containing a familiar vocabulary can write simple notes and messages on familiar situations but lacks clarity and focus. Sentence structure lacks variety but shows some control of basic grammar e.g. present and past tense and consistent use of punctuation e.g. periods capitalization . Individual can count add and subtract three- digit numbers can perform multiplication through 12 can identify simple fractions and perform other simple arithmetic operations. Individual is able to read simple directions signs and maps fill out simple forms requiring basic personal information write phone messages and make simple change. There is minimal knowledge of and experience with using computers and related technology. The individual can handle basic entry level jobs that require minimal literacy skills can recognize very short explicit pictorial texts e.g. understands logos related to worker safety before using a piece of machinery can read want ads and complete simple job applications. 24 LOW INTERMEDIATE BASIC EDUCATION Test benchmark TABE 5-6 scale scores grade level 4-5.9 Total reading 680-722 Total math 678-729 Total language 678-705 TABE 7-8 scale scores grade level 4-5.9 Reading 461-517 Total Math 442-505 Language 491-523 CASAS Life Skills 211-220 AMES B and C ABE scale scores grade level 4-5.9 Reading B 511-609 Reading C 514-521 Total Math B 494-603 Total Math C 493-508 Communication B 508-605 C 509-513 ABLE scale scores grade level 4-5.9 Reading 613-644 Math 593-641 Individual can read text on familiar subjects that have a simple and clear underlying structure e.g. clear main idea chronological order can use context to determine meaning can interpret actions required in specific written directions can write simple paragraphs with main idea and supporting detail on familiar topics e.g. daily activities personal issues by recombining learned vocabulary and structures can self and peer edit for spelling and punctuation errors. Individual can perform with high accuracy all four basic math operations using whole numbers up to three digits can identify and use all basic mathematical symbols. Individual is able to handle basic reading writing and computational tasks related to life roles such as completing medical forms order forms or job applications can read simple charts graphs labels and payroll stubs and simple authentic material if familiar with the topic. The individual can use simple computer programs and perform a sequence of routine tasks given direction using technology e.g. fax machine computer operation . The individual can qualify for entry-level jobs that require following basic written instructions and diagrams with assistance such as oral clarification can write a short report or message to fellow workers can read simple dials and scales and take routine measurements. HIGH INTERMEDIATE BASIC EDUCATION Test benchmark TABE 5-6 scale scores grade level 6-8.9 Total reading 723-761 Total math 730-776 Total language 706-730 TABE 7-8 scale scores grade level 6-8.9 Reading 518-566 Total Math 506-565 Language 524-559 CASAS Life Skills 221-235 AMES C and D ABE scale scores grade level 6-8.9 Reading C 525-612 Reading D 522-543 Total Math C 510-627 Total Math D 509-532 Communication C 516-611 Communication D 516-523 ABLE scale score grade level 6-8.9 Reading 646-680 Math 643-693 WorkKeys scale scores Reading for Information 75-78 Writing 75-77 Applied Mathematics 75-77 Individual is able to read simple descriptions and narratives on familiar subjects or from which new vocabulary can be determined by context can make some minimal inferences about familiar texts and compare and contrast information from such texts but not consistently. The individual can write simple narrative descriptions and short essays on familiar topics has consistent use of basic punctuation but makes grammatical errors with complex structures. Individual can perform all four basic math operations with whole numbers and fractions can determine correct math operations for solving narrative math problems and can convert fractions to decimals and decimals to fractions can perform basic operations on fractions. Individual is able to handle basic life skills tasks such as graphs charts and labels and can follow multi-step diagrams can read authentic materials on familiar topics such as simple employee handbooks and payroll stubs can complete forms such as a job application and reconcile a bank statement. Can handle jobs that involve following simple written instructions and diagrams can read procedural texts where the information is supported by diagrams to remedy a problem such as locating a problem with a machine or carrying out repairs using a repair manual. The individual can learn or work with most basic computer software such as using a word processor to produce own texts can follow simple instructions for using technology. 25 LOW ADULT SECONDARY EDUCATION Test benchmark TABE 5-6 scale scores grade level 9-10.9 Total reading 762-775 Total math 777-789 Total language 731-743 TABE 7-8 scale scores grade level 9-10.9 Reading 567-595 Total Math 566-594 Language 560-585 CASAS Life Skills 236-245 AMES E ABE scale scores grade level 9-10.9 Reading 544-561 Total Math 534-548 Communication 527-535 ABLE scale scores grade level 9-10.9 Reading 682-697 Math 694-716 WorkKeys scale scores Reading for Information 79-81 Writing 78-85 Applied Mathematics 78-81 Individual can comprehend expository writing and identify spelling punctuation and grammatical errors can comprehend a variety of materials such as periodicals and non-technical journals on common topics can comprehend library reference materials and compose multi-paragraph essays can listen to oral instructions and write an accurate synthesis of them can identify the main idea in reading selections and use a variety of context issues to determine meaning. Writing is organized and cohesive with few mechanical errors can write using a complex sentence structure can write personal notes and letters that accurately reflect thoughts. Individual can perform all basic math functions with whole numbers decimals and fractions can interpret and solve simple algebraic equations tables and graphs and can develop own tables and graphs can use math in business transactions. Individual is able or can learn to follow simple multi-step directions and read common legal forms and manuals can integrate information from texts charts and graphs can create and use tables and graphs can complete forms applications and resumes can perform jobs that require interpreting information from various sources and writing or explaining tasks to other workers is proficient using computers and can use most common computer applications can understand the impact of using different technologies can interpret the appropriate use of new software and technology. HIGH ADULT SECONDARY EDUCATION Test benchmark TABE 5-6 scale scores grade level 11-12.9 Total reading 776 and above Total math 790 and above Total language 744 and above TABE 7-8 scale scores grade level 11-12 Reading 596 and above Total Math 595 and above Language 586 and above CASAS Life Skills 246 and higher AMES E ABE scale scores grade level 11-12 Reading 565 and above Total Math 551 and above Communication 538 and above ABLE scale scores grade level 11-12 Reading 699 and above Math 717 and above WorkKeys scale scores Reading for Information 82-90 Writing 86-90 Applied Mathematics 82-90 Individual can comprehend explain and analyze information from a variety of literacy works including primary source materials and professional journals can use context cues and higher order processes to interpret meaning of written material. Writing is cohesive with clearly expressed ideas supported by relevant detail can use varied and complex sentence structures with few mechanical errors. Individual can make mathematical estimates of time and space and can apply principles of geometry to measure angles lines and surfaces can also apply trigonometric functions. Individuals are able to read technical information and complex manuals can comprehend some college level books and apprenticeship manuals can function in most job situations involving higher order thinking can read text and explain a procedure about a complex and unfamiliar work procedure such as operating a complex piece of machinery can evaluate new work situations and processes can work productively and collaboratively in groups and serve as facilitator and reporter of group work. The individual is able to use common software and learn new software applications can define the purpose of new technology and software and select appropriate technology can adapt use of software or technology to new situations and can instruct others in written or oral form on software and technology use. 26 EDUCATIONAL FUNCTIONING LEVEL DESCRIPTORS E NGLISH A S-A-S ECOND LANGUAGE LEVELS Literacy Level Speaking and Listening Basic Reading and Writing Functional and Workplace Skills BEGINNING ESL L ITERACY Test benchmark CASAS Life Skills 180 and below SPL Speaking 0-1 SPL Reading and Writing 0-1 Oral BEST 0-15 Literacy BEST 0-7 Individual cannot speak or understand English or understands only isolated words or phrases. Individual has no or minimal reading or writing skills in any language. May have little or no comprehension of how print corresponds to spoken language and may have difficulty using a writing instrument. Individual functions minimally or not at all in English and can communicate only through gestures or a few isolated words such as name and other personal information may recognize only common signs or symbols e.g. stop sign product logos can handle only very routine entry-level jobs that do not require oral or written communication in English. There is no knowledge or use of computers or technology. BEGINNING ESL Test benchmark CASAS Life Skills 181-200 SPL Speaking 2-3 SPL Reading and Writing 2-4 Oral BEST 16-41 Literacy BEST 8-46 Individual can understand frequently used words in context and very simple phrases spoken slowly and with some repetition there is little communicative output and only in the most routine situations little or no control over basic grammar survival needs can be communicated simply and there is some understanding of simple questions. Individual can recognize read and write numbers and letters but has a limited understanding of connected prose and may need frequent re-reading can write a limited number of basic sight words and familiar words and phrases may also be able to write simple sentences or phrases including very simple messages. Can write basic personal information. Narrative writing is disorganized and unclear inconsistently uses simple punctuation e.g. periods commas question marks contains frequent errors in spelling. Individual functions with difficulty in situations related to immediate needs and in limited social situations has some simple oral communication abilities using simple learned and repeated phrases may need frequent repetition can provide personal information on simple forms can recognize common forms of print found in the home and environment such as labels and product names can handle routine entry- level jobs that require only the most basic written or oral English communication and in which job tasks can be demonstrated. There is minimal knowledge or experience using computers or technology. LOW INTERMEDIATE ESL Test benchmark CASAS Life Skills 201-210 SPL Speaking 4 SPL Reading and Writing 5 Oral BEST 42-50 Literacy BEST 47-53 Individual can understand simple learned phrases and limited new phrases containing familiar vocabulary spoken slowly with frequent repetition can ask and respond to questions using such phrases can express basic survival needs and participate in some routine social conversations although with some difficulty has some control of basic grammar. Individual can read simple material on familiar subjects and comprehend simple and compound sentences in single or linked paragraphs containing a familiar vocabulary can write simple notes and messages on familiar situations but lacks clarity and focus. Sentence structure lacks variety but shows some control of basic grammar e.g. present and past tense and consistent use of punctuation e.g. periods capitalization . Individual can interpret simple directions schedules signs and maps can fill out simple forms but needs support on some documents that are not simplified can handle routine entry-level jobs that involve some written or oral English communication but in which job tasks can be demonstrated. Individual can use simple computer programs and can perform a sequence of routine tasks given directions using technology e.g. fax machine computer . 27 HIGH INTERMEDIATE ESL Test benchmark CASAS Life Skills 211-220 SPL Speaking 5 SPL Reading and Writing 6 Oral BEST 51-57 Literacy BEST 54-65 Individual can understand learned phrases and short new phrases containing familiar vocabulary spoken slowly and with some repetition can communicate basic survival needs with some help can participate in conversation in limited social situations and use new phrases with hesitation relies on description and concrete terms. There is inconsistent control of more complex grammar. Individual can read text on familiar subjects that have a simple and clear underlying structure e.g. clear main idea chronological order can use context to determine meaning can interpret actions required in specific written directions can write simple paragraphs with main idea and supporting detail on familiar topics e.g. daily activities personal issues by recombining learned vocabulary and structures can self and peer edit for spelling and punctuation errors. Individual can meet basic survival and social needs can follow some simple oral and written instruction and has some ability to communicate on the telephone on familiar subjects can write messages and notes related to basic needs complete basic medical forms and job applications can handle jobs that involve basic oral instructions and written communication in tasks that can be clarified orally. The individual can work with or learn basic computer software such as word processing can follow simple instructions for using technology. LOW ADVANCED ESL Test benchmark CASAS Life Skills 221-235 SPL Speaking 6 SPL Reading and Writing 7 Oral BEST 58-64 Literacy BEST 66 and above Individual can converse on many everyday subjects and some subjects with unfamiliar vocabulary but may need repetition rewording or slower speech can speak creatively but with hesitation can clarify general meaning by rewording and has control of basic grammar understands descriptive and spoken narrative and can comprehend abstract concepts in familiar contexts. Individual is able to read simple descriptions and narratives on familiar subjects or from which new vocabulary can be determined by context can make some minimal inferences about familiar texts and compare and contrast information from such texts but not consistently. The individual can write simple narrative descriptions and short essays on familiar topics such as customs in native country has consistent use of basic punctuation but makes grammatical errors with complex structures. Individual can function independently to meet most survival needs and can communicate on the telephone on familiar topics can interpret simple charts and graphics can handle jobs that require simple oral and written instructions multi-step diagrams and limited public interaction. The individual can use all basic software applications understand the impact of technology and select the correct technology in a new situation. HIGH ADVANCED ESL Test benchmark CASAS Life Skills 236-245 SPL Speaking 7 SPL Reading and Writing 8 Oral BEST 65 and above Individual can understand and participate effectively in face-to-face conversations on everyday subjects spoken at normal speed can converse and understand independently in survival work and social situations can expand on basic ideas in conversation but with some hesitation can clarify general meaning and control basic grammar although still lacks total control over complex structures. Individual can read authentic materials on everyday subjects and can handle most reading related to life roles can consistently and fully interpret descriptive narratives on familiar topics and gain meaning from unfamiliar topics uses increased control of language and meaning-making strategies to gain meaning of unfamiliar texts. The individual can write multi-paragraph essays with a clear introduction and development of ideas writing contains well-formed sentences appropriate mechanics and spelling and few grammatical errors. Individual has a general ability to use English effectively to meet most routine social and work situations can interpret routine charts graphs and tables and complete forms has high ability to communicate on the telephone and understand radio and television can meet work demands that require reading and writing and can interact with the public. The individual can use common software and learn new applications can define the purpose of software and select new applications appropriately can instruct others in use of software and technology.