U.S. Department of Labor - FY 2002 Annual Performance Plan

FY 2002 Annual Performance Plan

The Department of Labor has implemented GPRA using a systematic, staged approach based on sound business practices. Our first priority has been the establishment of enduring outcome goals which target continual improvement in the achievement of the core results the Department is committed to delivering on behalf of working men and women. After several years of piloting and refining goals, a majority of the Department’s programs have stabilized their goals, and DOL’s focus has progressively shifted toward ensuring the reliability of our performance measures and effectively using performance data to enhance our program results.

The Department recognizes that transitioning to a fully performance-based organization depends upon the availability of reliable and timely information concerning the results of DOL’s programs. The challenges to performance measurement vary significantly among DOL’s programs, with the data sources and the agencies’ level of control over the reporting systems being the primary factors influencing the reliability and usefulness of the Department’s performance information. Many DOL agencies collect critical program data from third parties, including State and local government agencies, community based organizations, private sector employers, and international organizations. The Department’s authorities to increase the frequency of reporting, establish data standards or verify the accuracy of the information reported by third parties are limited in some cases. In other programs, data important to fully assess the effectiveness of strategies adopted to achieve our GPRA goals are not readily available, or DOL’s information technology systems require enhancements to support routine access by program managers to key performance data.

A number of initiatives to improve performance measurement and performance based management will be undertaken in FY 2001 and additional efforts are planned for FY 2002. During FY 2001, the Department will initiate a series of workshops and seminars on GPRA implementation to facilitate an exchange of best practices among DOL agencies, and performance measurement related issues will be a featured topic. In addition, the Office of Inspector General will expand reviews of the reliability and quality of the performance data reported by DOL programs in FYs 2001 and 2002, and the Department will work closely with our agencies in addressing problems identified by the audits. The Department plans in FY 2002 to increase the use of program evaluations to meet a variety of performance management objectives, including supplementing information available through routinely collected performance information, reviewing the effectiveness of strategies in meeting key performance goals and analyzing the quality of data sources, indicators and reported program results.

Working together, DOL executives and program managers will continuously improve the quality of the Department’s performance information. However, success in overcoming some of the challenges the Department faces in ensuring the reliability of performance measures and effectively using performance data to enhance the results of DOL’s programs will also require support for data collection and verification authorities and a commitment of critical resources by OMB and the Congress.

5.1 Addressing Specific Performance Measurement Challenges

Within the larger Departmental framework, individual DOL agencies will address data challenges that are unique to the agency’s program environment and develop solutions that are consistent with the Department’s reporting requirements. While some DOL programs currently have adequate systems in place, others must overcome barriers to the production of timely, accurate, and relevant performance data. In FY 2002, DOL and its agencies will continue efforts to address four issues: the availability of data, validation of data, timeliness of reporting and the use of data in managing for results. The following examples describe several initiatives completed or in progress to further improve the measurement of our program results.

The Employment Standards Administration will continue its efforts to improve data measuring the results of three major programs in FY 2001. First, the Wage and Hour program has introduced a new computer system, the Wage and Hour Investigative and Reporting Database (WHISARD), permitting more timely access to extensive information. Because there is no unbiased database on labor standards violations or compliance, Wage and Hour faces a major challenge in determining industry-wide levels of compliance and measuring changes in that compliance. To determine the impact of Wage and Hour efforts, a statistically sound method has been developed for establishing baselines and measuring compliance using investigation-based compliance surveys of targeted industries and areas. Data on the outcomes of repeat investigations will also be used to evaluate the relative effectiveness, or return on investment, of the various types of interventions.
The new WHISARD system provides many advantages when compared to its predecessor. For example, data is entered into the system directly at the source (by Wage and Hour investigators) rather than manually batched and mailed to a central source for data entry later. As a result, data is much more current, which facilitates tracking the progress of investigative activity. Information on a complainant’s case is readily available which enhances customer service and satisfaction. All users of the WHISARD system have direct access, which was not previously possible and enhances efficiency and enforcement effectiveness. In addition, WHISARD can produce data on a much broader range of activities, such as local enforcement initiatives, which was not possible with the prior system, and assists in the agency’s strategic planning activities.

The Office of Federal Contract Compliance Programs’ (OFCCP) Case Management System (CMS) is used to measure program performance. Through software and hardware enhancements, field office managers are now able to access CMS data to track the accomplishments of individual organizational units. OFCCP has developed a database comprised of Federal contractor-submitted personnel activity data and other required records, with the contractor having the option to input directly to the database via the Internet. Following the final refinement of the analytical model (scheduled for completion in FY 2001) the response data will be analyzed and used in determining the scheduling of technical assistance, outreach and compliance evaluation actions, further increasing OFCCP’s efficiency and effectiveness.

Increasingly sophisticated databases detailing the case histories of injured Federal workers have been developed for the Federal Employees’ Compensation Act (FECA) program. These systems allow tracking of the goal of reducing lost production days, but also permit precise evaluation of various program initiatives such as the impact of alternative return to work techniques on various groupings of employees or injury categories.

FECA’s automated system is undergoing a complete redesign covering every major program operation. The redesign will replace a patchwork of loosely-linked programs each with its own database and rules, and provide a single automated system that will make data accessible to all users. This redesign will allow workers’ compensation claims staff to work more efficiently while providing improved customer service to injured workers, medical providers, and employing Federal agencies.
The Department continues to address the need to ensure the accuracy and reliability of performance data submitted by our employment and training system partners which serves as the foundation for key program decisions. The development of a comprehensive data validity system for the core indicators of the WIA program, Wagner-Peyser, and other key employment and training programs is expected to be substantially completed in FY 2002.
Accurate outcome oriented performance data has been an issue for VETS since performance planning under the GPRA commenced in 1999 and placed emphasis on outcomes. Reporting data prior to GPRA was developed to respond to the reporting requirements of Title 38, Section 4107, which is essentially activity based. In addition, reporting of placements on the ETA 9002 has been subject to error and under-counting because it requires that a veteran specialist (DVOP or LVER) make contact with a registrant to confirm that the person got a job before their employment may be reported on the ETA 9002.
VETS initiated a special study in Maryland that utilized Unemployment Insurance wage records to gain a better understanding of the outcome of services provided by veterans specialists to veterans. This study has shown that a much larger number of veterans get jobs than are likely to have been captured by the information collection and reporting systems now in place. This result has been corroborated by other State studies conducted by ETA. VETS has initiated performance measure pilots in seven States that will look at using UI wage records for a number of outcomes, including entered employments, earnings gain, and retention in employment. VETS is an active partner with ETA in designing new reporting systems that will use the results of these studies to better capture outcomes.

VETS will continue to implement the recommendations made in the July, 2000 data capacity report on VETS programs prepared by a contractor. This will help VETS ensure that its measurement of performance is accurate and verifiable to the best degree possible, considering the fact that for its biggest programs (DVOP and LVER) VETS must rely on data from the States. In FY 2001, VETS intends to have an updated evaluation system in place for use by VETS field staff in their reviews of grantee performance. This evaluation system will replace the current LESO (Local Employment Service Office) Evaluation Manual that VETS has been utilizing to verify service to veterans since 1989.

A key consideration that the new system will address is how to effectively verify services to veterans when the old paradigm has changed. In past years, all veterans entering a local office were registered and a formal file established. This made statistical sampling of files an effective way to verify services to veterans that were reported on the VETS 200 and the ETA 9002. However, in the One-Stop environment there has been an emphasis on self-service, where there is no registration. In fact, in some States, there is no registration even if a mediated service is provided. This may require replacing the traditional method of statistical sampling with other verification techniques, such as surveys.
Past reports by GAO and OIG raised concerns about the reliability and accuracy of placement and job retention data used to measure the outcome performance of Job Corps participants. Revisions to Job Corps' procedures have satisfactorily addressed all of the issues cited by both reports. As an example of enhanced procedures, neutral third party data verification is used to validate reported measures. A random sample of placements (75% of those reported) is verified using an independent placement verifier to ensure data integrity. Additionally, a centralized data system with numerous management information reports and system edit checks is used to minimize error in data reporting. Each Job Corps contractor reporting participant achievements is required to maintain systems to validate their data.
In FY 2002, progress will continue to be made for indicators where data are currently unavailable or incomplete. For example, the employment, retention and earnings gains outcomes for TAA/NAFTA program participants are being revised to reflect the implementation of the WIA. In addition, for indicators adopted with the implementation of WIA, baselines will be reviewed and indicators refined. This will be done at the agency and program level in conjunction with the Department-wide effort to improve the quality of indicators.
MSHA has a significant database and collection system that captures most of the information necessary to track performance under the strategic plan. Data has been collected for many years and the database is well established for performance measures under the strategic goal of reducing the number of mine fatalities and the nonfatal-days-lost incidence rate.. However, MSHA relies on mine operators and contractors to comply with legal requirements to report accurately and timely employment, injuries and accidents. MSHA conducts periodic audits to ensure compliance. The number of audits conducted may influence the degree of compliance. Audits were increased in FY2000. Due to the results of these audits, beginning in FY 2001 quarterly reports are generated for managers showing active mines which do not report quarterly employment and production information. Routine follow-up visits will be conducted at these mines and audits will be increased as appropriate depending upon findings. If analysis of these visits/audits determine any pattern or problem with the quality or reliability of the data, an action plan to address these findings will be generated in FY 2002.
MSHA’s system for determining compliance with the coal respirable dust standard has been in place since the 1970s and procedures are well established to ascertain the accuracy and reliability of the data. Automated devices are used to weigh the inspector dust samples and automatically enter the results into a custom designed program that updates the dust data files daily. A quality control program developed jointly by MSHA and the National Bureau of Standards assures that the weighing process continues to produce reliable results over time, and computer edit checks assure the accuracy of the database.

Metal and Nonmetal inspectors have conducted industry-wide sampling since the 1970s. Health policies and the management information system are well established and reliable. Automated devices are used to weigh inspector dust samples at MSHA’s analytical lab, which was certified by the American Board of Industrial Hygienists in FY 1998. Computer edits assure the accuracy of MIS data input The Agency workforce that inspects metal and nonmetal mines has steadily declined since 1994, and the number of metal and nonmetal mines has continually increased. As a result, metal and nonmetal samples are generally collected at the discretion of the inspector based upon the conditions observed at the mine. For FY 2002 and FY 2003 a new goal and baseline have been established. Designation of high risk occupations and new sampling procedures have been established for collecting samples at Metal and Nonmetal Mines. This replaces the previously used indexing method. Using the new procedures, samples collected in FY 2001 will be used to establish a new baseline. For MSHA’s current database and collection system will be used to capture data and samples relating to noise exposures.
In FY 2002, OSHA plans to continue to use program data more effectively to manage OSHA’s programs in improving the safety and health in the nation’s workplaces by ensuring the quality of safety and health data, and continuing to use program evaluations.
The Occupational Safety and Health Administration will continue to improve data validation procedures. Validation of data generated by the agency for current performance measures will continue to be addressed through a variety of means such as annual on-site audits of the injury and illnesses records of employers to determine the accuracy and reliability of the OSHA 200 Logs, the source of data for the OSHA Data Initiative and the BLS Annual Survey; information and outreach programs and enforcement of the injury and illness recordkeeping regulations; revision of injury and illness recordkeeping system (regulations, forms, and guidelines) to improve the quality of records by simplifying forms and regulations, providing clearer guidance for employers, and incorporating incentives for employers to maintain high quality records; and continuing the various methods OSHA developed for validating and verifying data in the OSHA Integrated Management Data System (IMIS).

Program evaluations will assess how well OSHA’s programs, policies and procedures are working, including the effectiveness of specific standards, customer satisfaction, and specific approaches towards reducing occupational injuries and illnesses.

5.2 Linking Costs to Performance

The Department has a solid financial systems infrastructure from which a cost accounting capability is being developed using the resources of a reliable, established accounting system—the Department of Labor Accounting and Related Systems (DOLAR$). DOLAR$, serving as the system of record for financial results throughout the Department, has been modified to capture, aggregate, allocate and report costs. A cost accounting module has been developed to allow aggregation of costs across agency lines and to allocate direct and indirect costs to the strategic and outcome goal levels established in the Department’s Strategic Plan.

The Department has maintained cost accounting information, beginning in FY 1999, for the outcome goals in the Department’s Strategic Plan. In FY 2002, the Department has for the first time linked budget authority and outlays to both strategic and outcome goals—Appendix B provides an overview of the linkage between budget activities that support the Department’s outcome goals. DOL will continue to develop the capability to consolidate data from a variety of program and financial system sources and link that data as needed to meet the performance reporting requirements of GPRA.

Table of Contents