FY 2004 Annual Performance Plan
The Department of Labor has implemented GPRA using a systematic, staged approach based on sound business practices. Our first priority has been the establishment of enduring outcome goals that target continual improvement in the achievement of the core results the Department is committed to delivering on behalf of working men and women. After several years of piloting and refining goals, a majority of the Department's programs have stabilized their goals, and DOL's focus has progressively shifted toward ensuring the reliability of our performance measures and effectively using performance data to enhance our program results.
The Department recognizes that transitioning to a fully performance-based organization depends upon the availability of reliable and timely information concerning the results of DOL's programs. The challenges to performance measurement vary significantly among DOL's programs, with the data sources and the agencies' level of control over the reporting systems being the primary factors influencing the reliability and usefulness of the Department's performance information. Many DOL agencies collect critical program data from third parties, including State and local government agencies, community-based organizations and private sector employers. The Department's authorities to increase the frequency of reporting, establish data standards or verify the accuracy of the information reported by third parties are limited in some cases. In other programs, data important to fully assess the effectiveness of strategies adopted to achieve our GPRA goals are not readily available, or DOL's information technology systems require enhancements to support routine access by program managers to key performance data.
Despite these measurement challenges, DOL is continuing its shift toward performance-based management through a number of initiatives in FY 2003. First, the Department plans to continue sponsoring workshops for program staff on performance management and measurement issues. Through these workshops, program managers will learn about measurement approaches from experts in the field and will also exchange ideas and best practices with each other. Second, the Office of Inspector General will continue its reviews of the reliability and quality of selected performance data used by DOL programs in FYs 2002 and 2003, and the Department will work closely with program staff in addressing problems identified in these reviews. Finally, the Department plans to further increase the use of program evaluations, particularly in areas identified by the PART and other sources as lacking critical performance data.
Working together, DOL executives and program managers are continuously improving the quality of the Department's performance information and the use of that information in management decisions. However, the success of these efforts largely depends on DOL's authorities to collect and verify data and the availability of resources needed to do so.
Within the larger Departmental framework, individual DOL agencies will address data challenges that are unique to their program environment and develop solutions that are consistent with the Department's reporting requirements. While some DOL programs currently have adequate systems in place, others must overcome barriers to the production of timely, accurate, and relevant performance data. In FY 2004, DOL and its agencies will continue efforts to address four issues: the availability of data, validation of data, timeliness of reporting, and the use of data in managing for results. Below are examples of several initiatives that agencies are undertaking to further improve the measurement of our program results.
Employment Standards Administration
FECA's automated system is undergoing a complete redesign covering every major program operation. The redesign will replace a patchwork of loosely linked programs each with its own database and rules, and provide a single automated system that will make data accessible to all users. This redesign will allow workers' compensation claims staff to work more efficiently while providing improved customer service to injured workers, medical providers, and employing Federal agencies.
Employment and Training Administration
The Department continues to address the need to ensure the accuracy and reliability of performance data submitted by our employment and training system partners, which serves as the foundation for key program decisions. The development of a comprehensive data validity system for the core indicators of the WIA program, Wagner-Peyser, and other key employment and training programs is expected to be substantially completed in FY 2002.
In FY 2003, the Department will establish a policy on data validity and verification that applies to its job training and employment programs. It will also focus attention on wide-scale training and technical assistance to deploy the data validity system among states and other grantees. Office of Inspector General reports have raised issues about the accuracy and reliability of data used to measure the outcome performance of the Welfare-to-Work competitive grant program and the Trade Adjustment Assistance and North American Free Trade Agreement Transitional Adjustment Assistance programs. Both programs are expected to be the subject of special attention with regard to the validity and verification system.
The Department has taken a leadership role in implementing common performance measures for several of its job training and employment programs under the President's Management Agenda goal of Linking Budget and Performance. The goals set forth in this plan are framed as the common measures. The Department will be undertaking steps to implement the measures during FY 2003 so that FY 2004 performance will establish the baseline using the new measures. The data validation and verification activities described above may be updated to reflect the common measures.
Veterans' Employment and Training Service
Accurate outcome oriented performance data has been an issue for VETS since performance planning under the GPRA commenced in 1999 and placed emphasis on outcomes. Reporting data prior to GPRA was developed to respond to the reporting requirements of Title 38, Section 4107, which is essentially activity based. In addition, reporting of placements on the ETA 9002 has been subject to error and under-counting because it requires that a veteran specialist (DVOP or LVER) make contact with a registrant to confirm that the person got a job before their employment may be reported on the ETA 9002.
VETS initiated a special study in Maryland that utilized Unemployment Insurance wage records to gain a better understanding of the outcome of services provided by veterans specialists to veterans. This study has shown that a much larger number of veterans get jobs than are likely to have been captured by the information collection and reporting systems now in place. This result has been corroborated by other State studies conducted by ETA.
VETS is an active partner with ETA in designing new reporting systems that will use the results of these studies to better capture performance outcomes. Both VETS and ETA will be implementing outcome oriented performance measures with the public labor exchange system, which has been fully integrated into the One-Stop environment. Unemployment Insurance wage records will be the key data source supporting the new performance measures.
VETS will continue to implement the recommendations made in the July, 2000 data capacity report on VETS programs prepared by a contractor. This will help VETS ensure that its measurement of performance is accurate and verifiable to the best degree possible, considering VETS biggest programs (DVOP and LVER) must rely on data from the States.
In FY 2003, VETS intends to have a new program evaluation system in place that will capture performance data on services provided to veterans by the public labor exchange system. A key consideration that the new system will address is how to effectively verify services to veterans when the old paradigm has changed. In past years, all veterans entering a local office were registered and a formal file established. This made statistical sampling of files an effective way to verify services to veterans that were reported on the VETS 200 and the ETA 9002. However, in the One-Stop environment there has been an emphasis on self-service, where there is no registration. In fact, in some States, there is no registration even if a mediated service is provided. This may require replacing the traditional method of statistical sampling with other verification techniques, such as surveys.
Mine Safety and Health Administration
MSHA has a significant database and collection system that captures most of the information necessary to track performance under the strategic plan. Data has been collected for many years and the database is well established for performance measures under the performance goal of reducing the number of mine fatalities and the nonfatal-days-lost injury incidence rate. However, MSHA relies on mine operators and contractors to comply with legal requirements to report accurately and timely employment, injuries and accidents. MSHA conducts periodic audits to ensure compliance. The number of audits conducted may influence the degree of compliance. Quarterly reports are generated for managers showing active mines that do not report quarterly employment and production information. Routine follow-up visits are conducted at these mines and audits increased as appropriate depending upon findings. If analysis of these visits/audits determine any pattern or problem with the quality or reliability of the data, MSHA takes corrective action.
MSHA's system for determining compliance with the coal respirable dust standard has been in place since the 1970s and procedures are well established to ascertain the accuracy and reliability of the data. Automated devices are used to weigh the inspector dust samples and automatically enter the results into a custom designed program that updates the dust data files daily. A quality control program developed jointly by MSHA and the National Bureau of Standards assures that the weighing process continues to produce reliable results over time, and computer edit checks assure the accuracy of the database.
Metal and Nonmetal inspectors have conducted industry-wide sampling since the 1970s. Health policies and the management information system are well established and reliable. Automated devices are used to weigh inspector dust samples at MSHA's analytical lab that was accredited by the American Industrial Hygiene Association in FY 1998. Computer edits assure the accuracy of Management Information System data input. Between 1994 and 1998 the Agency workforce that inspects metal and nonmetal mines declined and began to slightly increase during 1999 to 2002; however, the number of metal and nonmetal mines has continually increased. Metal and nonmetal samples are generally collected at the discretion of the inspector based upon the conditions observed at the mine. Designations of high-risk occupations and new sampling procedures have been established for targeting sampling at Metal and Nonmetal Mines. Using the new procedures, samples collected in FY 2001 are used to establish a baseline. A new goal was established in FY 2002 and continues in FY 2004.
MSHA's current database and collection system will be used to capture data and samples relating to noise exposures and diesel particulates.
Occupational Safety and Health Administration
In FY 2003, OSHA will continue to use program data to manage its programs and ensure the quality of safety and health data. The agency will rely on the results of program evaluations to identify the best strategies to achieve its strategic and performance goals.
OSHA will maintain its efforts to improve data validation procedures. Validation of data generated by the agency for current performance measures will be addressed through a variety of means such as annual on-site audits of the injury and illnesses records of employers to determine the accuracy and reliability of the OSHA 200 Logs, the source of data for the OSHA Data Initiative and the BLS Annual Survey; information and outreach programs and enforcement of the injury and illness recordkeeping regulations; revision of injury and illness record keeping system (regulations, forms, and guidelines) to improve the quality of records by simplifying forms and regulations, providing clearer guidance for employers, and incorporating incentives for employers to maintain high quality records; and continuing the various methods OSHA developed for validating and verifying data in the OSHA Integrated Management Data System (IMIS).
Program evaluations will assess how well OSHA's programs, policies, and procedures are working, including the effectiveness of specific standards, customer satisfaction, and specific approaches towards reducing occupational injuries and illnesses.
The Department has a solid financial systems infrastructure from which a cost accounting capability is being developed using the resources of a reliable, established accounting systemthe Department of Labor Accounting and Related Systems (DOLAR$). DOLAR$, serving as the system of record for financial results throughout the Department, has been modified to capture, aggregate, allocate and report costs. The Department's cost systems allow aggregation of costs across agency lines and the allocation of direct and indirect costs to the strategic and outcome goal levels established in the Department's Strategic Plan. In the FY 2001 Annual Report on Performance and Accountability, the Department provided cost data for FY 1999 through FY 2001 to accompany and provide perspective on each outcome goal presentation in the Performance Report section.
The Department has maintained cost accounting information, beginning in FY 1999, for the outcome goals in the Department's Strategic Plan. In this FY 2004 plan, the Department continued its practice of linking budget authority and outlays to both strategic and outcome goals, and for the first time this year, has added information on the cost to achieve each of the core performance goals shown in Chapter 4. In addition, Appendix C provides an overview of the linkage between budget activities and outcome goals. DOL will continue to develop the capability to consolidate data from a variety of program and financial system sources and link that data as needed to meet the performance reporting requirements of GPRA.
A significant accomplishment for the Department has been the development and implementation of its first integrated agency-level performance budget format for the FY 2004 budget cycle. In addition, the Department will develop for FY 2005 prototype goals to improve the cost effectiveness of a job training and one regulatory program, with the expectation that these prototypes could be adapted to other DOL job training and regulatory programs.