UIPL22-05_Attach.pdf

ETA Advisory File Text

Attachment Administration of Unemployment Insurance UI Data Validation DV A. Data Validation System Overview. At various intervals ranging from weekly to annually states submit information to the national office on over 40 required reports. Most of this information consists of summary counts of their UI-related transactions e.g. initial claims taken or statuses e.g. active employers . In conjunction with Employment and Training Administration ETA staff Mathematica Policy Research Inc. MPR designed an automated system to validate or assess the accuracy of most of the key summary count data on 13 of the UI required reports UIRR that supply workload data or data used for key performance measures ETA reports number 207 218 227 581 586 5130 5159 9050 9051 9052 9053 9054 9055. States also supply the results of individual sample reviews of the quality of nonmonetary determinations on the ETA 9056 report and of the quality of Lower Authority Appeals decisions quality on the 9057 report and of Tax Performance System TPS acceptance samples. These micro-database reports are validated differently from reports of summary counts. Since 2002 UI DV has been facilitated through the use of standardized software that handles processing and reporting for summary count validation and facilitates reporting for quality sample validation and wage item validation. At present states use PC-based software developed by MPR. Software developed by Office of Workforce Security OWS to run on the state Sun machines used for UI required reporting is scheduled for release in April 2005 and states must begin using it after June 30 2005. There are three components to validation depending on the type of report validated Validation of Summary Counts. This covers validation of selected report cells on the first group of 13 required reports noted above. There are two dimensions to validating summary counts report validation RV and data element validation DEV . Modules 1 2 and 3 of the DV Benefits and Tax Handbooks explain validation of summary counts which is the most complex and most highly automated. The standardized software is designed to support this aspect of validation. Validation of Quality Samples. Validation of the Benefits Timeliness and Quality BTQ quality samples for nonmonetary determinations and lower authority appeals and three of the TPS acceptance samples involves ensuring that their samples are random and are drawn from the correct universes. This approach is explained in Module 4 of the handbooks. Both the MPR and OWS software merely serve as the vehicle for transmitting the results of this largely manual validation process. - 2 - Validation of Wage Items. Wage items are counts of records of employed individuals that employers submit on UI wage records. The validation procedure is designed to ascertain whether the agency is correctly counting what the agency receives from employers it cannot ascertain whether employers submit accurate information. Because the wage item submissions are not fully automated the procedure relies on recounting small portions a day s worth or a batch of wage items from employer reports. Module 5 of the Tax Validation Handbook explains the validation procedure. Both the MPR and OWS software support this review by allowing the validator to record the data from this review and transmit them to the National Office. B. How Data Validation Is Administered States are required to validate certain reports data every third year except for data elements used for GPRA measures which are validated annually. The basic cycle presumes that the validated items pass validation items that do not pass must be revalidated the following year. The State Quality Service Plan SQSP is the vehicle through which states submit plans to implement validation or to revalidate failed items. The UI DV cycle will use a validation year concept. The validation year will coincide with the SQSP performance year. Validations pertaining to any reporting activity for the 12-month period that begins April 1 and ends the following March 31 will be considered a part of the same validation year. Because the ETA 581 report for March quarter is not due until May 20 to be submitted by May 10 tax validations must be done for the quarters ending July September or December. Validation reports for the validation year must be submitted to the national office by May 10 except for validation reports for validation year 2005 which are due by June 30 2005. This will allow sufficient time for data validation results to be included in the SQSP process. The critical variable for the state s validation cycle is which validation activities must be included in the SQSP because they did not pass validation in the just-completed validation year. The details of this determination depend on the structure of pass-fail groups and random samples. For benefits DV this structure will change when states cease using the MPR software and begin using the OWS software. The bulk of this attachment Section C below addresses the difference between the MPR and OWS structures. - 3 - March 31 2005 was a watershed date. It marked the end of validation year 2005. These are the main differences between validation year 2005 and all subsequent validation years Validation Year 2005 o States must use the MPR software for validation year 2005 validations and pass and fail will be determined using MPR categories. o All UI DV results for report periods through March 31 2005 are considered to be part of validation year 2005. o All validation year 2005 validation reports must be submitted to the national office by June 30 2005. No validation results submitted with the MPR software after this date will be accepted. o Except for GPRA elements passing validations for validation year 2005 do not need to be repeated until validation year 2008 and submitted until May 10 2008. Validation Year 2006 o This year includes all validations for report activities for April 1 2005 through March 31 2006 April 1 2005 through December 31 2005 for tax to be submitted timely . o Items validated during validation year 2006 include GPRA elements items that failed validation in 2005 and items not previously validated including Module 4 and 5 validations . o Validations will use the OWS software and results must be submitted by May 10 2006. o Subsequent validation year cycles will follow the 2006 pattern. The following sections of this attachment address the three components of validation Section C is the validation of summary counts Section D validation of quality samples and Section E validation of wage items. C. Validation of Summary Counts. 1. Basic Approach . Although UI reports typically combine more than one type of transaction for the sake of efficiency the MPR DV system examines all UI transactions or statuses by type each general type is called a population. For validation purposes it defines 15 benefits populations these enable the validation of 1 227 report cells on 12 benefits reports. There are five tax populations which relate to 45 report cells all on the ETA 581 report. The key building block of the MPR analysis is the subpopulation. A subpopulation is a unique subset of records within a population which have certain defined characteristics - 4 - e.g. final UI payments that were made for the state s maximum duration. There are 349 benefits and 46 tax subpopulations which the DV system combines to validate approximately 1 272 report cells. a. Steps in the Validation Process. The state first constructs extract files to compile all pertinent transactions for the desired report period. The extract files are built according to MPR s record layouts and follow the state-specific instructions in Module 3 and Handbook 401 reporting instructions. Each transaction contains the necessary characteristics or dimensions that enable the DV software to classify it into the appropriate subpopulation. Each transaction falls into only one subpopulation but given the nature of UI reporting it may be aggregated into the counts of several report cells. 2. Benefits Validation . After extract files for benefits populations are loaded into the system benefits validation proceeds in two stages Report Validation RV . The software counts all transactions within a subpopulation and combines subpopulations into corresponding report cells. The validation counts are compared with reported counts and the reported counts are considered valid if they fall within the established acceptable tolerance 1 for groups that include data used in Government Performance and Results Act GPRA measures and key time-lapsed variables and 2 for all other groups . Data Element Validation DEV . Portions of the extract files used to compute the validation counts for RV are also used as sampling frames for drawing random and non-random samples. The purpose of the samples is to determine whether the data elements in the extract file represent the appropriate information and are consistent with the source files from which they were derived and thus whether the validation counts can be trusted to reflect the state s activity according to Federal reporting requirements. They are the means for verifying whether for example every UI first payment for a full week of unemployment included in the extract file is truly a first payment it is for the UI program and it is not a partial payment. For the most important sets of transactions benefits validation relies on random samples of 100 or 200 cases taken from the extract files from which an inference can be made to the extract-file universe. The random sample s universe is considered valid if it contains no more than 5 misreported transactions. Random samples are examined in two steps. First an acceptance sample of the first 30 of 100 or 60 of 200 cases is reviewed. The remaining cases are reviewed only if the result of the acceptance sample review is not clearly a pass or fail. The other sample types Minimum or Missing Subpopulations or Outliers--are smaller comprising 1 to 10 transactions of each type. They may be selected randomly from selected subpopulations or as outliers of a sorted population. Although they are too small to tell whether a subpopulation passed or failed their - 5 - purpose is the same as the 100- or 200-case random samples to indicate whether system documentation supports the characteristics of the transactions in the extract file. 3. Tax Validation. The logic of tax validation is somewhat different. Instead of relying on random samples to tell whether the extract file used for RV has been properly constructed and contains correct data the entire extract file is subjected to a series of automated and manual DEV tests to eliminate errors and determine whether it contains only transactions that meet Federal definitions. These tests are The software tests each transaction to determine whether it contains all necessary data elements all data conform to specified coding conventions it is not a duplicate and it meets all parameters. Transactions that fail to meet all tests are removed for examination by the validator. Two transactions from each subpopulation are selected and each data element is checked against system documentation to test whether the extract file s data match the data in the state database. If any transactions fail this File Integrity Validation FIV review the extract file must be reconstructed under the assumption that in highly automated processes the errors are likely to be repeated. Because a failure of a FIV case requires that the extract file be rebuilt the software produces a Pass or Fail designation for FIV samples even though they are not the size of benefits random samples. Some states embody information in the coding structure of certain elements in the extract file. For example they may use certain values of the Employer Account Number for contributory employers and others for reimbursing. A secondary test to determine whether transactions in the extract file that have passed all the initial logic tests are properly classified is to sort them by these elements that state- specific supplementary codes. If the transactions are properly classified the coded elements will all be within the ranges designated by the state. The process of building an extract file making an RV comparison between reconstructed and reported counts and performing the DEV tests often involves multiple iterations. Only when the file passes all these checks can the validator have confidence that the extract file is correct and can treat the RV totals from it as the standard for judging reported counts. Thus for a reported count to pass RV the extract file must have passed all DEV tests. 4. Validation in the MPR Software Environment . - 6 - a. Description of the MPR Environment. In the current MPR PC-based environment States prepare extract files by population 15 benefits 5 tax and load them for analysis and processing by the standard DV software. States manually enter all reported counts. The MPR software performs various edits on the extract to remove invalid and duplicate transactions sorts all transactions into subpopulations and maps these into sets of counts that correspond to report cells and then computes differences between validation and reported counts. For benefits validation it o Creates 37 RV pass fail groups within the 15 benefits populations that are used to determine whether reports pass report validation o Draws 16 random samples and various non-random samples that are examined. The non-random samples are either minimum which consist of 2 transactions for certain subpopulations or missing subpopulation which consist of one transaction from each subpopulation not represented in the random sample or outliers which may include the same subpopulations selected in the random samples and consist of extreme values . Only seven of the 16 random samples are drawn from the same sets of transactions constituting the RV pass fail groups identified in the software system. For tax validation o There are 39 RV pass fail levels within the 5 tax populations that determine whether reports pass report validation. All but two of these are single report cells o There are no large random samples from which pass fail inferences can be made to a population for DEV purposes. Instead for an RV pass fail group to pass validation the extract file used for the validation count must pass all DEV tests described above in c 3 . The MPR software produces two sets of output reports that states transmit via e- mail to the OWS national and regional offices as evidence that they have completed validation o The RV output reports are summarized by population. They provide all validation counts reported counts count and differences and they indicate pass fail at the selected levels. o The DEV output reports are also summarized by population and provide a summary of the sample reviews and results. - 7 - o States submit the validation output reports to the national and regional offices by population. RV and DEV results are submitted separately. o Early versions of the software had the option of exporting the results as Excel spreadsheets or PDF files later versions provided a text-file option which we encouraged so that the reports could be more readily entered into a national office Access database. b. Administering the DV Process in the MPR Software Environment. Whether a portion of validation passes and is good for three years or must be redone and resubmitted during the next validation year depends on the following scoring rules Benefits RV o The national office tracks results for 15 benefits populations. The state prepares an extract file for each population and enters all reported counts. The software produces validation counts for every report cell and produces group tallies and the differences in group counts from which pass and fail are determined. Because validations are done by population and the process is largely automated once the extract file is loaded the DV software is designed to submit all RV results by population. The consequence is that although pass or fail is determined by whether key groups meet the validation criteria results of a revalidation cannot be submitted for only the failing group the state must submit redone report validation results for the entire population the next validation year. This leads to the following rules for conducting validations If all groups within the population obtained a pass score the state does not have to submit population results for three years i.e. until May 10 2008 unless the population contains a GPRA element i.e. populations 4 and 12 in which case population results must be submitted every validation year . If at least one group within the population fails validation in validation year 2005 then the state must revalidate the entire population and submit RV results by May 10 2006. For example if the group New Intra Total claims one of seven groups in Population 3 Claims--failed in a state during validation year 2005 the state would have to repeat RV for Population 3 during validation year 2006 and submit those new RV results by May 10 2006. This would include all results for Population 3 even those groups that passed in 2005. In the case where at least one group within the population is missing data the population will be flagged as incomplete and the state must resubmit results for the entire population as soon as possible. Failure to resubmit the completed population by June 30 2005 will cause the missing group to be assigned a fail score. Since the Sun-based software generates a new random sample for - 8 - population 14 when the population obtains a fail overall score states are required to revalidate both the RV and DEV components of it by May 10 2006. Benefits Data Element Validation DEV o The states submit data for random and non-random samples but only the 16 random samples are scored i.e. samples 1 2 5 7 11 12 16 18 20 23 28 30 33 35 39 and 40 in Table A.1 of the Handbook . Since the software provides for the individual submission of samples each random sample will be scored and handled individually. If the number of cases in error indicates that the universe error rate is less than or equal to 5 then the sample obtains a pass score and the results are valid for three years unless the sample includes a GPRA element samples 18 and 39 . The states will need to resubmit results by May 10 2008. If the number of cases in error indicates that the universe error rate is greater than 5 then the sample obtains a fail score and the state has to resubmit results for that sample by May 10 2006. Sample results are not linked to RV groups under the MPR software even for the seven random samples drawn from the same populations as RV groups. Thus if the random sample fails the state must only resubmit the random sample and not the related RV group and population. Sixteen samples and groups are linked under the OWS software however. See 3b below. In the case where at least one group within the population is missing data the population will be flagged as incomplete and the state must resubmit results for the entire population as soon as possible. Failure to resubmit the completed population by June 30 2005 will cause the missing group to be assigned a fail score. Tax Validation o In tax report validation is linked with data element validation. Because DEV checks that the extract file was constructed properly all tests must be passed to indicate that the extract file is valid. As with benefits the software constrains the user to submit a population in its entirety. This results in the following rules for revalidation and resubmission of validation results If all DEV and report validation items within the population obtained a pass score then the population obtains a pass and the state does not have to submit population results for three years i.e. by May 10 2008 except for Population 3 which contains a GPRA element Population 3 results must be submitted every validation year independently of the score . If any DEV test fails the population must be revalidated in the - 9 - next validation year and the state has to resubmit population results by May 10 2006. If any report validation element fails even if all DEV tests pass the population must be revalidated in the next validation year and the state has to resubmit population results by May 10 2006. In the case where at least one group within the population is missing data the population will be flagged as incomplete and the state must resubmit results for the entire population as soon as possible. Failure to resubmit the completed population by June 30 2005 will cause the missing group to be assigned a fail score. Because the ETA 581 report for the March quarter is not due until May 20 tax validations must be for the quarters ending July September or December to be submitted by May 10. 3. Validation in the Sun-based Software Environment a. Description of the OWS Sun-based Software Environment. The basic population subpopulation structure of UI DV and the number of report cells validated will remain unchanged and states will not have to modify their extract files. Several changes have been made in the structure of benefits samples and RV pass fail groups however that affect how the system can be administered and operated. Tables 1 and 2 compare the structure of groups and samples under the MPR and Sun versions of the benefits software the tax structure is the same for both software environments . Table 3 gives detail on samples and pass fail groups for the Sun environment. For sixteen random samples the universe for the random sample is the same as one or more RV pass fail groups i.e. there is a 1 1 relationship between them . These relationships are highlighted. The main changes are Many of the benefits RV pass fail groups have been redefined to better align with key measures and workload items and to emphasize groups of report cells that are generally important from a programmatic standpoint. There are now 68 such benefits groups. o The tax validation structure is unchanged. Some of the benefits random samples have been reconfigured to align with the RV pass fail groups and a new random sample has been added for population 14. In the new structure there will be 17 random samples instead of 16. Non- random samples are drawn partly in response to observed omissions in the samples and may result in different numbers of sample data sets. Sixteen of the random samples have a 1 1 relationship with RV pass fail groups. As described in the next section this affects the scoring rules for determining when validations must be repeated the following year. - 10 - Although the basic reports summarizing quantitative and qualitative results will remain largely the same as those currently transmitted to the national office as output reports from the MPR software a suite of reports summarizing validation results for key performance measures and workload items will be developed to track and summarize the DV progress more easily. The software will automatically retrieve and load the report counts to be validated from the state s copy of the UI reports database UI DB eliminating the need for data entry. Instead of states submitting results by e-mail the results from completed validations will be picked up from the state Sun machine and transferred to the UI DB from which reports can be produced. To minimize repeat validation effort the software allows random samples to be drawn and submitted individually. Like the MPR software it does however require that all RV calculations in a population be completed and submitted as a bloc. This process is not labor-intensive because the extract file must be prepared in any case for the entire population and the software produces all validation counts and obtains reported all counts. b. Administering the DV Process in the Sun Software Environment. The scoring rules for determining whether a portion of validation passes and is good for three years or must be redone and resubmitted within the next validation year are basically the same as the rules used with the MPR software. Like the MPR software the Sun-based software also requires that all report validations for a population be submitted as a bloc. The essential difference in scoring involves the 16 benefits groups for which a 1 1 relationship exists between a benefits RV pass fail group and a random sample. For these groups RV will be linked to DEV and If the random sample fails the random sample and the RV pass fail groups in the population must be revalidated within one year. UI DV s predecessor Workload Validation applied the same principle. If the random sample passes but the related RV group fails only the RV for the population must be repeated next year. There is no link between the other random sample findings and the RV groups that do not have a 1 1 relationship because of the uncertainty in making inferences from samples to specific RV groups. Therefore a failure of the random sample has no implications for the pass fail determination made on the RV groups and these are scored as under the MPR environment. Thus for example if the random sample for Overpayment Reconciliation activities fails none of the five related groups of overpayment reconciliation report cells need to be revalidated for quantity in the next year if their RV - 11 - results were pass. Only the Overpayment Reconciliation sample needs to be repeated in the following year. 4. Future Developments. As experience accumulates we will evaluate the validation structure used for both benefits and tax with a view toward making validation more efficient and effective. This evaluation will include the structure of samples and RV pass fail groups the logical relationship among them and the role to be played by outside samples the Benefits Timeliness and Quality BTQ review of nonmonetary determinations quality and the Tax Performance System TPS acceptance samples for status determination and field audit accuracy. Pending changes in the key UI Performs measures and in required reports elements will also entail changes in validation. D. Validating Quality Samples. 1. Procedure . As explained in Module 4 of the handbooks there are two steps to validating the BTQ quality samples for nonmonetary determinations and lower authority appeals the three TPS status determination acceptance samples and the TPS field audit acceptance sample. Validating the universe. The first is comparing the count of the sampling frame universe from which the samples are drawn with the appropriate report count to ensure that the samples are drawn from the proper universe. The samples are to be drawn from the transactions used to compile the following reports o Nonmonetary determinations quality sample ETA 9052 report o Lower Authority Appeals quality sample ETA 9054 o TPS Acceptance Samples for new successor and inactive terminated status determinations and field audits ETA 581. Checking for randomness. Module 4 explains how the validators are to test to ensure that the sample was randomly drawn. 2. Pass Fail Criteria . The sampling frame must be within 2 of the benchmark report count. The sample selection procedure must pass the test of randomness described in Module 4. 3. Timing and Frequency . Because the validation must precede investigation of the sample validation must occur as soon as the appropriate sample can be drawn. Module 4 validations are to be conducted every three years as follows and with the following exceptions - 12 - BTQ quality sample validations may be done for any quarter as soon as the data are available. o If the universe is not within 2 of the benchmark report count the review must be repeated the following year although the sample may be evaluated for quality. o If the sampling procedure is not random the sample must be redrawn and must pass the appropriate tests of randomness before the samples can be evaluated by the BTQ reviewers. o Validators must arrange procedures with the BTQ unit to ensure that this procedure is followed. The TPS reviews must be conducted as follows o The status determinations reviews must be done in late January or February whenever the calendar year s data are available to draw the calendar-year sample o The review for the TPS field audit acceptance sample must be done in October or November whenever the data for the third quarter of the calendar year become available to draw the sample from this universe of audits for the first three quarters of the calendar year. o If the universe is not within 2 of the benchmark report count the review must be repeated the following year although the sample may be evaluated for quality. o If the sampling procedure is not random the sample must be redrawn and must pass the appropriate tests of randomness before the TPS reviewers can evaluate the samples. o Validators must arrange procedures with the TPS unit to ensure that this procedure is followed. 4. Reporting . Under the MPR software results of these quality reviews are reported in the comments field to the population to which they pertain e.g. Benefits Population 5 for the nonmonetary determinations review. The OWS software will contain a similar function. Validation of quality reviews conducted by June 30 2005 must be submitted using the MPR software. E. Validation of Wage Items. 1. Procedure . As explained in Module 5 of the tax handbook wage items received by the state in a short period of time in the major submission forms are recounted and compared with the count in agency records. This is the same procedure used under Workload Validation. 2. Pass Fail Criterion . The recounts must be within 2 of the benchmark report count. - 13 - 3. Timing and Frequency. This validation is to be conducted every three years. It may be done at any time of year. If the validation indicates difficulties with either the universe or sampling methodology the validators must work with the TPS reviewers to ensure that the appropriate corrections are made. The validation must then be repeated the following year. 4. Reporting . The MPR software contains a simple worksheet for completing this review and submitting the results. The OWS software will have a similar function. Validations of wage items conducted before by June 30 2005 must be submitted using the MPR software. - 14 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 5159A 201-10 5159A 201-10 5159A 202-10 5159A 201-11 5159 Intrastate Weeks Claimed 5159A 203-10 5159A 201-12 5159A 201-12 5159A 202-10 5159A 202-12 5159A 202-11 5159 Interstate Liable Weeks Claimed 5159A 203-12 5159A 202-12 5159A 201-11 5159A 203-10 5159A 202-11 5159A 203-11 1 Weeks Claimed 5159 Interstate Weeks Claimed Filed from Agent State 5159A 203-11 Total 5159A 203-12 5159B 303-26 5159B 303-26 5159B 303-27 5159B 303-27 5159 Final Payments 5159B 303-28 5159 Total 5159B 303-28 218B 102-8 218B 102- 8-13 218B 102-9 218B 104- 14 19 218B 102-10 218B 104-20 218B 102-11 218B 102-12 218B 102-13 218B 104-14 218B 104-15 218B 104-16 218B 104-17 218 Final Payments 218B 104-18 2 Final Payments Non-Pass Items 218B 104-20 218 Total 5159A 101-2 5159A 101-2 5159A 101-7 5159A 102-2 5159A 102-2 New Intra Total 5159A 103-2 5159A 102-7 5159A 101-7 5159A 103-2 5159A 102-7 3 Claims 5159 New Intrastate Interstate Received as Liable 5159A 103-7 Received as Liable Total 5159A 103-7 - 15 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 5159A 101-4 5159A 101-4 5159A 102-4 5159A 102-4 5159 Filed from Agent 5159A 103-4 Filed from Agent Total 5159A 103-4 5159A 101-5 5159A 101-5 5159A 102-5 5159A 102-5 5159 Taken as Agent 5159A 103-5 Taken as Agent Total 5159A 103-5 5159A 101-6 5159A 101-6 5159A 102-6 5159A 102-6 5159 Transitional 5159A 103-6 Transitional Total 5159A 103-6 586A 101-1 586A 101-1 586 New CWC 586A 102-1 New CWC Total 586A 102-1 586A 101-2 586A 101-2 586 New CWC BY Established 586A 102-2 New CWC BY Total 586A 102-2 218A 100-2 5159A 201-13 218 Total Determinations 218A 100-3 218A 100-2 218 Total Benefit Year Established 218A 100-4 218A 100-3 218B 101-8 218A 100-4 218B 101-9 218A 100-5 218B 101-10 218A 100-6 218B 101-12 218B 101 8-13 218B 101-13 218B 103 14 19 218B 103-14 218B 103-15 218B 103-16 218B 103-17 218 Total Benefit Year Established 218B 103-18 218A 100-5 218A 100-6 Non-Pass Items 5159A 201-13 not grouped - 16 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 5159A 101-3 5159A 101-3 5159A 102-3 5159A 102-3 3a Additional Claims 5159 Additional Intrastate 5159A 103-3 Additional Intra Total 5159A 103-3 9050ALL 2 9050ALL 2 9050ALL 3 9050ALL 3 9050ALL 4 9050ALL 4 9050ALL 6 9050ALL 6 9050ALL 7 9050ALL 7 9050 First Payments 9050ALL 8 9050 Total 9050ALL 8 9050PART 2 9050PART 2 9050PART 3 9050PART 3 9050PART 4 9050PART 4 9050PART 6 9050PART 6 9050PART 7 9050PART 7 9050 Partial First Payments 9050PART 8 9050 Partial 9050PART 8 9051ALL 2 9051ALL 2 9051ALL 3 9051ALL 3 9051ALL 4 9051ALL 4 9051ALL 6 9051ALL 6 9051ALL 7 9051ALL 7 9051 Continued Weeks Compensated 9051ALL 8 9051 Total 9051ALL 8 9051PART 2 9051PART 2 9051PART 3 9051PART 3 9051PART 4 9051PART 4 9051PART 6 9051PART 6 9051PART 7 9051PART 7 9051 Continued Partial Weeks Compensated 9051PART 8 9051 Partial 9051PART 8 586A 101-4 586A 102-4 4 Payments 586 Weeks Compensated 586A 102-4 586 Total 586A 101-4 - 17 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 586A 101-6 586A 102-6 586 Prior Weeks Compensated 586A 102-6 586A 101-6 586B 8-Total 586B 8-Total 586 CWC First Payments 586B 9-Total 586B 9-Total 586A 101-5 586A 101-5 586 Benefits Paid 586A 102-5 586A 101-7 586A 101-7 586A 102-5 586 Prior Benefits Paid 586A 102-7 586 Total 586A 102-7 5159B 301-14 5159B 301-14 5159B 301-15 5159B 301-15 5159 UI Weeks Compensated 5159B 301-16 5159B 301-16 5159B 301-17 5159B 301-17 5159B 301-18 5159B 301-18 5159 UCFE UCX Weeks Compensated 5159B 301-19 AR5159B Total 5159B 301-19 5159B 302-14 5159B 302-14 5159B 302-15 5159B 302-15 5159 UI Amount Paid 5159B 302-16 5159B 302-16 5159B 302-17 5159B 302-17 5159B 302-18 5159B 302-18 5159 UCFE UCX Amount Paid 5159B 302-19 AR5159B Total 5159B 302-19 207A 101-2 207A 101-2 207A 103-1 207A 101-3 207 Total Single Claimant Dets and UCFE UCX Redets 207A 105-1 207A 101-5 207 UI Single Claimant Redeterminations 207A 101-3 207A 101-6 207A 102-2 207A 102-2 207A 102-3 207A 102-3 207A 104-1 207A 102-5 5 Nonmonetary Determinations 207 Single Claimant Denials 207A 106-1 207 Total Total 207A 102-6 - 18 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 207A 101-5 207A 103-1 207 UI Multi-Claimant Determinations 207A 101-6 207A 104-1 207B 201-8 207A 105-1 207B 201-9 207A 106-1 207B 201-10 207B 201-8 207B 203-8 207B 201-9 207B 203-9 207B 201-10 207 Separation Determinations 207B 203-10 207B 202-8 207B 202-8 207B 202-9 207B 202-9 207B 202-10 207B 202-10 207B 203-8 207B 204-8 207B 203-9 207B 204-9 207B 203-10 207 Separation Denials 207B 204-10 207B 204-8 207C 301-12 207B 204-9 207C 301-13 207B 204-10 207C 301-14 207C 301-12 207C 301-15 207C 301-13 207C 301-16 207C 301-14 207 UI Non Separation Determinations 207C 301-17 207C 301-15 207C 302-12 207C 301-16 207C 302-13 207C 301-17 207C 302-14 207C 302-12 207C 302-15 207C 302-13 207C 302-16 207C 302-14 207 UI Non Separation Denials 207C 302-17 207C 302-15 207A 102-5 207C 302-16 Non-Pass Items 207A 102-6 207C 302-17 - 19 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 9052A 2 9052A 2 9052A 3 9052A 3 9052A 4 9052A 4 9052A 6 9052A 6 9052A 7 9052A 7 9052A 8 9052A 8 9052B 100 9052B 98 9052B 102 9052B 99 9052B 103 9052B 100 9052B 104 9052B 102 9052B 98 9052B 103 9052 Single Claimant Non Monetary Determinations 9052B 99 9052 Total Total 9052B 104 9052C 193 9053A 2 9052 Multi-Claimant Non Monetary Determinations 9052C 194 9053A 3 9053A 2 9053A 4 9053A 3 9053A 6 9053A 4 9053A 7 9053A 6 9053A 8 9053A 7 9053B 98 9053A 8 9053B 99 9053B 100 9053B 100 9053B 102 9053B 102 9053B 103 9053B 103 9053B 104 9053B 104 9053B 98 9052C 193 9053 Single Claimant Non Monetary Determinations 9053B 99 9052C 194 9053C 193 9053C 193 9053 Multi-Claimant Non Monetary Determinations 9053C 194 9053 Total Total 9053C 194 6 Appeals Filed Lower 5130 Lower Authority Appeals AR5130B 200-10 Total AR5130B 200-8 - 20 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells Authority AR5130B 200-8 AR5130B 200-10 AR5130B 200-11 AR5130B 200-9 7 Appeals Filed Higher Authority 5130 Higher Authority Appeals AR5130B 200-9 Total AR5130B 200-11 5130A 100-1 5130A 100-1 5130A 100-3 5130A 100-3 5130 All Decisions 5130A 100-5 5130A 100-5 5130B 210-10 5130B 210-8 5130 UI Disposed of During Month 5130B 210-8 5130B 210-10 5130C 300-14 5130C 300-14 5130C 300-16 5130C 300-16 5130 UI Decisions by Type 5130C 300-18 5130C 300-18 5130C 310-14 5130C 310-14 5130C 310-16 5130C 310-16 5130 UI in Favor of Appellant 5130C 310-18 5130C 310-18 5130D 400-21 5130D 400-21 5130D 400-22 5130D 400-22 5130D 400-23 5130D 400-23 5130D 400-24 5130D 400-24 5130D 400-25 5130D 400-25 5130 UI Decisions by Issue 5130D 400-26 5130 Total 5130D 400-26 9054A 2 9054A 2 8 Appeals Decisions Lower Authority 9054 All Appeals 9054A 3 9054 Total 9054A 3 5130A 100-2 5130A 100-2 5130A 100-4 5130A 100-4 5130 All Decisions 5130A 100-6 5130A 100-6 5130B 210-11 5130B 210-9 5130 UI Disposed of During Month 5130B 210-9 5130B 210-11 5130C 300-15 5130C 300-15 5130C 300-17 5130C 300-17 9 Appeals Decisions Higher Authority 5130 UI Decisions by Type 5130C 300-19 AR5130 Total 5130C 300-19 - 21 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 5130C 310-15 5130C 310-15 5130 UI in Favor of Appellant 5130C 310-17 5130C 310-17 9054B 2 9054B 2 9054B All Appeals 9054B 3 9054B Total 9054B 3 10 Appeals Case Aging Lower Authority 9055 Lower Authority Appeals Case Aging 9055A 1 Total 9055A 1 11 Appeals Case Aging Higher Authority 9055 Higher Authority Appeals Case Aging 9055B 1 Total 9055B 1 227A 102-2 227A 101-2 227 Multi Schemes Cases 227A 102-3 227A 101-3 227A 101-2 227A 102-2 227 Fraud 227A 101-3 227A 102-3 227A 104-2 227A 104-2 227A 104-3 227A 104-3 227A 105-2 227A 105-2 227A 105-3 227A 105-3 227A 106-2 227A 106-2 227A 106-3 227A 106-3 227A 107-2 227A 107-2 227A 107-3 227A 107-3 227A 108-2 227A 108-2 227A 108-3 227A 108-3 227 Non-fraud Total 227A 102-1 227A 101-4 227A 101-4 227A 101-5 227A 101-5 227A 102-4 227A 102-4 227A 102-5 227A 102-5 227A 104-4 227A 104-4 227A 104-5 227A 104-5 12 Overpayments Established 227 Total Dollars Established 227A 105-4 Total 227A 105-4 - 22 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 227A 105-5 227A 105-5 227A 106-4 227A 106-4 227A 106-5 227A 106-5 227A 107-4 227A 107-4 227A 107-5 227A 107-5 227A 108-4 227A 108-4 227A 108-5 227A 108-5 227A 109-4 227A 109-4 227 Penalty 227A 109-5 227A 109-5 227C 303-11 227C 303-11 227C 303-12 227C 303-12 227C 303-13 227C 303-13 227C 303-14 227C 303-14 227C 304-11 227C 304-11 227C 304-12 227C 304-12 227C 304-13 227C 304-13 227C 304-14 227C 304-14 227C 305-11 227C 305-11 227C 305-12 227C 305-12 227C 305-13 227C 305-13 227C 305-14 227C 305-14 227C 306-11 227C 306-11 227C 306-12 227C 306-12 227C 306-13 227C 306-13 227C 306-14 227C 306-14 227C 307-11 227C 307-11 227C 307-12 227C 307-12 227C 307-13 227C 307-13 227 Recovered 227C 307-14 227C 307-14 13 Overpayment Reconciliation Activities 227 Waived 227C 308-13 Total 227C 308-13 - 23 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 227C 308-14 227C 308-14 - 24 - Table 1 Report Validation Pass Fail Groups Under the MPR and OWS-Sun System Environments Benefits Population OWS-Sun System MPR Number Type of Transaction Group Title Report Cells Group Title Report Cells 227C 309-11 227C 309-11 227C 309-12 227C 309-12 227C 309-13 227C 309-13 227 Written-Off 227C 309-14 227C 309-14 227C 310-11 227C 310-11 227C 310-12 227C 310-12 227C 310-13 227C 310-13 227 Addition 227C 310-14 227C 310-14 227C 311-11 227C 311-11 227C 311-12 227C 311-12 227C 311-13 227C 311-13 227 Subtraction 227C 311-14 227C 311-14 227E 501-18 227E 501-18 227E 502-18 227E 502-18 227E 503-18 227E 503-18 227E 504-18 227E 504-18 227E 505-18 227E 505-18 227 UI Total Accounts Receivable 227E 506-18 UI Total Accounts Receivable 227E 506-18 227E 501-19 227E 501-19 227E 502-19 227E 502-19 227E 503-19 227E 503-19 227E 504-19 227E 504-19 227E 505-19 227E 505-19 227 UCFE UCX Total Accounts Receivable 227E 506-19 FE X Total Accounts Receivable 227E 506-19 227C 312-11 227C 312-11 227C 312-12 227C 312-12 227C 312-13 227C 312-13 14 Aged Overpayments 227 Receivables Removed at End of Period 227C 312-14 Receivables Removed at End of Period Total 227C 312-14 - 25 - Table 2 Data Element Validation Samples Under the MPR and OWS-Sun System Environments Benefits Population Data Element Validation Samples MPR Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Sample Name Type and Size Universe subpops 100 Intrastate Weeks Claimed Random 60 200 1.1-1.3 Intrastate Weeks Claimed Random 60 200 1.1-1.3 110 Interstate Liable Weeks Claimed Random 30 100 1.4-1.6 Interstate Liable Weeks Claimed Random 30 100 1.4-1.6 1 Weeks Claimed 120 Interstate Weeks Claimed from Agent Minimum--6 1.7-1.9 Interstate Weeks Claimed filed from Agent State Minimum-- 6 1.7-1.9 2 Final Payments 200 Final Payments Random 30 100 2.1-2.4 Final Payments Random 30 100 2.1-2.4 300 New Intra Inter Liable Claims Random 60 200 3.1-3.18 New UI Claims Random 60 200 3.1-3.14 301 New Intra Inter Liable Claims Missing Subpops 17 3.1-3.18 New UI Claims Missing 3.1-3.14 New UCFE UCX Claims Minimum-- 8 3.15-3.18 305 Interstate Filed from Agent Minimum--6 3.19-3.21 Interstate Filed from Agent State Claims Minimum-- 6 3.19-3.21 310 Interstate Taken as Agent Minimum--6 3.22-3.24 Interstate Claims Taken As Agent State Minimum-- 6 3.22-3.24 315 Intra and Inter Transitional Claims Random 30 100 3.25-3.33 Intrastate and Interstate Transitional Claims Random 30 100 3.25-3.33 320 CWC Claims Random 30 100 3.34-3.39 CWC Claims Random 30 100 3.34-3.39 321 CWC Claims Missing Subpops 5 3.34-3.39 3 Claims 325 Monetary Sent w o New Claim Minimum-- 12 3.40-3.45 Monetary Sent Without New Claim Minimum-- 12 3.40-3.45 - 26 - Table 2 Data Element Validation Samples Under the MPR and OWS-Sun System Environments Benefits Population Data Element Validation Samples MPR Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Sample Name Type and Size Universe subpops 330 Entering Self Employment Program Minimum--2 3.46 Entering Self Employment Pgm Minimum-- 2 3.46 350 Intrastate Additional Claims Random 60 200 3A.1-3A.3 Additional Claims Random 60 200 3a.49-3a.54 3a Additional Claims 360 Interstate Liable Additional Claims Minimum--6 3A.4-3A.6 400 First Payments Random 60 200 4.1-4.16 First Payments Random 60 200 4.1-4.16 405 First Payments Missing Subpops 15 4.1-4.16 First Payments Missing 4.1-4.16 410 First Payments Outliers--10 4.1 4.3 4.5 4.7 4.9 4.11 4.13 4.15 First Payments Intrastate only Outliers-- 10 415 Continued Weeks Total Payments Outliers--10 4.17-4.24 Continued Weeks Total Payments Outliers-- 10 4.17-4.24 420 Continued Weeks Partial Payments Random 30 100 4.25-4.32 Continued Weeks Partial Payments Random 30 100 4.25-4.32 425 Adjusted Payments Outliers--10 4.33-4.42 Adjusted Payments Outliers-- 10 4.33.4.42 430 Self Employment Payments Minimum--2 4.43 Self-Employment Payments Minimum-- 2 4.43 435 CWC First Payments Random 30 100 4.44-4.45 CWC First Payments Random 30 100 4.44-4.45 440 CWC Continued Payments Minimum--4 4.46-4.47 CWC Continued Payments Minimum-- 4 4.46-4.47 445 CWC Adjusted Payments Minimum--4 4.48-4.49 CWC Adjusted Payments Minimum-- 4 4.48-4.49 4 Payments 450 CWC Prior Weeks Compensated Minimum--4 4.50-4.51 CWC Prior Weeks Compensated Minimum-- 4 4.50-4.51 - 27 - Table 2 Data Element Validation Samples Under the MPR and OWS-Sun System Environments Benefits Population Data Element Validation Samples MPR Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Sample Name Type and Size Universe subpops 500 Single Claimant Nonmon Determinations Random 30 100 5.1-5.60 Single Claimant Nonmonetary Determinations Random 30 100 5.1-5.60 501 Single Claimant Nonmon Determinations Missing Subpops 59 5.1-5.60 Single Claimant Nonmonetary Determinations Missing 5.1-5.60 505 Single Claimant Nonmon Determinations Outliers--10 5.1-5.60 Single Claimant Nonmonetary Determinations Outliers-- 10 5.1-5.60 510 UI Multi-Claimant Determinations Minimum--8 5.61-5.64 Multi-Claimant Nonmonetary Determinations Minimum-- 8 5.61-5.64 5 Nonmonetary Determinations 520 Single Claimant Redeterminations Random 30 100 5.65-5.70 Redeterminations Random 30 100 5.65-5.70 6 Appeals Filed Lower Authority 600 Appeals Filed Lower Authority Minimum--4 6.1-6.2 Appeals Filed Lower Authority Minimum-- 4 6.1-6.2 7 Appeals Filed Higher Authority 700 Appeals Filed Higher Authority Minimum--4 7.1-7.2 Appeals Filed Higher Authority Minimum-- 4 7.1-7.2 800 Lower Authority Appeals Decisions Random 60 200 8.1-8.52 8.54-8.55 Single Claimant Appeals Decisions Lower Authority Random 60 200 8.1-8.44 810 Lower Authority Appeals Decisions Missing Subpops 21 8.33-8.52 8.54-8.55 Single Claimant Appeals Decisions Lower Authority Missing 8.33-8.40 820 Lower Authority Appeals Decisions Outliers--10 8.1-8.52 8.54-8.55 Single Claimant Appeals Decisions Lower Authority Outliers-- 10 8.1-8.44 8 Appeals Decisions Lower Authority Multiclaimant Appeals Decisions Lower Authority Minimum-- 22 8.45-8.55 900 Higher Authority Appeals Decisions Random 30 100 9.1-9.20 9.22-9.23 Single Claimant Appeals Decisions Higher Authority Random 30 100 9.1-9.12 9 Appeals Decisions Higher Authority 910 Multi Claimant Appeals Higher Authority Missing Subpops 9 9.13 9.20 9.22-9.23 - 28 - Table 2 Data Element Validation Samples Under the MPR and OWS-Sun System Environments Benefits Population Data Element Validation Samples MPR Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Sample Name Type and Size Universe subpops 920 Higher Authority Appeals Decisions Outliers--10 9.13 9.20 9.22-9.23 Single Claimant Appeals Decisions Higher Authority Outliers-- 10 9.1-9.12 Multi-claimant Appeals Decisions Higher Authority Minimum-- 22 9.13-9.23 10 Appeals Case Aging Lower Authority 1000 Appeals Case Aging Lower Authority Outliers--10 10.1-10.7 Appeals Case Aging Lower Authority Outliers-- 10 10.1-10.7 11 Appeals Case Aging Higher Authority 1100 Appeals Case Aging Higher Authority Outliers--10 11.1-11.6 Appeals Case Aging Higher Authority Outliers-- 10 11.1-11.6 1200 Overpayment Established Random 60 200 12.1-12.7 12.9-12.15 Overpayments Established Random 60 200 12.1-12.16 1210 Overpayment Established Missing Subpops 13 12.1-12.7 12.9-12.15 12 Overpayments Established 1220 Overpayment Established Outliers--10 12.1-12.7 12.9-12.15 Overpayments Established Outliers-- 10 12.1-12.16 1300 Overpayment Reconciliation Activities Random 30 100 13.1-13.34 Overpayment Reconciliation Activities Random 30 100 13.1-13.34 1310 Overpayment Reconciliation Activities Missing Subpops 33 13.1-13.34 Overpayment Reconciliation Activities Missing 13.1-13.34 13 Overpayment Reconciliation Activities 1320 Overpayment Reconciliation Activities Outliers--10 13.1-13.34 Overpayment Reconciliation Activities Outliers-- 10 13.1-13.34 1400 Aged Overpayments Random 30 100 14.1-14.12 14 Aged Overpayments 1410 Aged Overpayments Outliers--10 14.1-14.12 Overpayments Aging Outliers-- 10 14.1-14.12 - 29 - Table 2 Data Element Validation Samples Under the MPR and OWS-Sun System Environments Benefits Population Data Element Validation Samples MPR Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Sample Name Type and Size Universe subpops 1420 Aged Overpayments Missing Subpops 11 14.1-14.12 - 30 - Table 3 -- Detail on Samples and Pass Fail Groups for OWS-Sun Software with 1 1 Relationships Highlighted Benefits Population Data Element Validation Samples Pass Fail Groups Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Group Number Group Name 100 Intrastate Weeks Claimed Random 60 200 1.1-1.3 1.01 5159 Intrastate Weeks Claimed 110 Interstate Liable Weeks Claimed Random 30 100 1.4-1.6 1.02 5159 Interstate Liable Weeks Claimed 1 Weeks Claimed 120 Interstate Weeks Claimed from Agent Minimum--6 1.7-1.9 1.03 5159 Interstate Weeks Claimed Filed from Agent State 200 Final Payments Random 30 100 2.1-2.4 2.01 5159 Final Payments 2 Final Payments 2.02 218 Final Payments 300 New Intra Inter Liable Claims Random 60 200 3.1-3.18 3.01 5159 New Intrastate Interstate Received as Liable 301 New Intra Inter Liable Claims Missing Subpops 17 3.1-3.18 3.02 5159 Filed from Agent 305 Interstate Filed from Agent Minimum--6 3.19-3.21 3.03 5159 Taken as Agent 310 Interstate Taken as Agent Minimum--6 3.22-3.24 3.08 218 Total Determinations 315 Intra and Inter Transitional Claims Random 30 100 3.25-3.33 3.04 5159 Transitional 320 CWC Claims Random 30 100 3.34-3.39 3.06 586 New CWC 321 CWC Claims Missing Subpops 5 3.34-3.39 3.07 586 New CWC BY Established 325 Monetary Sent w o New Claim Minimum-- 12 3.40-3.45 3.09 218 Total Benefit Year Established 3 Claims 330 Entering Self Employment Program Minimum--2 3.46 3.10 218 BY Established by Weeks 3a Additional Claims 350 Intrastate Additional Claims Random 60 200 3A.1-3A.3 3a.01 5159 Additional Intrastate 360 Interstate Liable Additional Minimum--6 3A.4-3A.6 - 31 - Table 3 -- Detail on Samples and Pass Fail Groups for OWS-Sun Software with 1 1 Relationships Highlighted Benefits Population Data Element Validation Samples Pass Fail Groups Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Group Number Group Name Claims 400 First Payments Random 60 200 4.1-4.16 4.01 9050 First Payments 405 First Payments Missing Subpops 15 4.1-4.16 4.02 9050 Partial First Payments 410 First Payments Outliers--10 4.1 4.3 4.5 4.7 4.9 4.11 4.13 4.15 4.03 9051 Continued Weeks Compensated 415 Continued Weeks Total Payments Outliers--10 4.17-4.24 4.05 586 Weeks Compensated 420 Continued Weeks Partial Payments Random 30 100 4.25-4.32 4.04 9051 Continued Partial Weeks Compensated 425 Adjusted Payments Outliers--10 4.33-4.42 4.06 586 Prior Weeks Compensated 430 Self Employment Payments Minimum--2 4.43 4.07 586 Benefits Paid 435 CWC First Payments Random 30 100 4.44-4.45 4.09 586 CWC First Payments 440 CWC Continued Payments Minimum--4 4.46-4.47 4.08 586 Prior Benefits Paid 445 CWC Adjusted Payments Minimum--4 4.48-4.49 4.10 5159 UI Weeks Compensated 450 CWC Prior Weeks Compensated Minimum--4 4.50-4.51 4.11 5159 UCFE UCX Weeks Compensated 4.12 5159 UI Amount Paid 4 Payments 4.13 5159 UCFE UCX Amount Paid 5.09 9052 Single Claimant Non Monetary Determinations 500 Single Claimant Nonmon Determinations Random 30 100 5.1-5.60 5.11 9053 Single Claimant Non Monetary Determinations 5 Nonmonetary Determinations 501 Single Claimant Nonmon Determinations Missing Subpops 5.1-5.60 5.01 207 Total Single Claimant Dets and UCFE UCX Redets - 32 - Table 3 -- Detail on Samples and Pass Fail Groups for OWS-Sun Software with 1 1 Relationships Highlighted Benefits Population Data Element Validation Samples Pass Fail Groups Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Group Number Group Name 59 505 Single Claimant Nonmon Determinations Outliers--10 5.1-5.60 5.03 207 Single Claimant Denials 510 UI Multi-Claimant Determinations Minimum--8 5.61-5.64 5.11 9053 Multi-Claimant Non Monetary Determinations 5.12 9052 Multi-Claimant Non Monetary Determinations 520 Single Claimant Redeterminations Random 30 100 5.65-5.70 5.02 207 UI Single Claimant Redeterminations 5.04 207 UI Multi-Claimant Determinations 5.05 207 Separation Determinations 5.06 207 Separation Denials 5.07 207 UI Non Separation Determinations 5.08 207 UI Non Separation Denials 6 Appeals Filed Lower Authority 600 Appeals Filed Lower Authority Minimum--4 6.1-6.2 6.01 5130 Lower Authority Appeals 7 Appeals Filed Higher Authority 700 Appeals Filed Higher Authority Minimum--4 7.1-7.2 7.01 5130 Higher Authority Appeals 8.01 5130 All Decisions 800 Lower Authority Appeals Decisions Random 60 200 8.1-8.52 8.54-8.55 8.06 9054 All Appeals 810 Lower Authority Appeals Decisions Missing Subpops 21 8.33-8.52 8.54-8.55 8.02 5130 UI Disposed of During Month 8 Appeals Decisions Lower Authority 820 Lower Authority Appeals Decisions Outliers--10 8.1-8.52 8.54-8.55 8.03 5130 UI Decisions by Type - 33 - Table 3 -- Detail on Samples and Pass Fail Groups for OWS-Sun Software with 1 1 Relationships Highlighted Benefits Population Data Element Validation Samples Pass Fail Groups Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Group Number Group Name 8.04 5130 UI in Favor of Appellant 8.05 5130 UI Decisions by Issue 9.01 5130 All Decisions 900 Higher Authority Appeals Decisions Random 30 100 9.1-9.20 9.22-9.23 9.05 9054B All Appeals 910 Multi Claimant Appeals Higher Authority Missing Subpops 9 9.13 9.20 9.22- 9.23 9.02 5130 UI Disposed of During Month 920 Higher Authority Appeals Decisions Outliers--10 9.13 9.20 9.22-9.23 9.03 5130 UI Decisions by Type 9 Appeals Decisions Higher Authority 9.04 5130 UI in Favor of Appellant 10 Appeals Case Aging Lower Authority 1000 Appeals Case Aging Lower Authority Outliers--10 10.1-10.7 10.01 9055 Lower Authority Appeals Case Aging 11 Appeals Case Aging Higher Authority 1100 Appeals Case Aging Higher Authority Outliers--10 11.1-11.6 11.01 9055 Higher Authority Appeals Case Aging 1200 Overpayment Established Random 60 200 12.1-12.7 12.9-12.15 12.04 227 Total Dollars Established 1210 Overpayment Established Missing Subpops 13 12.1-12.7 12.9-12.15 12.01 227 Multi Schemes Cases 1220 Overpayment Established Outliers--10 12.1-12.7 12.9-12.15 12.02 227 Fraud 12.03 227 Non-fraud 12 Overpayments Established 12.05 227 Penalty 1300 Overpayment Reconciliation Activities Random 30 100 13.1-13.34 13.01 227 Recovered 13 Overpayment Reconciliation Activities 1310 Overpayment Reconciliation Activities Missing Subpops 13.1-13.34 13.02 227 Waived - 34 - Table 3 -- Detail on Samples and Pass Fail Groups for OWS-Sun Software with 1 1 Relationships Highlighted Benefits Population Data Element Validation Samples Pass Fail Groups Number Type of Transaction Sample ID Sample Name Type and Size Universe subpops Group Number Group Name 33 1320 Overpayment Reconciliation Activities Outliers--10 13.1-13.34 13.03 227 Written-Off 13.04 227 Addition 13.05 227 Subtraction 1400 Aged Overpayments Random 30 100 14.1-14.12 14.01 227 UI Total Accounts Receivable 1410 Aged Overpayments Outliers--10 14.1-14.12 14.03 227 Receivables Removed at End of Period 14 Aged Overpayments 1420 Aged Overpayments Missing Subpops 11 14.1-14.12 14.02 227 UCFE UCX Total Accounts Receivable