Skip to page content
Wage and Hour Division
Bookmark and Share

Wage and Hour Division (WHD)

APPENDIX C

METHODOLOGICAL ISSUES

Methodological Issues

This appendix summarizes the methods used by Westat to conduct the 2000 Surveys of Employee and Establishments (1). For both surveys, summaries of the questionnaires, sample designs, data collection procedures, response rate calculations, and weighting activities are presented. In addition, this Appendix includes a discussion of issues related to comparing the surveys to the 1995 Surveys of Employees (2) and Establishments (3) as well as the methods used to define key measures used in the analysis (e.g., coverage and eligibility under the Family and Medical Leave Act).

1. 2000 Survey of Employees

1.1 Questionnaire

The instrument for the 2000 Survey of Employees consisted of five major sections: (a) the screener, which served to classify potential respondents as being either "leave-takers," "leave-needers," or "employed only" since January 1, 1999; (b) a series of items specifically for leave-takers which asked about their experience with leave; (c) items for leave-needers regarding why they needed leave and did not take it; (d) items asked of all respondents, including items about their employment and their opinions about family and medical leave; and (e) items obtaining demographic information on respondents. The questionnaire for the 2000 Survey of Employees is shown in Appendix D.

1.2 Sample Design and Population Universe

The survey was conducted with a sample of individuals aged 18 or older in U.S. households who were employed at any time between January 1, 1999 and the time of the survey (between 18 and 20 months, depending on when the interview occurred). The sample was drawn from the universe of all known U.S. households with telephones. The sample frame represented all employees that had a telephone. This includes those employed in both the public and private sectors. Approximately 24,500 telephone numbers were selected using a list-assisted Random Digit Dial (RDD) method.

For each telephone number, an interviewer attempted to screen for eligibility by determining whether the household contained at least one person 18 years of age or older who had been employed since January 1, 1999. Furthermore, for all persons within a household meeting these criteria, the interviewer attempted to determine if they had taken (or needed without taking) family or medical leave since January 1, 1999. All persons said to have taken or needed this type of leave were eligible for the extended interview. Those not having taken or needed leave (i.e., those who were employed only) were sub-sampled for the extended interview.

1.3 Data Collection and Response Rates

Data for the Survey of Employees were collected by interviewers specially trained for the project using a Computer Assisted Telephone Interviewing (CATI) system. Interviewing began on July 15, 2000 and continued for approximately 10 weeks. A total of 2,558 interviews were completed: 1,229 with persons who took leave (for reasons covered by FMLA) since January 1, 1999; 203 with persons who needed leave (for a covered reason) but did not take it; and 1,126 with persons that were employed only and had not taken any family or medical leave.

The response rate for the Survey of Employees was computed in three steps. In the first step, a response rate was calculated for the screening interview, which identified eligible respondents in the household. In the second step, a response rate was calculated for the extended interview, which collected the data from the selected household respondent. In the third step, the two response rates were combined to produce the overall survey response rate.

Calculating the Screening Interview Response Rate. The weighted response rate for the screener ranged from 67.5 percent to 69.3 percent. The range reflects different assumptions made about the eligibility of those telephone numbers where no respondent ever answered the telephone.

The lower screener weighted response rate (67.5%) was calculated using the following formula:

C/(C+R+.27NA+.6M + ONR)

where:

C=complete,
R=refusal,
NA=no answer,
M=message machine,
ONR=other non-response

This assumes that a residential household existed for 27 percent of those calls where someone never answered the telephone and for 60 percent of those calls where the interviewers only encountered an answering machine. This is the standard formula used by Westat when computing response rates for random digit dial surveys. It is based, in part, on guidelines published by the Council of American Survey Research Organizations (CASRO). It modifies these guidelines by reducing the number of "no answers" that are classified as eligible, based on research tracking telephone numbers through the telephone company.

The higher response rate of 69.3 percent was computed using the following formula:

C/(C+R+M + ONR)

This formula excludes the calls where someone did not answer the telephone (NAs above), but includes calls that reached an answering machine. This rate is comparable to the method used by the University of Michigan for the 1995 survey.

Calculating the Extended Interview Response Rate. The final weighted response rates for completing the extended interviews represent the proportion of interviews that were completed among those eligible and selected for the study. The rates, calculated separately for each type of respondent, were:

Extended Interview Response Rate
Overall: 84.2%
Leave-taker 83.9%
Leave-needer 85.3%
Employed only 84.2%

Calculating the Final Response Rate. The final response rate is computed by multiplying each respective extended interview response rate by the screener response rate. For this step , the higher screener response rate of 69.3 percent was used to maintain comparability with the 1995 survey. The final weighted response rates for each type of respondent were:

Final
Overall: 58.3%
Leave-taker 58.1%
Leave-needer 59.1%
Employed only 58.3%

1.4 Weighting

For each interview, a sampling weight was attached which was derived from the following components:

  1. A base weight reflecting the overall probability of selection of the household;
  2. An adjustment for the number of telephones in the household;
  3. A non-response adjustment for the household (i.e., screener) based on census data for the telephone exchange;
  4. The probability of selection of the person within the household;
  5. A non-response adjustment for the person, based on age and gender; and
  6. A post-stratification adjustment to the gender distribution of the U.S. non-institutional population, based on U.S. Census figures.

The post-stratification adjustment was done for several reasons. First, it inflated the population estimates from the survey to reflect all households in the country, including those without telephones. Second, by using the U.S. Census totals, the estimates incorporate a correction for undercounting particular subgroups in the population.

The 1995 Survey of Employees followed a very similar process in the weighting with two exceptions. It did not adjust for the number of telephones in the household (step b), and it did not implement a poststratification adjustment (step f).

In order to maintain comparability for analyses between 1995 and 2000 data, both of these adjustments were done for the 1995 survey weights. All of the 1995 weights were globally down-weighted using an estimate of the average number of telephones per household in U.S. households in 1995 (approximately a factor of 92.5%). The weights were then inflated separately by gender to reflect the same post-stratification factors used for the 2000 data-set (factor 1.06 for females and 1.12 for males).

1.5 Estimates of Variances

Appendix B provides the standard errors and unweighted sample sizes for each of the estimates published in the report. The standard errors were computed using replicate variance estimation methods. The program used to estimate the standard errors was a Westat-authored program, WESVAR. The variance estimation procedures account for both the complex sample design and the use of weights in the estimation process.

1.6 Comparisons Between 1995 and 2000 Surveys of Employees

As noted above, efforts were made to keep the 1995 and 2000 surveys as comparable as possible. Nevertheless, the comparisons between the surveys may still have been affected by several differences between the two surveys. This section discusses two of these differences: (1) differences in the response rates, and (2) differences in question wording for key items. This section also discusses the implications of these differences for comparing the 1995 and 2000 surveys.

1.6.1 Differences in Response Rates

The 1995 survey had a combined response rates of 73.1 percent for leave-takers, 75.9 percent for leave-needers, and 70.6 percent for those who were employed but did not take any leave. This is between 11 and 16 percentage points higher than the 2000 survey, depending on the group (4). The differences in response rates may have led to different patterns of non-response error across the two surveys (5). In order to investigate the extent that comparisons are affected by the response rate differentials, several analyses related to the non-response were conducted. The results of these analyses are reported in full detail in the main methodology report for the survey (in press). In this section, the primary results from these analyses are reported.

Three different types of non-response analyses were conducted. Each provides a different perspective on the potential non-response problems in the 2000 survey. The analyses, and the non-response problem they are designed to address, include:

  • Demographic distribution comparison. This should reveal any differences in the types of respondents captured in each survey.
  • Non-response follow-up survey. A survey of a sample of non-respondents to the 2000 survey was completed. This provides a direct measure of a sample of persons that the main survey missed.
  • Level-of-effort comparison. A comparison of differences between respondents to the 2000 survey by the level of effort it took to complete the interview. The assumption in this analysis is that those that required the most effort to interview resemble those that the survey was unable to interview.

In the section that follows, the comparison of the 1995 and 2000 demographic distributions is discussed. In the last part of the section, the results from the other two analyses are described, along with conclusions related to the potential non-response error.

Comparing Demographic Distributions

Table C-1 displays the weighted levels of the three groups analyzed in the 1995 and 2000 surveys (i.e., leave-taker, leave-needer, and all employees) by whether they work for a covered establishment and by their eligibility status. Table C-2 displays the unweighted demographic distributions of the three groups. The unweighted responses are used so that none of the non-response or post-stratification adjustments embedded in the weighting influence the observed distributions. The "all employees" column was standardized to the distribution observed in the 2000 survey. (6)

Table C.1. Distribution by Study Groups (Weighted Sample)
1995 and 2000 Surveys, (In Millions)

Leave-Takers

Leave-Needers

All Employees

1995
Survey

2000
Survey

1995
Survey

2000
Survey

1995
Survey

2000
Survey

All Employees

20.4

23.8

3.9

3.5

127.3

144.0

Employees in
covered worksites

14.9

18.1

2.6

2.9

84.0

110.4

Eligible employees in
covered worksites

12.6

15.5

2.3

2.4

69.8

88.9

Source: Survey of Employees

These distributions show relatively small differences across the demographic groups. The largest differences are for gender and income. The 2000 survey found more females and higher income groups. Both of these differences can be partially explained by trends over the five year period between the two surveys. As noted in the introduction to the report, women constitute a greater proportion of the workforce in 2000, relative to 1995. Similarly, the higher income of those in 2000 reflects inflation and actual growth in income over this time period.

Table C.2. Demographic Distributions by Study Groups (Unweighted Sample):
1995 and 2000 Surveys

Percent of
Leave-Takers

Percent of
Leave-Needers

Percent of
All Employees

1995
Survey

2000
Survey

1995
Survey

2000
Survey

1995
Survey

2000
Survey

Gender * xx

Female

58.7%

62.5%

50.3%

57.6%

48.0%

52.5%

Male

41.3%

37.5%

49.7%

42.4%

52.0%

47.5%

Age * +

18-24

12.2%

7.1%

9.6%

8.0%

12.9%

13.0%

25-34

31.5%

26.8%

29.4%

25.9%

23.2%

21.2%

35-49

38.5%

41.1%

39.6%

42.8%

42.6%

42.0%

50-64

14.5%

22.6%

20.3%

21.4%

18.2%

21.1%

65+

3.3%

2.4%

1.1%

2.0%

3.2%

2.7%

Race * ++

White, Non-Hispanic

79.4%

77.2%

71.0%

74.7%

81.1%

78.5%

Black, Non-Hispanic

10.6%

10.2%

17.5%

12.9%

9.5%

9.5%

Hispanic

8.1%

7.0%

8.7%

9.4%

7.5%

6.9%

Asian

(1)

2.5%

(1)

2.0%

(1)

2.7%

Other

1.9%

3.1%

2.7%

1.0%

1.9%

2.4%

Married xx

Married

72.7%

75.2%

63.1%

68.5%

70.9%

69.8%

Separated/Divorced

15.2%

14.0%

23.5%

20.2%

13.1%

10.2%

Never married

12.0%

10.8%

13.4%

11.3%

16.0%

20.0%

Family with Children

No

43.3%

42.2%

47.1%

45.8%

55.0%

57.1%

Yes

56.7%

57.8%

52.9%

54.2%

45.0%

42.9%

Education ** + xx

Less than high school

8.7%

5.0%

9.6%

6.9%

8.4%

5.4%

High school graduate

26.9%

28.0%

27.3%

28.1%

28.8%

29.4%

Some college

31.0%

31.3%

34.2%

26.6%

29.8%

29.2%

College graduate

20.8%

24.0%

16.6%

28.1%

20.9%

25.4%

Graduate school

12.6%

11.6%

12.3%

10.3%

12.1%

10.7%

Income ** ++ xx

Less than $20,000

19.6%

13.5%

24.3%

15.5%

18.8%

14.8%

$20,000 to less than $30,000

17.3%

12.3%

19.2%

16.6%

16.1%

12.3%

$30,000 to less than $50,000

29.9%

24.7%

28.8%

24.1%

32.2%

25.7%

$50,000 to less than $75,000

19.6%

26.7%

15.2%

27.8%

20.2%

24.6%

$75,000 to less than $100,000

9.0%

12.4%

9.0%

9.1%

8.2%

11.9%

$100,000 or more

4.6%

10.3%

3.4%

6.9%

4.6%

10.6%

Compensation

Salaried

36.5%

36.4%

29.4%

25.2%

37.0%

36.5%

Hourly

55.0%

53.6%

61.0%

59.9%

51.6%

52.4%

Other

8.5%

10.0%

9.6%

14.8%

11.5%

11.1%

(1) Asian was not a race category in the 1995 survey. Asians are included in "All Others."
* Difference between years for leave-takers is significant at p<.10; ** is significant at p<.05.
+ Difference between years for leave-needers is significant at p<.10; ++ is significant at p<.05.
x Difference between years for all employees is significant at p<.10; xx is significant at p<.05.
Note: "All employees" was calculated by weighting leave-takers by 16.5%, leave-needers by 2.4% and employed-only by 81.1%. Column percents may not total to 100% due to rounding.

There are several differences that are not as easily explained by trends in employment. Leave-takers for the 2000 Survey were more likely to be in the older age groups. In addition, the 2000 survey has a lower proportion of persons that are white (for employees 78.5% vs. 81.1%), a lower proportion with less than high school education (5.4% vs. 8.4%), and a higher proportion that are never-married (20% vs. 16%).

Overall, therefore, the 1995 and 2000 survey samples differ in terms of demographics. However, considering shifts in the economy over the time period, these differences are relatively small and do not indicate large differences in non-response error between the 1995 and 2000 employee surveys.

Results of Other Analyses and Comparisons to the 1995 Data

Two other analyses were conducted to assess non-response error. One analysis was based on a survey of persons that did not respond to the 2000 survey. The other analysis was of the 2000 survey information by the amount of effort it took to complete the interviews (i.e., number of contacts to complete and whether respondent initially refused to complete).

These analyses did not find a great deal of evidence that non-response error significantly affects the comparisons between the 1995 and 2000 surveys. Two general patterns were detected. The first was that non-response to the screener was slightly more likely to have occurred among households without employed persons. This could have led to overestimating the number of employed persons on the 2000 survey. The second pattern was that non-response at the extended level may have been more likely among selected demographic groups. The most significant differences implied that non-respondents were more likely to be age 18-24 and male. Other, less consistent patterns implied that non-respondents to the extended interview were employees with children and with less than a college degree. No consistent differences were found across a number of outcome variables, such as coverage status, eligibility status, whether the employee heard of FMLA and whether the leave-taker reported a serious health condition for their longest leave.

Comparing the 1995 and 2000 surveys should be done with caution. While the analyses summarized above are standard ways to assess potential non-response error, these methods all rely on untested assumptions about those persons that are never interviewed. With this caveat in mind, the analyses discussed above provide little indication that significant non-response bias exists when comparing the 1995 and 2000 employee surveys.

1.6.2 Differences in Survey Instruments

The 1995 and 2000 survey instruments are very similar. In most cases, comparable items are worded identically in both surveys. The primary differences between the two surveys are additional questions included in the 2000 survey. For example, the 2000 survey included items about the details for the longest and second longest leaves reported by the respondent. The 1995 survey only asked detailed questions about the longest leave.

There are two exceptions to this general rule. First, the initial items used to classify respondents as a leave-taker or leave-needer were modified in 2000 (7). In 1995, the item read:

Since January 1, 1994, have you taken leave from work to care for a newborn, newly adopted, or new foster child; for your own serious health condition or the serious health condition of your child, spouse, or parent that lasted more than three days or required an overnight hospital stay?

In 2000, the item was changed to read:

Since January 1, 1999, have you taken leave from work

  • to care for a newborn, newly adopted, or new foster child;

  • for reasons related to your or a family member’s pregnancy; or

  • for your own serious health condition or the serious health condition of your child, spouse, or parent? A serious health condition is one that lasted more than 3 days or required an overnight hospital stay.

One difference between the two surveys is that the 2000 item included the extra condition "for reasons related to your or a family member’s pregnancy" (see second bullet above). This change was made to clarify that pregnancy disability leave is covered by the FMLA.

A second difference is the modification of the sentences defining a serious health condition. In 2000, this definition was split out as a separate sentence (see second sentence in the last bullet above). In 1995, this was stated as part of the last phrase of the item. This change was made to clarify that such leave is conditioned upon taking time off for more than three days or for an overnight hospital stay.

The second questionnaire item that was changed asked about the size of the establishment for which the respondent worked. These items were used to classify respondents into a covered and non-covered status with respect to the FMLA. In 1995, this item read:

At the place where you work(ed) (i.e., the site¾ store, building) would you say there were fewer than 50 permanent employees or 50 or more permanent employees?

In 2000, the word "permanent" was dropped from the item. This change was made because the Act does not require that employees be permanent for the establishment to be covered by the Act.

1.6.3 Caveats for Comparing the 1995 and 2000 Survey of Employees

Given the differences in response rates between 1995 and 2000, as well as the above questionnaire changes, some caution should be exercised when estimating change between the two surveys. The areas of concern related to the analyses discussed in this report are described below.

Changes in the Number of Employees

Households that refused to complete the 2000 screener tended to consist of persons that were not employed during the reference period. All other things being equal, this would lead to a higher estimate of the total number of employed persons in the 2000 survey. As a result, comparing the 1995 and 2000 surveys may overestimate the amount of growth in employment over this time period. This is reflected in changes observed in the Current Population Survey (CPS). The CPS, which measures employment over a one week period in each month, estimated growth in employment from 125 million in 1995 to 133 million in 2000 (difference of 8 million). Comparing the two FMLA surveys, which measure employment over an 18-20 month period, the growth is from 127 million to 144 million, a growth of approximately 17 million.

A larger estimate of the growth in the number of employees may lead to overestimates of growth in important subgroups, such as covered employees and covered and eligible employees.

Estimates of Covered Employees

The proportion of persons who were covered by the FMLA increased from 66 percent to 77 percent, based on estimates from the 1995 and 2000 employee surveys. This increase diverges from results from the 1995/2000 Survey of Establishments and data published by the Bureau of Labor Statistics, neither of which found a change in the proportion covered by FMLA.

The change observed between 1995 and 2000 may reflect the more inclusive wording of the item on establishment size in the 2000 Survey (8), rather than a true increase in covered employees. A comparison of this estimate to that from the 2000 Survey of Establishments suggests that the estimate from the Employee survey is too high, although it is difficult to estimate the magnitude of any over-estimate because the two sources of information are not directly comparable.

Regardless of the precise accuracy of the number of covered employees estimated from the 2000 Survey of Employees, it is likely that the estimate of change between 1995 and 2000 using these data is an over-estimate.

Covered and Eligible Employees

The estimates of covered and eligible employees are computed by restricting the covered population to those that worked at least 1,250 hours over the previous 12 month period (9). As noted above, the estimate of change for the proportion and number of covered employees may be an over-estimate. Consequently, the estimate of change for the number of covered and eligible employees may also be an overestimate.

Covered and Eligible Leave-Takers

The estimates of covered and eligible leave-takers is computed by restricting the covered leave-takers to those that worked at least 1,250 hours over the pervious 12 month period. As noted above, the estimate for the proportion and number of covered employees may be an over-estimate. Consequently, the estimate for the number of covered and eligible leave-takers may also be an overestimate. Comparisons between 1995 and 2000 of these also over-estimate the change.

Characteristics of Leave-Takers

The non-response analysis suggested that some of the changes in the characteristics of leave-takers were due to differential non-response patterns across demographic groups. While the analysis did not find a great deal of evidence of a large non-response bias in this regard, it does not totally rule it out either. For example, some of the observed increase in the proportion of female leave-takers may be due to the lower response rate for males in the 2000 survey.

The change in the screening instrument (described above) may have also affected the reasons respondents reported they took leave. As noted in the report, the proportion of persons that reported taking leave for a serious health reason decreased between 1995 and 2000 (61.4% in 1995 vs. 47.2% in 2000; see Table 2.7). One possible explanation for this pattern is the addition of the extra phrase in the screener referring to "…reasons related to your or a family member’s pregnancy" (see above discussion). Respondents in 1995 might have reported pregnancy-related leave as their own serious health condition.

Analysis of the data from the survey does not seem to indicate that this change was major reason for the decrease in the proportion reporting a leave for a serious health condition. First, while the reasons for leave shifted, the demographic distributions of the samples did not shift significantly (as discussed above). If the screener was the cause of the shift, one would have expected those individuals who tend to take this type of leave (e.g., women, married persons) would constitute a larger portion of the sample than expected. More women were interviewed in 2000. However, most of this increase can be explained by changes in the labor force. Furthermore, the decrease in the number of leaves taken for personal health related reasons occurred across almost all demographic groups (see Chapter 2). For example, while females are less likely to take leave for their own health condition, both females and males showed a significant decrease in taking this type of leave between 1995 and 2000. If the screener affected how leave-takers were initially identified, this effect seems to have occurred across all demographic groups.

Second, if this change were responsible for the decrease in leaves taken for the employee’s own health, then one would expect the shift to occur primarily in the categories related to pregnancy and maternity. However, as described in Chapter 2, this shift occurred across all of the other reasons, including care for ill parent and care for ill spouse (e.g., see Table 2.7).

It is also possible that the shift in the distribution of the reasons for leave is at least partly due to re-structuring the screening item that defined a "serious health condition." As noted above (section 1.6.2), the statement defining a serious health condition was changed by separating the definition into a separate sentence to clarify that this type of leave is dependent on specific conditions. These conditions were in the 1995 question, but were not separated into a separate sentence as in 2000. One possible scenario is that this change resulted in respondents in 2000 using a more restrictive definition for serious health condition (e.g., reporting leave as a result of more severe conditions). If true, then respondents may have reported relatively fewer leaves for their own illnesses in 2000 relative to 1995.

If the restructuring of the item in 2000 had this type of effect, one would have expected some change in the demographic distribution of leave-takers between the two surveys, since certain groups are more likely to take leave for this type of reason (e.g., younger, males, non-married employees). As noted above, the changes observed in the demographics were relatively small. On it’s face, therefore, the change in the questions is not clearly related to the changes observed on the two surveys. However, analysis of changes in the demographics is not definitive and more research into this hypothesis needs to be conducted.

In summary, it is possible that changing the screening question, and the associated questions on the extended interview, did affect the reasons employees reported for taking leave. Analysis reported above does not find strong evidence for this. However, as noted in chapter 2, there also is not a clear substantive explanation for why there was a decrease in reports of "serious health conditions" between 1995 and 2000. Further research, investigating both the substantive and methodological causes for the change, needs to be conducted before definitively explaining this trend.

2. 2000 Survey of Establishments

2.1 Questionnaires

The 2000 Survey of Establishments was conducted using two instruments, administered at separate points in time (both questionnaires are shown in Appendix D). First, the screener served to confirm that the establishment still existed and, if necessary, obtain the correct name, address, and phone number for the business. Second, the screening instrument obtained the name, address, telephone number, and fax number of the person most knowledgeable about employee benefits for that establishment. This person was then recruited for the extended interview. Often, this individual was at a location other than the sampled establishment. Finally the screener confirmed that the establishment did in fact have employees and was neither a government nor a quasi-governmental organization.

The second instrument, the extended questionnaire, collected the data of interest for the project. This questionnaire was similar to that used in the 1995 project and repeated many of the same questions. The wording of most items remained the same so that valid comparisons could be made between the two surveys. The discussion below points to several key changes that may affect the comparisons between the surveys.

2.2 Sample Design and Population Universe

The sample for the 2000 Survey of Establishments was designed to cover all private business establishments excluding self-employed without employees, government and quasi-government units (federal, state, and local governments, public educational institutions, and post offices). Note this universe differs from the employee survey, which includes both private and public employees. The sample frame was the Dun and Bradstreet’s Dun’s Market Identifiers (DMI). This is considered to be the most comprehensive commercially available list of U.S. businesses. Most out-of-scope establishments could be identified using information available on the DMI.

The sample frame was stratified by establishment size and by industry grouping (five groups). The five industry groups were formed by grouping establishments using their Standard Industrial Classification (SIC) codes as given below:

  1. SIC group I: Agriculture, forestry, and fishery (SICs 01-09); Mining (SICs 10-14; Construction (SICs 15-17);
  2. SIC group II: Manufacturing (SICs 20-39);
  3. SIC group III: Transportation, communication, and utilities (SICs 40-49 except SIC 43, U.S. postal service); Wholesale (SICs 50-51); Finance, insurance, and real estate (SICs 60-67);
  4. SIC group IV: Retail (SICs 52-59);
  5. SIC group V: Services (SICs 70-89 except public units from SIC 82).

The sampling strata were then defined by cross-classification of the size classes and 5 SIC groups.

As is commonly done in an establishment survey, larger establishments were sampled with a higher probability than smaller establishments. This ensured that enough large establishments would be available for analysis. All estimates in this report adjust for over-sampling large establishments by weighting establishments by their probability of selection in order to produce unbiased estimates of establishments in the United States (see Section 2.4).

2.3 Data Collection and Response Rates

The 2000 Survey of Establishments was conducted in two phases. As noted above, establishments were first screened to confirm their eligibility and obtain contact information for the person said to be most knowledgeable about employee benefits. This person was contacted a few weeks later for the main survey interview. All data were collected by interviewers who were specially trained for the project. Interviewing for the main study began on July 13, 2000 and continued for approximately 10 weeks. A total of 1,839 interviews were completed.

The final weighted response rate for the 2000 Survey of Establishments was 65.0 percent. This combines both the screener and extended interview response rates.

2.4 Weighting

Weighting for the 2000 Survey of Establishments involved three main steps:

  1. Assigning sampling base weights equal to the reciprocal of the probabilities of selection;
  2. Adjusting the base weights to compensate for non-response in the screener and extended interviews; and
  3. Post-stratifying the weights so that weighted counts from the survey agreed with Bureau of Labor Statistics (BLS) establishment counts (10) within broad size and industry groups.

2.5 Estimates of Variance

Appendix B provides the standard errors and unweighted sample sizes for each of the estimates published in the report. The standard errors were computed using replicate variance estimation methods. The program used to estimate the standard errors was a Westat-authored program, WESVAR. The variance estimation procedures account for both the complex sample design and the use of weights in the estimation process.

2.6 Comparing the 1995 and 2000 Surveys of Establishments

This section describes the issues related to comparing the 1995 and 2000 Surveys of Establishments, including their response rates and survey instruments.

Response Rates

The weighted response rate for the 1995 survey was 73.2 percent. This is 8.2 percent higher than the 65.0 percent achieved on the 2000 survey. At the time of writing this report, a detailed analysis of the possible effects these differences have on the estimates has not been completed. The reader should therefore be cautious when making comparisons between the two surveys.

Survey Instruments

The 2000 survey instrument was based primarily on that used for the 1995 survey. In most cases, differences between the two instruments reflect questions added to the 2000 instrument. Two questions were changed, however.

The first change was to reorder responses to the series of items about benefits provided under the FMLA by employers (Q6 series; see instrument in Appendix E). This question asked employers about policies covering five of the reasons employees could take leave under the FMLA (e.g., own serious health reason). In 2000, the order of reasons was:

  1. To care for a newborn;
  2. For adoption or foster care placement;
  3. For the employee’s own serious health condition other than maternity-related conditions;
  4. For mothers for maternity related reasons; and
  5. For care of a child, spouse, or parent with a serious health condition.

In 1995, the order of these items had been (3), (4), (1), (2), and (5). Items were reordered to help respondents understand the question by grouping together the health related conditions.

The second change was made to one item within the series that asked whether different administrative activities were easy or difficult (Question 28 series; see Appendix E). In the 1995 survey, this item read: "coordinating the Act with pre-existing leave policies," while in the 2000 survey, the item was changed to read: "coordinating the Act with other leave policies." This change was made because coordinating with pre-Act leave policies was no longer relevant in 2000.

3. Survey Definitions of Coverage, Eligibility and Use of FMLA

Throughout the report, estimates are presented on worksites that are covered by the FMLA, employees who are eligible under the FMLA and leave-takers who have taken leave under the FMLA. This section describes how these were defined for purposes of the analysis.

3.1 Employee Survey

The employee survey identified persons who were employed between January 1, 1999 and the time the interview was completed (between 18 and 20 months). This subsection describes how FMLA-related estimates were defined in the analysis of the Survey of Employees.

Coverage and Eligibility

To determine coverage under the FMLA, the respondent was asked whether his/her employer had at least 50 employees within 75 miles of the worksite (Question C15). If the answer to this question was "yes," the respondent was defined as covered under the Act. This operational definition is not in precise conformance with the requirements of the FMLA. The Act defines an employee as working for a covered employer when the business has at least 50 employees. The definition used on the survey, however, did not include those situations when a worksite did not have 50 employees within a 75-mile radius, but was part of a business that did have employees at other worksites and thus would have met this criteria. This may have pushed the estimate of the number of covered employees downward.

Employees were classified as being eligible under the Act if the respondent reported working at least 1,250 hours in the previous 12 months (Question C17). This also does not conform precisely to the definition of eligibility in the FMLA. The Act also specifies that the employee must have worked for the same employer for at least 12 months and have worked at least 1,250 hours for that employer within the past 12 months.

Characteristics of Leave-Takers and Leave-Needers

Leave-takers were asked to report about any leaves that occurred since January 1, 1999 and the time of the interview (Section A of the questionnaire). Leave-takers were asked detailed questions about the two longest leaves that occurred during this period (e.g., reason for leave and length of leave). Summary information was collected for any other leaves that were taken during the reference period. To determine whether the leave-taker was covered under the FMLA, the respondent was asked about the size of their employer at the time of the longest leave (Question C15). To determine eligibility, the respondent was asked about the number of hours worked at the time of the longest leave (Question C17, C20).

A similar definition was followed for leave-needers. These respondents were asked about all of the times they needed but did not take leave during the survey reference period (Question B1). If more than one leave was needed, the respondent was asked detailed questions about their most recent need for leave. To determine whether the person was covered under the FMLA, the respondent was asked about the size of the employer he/she was working for at the time the most recent leave was needed (Question C15). Similarly, to determine eligibility under FMLA, the respondent was asked about the number of hours worked for the employer at the time the most recent leave was needed (Questions C17 – C20).

Characteristics of Employed Population

The employed population was defined as all of the leave-takers, leaven-needers, and all others who were employed but did not take or need leave between January 1, 1999 and the survey. To determine the coverage and eligibility of the "employed-only" group, the respondent was asked about his/her current employer (Question C15). If the respondent was not currently employed, he/she was asked about the employer he/she had worked for the longest period of time since January 1, 1999 (Question C17-C20).

Leaves Taken Under the FMLA

To determine if a leave-taker had taken leave under the FMLA, the respondent was first asked if he/she had heard of the FMLA (Question C3). If the respondent said "yes" to this question, he/she was then asked if the longest leave reported was taken under the FMLA (Question C6). The number of persons that took leave under the FMLA was estimated by counting those persons who:

  1. Reported taking leave under the FMLA;
  2. Were defined as working in a covered worksite; and
  3. Were classified as being eligible under the law.

3.2 Establishment Survey

The Establishment Survey also provided estimates of the number of FMLA-covered establishments and the number of persons taking leave under the FMLA. This sub-section briefly describes how these were defined for the estimates discussed in the report.

Coverage

An establishment was classified as being covered under the FMLA if the respondent reported as having at least 50 employees within a 75 mile radius of the worksite. As with the employee survey, this definition is not in strict conformance to the definitions under the Act. See the discussion of coverage for the Employee Survey above for more details.

Note that this definition defines as "non-covered" those establishments that report fewer than 50 employees within a 75-mile radius of the worksite but reported on the survey as being covered under the Act. Some of these establishments actually may be covered because the business has some other worksite that meets the coverage criteria. As indicated above, 16 percent of establishments that were classified as non-covered by this definition reported in the survey that they were covered by the Act. To be consistent with the 1995 analysis, these establishments were not classified as covered. To the extent that these firms are in fact covered, estimates of FMLA coverage of employee and employers based on the establishment survey are too low.

Exploratory analyses were conducted that examined the characteristics of these establishments and the implications of including them as "covered" for selected findings discussed in the report. These analyses found that:

  1. Approximately half of these establishments reported having more than one worksite. Theoretically, then, somewhere between 50 percent and 100 percent of this group of establishments could be covered under the law because they have more than 50 employees across all sites (11).

  2. The leave policies for these establishments resemble those that were defined as "covered" in the report. A majority of them report providing up to 12 weeks of leave for many of the FMLA reasons (e.g., see Table 5.1). Counting all of these establishments within the covered category decreases the estimate of the number of non-covered establishments providing leave consistent with the FMLA requirements. This also reduces the estimate of the number of covered establishments that provide this type of leave.

  3. The finding that the proportion of non-covered establishments with leave policies consistent with the FMLA has increased between 1995 and 2000 is not affected when re-classifying these establishments as covered (e.g., see discussion in section 5.1.1; Appendix Table A5.2).

Leaves Taken Under the FMLA

To estimate the number of leaves taken under the FMLA, the respondent was asked to provide data for the period between January 1, 1999 and the interview. The establishment’s estimated number of leaves taken could include multiple leaves for the same person. The estimate uses these data by aggregating this item for those establishments that reported they were covered by the law and that were classified as being covered based on the size of their establishment.

As noted in the report (Section 3.5.1), a number of establishments had difficulties retrieving this information from their records. Approximately 45 percent of covered establishments did not provide these data at all. These establishments were excluded from the estimates for the amount of leave taken under the FMLA.

References

Groves, R and Wissoker, D.W. (1999). "Early Nonresponse Studies of the 1997 National Survey of America’s Families," Report #7, Assessing the New Federalism. Urban Institute: Washington, DC.

Keeter, S., Miller, C., Kohut, A., Groves, R.M., and Presser, S. (2000). "Consequences of Reducing Nonresponse in a National Telephone Survey." Public Opinion Quarterly, 64:125-148.


(1) A more detailed description of the technical characteristics of each survey is provided in the full methodological report (in press) for this project.

(2) The 1995 Survey of Employees was conducted by the University of Michigan.

(3) The 1995 Survey of Establishments was conducted by Westat.

(4) It is important to note the difference between non-response error and a low response rate. A low response rate increases the chances that significant non-response error exists. However, non-response error occurs only when the non-respondents actually differ from the respondents along the characteristics that are important to the survey. For example, a recent analysis comparing two surveys which differed by approximately 20 percentage points did not find significant evidence of more non-response error for the survey with the lower response rate (Keeter, et al., 2000). Similarly, analysis of a large survey on welfare reform came to similar conclusions when comparing results among estimates based on response rates that differed by as much as 20 percentage points (Groves, et al., 1997).

(5) For example, it is possible that the 2000 survey missed more employed males in the populations that took leave for family and medical reasons. This would affect the comparison of leave-takers by gender between the 1995 and 2000 surveys. The extent that this may be the case depends on the differentials in response rates by gender between the two surveys.

(6) For this purpose, the leave-takers were counted as 16.5 percent of the total employed population, leave-needers as 2.4 percent and employed-only as 81.1 percent. These percentages approximate what was observed for the 2000 distribution across these groups.

(7) These items were asked during the screener (referring to each person living in the household) and the extended interview (referring to the respondent).

(8) The definition used in the survey is not in precise conformance with the requirements of the FMLA. The Act defines an employer as covered only when the employer has 50 or more employees for at least 20 workweeks in the current or preceding calendar year. The survey, however, counted as covered all establishments with at least 50 employees within 75 miles of the sampled location. Thus, the survey did not count as covered those employers with at least 50 employees beyond 75 miles of the sampled location.

(9) The employee eligibility test also requires employees to work at a location where at least 50 employees are employed within 75 miles. This part of the eligibility requirement was actually applied when classifying establishments as covered or not covered. Requirements also include 12 months for same employer which we did not take into account.

(10) BLS data used for this purpose are derived from the Covered Employment Wages program.

(11) This group of establishments was defined as non-covered in the report because they have fewer than 50 employees within 75 miles of the worksite. For these establishments to qualify as covered under the FMLA, there would have to be another worksite that increased the total number of employees for the business over the minimum of 50 required by the law.

Previous Section Next Section