Convenience v. Random Sampling in Wage and Hour

Convience sampling v. Random sampling:

A convenience sample is a sample where individuals are selected, in part or in whole, at the convenience of the researcher. The researcher makes no attempt, or only a limited attempt, to insure that this sample is an accurate representation of some larger group or population.

The classic example of a convenience sample is standing at a shopping mall and selecting shoppers as they walk by to fill out a survey.

In contrast, a random sample is one where the researcher insures (usually through the use of random numbers applied to a list of the entire population) that each member of that population has an equal probability of being selected.

See how convenience samples can potentiallly help improve a random sample 10498-en.pdf

The last word on survey evidence in wage and hour cases

What do the courts say about survey evidence?

Take a look at:

sciman00.pdf

Reference Manual on Scientific Evidence, Second Edition 2000, 638 pages

The second edition of Reference Manual on Scientific Evidence has been published and distributed to federal judges.

Attack II: Your expert is not an expert

Is the bar too high for survey evidence in wage and hour cases?

The second approach that the defendant's in Marlo v. UPS employed was an attack on the plaintiff's expert's qualifications as a survey expert. Although the expert had done a lot of surveys, the defendant's were able to convince the court that she was not quite expert enough.

Specifically,

According to the defendant's, Rules 702 and Daubert, the Reference Manual
details specific expertise requirements for survey experts and provides that experts
who design, conduct, or analyze a survey be appropriately skilled with the
following qualifications:

• Experts prepared to design, conduct, and analyze a survey generally
should have graduate training in psychology (especially social,
cognitive, or consumer psychology), sociology, marketing,
communication sciences, statistics, or a related discipline.

• That training should include courses in survey research methods,
sampling, measurement, interviewing, and statistics.

• In all cases, the expert must demonstrate an understanding of survey
methodology, including sampling, instrument design (questionnaire
and interview construction), and statistical analysis.

• Publication in peer-reviewed journals, authored books, membership in
professional organizations, faculty appointments, consulting
experience, research grants, and membership on scientific advisory
panels for government agencies or private foundations are indications
of a professional's area and level of expertise.

• In addition, if the survey involves highly technical subject matter (e.g.,
developmentally disabled adults with limited cognitive skills), the
survey expert also should be able to demonstrate sufficient familiarity
with the topic or population (or assistance from an individual on the
research team with suitable expertise) to design a survey instrument
that will communicate clearly with relevant respondents.

In short, the defense said the expert met none of the above.

Attack I: - more details..

Is the bar for wage and hour survey evidence too high?

As mentioned in the last posted the defendant's in Marlo v. UPS attacked the plaintiff's wage and hour survey on several levels. The following discusses the details of the attack.

(see yesterday for discussion on 1. sample not representative)

2. The survey questions were not framed in a clear, precise,
and non-leading manner


The questions to be asked of interviewees were not framed in a clear,
precise, and non-leading manner. First, many questions in the Telephone Survey
use biased words and phrasing that were designed to lead to a specific or desired
response.

Second, the [Federal] Reference Manual explains that the "yes/no/not applicable"
response format results in inaccurate data because it encourages "acquiescence"
(the tendency of respondents in some surveys to agree or disagree, known as "yeahsaying"
and "nay-saying") and "social desirability" (the desire to present oneself in
a favorable light). Reference Manual at 375-377

Third, the Telephone Survey elicited responses to all questions, even
though some questions clearly did not apply to all respondents because they did not
pertain to all three of the relevant FTS positions.

....The fact that most of the respondents answered every question regardless of whether it applied to them confirms that the questions encouraged acquiescence and guessing.

Attack I: Inadmissible Hearsay - a detailed look

Is the bar for wage and hour survey evidence too high?

As mentioned in the last posted the defendant's in Marlo v. UPS attacked the plaintiff's wage and hour survey on several levels. The following discusses the details of the attack.

1. A representative sample was not selected.

First, no precautions were
taken to ensure that only qualified respondents were included in the Telephone
Survey....the Telephone Survey did not ask any questions to ensure that
respondents were qualified to participate in the survey, such as whether they were
currently employed at UPS in one ofthe FTS job positions, or if they ever held one
of the relevant FTS job positions at UPS.


Second, there is no evidence that the respondents who participated in
the Telephone Survey are representative of all class members. Of the 614
"qualified respondents" provided by Plaintiffs counsel, only 160 interviews were
completed. The plaintiff's expert does not provide information about relevant characteristics of the 160 respondents, such as the region or district where they worked, whether they worked as a full-time on-road, hub, or preload supervisor, or the length oftheir employment. fd. Nor does it compare the characteristics of the respondents to the population the Telephone Survey was
intended to represent.

The plaintiff's expert confirmed at deposition that she did not know whether
the sampling frame was representative of the universe, whether the sample was
random from any other larger universe of people who were being surveyed, or
whether the sample was stratified as to any larger list, and that she did nothing at all
to evaluate the representative nature of the sample.

Attack I: The survey is inadmissible hearsay

Is the bar too high for survey evidence in wage and hour cases?

The defendant's first attack was the survey was inadmissible hearsay of the type not reasonalbly relied on by experts in the field.
According to the defendants:

The results of the Telephone Survey are out-of-court statements by
respondents purportedly discussing their knowledge and application of UPS's
methods and procedures. Plaintiff seeks to use the results ofthe Telephone Survey
to prove the truth of those statements, i.e., that the respondents know and apply
UPS' methods and procedures. This is classic hearsay. FED. R. EVID. 801(c)

Some courts have allowed the admission of survey evidence under the
residual or "catch all" exception to the hearsay rule upon a finding that the survey is
"material; . .. more probative on the issue than any other evidence; and ... has
'circumstantial guarantees of trustworthiness. "

A survey is trustworthy if it "is conducted in accordance with
generally accepted survey principles, and if the results are used in a statistically
correct way, since proper survey and statistical methods are intended to assure a
poll's reliability."

The defendant's attacked the plaintiff's wage and hour survey on several levels. Using the defendant's definition of 'trustworthy' means that a trustworthy survey is one where:

(1) the "universe" was properly defined;
(2) a representative sample of that universe was selected;
(3) the questions to be asked of interviewees were framed in a clear, precise, and nonleading
manner;
(4) sound interview procedures were followed by competent
interviewers who had no knowledge ofthe litigation or the purpose for which the
survey was conducted;
(5) the data gathered was accurately reported;
(6) the data was analyzed in accordance with accepted statistical principles; and
(7) the objectivity of the entire process was ensured.

According to the defendant's motion, the plaintiff's expert's survey did not meet any of these requirements for a sound study.

Survey adminstration

Is the bar too high for survey evidence in wage and hour cases?

Here are the details of how the survey was conducted:

SAMPLE

A list of 614 cases was provided. This list was checked for duplicate phone numbers and valid area codes and 64 cases were thrown out. Of the 550 remaining numbers, 166 were unusable, leaving 384 viable phone numbers. The remaining 384 cases we dialed up to 5 attempts in an effort to complete the questionnaire with as many as possible.

INTERVIEWING·
Interviewing took place between the dates of November 18th and November 24",2004. Calling times were scheduled from 6:00-9:00 PST weekdays, 9:00 am-4:00 pm PST on Saturday and 2:00-8:00 PST Sunday.

Interviewing took place at various times during these calling windows. Daytime dialing also occurred on the 220d and 23'd of November.

Interviewing was completed following sound, standard marketing research call center practices Including reading verbatim and administering the interview in a non-biased fashion. 20% of completed interviewers were silently monitored (10% is industry standard). 13% of respondents were called back to confirm qualification and study participation (10% is industry standard).

Supervision ratio was 1 hour for every 3.5 interviewing hours. Industry average is approximately 1:8.
A total of 7 Intervij1wers were briefed and dialed on this study at some time during its 7days in field ranging from 1to 5 during any given shift.

QUESTIONAIRE·

Aquestionnaire wjth 42 yes/no/na questions was provided to ORI, atrue and correct copy of which Is attached hereto ap Exhibit 1. This questionnaire was programmed Into CATI (computer assisted telephone interviewing) software commonly used within the industry to administer telephone interviews. In addition to providing the script and capturing data the software also regulates the phone numbers to be called in reference to callbacks times, resolved vs. non resolved numbers, etc.

Part I : Background on survey in wage and hour case

Is the bar too high for survey evidence in wage and hour cases? Part I


In November 2004, Plaintiff Michael Marlo's counsel of record, John
Furutani, retained a "data collection execution and management" to conduct a telephone survey ("Telephone Survey") of current and former UPS employees.

The survey asked questions related to procedures that the UPS employees followed. The survey also asked about the authority that the UPS employees had to fire, change schedules, etc. of other UPS employees.

Plaintiff intends to offer the expert testimony of the president of the survey firm regarding the fact, content, results, or purported scientific validity of the Telephone Survey.


The survey:

Application of knowledge and training!manager's desires
1) Do you apply your knowledge ofUPS' prescribed procedures when
determining which procedure to follow in a given situation?
2) Are the decisions you make on a daily basis as a UPS full-time supervisor
based on UPS' rules, procedures or methods that you have been trained on?
3) Are the decisions which you routinely make as a UPS full-time supervisor
based on the experience you have gained working for UPS?
4) Have you been traioed in UPS' methods and procedures?
5) Are the decisions which you routinely make as a UPS full-time supervisor
based on the training you have received while working for UPS?
6) Are the decisions of consequence you make in the operation dictated by the
training you have received on the proper procedures and methods?
Training on the methods
7) Do you have the authority to modify training methods if you feel you have a
better way of doing something?
8) Do you train employees on UPS' methods?
9) Do you ever disregard the methods while training?
10) Do you train outside of the prescribed methods?
11) Do you train outside of the prescribed procedures?

12) Do you have the authority to train an employee in your own methods if you
feel they are better than UPS'?
Changing and adherence to goals set by UPS
13) Do you have the authority to unilaterally change the goals set by UPS?
14) Do you have the authority to unilaterally change the goals given to you by
Industrial Engineering (IE)?
15) As a full-time supervisor, do you have the authority to disregard "Packages
Per Hour" (PPH) goals?
16) As a full-time supervisor, do you have the authority to disregard "Stops-peron-
mad-hour" (SPORR) goals?
Assigning staff/vacation/salary decisions
17) If staffing is low going into the following week, do you have the authority to
unilaterally cancel the planned vacations of your employees for the following
week?
18) If one ofyour drivers has an accident, do you have the authority to decide that
the driver does not need a ride along the next day?
19) If a driver has a minor accident, do you have the authority to instruct the
driver to ignore it and not report it?
20) Do you determine the hourly wages for the hourly employees assigned to you?
21) Do you have the authority to supervise those employees who do not work
directly under you?
22) Does the Collective Bargaining Agreement dictate how many hourly
employees can be off on vacation at any given time?
23) Does the Collective Bargaining Agreement dictate which hourly employees
get to pick vacation first?
24) Are the decisions which you make as a UPS full-time supervisor limited in
any way by the Collective Bargaining Agreement for hourly employees?
25) Do you have the authority to ignore the Collective Bargaining Agreement?
26) As a full-time hub, preload or on-road supervisor, do you have the authority to
hire more employees to your rosters without Industrial Engineering (IE) or
Human Resource (RR) approval? '.
27) As a full-time hub, preload or on-road supervisor, do you have the authority to
hire a new employee on behalf ofUPS?
Disciplining staff
28) If you see an employee violating a method, such as walking on a moving belt,
do you have the authority to iguore the behavior and allow it to continue?
29) Do other supervisors have the authority to discipline your hourly employees?
30) Do you have the authority, as a full time supervisor, to discipline the hourly
employees of another supervisor?
Working as directed
31) Are you directed by your manager or division manager to do traininglon-jobsupervision?
32) Are you required to follow UPS' methods?
33) Are you required to make sure that UPS' hourly employees are following
UPS' methods?
34) Are you required to follow UPS' methods and procedures in achieving the
goals dictated by corporate?
35) Are you required to work as directed by your manager or division manager?
36) Do you have the authority to disregard any safety method prescribed by UPS?
37) Are you required to do daily audits?
38) Do you use a checklist to perform these daily audits?
39) When you are doing a checklist audit, do you have the authority to disregard
any methods or procedures?
40) Are you required to document audits on UPS checklist or audit forms?
41) Does your manager have the final say for operational issues?
42) Does your division manager have the ultimate decision-making authority for
the facility?

Is the bar too high for survey evidence in wage and hour cases?

In 2004 California Federal Court case, Marlo v. UPS, the court excluded the plaintiff's survey evidence in a misclassification case. The plaintiff's hired a survey firm to ask UPS employees about the duties that they performed. Based on the survey, the plaintiffs expert opined that the UPS workers were misclassified as exempt from overtime payments.

The defendant's motion questioned the reliability of the survey and the qualifications of the plaintiff's expert.

Over the next few post, we will be discussing the ins and outs of this case.

Statistical evidence in wage and hour cases after Brinker

Will statistical and survey evidence become less important after Brinker?

(from Akin Gump newsletter)

The California Court of Appeal for the Fourth Appellate District on July 22, 2008, in Brinker Restaurant Corp. v. Superior Court of San Diego County, issued a unanimous, published opinion that defines the elements of meal period, rest period and off-the-clock claims under California law. The court concluded that an employer must only provide meal and rest breaks to employees, not ensure that employees take them, as lawyers for plaintiffs have argued in many cases. The court held that why an employee did not take a particular break was an individual question that could not be resolved on a class basis. The court thus reversed class certification.

In terms of statistical evidence...the Court of Appeal rejected an argument that plaintiffs raised with respect to all of their claims: that “expert statistical and survey evidence” rendered their claims amenable to class treatment. The Court of Appeal explained that such evidence, while sometimes useful in the class certification context, in this case could not show why meal and rest breaks were not taken, whythey were interrupted or why employees worked off the clock. Only individual inquiries could resolve those questions, making class certification of plaintiffs’ claims inappropriate. 

Wage and hour survey discussion Part I

From the previous post (9/2/2008)According to the defense's Daubert motion, the wage and hour survey in the Big Lots case suffered from a number of flaws.

As mentioned, the court disagreed and denied the defense's motion.

List of criticisms:

* The survey responses conflicted with opt-in actual deposition testimony.

* Survey had “non-response bias and coverage error.”

* Another flaw with the survey is that it was not pretested, with the
exception of internally and with Plaintiffs’ counsel

*Recency bias, or the tendency to overweight recent experience in our perception of general experience.

* “Coverage error occurs when members of the survey population do not have an equal probability of being chosen to be part ofthe sample studied.” (Survey did not include non-opt-ins)

* Numerous survey responses that indicated confusion or dissatisfaction with survey questions. Of the 553 responses he reviewed, over 55 percent (306)contained missing or incomplete answers...

*Plaintiff expert did not provide a follow-up to questions

* Survey used the FLSA test qualifications. Focused on the long test (for exemption classification) when the short test questions were arguably the correct ones.

* Questions failed to seeks information regarding the respondent’s employment
history with BigLots. It fails to draw any distinction between time employed at Big Lots
that is at issue in thisaction and time that falls outside the statute of limitations.

*Use of the word “regularly” in the survey question resulted in a great
deal of confusion,destroying any confidence that the survey responses on the relevant
issues (hiring and firing) arereliable in terms of what is required by the regulations.

*“Regularly” is not defined in the survey instrument, leaving the respondents to formulate their own interpretations

* Term refers to average work week but does not define them.

* Some questions do not add correctly. Either their totals failed
to equal 100 percent or, as in the case of numerous respondents they listed very high percentages (between 75% to 100% foreach of the activities) evidencing total confusion about the question.

*Wrong defintion of work week. The employer defined a full time work week as 32 hours and not 40 as in the survey

* Disagreed with damage method.

Plaintiff expert calculated that, based upon the survey responses, the respondents worked anaverage of 59.3 hours per week. He further claimed, based upon a 95 percent confidenceinterval, that this means that 95 percent of all opt-ins worked 58.5 to 60 hours a week. He then concludes that all of the opt-ins should be awarded damages simplyby determining from payroll records how many weeks each Assistant Store Manager had worked, by applying theappropriate overtime factor to the base hourly rate and then multiplying the resulting overtimerate by the number of hours worked in excess of forty per week.

Sponsered Link: Looking for a survey and damages expert in a wage and hour case? Go to http://www.employstats.com/

Wage and hour survey is admitted in FLSA case

John Johnson and Robert Burden v. Big Lots Stores, Inc., United States
District Court, Eastern District of Louisiana

In this case, the plaintiff's allege that the assistant store managers
are missclassifed as exempt from OT. According the plaintiff's report they
assistant managers worked on average 58 hours a week but were paid for
only 40 hours.

Plaintif's hired an ecoonomist to perform a survey of the opt-in members
of the class. The plaintiff's damage analysis was based on the survey.

Summary:

In a nutshell the defense argues that the plaintiff's survey is flawed. Argue to the brief, the flaws are as follow. First, the respondents are interested in the outcome of the litigation for
which the survey was conducted, and were told that the survey was to be used to help with their case, rendering the results unreliable. Second, additional methodological defects infected the survey with bias, rendering the analysis based upon it unreliable. Finally, the plaintiff's expert asked the wrong questions, resulting in misleading and unreliable answers.

The court disagreed and denied the defense's Daubert motion. In the coming post, we will discuss this case in more detail.

Sponsered Link: Looking for a survey and damages expert in a wage and hour case? Go to www.employstats.com