Quantitative Methods for Lawyers - Class #5 - Research Design Part V - Professor Daniel Martin Katz

1. Quantitative
Methods
for
Lawyers
Research Design - Part V
Class #5
@ computational
computationallegalstudies.com
professor daniel martin katz danielmartinkatz.com
lexpredict.com slideshare.net/DanielKatz

2. The Origins of Modern
Research Design /
Statistical Inference

3. https://www.youtube.com/watch?v=lgs7d5saFFc

5. Correlation
vs.
Inference

8. https://www.youtube.com/watch?v=t8ADnyw5ou8

9. https://www.youtube.com/watch?v=FJcUU0GXsms

10.
A higher education (independent variable) typically
leads to a higher income (dependent variable).
There is a correlation between education and income.
This allows people to say they make a great income
because of their great education.
Example:

11.
Occupation is a potential intervening variable.
Between education and income is occupation. The level
of a person’s education affects the chances for a good
occupation, and that occupation then affects the
income.
Education does not directly cause income an indirect
cause, but not a direct cause.

12.
Occupation is a potential intervening variable.
Between education and income is occupation. The level
of a person’s education affects the chances for a good
occupation, and that occupation then affects the
income.
Education does not directly cause income an indirect
cause, but not a direct cause.
Indep. variable > intervening variable > dep. variable
[education > occupation > income

13. Another Example:
Ice Cream Sales
and
Crime Rate ?

14.
Extraneous variables are defined as any variable
other than the independent variable that could cause
the change in the dependent variable.
Extraneous (as Independent var) > dependent variable
[ Ice Cream Sales > deaths ]

15. Visual
of
the
Example:
IceCream
Consumption
Crime
Heat

16.
One group uses the researcher’s new learning strategy.
The other group uses a strategy of their choice. Finally, all
students are tested over the materials. The psychologist
testifies as to the benefits of this new learning strategy.
How does the attorney challenge this study?
An educational psychologist develops a new learning
strategy. First, the experimenter randomly assigns
students to two groups. Second, both groups study text
materials for thirty minutes on a biology topic.

17. ECOLOGICAL FALLACY

18. ECOLOGICAL FALLACY
Concept: The ecological fallacy arises when group
data is used to draw conclusions about individuals.
The fallacy arises when an individual assumes
something is true of one or more of the parts because
it is true of the whole.

19. For example, the average income of residents in a particular
region is $40,000.
Closer examination of the area shows the neighborhood is
actually composed of two housing estates.
One is a lower socio-economic group of residents, and the
other is a higher socio-economic group. The residents in the
poorer part of town earn on average $13,000 while the more
affluent citizens average $61,000.
the average in this example is constructed from two disparate
groups. In the end, at the individual level, it actually is
unlikely that any one person in the group earns $40,000.

20. The
Schelling Social
Segregation Model

22. Schelling Social Segregation Model

23. Schelling Social Segregation Model

24. Schelling Social Segregation Model

25. Schelling Social Segregation Model

26. Schelling Social Segregation Model

27. Schelling Social Segregation Model

28. Schelling Social Segregation Model
versus
note: stopped model before % unhappy = 0

29. Schelling Social Segregation Model
Test these
Parameters and
report Results

30. Schelling Social Segregation Model

31. Schelling Social Segregation Model

32. Schelling Social Segregation Model

33. Schelling Social Segregation Model
versus

34. What is the micro to
macro disconnect?
relationship to the
ecological inference
fallacy?

35. Question: Assume that statistics show that the largest
number of robberies occurs in the downtown district
from the studied city.
Assume also that the census shows that the average
incomes of persons living in that downtown district are
substantially higher than any other area in the city.
Can we conclude that in the downtown region, the high
income residents are more likely to be robbed?

36. It is tempting to conclude that high income residents
are more likely to be robbed in that location, but this
data does not establish who was robbed.
The group data only establishes that rich people live in
a high crime area.
The robbers might select rich victims or they might
select lower income people commuting into the
downtown to work.

37. RELIABILITY AND VALIDITY

38. Reliability is the degree of consistency.
For example, when a group of people weigh
themselves twice on the same scale, if the scale
readings are consistent between the first and second
weighing, there is total reliability.
Suppose a group of people weigh themselves twice,
once on the first scale and then on a second scale.
Suppose the recorded rates vary between the two
scales. In such an example, there is variance, and as
such, a measurable lesser degree of reliability. Again,
reliability is a measurement of consistency.

39. Concept: Validity is a question of truthfulness.
Does the test actually measure what it purports to
measure?
Using the prior example, does the scale actually
weigh the true weight of the individual?

40. If a single bathroom scale cannot even give a
consistent weight, it is certain not to be accurate.
In such a case, that single bathroom scale lacks
both reliability and validity because that single scale
is inconsistent, and that single scale does not read
accurately.

41. Question: Suppose a group of people test on
two scales and those the two scales are
consistently five pounds apart.
Is there reliability?
Can there be validity?

42. Two scales consistently read five pound apart.
Looking at this result from the standpoint of each scale,
the two scales are reliable.
In other words, one scale is always five pound heavier
and therefore is always consistent within it. The same
reliability exists for the other scale which is always five
pound lighter.
Still, both scales cannot be truthful, but it is possible in
this example, that one scale is truthful. Possibly, one
scale has validity and reliability, but both scales
cannot have validity.

43. Internal validity asks how valid is the research?
The answer to this question lies within the research
project. Examples of threats to internal validity are
selection bias, incorrect recording of data, and
participants dropping out of the study. Examples of
control against these threats are random selection
and a narrow study.

44.
External Validity addresses how much the researcher
can generalize the outcome to the greater population.
In more plain words, just how broadly can the
outcome of this study be read?
All researchers want to generalize the outcome as
broadly is possible.

45.
Challengers to this early smoking study could legitimately
question, are moderate smoking levels safe?
Without breaking the data down by smoking qualities,
the answer is unknown.
By restricting this early study to two general variables of
cigarette smokers and lung diseases, there was no
generalization capacity to show outcomes to societal
populations with multiple variables. This is an example
of a threat to the internal validity because of variable
selection error. If the internal validity is not solid, there
cannot be external validity.

46. Describe some of the
RELIABILITY AND VALIDITY
Issues associated with
the LSAT?
The Bar Exam?

48. Surveys

49. There are four basic types of surveys:
1) mail, 2) telephone, 3) online, and 4) in person.
Mail surveys are paper and pencil documents that are
mailed to respondents.
They are self-administered by the recipient.
Based upon the fact that the questionnaire is filled out
by the recipient, what is the strength and weakness of
that point?

50. Answer:
The researcher has little control over the answer.
The ability to clarify the answer is less since the
respondent has no person present to ask clarifying
questions. The absence of a researcher
representative can result is erroneous responses.
On the other hand, the recipient might feel a sense
of anonymity which allows for more correct
responses to sensitive questions.

51. Surveys by telephone might be conducted by trained
interviewers. Data collected through telephone
surveys usually has minimal missing or erroneous
data primarily because it offers the opportunity for
personal assistance.
Those interviewers typically follow a regularized
protocol or an actual script.
New automated random dialing systems increase the
“randomness of the sample.” but there are still
additional Issues ...

52. In today’s world, what is the
current downfall for researchers
trying to contact respondents
through telephones?

53. In 2011, land line telephones were unavailable to
about a third of all households.
This shows the great rise of cell phones as the
primary phone system. Further, scientific
researchers using automated phone surveys are
barred by “do not call” restrictions.
Finally, many phone respondents have the capacity
to advance read the caller ID. They simply do not
answer. On a national basis, the use of the
telephone survey may no longer be the
predominate method for surveys

54. Online surveys are experiencing
the largest growth. However, it
still has some limitations. Why?

55. Surveys can also be administered by computer and the
Internet. All provide the potential to conduct complicated
research because “help menus” can assist respondents
through the survey. The questionnaires also can include
visual aids or images as part of those surveys. And
perhaps most importantly, they are the least expensive
format.
To achieve validity, however, they are best used with
respondents who are pre-recruited. Otherwise, predicting
the responding sample can be difficult. By definition,
online surveys are only available to those with access to
computers. Also, the surveys can be filtered by programs
barring unwanted span. The big issue involving online
surveys is “what determines your sample?”

56. Drafting questions for a scientific questionnaire is a
methodological process. There are steps to be
considered prior to drafting a single question.
Where do you think the researcher should start this
process?

57. Questions in a survey should have an internal validity
check.
Suppose the hypothesis is that the educational experience
in a specific, demographically changing community is
characterized by a high level of educational instability.
The researcher decides this community’s educational
instability is a result of rapid population growth which
corresponds to a rapid growth in the school population.
The research hypothesizes that this rapid growth developed
instability in the education plan as educators struggled to
deal with the rapidly growing school population. As
stated, one of the researcher’s pre-determined specific
characteristics is a rapid growth.

58. Give three objective questions for the
questionnaire which will allow the researcher to
validate the portion of the hypothesis which
claims a rapid growth in the school population.

59. Three objective indicator questions of this rapid
growth hypothesis element are:
(1) Total student enrollments of the respondent’s
school for each of the last 3 years.
(2) The number of teachers in the school for each of
the last 3 years.
(3) Dollar figures for construction of the school for
each of the last 3 years.

60. By asking those three objective questions in the
questionnaire, the researcher can help validate the
portion of the hypothesis which claims a rapid
growth of the school population.
If all three answers show substantial growth in each
of the last three years, this element will be validity
by those three different approaches: student
enrollments, teacher numbers and construction
expenses.

61. Assume the above objective indicator questions
validated the school’s rapid population growth.
The researcher now wants objective questions to
validate another element of the hypothesis, that is, this
rapid growth caused some educational instability.
Give objective questions to act as an internal validity
check for that educational instability element of the
hypothesis?

62. Objective indicator questions of this educational
instability element are:
(1) What is the average number of years of
teaching experience for employed teachers for
each of the last three years?
(2) What is the average classroom size for each
of the last three years?
(3) How many teachers had to be replaced after
choosing to leave the school during the last three
years?

63. The researcher decides the
type of questions.
They can be closed, open or
contingent questions.

64. Closed Question
How often do your parents question whether
you have homework?
(1) Never
(2) Less than once per week
(3) Once per week
(4) Two or more times per week

65. Open Question
What do you like most about school?
Contingent Question
Are you on the honor roll?
(1) Yes … [If yes, answer the following question]
(2) No … [If no, skip to question # 9]

66. What are some of the advantages /
disadvantages of each question type?

67. As a general rule, questions in the questionnaire
should be short and concrete.
Do not combine concepts.
For example, do not ask: “Do you think the school
should teach geography and art?” Respondents
do not know how to respond if they think yes to art
and no to geography.

68. Be careful about historical questions.
“How many times in the last 5 years did you get a
flu shot?”
Many historical questions are important, but faulty
memories can make the answers into just guesses.

69. Leading questions can bias the data. “Do you
oppose a longer school day?” This suggests the
researcher’s opinion. The question should be, “Do
you favor or oppose a longer school day?” Please
note, there is a vague term. What is a longer
school day? The question must be premised with a
neutral definition.

70. Using Members of your
target population
PRETEST,
PRETEST,
PRETEST Your Questions !

71. Daniel Martin Katz
@ computational
computationallegalstudies.com
lexpredict.com
danielmartinkatz.com
illinois tech - chicago kent college of law@