Cornell University Law School Social Science and Law

Research Methods

I. Introduction

Quantitative research has gained wider and wider acceptance both in legal scholarship and in the courtroom.  This website is intended to function as a resource for law students interested in learning more about the use of quantitative methods in law.  Each of the following sections detail the various ways in which statistical data can be analyzed, giving some background and useful online resources.

II. Quantitative Research Methods in Legal Scholarship

Quantitative research has gained wider and wider acceptance in legal academia over time. The following articles serve as something of an introduction to the history and use of quantitative methods in legal academia.  The articles vary in their scope and focus, ranging from pieces discussing the history of empirical scholarship in legal academia to proposals for establishing a stronger foundation for empirical legal studies.  They are very useful to read if you are interested in understanding the problems that initially led legal scholars to use quantitative methods.

Articles

“Building an Infrastructure for Empirical Research in the Law” by Lee Epstein & Gary King, Journal of Legal Education, Vol. 53, No. 3 (2003)

http://epstein.law.northwestern.edu/research/rules2.pdf
Epstein and King argue that the current organization of legal scholarship is ill equipped for producing high quality empirical scholarship and suggest that changes in the policies of law reviews as well as the archiving of data might ameliorate this problem.

“The Past, Present, and Future of Empirical Legal Scholarship: Judicial Decision Making and the New Empiricism” by Michael Heise, University of Illinois Law Review, Vol. 2002, No. 4 (2002)

http://www.law.uiuc.edu/lrev/publications/2000s/2002/2002_4/Heise.pdf
Heise provides a history of empirical legal research on judicial decision-making, ranging from work conducted before the Second World-War to the present day.

“Introduction to the Symposium on Empirical and Experimental Methods in Law” by Richard H. McAdams & Thomas S. Ulen, University of Illinois Law Review, Vol. 2002, No. 4

 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=419980
Experimental methods have become more and more popular in economics. This article introduces a symposium on whether such methods might become more popular in empirical legal studies in the future.

“Can Statistics Tell Us What We Do Not Want to Hear? The Case of Complex Salary Structures” by Mary W. Gray, Statistical Science, Vol. 8, No. 2 (1993), pp. 144-179

http://www.stat.umn.edu/~sandy/courses/8801/articles/Law/graylaw.pdf
Gray demonstrates some of the dangers that arise when courts disregard data and analyses that conflict with the intuitions. She draws on the interesting example of regression analyses of faculty salaries and discrimination to make her argument.

 “Statistical Reasoning in the Legal Setting” by Joseph L. Gastwirth, The American Statistician, Vol. 46, No. 1. (Feb., 1992), pp. 55-69

http://www.stat.umn.edu/~sandy/courses/8801/articles/Law/gastwirthonlaw.pdf
This is a statistician’s view of how quantitative evidence is presented and evaluated in the legal context.

“Damned Liars and Expert Witnesses” by Paul Meier, Journal of the American Statistical Association, Vol. 81, No. 394. (Jun., 1986), pp. 269-276

http://www.stat.umn.edu/~sandy/courses/8801/articles/Law/meierlaw.pdf
A statistician’s view of some of the ways in which practices within the statistics community conflict with the views of courts regarding inferential statistics.

“Trial by Mathematics: Precision and Ritual in the Legal Process” by Laurence H. Tribe, Harvard Law Review, Vol. 84, No. 6 (Apr. 1971)

http://www.jstor.org/pss/1339610
This is a classic article in which Tribe argues that mathematical reasoning and methods of proof may be insufficient or at least inapplicable in the legal context.

The Daubert Trilogy

The Daubert Trilogy describes three cornerstone cases in which the Supreme Court laid down the standard for the admissibility of expert testimony in United States’ Courts.  The Daubert case itself dealt the admissibility of expert epidemiological testimony and quantitative evidence in the context of a suit over whether the drug Bendectin caused birth defects.  These cases are essential reading for any student interested in understanding how quantitative evidence (and other expert evidence more generally) is evaluated for accuracy and reliability by courts. 

Since the cases deal with the Federal Rules of Evidence, a link is provided to the relevant Article.  Make sure to read the Committee Notes as well (especially the notes for Rule 702).  Also linked below is a study by the RAND instituted summarizing this standard as well as providing an analysis of its resulting effect on expert testimony in Federal Courts. It is a useful resource for understanding how courts have come to use the Daubert standard in practice.

Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993)
This may be the most important case in evidence law of the last 20 years. It deals with the use of expert scientific testimony in the context of a pharmaceutical liability case but its effects have spread far beyond this narrow purview.

General Electric Co. v. Joiner, 522 U.S. 136 (1997)
This follow-up case gave the standard of review for a motion filed under the Daubert standards for evidence.

Kumho Tire v. Carmichael, 526 U.S. 137 (1999)
This case expanded the reach of Daubert to non-scientific experts. The case dealt an automobile accident and the testimony of a purported expert on the design and function of tires.

Federal Rules of Evidence, Article VII
http://www.law.cornell.edu/rules/fre/rules.htm#Rule701
The Daubert trilogy of cases above deal specifically with Article VII or the Federal Rules of Evidence. The rules and their commentary should be read alongside the cases as they have changed and responded to the determinations of the Supreme Court.

“Changes in the Standards for Admitting Expert Evidence in Federal Civil Cases Since the Daubert Decision” by Lloyd Dixon & Brian Gill, Report Prepared for the RAND Instituted for Civil Justice (2001)
http://www.rand.org/pubs/monograph_reports/MR1439/MR1439.pdf
This study represents one of the most complete analyses of the effects on the Daubert standards on the admission of expert evidence. It provides a comprehensive theory on Daubert’s effects and makes use of a large and novel data to explore its hypotheses.

Additional Cases

While there are numerous other cases based on and involving statistical and quantitative evidence and methods, relatively few of them directly discuss how courts should evaluate such data.  The following cases, however, are notable exceptions.  In these cases, courts have extensively discussed how such data should be treated, particularly in the context of expert witness.  They are useful cases to read for some perspective on how courts have wrestled with the particular challenges that quantitative research raises in the litigation context.

McCleskey v. Kemp, 481 U.S. 279 (1987)
This is an immensely famous case in which the Supreme Court rejected the use of statistical evidence that African-Americans were more likely to receive the death penalty than whites as grounds for reversing a petitioner’s death sentence.

Hazelwood School District v. United States, 433 U.S. 299 (1977)
This case is an example of the use of statistical data in the civil rights context, focusing on data of pupils and teachers.

Alabama v. United States, 304 F.2d 583 (1962)
This case is another example of the use of statistical data in the civil rights context, focusing this time on voting rights and access.

Frye v. United States, 54 App.D.C. 46(1923)
Prior to the ascendency of Daubert and its progeny, the Frye test was the standard for evaluating the admissibility of scientific evidence. It is still good law in some jurisdications.

Web Resources

Empirical Legal Studies Bibliography (searchable)
http://www.law.ucla.edu/home/apps/els/
The Empirical Legal Studies Bibliography is a powerful tool for students seeking a topic for seminar paper or law review note.  It covers an immense range of empirical research, including quantitative research, across topics as diverse as antitrust, psychology and the law, death penalty and civil rights studies, and corporate law.

III. Using Quantitative Methods

Reference Manual on Scientific Evidence – The Federal Judicial Center (2d ed. 2000) (especially pp. 83–276)
http://www.fjc.gov/public/pdf.nsf/lookup/sciman00.pdf/$file/sciman00.pdf
The Reference Manual on Scientific Evidence is a major resource for judges, lawyers and other legal professionals.  The Manual covers all types of scientific evidence from DNA evidence to toxicology tests to economic evidence.  The topics covered from pages 83-276 are particularly important as they cover basic statistics, linear regression and survey research. The Manual is an excellent starting point for understanding the basics of quantitative research.


Books

There are very few books that deal specifically with the use of statistics in the law.  This poses something for a challenge for law students as the examples employed by textbooks in statistics or econometrics are often based on scientific and economic concepts that students may be unfamiliar with. 

Basic Concepts of Probability and Statistics in the Law by Michael O. Finkelstein, (Springer Press 2009)

Statistics for Lawyers by Michael O. Finkelstein & Bruce Levin (Springer-Verlag 2d ed. 2001)
The books written by Michael Finkelstein are written specifically for law students and use examples that deal with legal concepts.  Furthermore, Statistics for Lawyers covers a substantial range of statistical methods and can serve as an excellent reference text for a student interested in quantitative methods. Designing Social Inquiry deals mainly with rules of inference.  It is a useful for understanding how social scientists approach a wide range of research questions, from either a quantitative or qualitative perspective.

Designing Social Inquiry by Gary King, Robert O. Keohane, & Sidney Verba (Princeton University Press 1994)
This book explains the basics of statistical inference demonstrating that the same logic underpins both quantitative and qualitative evidence. It is an excellent for understanding the fundamentals of research design.

Articles

“The Rules of Inference” by Lee Epstein & Gary King, The University of Chicago Law Review, Vol. 69, No. 1 (Winter 2002)

http://aals.org.cnchost.com/2005midyear/commercial/EpsteinSupplementaryMaterials.pdf
This article represents a shortened version of the King, Keohane and Verba text, specifically adjusted to suit the legal context.

“Above the Rules: A Response to Epstein and King” by Frank Cross, Michael Heise, Gregory C. Sisk, The University of Chicago Law Review, Vol. 69, No. 1 (Winter 2002)

http://personal2.stthomas.edu/gcsisk/Siskwebpagestuff/AbovetheRules.pdf
This is a largely positive response to the Epstein and King article above. It tends to emphasize some of the limitations of the article regarding the incentives of legal scholars.

“Recent Developments in the Econometrics of Program Evaluation” by Guido W. Imbens & Jeffrey Woolridge, Journal of Economic Literature, Vol. 47, No. 1 (March 2009)
This article has nothing to do with empirical legal studies but is an excellent explanation of some of the challenges associated with using quantitative data to show causality. The authors are two of the most highly respected econometricians in the world.

“A Bayesian Approach to Identification Evidence” by Michael O. Finkelstein & William B. Fairley, Harvard Law Review Vol.83 pp. 489 (1970)
This article brings the idea of Bayes theorem to bear on the idea of evidence. While more work has been done on this topic in the intervening years, this article remains a classic.

Textual Websites

The Little Handbook of Statistical Practice by Gerard E. Dallal
http://www.StatisticalPractice.com
This website contains a wealth of information on different basic statistical concepts like distributions, tests of means, and correlation.

Statistics at Square One
http://www.bmj.com/statsbk/
This is another website that describes a range of statistical concepts.

The Data Analysis BriefBook
http://physics.web.cern.ch/Physics/DataAnalysis/BriefBook/
This book is a more advanced version of the two websites above. It tends to cover more the mathematical architecture behind quantitative methods.

Webcenter for Social Research Methods
http://www.socialresearchmethods.net/
This website has much of the same information as the other three, but has a more graphically intense layout.

Introductory Statistics: Concepts, Models and Applications
http://www.psychstat.missouristate.edu/sbk00.htm
This is a nice website for information on distributions and statistical tests.

Multivariate Statistics: Concepts, Models and Applications
http://www.psychstat.missouristate.edu/multibook/mlt00.htm
This is a good website for the fundamentals of regression analysis.

HyperStat Online Statistics Textbook
http://davidmlane.com/hyperstat/
This is another website that goes the basics of ANOVA analysis, a common method in psychological studies.

StatPrimer
http://www.sjsu.edu/faculty/gerstman/StatPrimer/
This website also covers many of the basics of statistical analysis leading up to ANOVA and correlation analyses.

Calculators and Other Demonstration Websites
The following websites do not provide any real description of statistical methods but are useful for visualizing and understanding various statistical concepts. While they do not replace a full statistical program like SAS, SPSS or Stata, they provide some useful insight into the basics of statistical analysis.

Seeing Statistics
http://www.seeingstatistics.com/

Statistical Calculators
http://calculators.stat.ucla.edu/

Correlation Calculators
http://faculty.vassar.edu/lowry/VassarStats.html

Information on Probability Distributions
http://faculty.vassar.edu/lowry/VassarStats.html

Probability Calculator
http://www.psychstat.missouristate.edu/aspx/ProbabilityCalculators.aspx