1. Measuring the
Complexity
of the Law:
The United
States Code
daniel martin katz
michael j bommarito ii
computationallegalstudies.com
@ computational

2. This is the Era of “Big” Data
Increasing Computing Power
Decreasing Data Storage Costs
Fundamentally Altering the Scope of Scientific Inquiry
and Technical Possibility

3. Highlighting the Data Deluge
2008
2009
2010

4. Highlighting the Data Deluge
2011
2011

5. Legal Studies
in the Era of Big Data
US Patent D-Base
Fed Courts Pacer
~500+ million Docs
PublicResource.org
Law.Gov ?

6. Legal Studies
in the Era of Big Data
US Patent D-Base
Fed Courts Pacer
~500+ million Docs
PublicResource.org
Law.Gov ?

7. But What Can We Learn
About the Nature of Law
With All of this Information?

8. Perhaps Generate
New Perspectives
on Old Questions?

9. The Complexity of the Law
is a Canonical Question
Legal and Political Theory
Justifications for Law and the State
Optimal Precision in Legal Rules
Kaplow (1995) Tullock (1995)
Applied Scholarship -Tax, Environmental Law, Admin Law, etc.
Economics of the Legal Profession
Complexity ~ Bespoke Legal Work
=

10. Law as a Means of
Solving Social Dilemmas
Aligns Incentives / Channels Behavior
Offers Focal Points / Coordination Mechanisms
Encourages Actors to Internalize Costs
Protects Individual Rights and Liberties
Maintains Monopoly on Legitimated Violence

11. Complexity of Society &
Complexity of the Law
What conditions must be met for law
to achieve these ends?
Legal Rules Should Reflect The Nature of:
Economic Exchange
Political Behavior
Social Interaction

12. A Perspective on the Scope
of Legal Rules in a Modern Society
This Paper is an Effort (Albeit Imperfect)
to Measure the Complexity of the Law
Applying the Computational Tools of the
‘Big Data’ Era
Using an Important Corpus of Written Law
That is Large and Cross Cutting

13. This is the United States Code

14. This is the United States Code
Drawn from the U.S. Statutes at Large
Compiled Version of Federal Statutory Law
Does not Include Fed Admin Regulations

15. The 50 Titles in US Code

16. Some Potential Questions
How Large is the United States Code?
How Complex is the Code and its Components?
How has its Composition Changed Over Time?
Can we understand if those changes scale to changes
in the complexity of broader society?

17. How Large is the United States Code?

18. How Large is the United States Code?

19. How Large is the United States Code?

20. How Large is the United States Code?

21. How Large is the United States Code?
~Title 29 - Labor

22. How Large is the United States Code?

23. This is the United
States Code

24. The Case for a
Computational Approach
The United States Code is Large ...
Need Tools and Methods that Scale
to the Size and Scope of this Body of
Information
Computational Methods are Arguably Required

25. Computational Approach
to the Measurement
of Complexity
(1) Provide a Mathematical
Representation for an Object
(2) Generate a Measurement Strategy for
that Object

26. A Mathematical Approach to the Study
of the United States Code
389 Physica A 4195 (2010)

27. Additional Treatment Available Here

28. The US Code as a
Mathematical Object
Hierarchical Structure
Title 26
Subtitle A
Chapter 1
Subchapter F
Part I
Section 501
Subsection (c)
Paragraph (3)

29. The US Code as a
Mathematical Object
Citation
Network
Example: Tax Evasion
Title 26 Tax
Might Cite
Title 18 Crimes &
Criminal Procedure
http://computationallegalstudies.com/2009/09/14/
the-structure-of-the-united-states-code/

30. Hierarchical
Structure

31. Hierarchical
Structure
+
Citation
Network

32. Hierarchical
Structure
+
Citation
Network

33. Hierarchical
Structure
+
Citation
Network

34. Hierarchical
Structure
+
Citation
Network

36. +

37. +
Linguistic Content
United States Code
Features
~24 Million tokens

38. The Complexity of Law
The sheer difficulty of searching for and assimilating
the information content of a relevant body of legal
rules
Related to But Distinct from other important features:
Ambiguity
Uncertainty
Costs of Compliance

39. Proxy for an Information
Acquisition Process
(1) Work Through a Hierarchy
(2) Look up any Citations you encounter
(3) Confront Language of
varying length and diversity

40. Composite Measure
through Weighted Ranks
(A) For each factor - Measure each of the
49 Active titles
(B) Then, rank each title from
(1) Most to (49) Least
(C) Apply a Weight to each of these factors
(Simplest Case is to Average)

41. Toward a Composite Measure
Multidimensional object --> Composite Measure
Full Code and Individual Titles
Weighted Ranks
Imperfect for certain applications but a good start
A Positive Feature of weighted ranks
(1) Change Weights
(2) Add Additional Dimensions

42. Toward a Composite Measure
Structure
Dependence
Composite Measure
Language

43. Two Forms of Weighted Ranks
Unnormalized Measure
Considers the Complexity of
reviewing a full title
Does not Control for Title Size
Normalized Measure
Considers the Average or Emblematic
Provision within the Title
Thus, we control for title size

44. The Unnormalized Measure
Structure
Dependence
Language
Vertices
(Provisions)
Net Flow
Entropy
Unnormalized
Measure

45. Unnormalized
Measure:
Structure
Structure
Vertices
(Provisions)
!

46. The Unnormalized Measure:
Structure
Title
Vertices
(Provisions)
|V*|
|VS|
|V*|
42
116,237
1,418
7,400
107,419
26
52,838
531
2,088
50,219
16
Structure
|V|
33,925
475
4,726
28,724
20
32,888
715
2,232
29,941
7
31,468
351
2,743
28,374
Title
|V|
|V*|
|VS|
|V*|
13
272
22
70
180
27
224
13
45
166
4
221
6
47
168
1
84
4
39
41
9
68
4
31
33

47. Unnormalized
Measure:
Net Flow
Dependence
Net Flow

48. The Unnormalized Measure:
Net Flow
Title
42
-846
-0.11432
22
-719
-0.24904
-561
-0.78025
16
Net Flow
NetFlowPerSection
50
Dependence
NetFlow
-534
-0.11299
12
-514
-0.28429
Title
NetFlow
NetFlowPerSection
5
2654
2.57669
18
836
0.62109
31
751
1.59447
28
659
0.83102
26
576
0.27586

49. The Unnormalized Measure:
Language
Language
Entropy
!

50. The Unnormalized Measure:
Language
Language
Entropy
Uniform
Random

51. Understanding Linguistic Entropy
Uniform
Dog,
Dog,
Dog,
Dog,
Dog,
Dog,
Dog,
Dog,
Dog,
Dog,
...
Random
snug
pizza
pasture
it
of
observation
hand
seattle
middle
arm
...

52. The Unnormalized Measure:
Language
Title
10.80
10.79
16
Entropy
15
42
Language
NetFlow
10.75
33
10.67
22
10.67
Title
NetFlow
9
8.24
1
8.49
32
8.5
3
8.89
27
9.01

53. The Unnormalized Measure
Structure
Dependence
Language
Vertices
(Provisions)
Net Flow
Entropy
Unnormalized
Measure

54. The Unnormalized Measure:
Language
Language
Entropy ≈ Diversity
Title 42 - Public Health & Welfare
Entropy
Leprosy
Social Security
National Flood Insurance
US Synthetic Fuels Corp
Intl Child Abduction Remedies

55. The Unnormalized Measure
Title
Vertices
Entropy
Flow
42
16
22
7
15
18
12
5
26
10
20
25
31
49
19
29
33
50
28
38
1
3
9
5
6
16
8
13
2
7
4
12
22
10
14
11
18
21
27
15
2
3
5
6
1
8
15
23
29
13
19
14
18
9
7
16
4
22
20
25
2
9
5
11
19
3
10
1
7
21
20
18
4
25
28
22
29
8
6
17
Composite Score
1.67
5.00
6.33
7.33
8.67
9.00
11.00
12.33
12.67
13.67
14.33
14.67
14.67
14.67
16.33
16.33
17.00
17.00
17.67
19.00
Composite Rank
1
2
3
4
5
6
7
8
9
10
11
12
12
12
15
15
17
17
19
20

56. The Normalized Measure
Dependence
Language
Net Flow
Per
Section
Tokens
Per
Section
Language
Structure
Entropy
Mean
Depth
Normalized Measure

57. The Normalized Measure
Title
Net
Flow
42
26
5
49
31
29
12
20
21
18
19
7
15
8
47
22
6
16
10
2
7
1
25
4
22
10
20
35
3
28
11
19
31
37
5
16
9
21
Tokens
Per
Section
8
2
21
11
17
7
9
12
3
25
13
19
10
4
5
32
14
29
31
Entropy
Flow
2
29
23
9
18
16
15
19
12
8
7
6
1
28
17
5
27
3
13
Depth
Rank
10
1
2
5
12
6
18
4
7
22
11
23
30
3
8
27
14
32
9
Composite
Score
5.5
9.75
11.75
12.5
12.75
12.75
13
13.75
14.25
14.5
14.75
14.75
15
16.5
16.75
17.25
17.75
18.25
18.5
Composite
Rank
1
2
3
4
5
5
7
8
9
10
11
11
13
14
15
16
17
18
19

58. Open Access,
Replication,
Extension

59. All Code + All Data - Located Here:
https://github.com/mjbommar/us-code-complexity

60. Website Still in Progress:
http://s3.amazonaws.com/katzcloud/us-code-complexity/index.html

62. Summary
Offered a framework designed to
measure the complexity of the legal rules
Designed to open a dialogue regarding Legal
complexity and its measurement
Demonstrated measurement principles that are
potentially applicable to other classes of legal
documents
Highlighted what computation might be able to
offer to empirical legal studies

63. Thanks to the Following

64. Measuring the
Complexity
of the Law:
The United
States Code
daniel martin katz
michael j bommarito ii
computationallegalstudies.com
@ computational