Clustering Methods and Community Detection with NetworkX. A slide deck for the NTU Complexity Science Winter School. For the accompanying iPython Notebook, visit: http://github.com/eflegara/NetStruc

1. Community Structures
Erika Fille T. Legara, Ph.D.
@eflegara | h5p://eflegara.github.io
Scien&st, IHPC-A*STAR
http://tiny.cc/commdet2017
http://tinyurl.com/commdet2017

2. Walkthrough
• Network structure and function
• Communities
• Community detection algorithms
• Divisive
• Agglomeratxive
• Application: Community detection in framing analysis
• Hands-on http://tiny.cc/commdet2017
http://tinyurl.com/commdet2017

3. Network Structure
• Path length
• Diameter
• Clustering coefficient
• Degree distribution
• Centrality measures
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
What would the behaviour of
networked systems be based on their
structural properties?
Structure Func*on

4. Communities
• Cluster, module, group
• A group of nodes that have a
higher likelihood of connecting
to each other than to nodes
from other communities
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

5. Example of Communities
• Network: World Wide Web
• Communities: Sites on related topics
• Network: Friendship network
• Communities: Group formation among people
• Network: Metabolic networks
• Communities: Functional modules
• Network: Collaboration network
• Communities: Research fields
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

6. Social Networks: Politics, Ideologies
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
The political blogosphere and the 2004 U.S. election:
divided they blog
by Lada Adamic, Natalie Glance
Social media, political change and the #RapplerDebate
by Maria A. Ressa and Russell Shepherd

7. How does one find the
Communities?
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

8. Zachary’s karate club
34 members of a karate club
78 pairwise links between members who
interacted outside the club
“Mr. Hi”
“John A”

9. Methods
Agglomera]ve Divisive
Cut 3 classes
Cut 4 classes
Cut 6 classes
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

10. Modularity
• Measure the quality of a partition
“the fraction of edges that fall within communities, minus the
expected value of the same quantity if edges fall at random
without regard for the community structure”
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
Mc =
ncX
c=1
"
Lc
L
✓
kc
2L
◆2
#

11. Modularity
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
Mc =
ncX
c=1
"
Lc
L
✓
kc
2L
◆2
#
Number of
communities
Total number
of links in
Community c
Total node
degrees in
Community c
Total number of
links

12. Modularity
Community Structures | EF Legara | 2017 NTU Winter School on Complexity Science
Mc =
ncX
c=1
"
Lc
L
✓
kc
2L
◆2
#
Number of
communities
Total number
of links in
Community c
Total node
degrees in
Community c
Total number of
links
1 2
3
Community 1:
"
6
20
✓
14
40
◆2
#
"
4
20
✓
10
40
◆2
#
Community 3:
Mc = 0.50
Community 2:
"
7
20
✓
16
40
◆2
#

13. Modularity
Community Structures | EF Legara | 2017 NTU Winter School on Complexity Science
Mc =
ncX
c=1
"
Lc
L
✓
kc
2L
◆2
#
Number of
communities
Total number
of links in
Community c
Total node
degrees in
Community c
Total number of
links
1 2
3
Mc = 0.35
Community 1:
"
4
20
✓
10
40
◆2
#
Community 3:
"
2
20
✓
8
40
◆2
#
Community 2:
"
9
20
✓
22
40
◆2
#

14. Modularity
Community Structures | EF Legara | 2017 NTU Winter School on Complexity Science
Mc =
ncX
c=1
"
Lc
L
✓
kc
2L
◆2
#
Number of
communities
Total number
of links in
Community c
Total node
degrees in
Community c
Total number of
links
M = 0

15. 1 2
3
Mc = 0.50
M = 0
1 2
3
Mc = 0.35
Optimal Partition Suboptimal Partition
One Community
Modularity
Mc =
ncX
c=1
"
Lc
L
✓
kc
2L
◆2
#
Community Structures | EF Legara | 2017 NTU Winter School on Complexity Science

16. Quantifying Community Structures
Divisive
• Girvan-Newman Algorithm (edge betweenness)
Agglomerative
• Hierarchical clustering (distance matrix)
• Louvain Method (modularity-maximization)
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
Cut 3 classes
Cut 4 classes
Cut 6 classes

17. Community Detection in Framing Analysis
Framing - the “central organizing idea for
making sense of an issue or conflict and
suggesting what is at stake”
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
EF Legara, C Monterola, C David, and JM Atun, "News Framing of
Population and Family Planning Issues via Syntactic Network
Analysis", Intl J of Mod Phys C 21, Issue: 1, pp. 51-65 (2010).
A Network of Words

18. “Frames have been shown experimentally to cause differences
in attributions of responsibility among readers or audiences.”
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

19. • 346 news articles (randomly sampled)
• Population, family planning, reproductive health, and
contraception published between 1987 and 2007
Framing in the News
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

20. Every article assigned to
a single dominant frame
q Population and Development
q Family Planning as Conflict between
Government and Church
q Women’s and Reproductive Health
q Population Management Threatens
Morals and Values
q Population Growth and
Demographic Trends
q Others
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

21. Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
Collected Articles
Delete stop words and replace
with placeholder “xxx”
Generalize conceptually
similar terms
Obtain high-frequency words
A Network of Words

22. Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
Map words: add edge weight
(1u) between nodes if words co-
exist in a scanning window
Create co-occurrence
network
Perform cluster analysis

23. Co-Occurrence
Network of
Words
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

24. F1: Popula]on and Development
F2: Popula]on Growth and
Demographic Trends
F3: Women’s and Reproduc]ve Health
F4: Family Planning as Conflict
between Government and Church
F5: Popula]on Management
Threatens Morals and Values
F6: Others
Development Frame
Maternal Health Frame
Framing by the Church
F1: Popula]on and Development
F2: Popula]on Growth and
Demographic Trends
F3: Women’s and Reproduc]ve Health
F4: Family Planning as Conflict
between Government and Church
F5: Popula]on Management
Threatens Morals and Values
F6: Others
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science
Coding done by communica*on
research experts

25. http://tinyurl.com/NetStruc2016
http://tiny.cc/NetStruc2016
Community Structures | EF Legara | 2016 NTU Winter School on Complexity Science

26. Community Structures
Erika Fille T. Legara, Ph.D.
@eflegara
Scien&st, IHPC-A*STAR
http://tiny.cc/commdet2017
http://tinyurl.com/commdet2017