This talk was delivered by Mostafa Zien, a master student at SRGE in the monthly workshop of the SRGE that held on Saturday 19 Sept 2015 at the Conf. center of Cairo University

1. The Circles of Relations Modifiers Modeller from
Smartphone Photo Gallery
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Moustafa Zein
http://www.egyptscience.net
1

2. Scientific Research Group in Egypt
www.egyptscience.net
2
SRGE workshop in Cairo University Conference Hall (19-September-2015)

3. Overview
 Introduction
 Problem Definition
 Motivation
 Related Work
 Proposed Approaches
 Identifying Circles of Relations
 Relationship Modifiers Modeller
 Results and Discussion
 Conclusion and Future Works
SRGE workshop in Cairo University Conference Hall (12-September-2015)
3

4. Introduction
SRGE workshop in Cairo University Conference Hall (12-September-2015)
4

5. Introduction
• Expanding number of different social devices such as smartphones
offers metadata for dissecting human interaction and support
psychology and human behavior studies
• Smartphone data resources are (recording GPS logs, calls,
messages, and applications logs) and recently added Geotagged
photos.
• Recently, Geotagged photos used to study tourist travel behaviours to
improve travel planning and recommendations for tourists.
• Geotagged photos have background information about the userSRGE workshop in Cairo University Conference Hall (12-September-2015)
5

6. Problem Definition
• Extracting people social relations and interests are done with mobile
data (calls, SMS, BT, infrared, Wi-Fi, GPS sensor logs, and text
content).
• These data resources lack background information about the
surrounding environment about the owner of the photo.
• Identifying circles of social relations may need to a model to deals
with relation factors such as (location, date, time, and distance)and
modifiers.
• The relation modifiers represent relation attitude and change. Trust,SRGE workshop in Cairo University Conference Hall (12-September-2015)
6

7. Motivation
• Identifying the circles of relations from smartphone photo gallery and
There is no any mention about extract circles of relations from
Geotagged photo.
• Predicting attitude and change of social relationships represents a
core issue of many psychological and economics studies.
• Concentrating on mining degree of relations for smartphone users
with closest people, places where is visited, and interests.
• Modeling the circles of relations according to relation modifiers and
factors to get a real and accurate representation for humanSRGE workshop in Cairo University Conference Hall (12-September-2015)
7

8. Related Work
SRGE workshop in Cairo University Conference Hall (12-September-2015)
8

9. Related Work
• Most of studies focus on Geotagged photo logs to build places
recommendation systems to make effective target organization and
transportation plans.
• Few studies introduced an GPS logs as a feature with another features
(e.g. call logs, SMS logs )to identify human relationship.
• There are studies modeled social relations based on social network and
trust modifier
• There is a study presented an analysis of trust propagation based on
the notion of transitivity from mobile social network.SRGE workshop in Cairo University Conference Hall (12-September-2015)
9

10. Proposed
Approaches
SRGE workshop in Cairo University Conference Hall (12-September-2015)
10

11. Identifying Circles of Relations
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Fig.1: The Algorithmic form for Circles of relations identification framework.
11

12. Identifying Circles of Relations Cont.
• Data collection:
• The dataset is collected from photo galleries of several Arabic
participants.
• We asked every participant to choose around (150) photo or more
from photo gallery over 6 months. The dataset consist of 1432
Geotagged photosParticipant
ID
Photo ID Time Taken Date
Taken
Geolocation
Latitude Longitude
001 001-1 12:57:30 2014:10:21 29.98563 31.25127
001-2 22:05:08 2014:10:31 30.86151 30.58107
002 002-1 13:13:37 2014:09:07 30.01689 31.37703
Table 1. Participant's Geotagged Photos Metadata.
SRGE workshop in Cairo University Conference Hall (12-September-2015)
12

13. Relationship Modifiers Modeller
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Fig.2: The Algorithmic form for Circles of relations modifiers modeller.
13

14. Relationship Modifiers Modeller
• Degree of relation:
o Is classified to three level of relation (high class, medium class, low class).
o Equation 1 showed the positive relationship between importance of relation and
(frequency of visiting and the distance) or between relation and frequency of
visiting only.
o Equation 1:
𝐼 ∝ 𝐷 & 𝐹 | 𝐼 ∝ 𝐹
where (I) represents importance of relation, (D) represents distance and
(F) represents Frequency.
• Transitivity:
o Is a parallel connection in which if entity (A) is identified with entity (B) and entity
(B) is identified with entity (C), then (A) is identified with entity (C) as in equation
2.
o Equation 2:
∀ A, B, C ∈ X : (ARB ∩ BRC) → ARC R is a relation
SRGE workshop in Cairo University Conference Hall (12-September-2015)
14

15. Relationship Modifiers Modeller
• Transitivity:
o Transitivity refers to the probability of a relationship between two faces in same photo from
participant photo gallery.
o Equation 3 represented the transitivity relation mathematically. In equation 3, A, B, and C
represent faces in photos, X1 represents the same photo, and X2 represents the same
location, where the faces must appear in the same photo and may be in different locations or
the same location.
o Equation 3:
∀ A, B, C ∈ X1|| X2 : (ARB ∩ BRC) → ARC
• Decay or Lifetime:
o Equation 4: 𝑡ℎ =
𝑡=1
𝑚 𝑁 𝑡
𝑚
𝑛 where (𝑡ℎ) represents threshold of continuity, (m) represents
number of months, (N) refers total number of photo per month, and (n) represents the total
number of photos.
SRGE workshop in Cairo University Conference Hall (12-September-2015
15

16. Relationship Modifiers Modeller
• Trust:
o Trust is the measure of conviction that a given entity will go about as one anticipates
o There is interference between trust modifier and decay modifier, which is a negative
relationship between them. If any relation is passed from the decay modifier, this relation
classified based on the trust modifier.
o To calculate trust modifier, some parameters will be provided from the decay modifier.
Parameters such as minimum and maximum thresholds which are passed from decay
modifiers.
o In Equation 5, (𝑝𝑡𝑟𝑢𝑠𝑡) is the percentage of trust of relationship, (𝑡ℎ 𝑚𝑖𝑛, 𝑡ℎ 𝑚𝑎𝑥) represent
minimum and maximum thresholds, that are taken from decay modifier, and (𝑡ℎ 𝑐𝑢𝑟𝑟𝑒𝑛𝑡) refers
to current threshold of a relation
o Equation 5:
𝑝𝑡𝑟𝑢𝑠𝑡 = 𝑡ℎ 𝑚𝑎𝑥 −𝑡ℎ 𝑚𝑖𝑛
𝑡ℎ 𝑚𝑎𝑥 −𝑡ℎ 𝑐𝑢𝑟𝑟𝑒𝑛𝑡
SRGE workshop in Cairo University Conference Hall (12-September-2015
16

17. Results and
Discussion
SRGE workshop in Cairo University Conference Hall (12-September-2015)
17

18. Results and Discussion
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Fig. 3. Face and interest detection stage.
18

19. Results and Discussion Cont.
• The Communication Patterns:
SRGE workshop in Cairo University Conference Hall (12-September-2015)
ID Intensity
Regularity
photo per week
Temporal tendency Maintenance
In the
Morning Afternoon Weekend Workday
Last month
meta_1 21 5 8 13 18 3 15
meta_2 3 1 0 3 3 0 3
meta_3 3 1 0 3 3 0 1
meta_4 5 1 5 0 5 0 0
meta_5 4 2 4 0 0 4 0
Table 2. The factors of communication patterns are applied on photo gallery to a participant.
19

20. Results and Discussion Cont.
• Clustering the circles of relations:
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Faces Places Interests
Node ID Weight
Cluster
Number Node ID Weight Cluster Number Node ID Weight Cluster Number
1 0.9998 1 1 0.9098 1 1 0.7811 1
2 0.6871 2 2 0.6470 2 2 0.6141 2
3 0.7261 2 3 0.8061 2 3 0.7661 2
4 0.596 3 4 0.596 3 4 0.596 3
Table 3. The relation weights assigned to all places, interests, and faces from one of the photo galleries.
20

21. Results and Discussion Cont.
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Fig. 5. Frequency of appearance to a face in a participant photo gallery.
21

22. Results and Discussion Cont.
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Table. 4. Degree classes values and ranked according to the furthest distance.
22
Low class Medium class High class
Face Interest Place Face Interest Place Face Interest Place
0.2134 0.0212 0.0223 0.6433 0.4155 0.5919 0.8473 0.9873 0.8309
0.3221 0.2342 0.1399 0.4997 0.5991 0.4542 0.7300 0.8117 0.8519
0.1222 0.3437 0.0432 0.4002 0.6332 0.6647 0.9421 0.9109 0.8354
0.1888 0.1637 0.0449 0.7125 0.6918 0.7736 0.6303 0.9997 0.1029
0.0998 0.0089 0.0056 0.5141 0.6023 0.6068 0.9052 0.8422 0.9919
0.0598 0.3914 0.596 0.7087 0.7322 0.5888 0.8303 0.8918 0.9148

23. Results and Discussion Cont.
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Table. 5. A sample decay threshold, the probability of passed months of threshold, and
percentage of relation trust.
23
Mate ID Threshold Passed months per total Status Trust Percentage
Mate 1 0.532 0.60 passed 50.3%
Mate 2 0.433 0.20 decay -
Mate 3 0.292 0.30 decay -
Mate 4 0.654 0.78 passed 87.5%
Mate 5 0.492 0.64 passed 67.1%
Mate 6 0.419 0.76 decay -
Mate 8 0.536 0.59 passed 74.9%
Mate 7 0.129 0.77 decay -

24. Results and Discussion Cont.
SRGE workshop in Cairo University Conference Hall (12-September-2015)
Fig. 6. Circles of relations of faces for participant (1) highest degree class, (2) medium degree class, and
lowest degree class. All of faces positions are sorted according to distance factor.
24

25. Conclusion And Future
Work
SRGE workshop in Cairo University Conference Hall (12-September-2015)
25

26. Conclusion And Future Work
SRGE workshop in Cairo University Conference Hall (12-September-2015)
26
• Our solution is based on a framework to extract the circles of relations.
• The first step of the framework is to extract the information about
favorite places, time, and date.
• The second step of the framework is detecting social interests and
faces from photo content.
• At the third step in the framework, the factors of communication
patterns and Fuzzy C Mean are used to assign degree of relations.
• A relation modifiers modeller is presented to process the circles of
relations for any smartphone user.
• A relation modifiers modeller has mainly two parts. The first part is
processing the inputs to model, which takes framework output, and
photo details.

27. Conclusion And Future Work
SRGE workshop in Cairo University Conference Hall (12-September-2015)
27
• At the same part, relation factors are extracted from photo metadata. In
the second part, degree class, and the relation factors entered to the
rest of relation modifiers.
• The output of applying step of the relation modifiers is circles of
relations with degree classes and outputs of every modifier.
• The accuracy of the model is distributed on decay with accuracy 70%
and accuracy of trust modifier reached to 98%.
• As a future work, we look forward to following face expressions on
photos to predict participant moods with relation attitudes.
• predict the social life facets for smartphone users from smartphone
gallery.

28. Publications
• Moustafa Zein, Fatma Yakoub, Ammar Adl, Aboul Ella Hassanien, Vaclav Snase.
“Identifying Circles of Relations from Smartphone Photo Gallery”, International
Conference on Communication, Management and Information Technology (ICCMIT
2015), Procedia Computer Science, 2015.
• Moustafa Zein, Ammar Adl, Amr Badr, Aboul Ella Hassanien, Tai-Hoon Kim, “A
Social Relationship Modifiers Modeller”, CIA 2015 (2015 3rd IEEE International
Conference on Computer, Information and Application (CIA 2015), 21-23 September
2015 in Yeosoo, Yeosoo, South Korea).
28
SRGE workshop in Cairo University Conference Hall (12-SeptemSeptmper 5)

29. Thanks and Acknowledgement
Authors: Moustafa Zein, Fatma Yakoub, Ammar Adl, Aboul
Ella Hassanien, Ammer Badr and Vaclav Snaseld
29
SRGE workshop in Cairo University Conference Hall (12-September-2015)