Personal Understanding & Enhancement from Emerging forms of Personal Data

1. “We will all have Human
Ledgers in the Future”
Associate Professor & Asst HoS at the School of Computing, Dublin City University, Ireland
Principal co-Investigator at the Insight Centre for Data Analytics
(@cathal - cathal@gmail.com)
Cathal Gurrin
Researcher
Personal Data
Analytics and e-health
Educator
Information Retrieval
and Data Analytics
Lifelogger
A decade of visual
lifelogging experience
DCU - 18th May 2018
”Personal Understanding & Enhancement from Emerging forms of Personal Data”

2. In 2006 I put on a wearable camera. 12 years and billions of data
points later, I am still doing it….. WHY?

3. We leave trails of data
all the time now….

4. But we as individuals are beginning to capture our own data archives …
… the Human Ledger…

5. 1928
Bucky-
Fuller
1980s
Steve
Mann
2004
Williams
(Sensecam)
2015
Early
Products
2006+
Memory
Studies
2004
G. Bell
(MyLifeBits)
1946
Vannevar
Bush

6. “If somebody kept a very accurate
record of a human being … I decided
to make myself a good case history of
such a human being and it meant that
I could not be judge of what was valid
to put in or not. I must put everything
in, so I started a very rigorous record”
A Physical Ledger
Richard Buckminster Fuller

7. “A record if it is to be useful to science, must be
continuously extended, it must be stored, and
above all it must be consulted… Consider a
future device for individual use, which is a sort
of mechanized private file and library… a device
in which an individual stores all his books,
records, and communications, and which is
mechanized so that it may be consulted with
exceeding speed and flexibility. It is an
enlarged intimate supplement to his memory.”
Memex
Vannevar Bush

8. “Sensors can also log the three billion
or so heartbeats in a person's lifetime,
along with other physiological
indicators, and warn of a possible
heart attack... spot irregularities early,
providing warnings before an illness
becomes serious. Your physician
would have access to a detailed,
ongoing health record, and you would
no longer have to rack your brain to
answer questions such as "When did
you first feel this way?”
My Life Bits
Gordon Bell

9. “What I argue is that if I'm going to be
held accountable for my actions that I
should be allowed to record... my
actions. Especially if somebody else is
keeping a record of my actions”
Father of Wearable Computing
Steve Mann

10. This is all kind of mad? Well, lets have a look…

11. Lifelogging refers to the process of storing data concerning the life
activities of an individual for future use, human ledgers & self-search
And it puts the ledger in control of the individual…

12. Human Ledger
Life Experience
The individual will have a human ledger &
personal search engine for all life
experience… activities, experiences,
behaviours, information, biometrics… huge
volumes of data captured passively.

13. Quantified Self
Personal Insights
Data-driven Health
Augmented
Wellness
Behaviour Change
Enhanced Security
Population-wide
Analytics
Augmented
Community
Augmenting Human
Memory
Nomenclators
Augmented
Memory
Enhanced Productivity
& Education
Enhanced Interactions
Rich Sharing and
Reminiscing
Augmented
Cognition
Many (Individual) Use Cases
Health & Wellbeing Memory & Cognition

14. Focus on supporting knowledge
acquisition and learning in the
early years.
1. Knowledge Support
From education to the workplace,
providing information and
insights to assist productivity and
fitness.
2. Productivity
Into old age, providing support
for cognition and health to
maintain independence and
activity.
3. Health
CHILD
ADULT
ELDERLY
In reality, we can’t yet imagine the use-cases of human ledgers,
but it could become a permanent companion assisting you
throughout life. Constantly growing in size.

15. Potential for Enhanced Health & Productivity
New Forms of Epidemiological Study
New Pedagogic Methodologies
New Challenges & Concerns
Additional benefits come when we examine populations (Horizontal)

16. So… what types of data are becoming available?
Driven by low-cost sensors, massive volumes of data are being created
and years of data can fit on a $100 hard disk.

17. QS
Quantified Self

18. QS
Quantified Self

19. Rich Sensing

20. Google Glass, Snapchat Spectacles,
ion SnapCam, Narrative Clip, GoPro,
Sony Xperia Eye, etc… capturing
experiences not just biometrics…

21. Writing Content
Reading Content
Device Interactions
Loggerman: Privacy-
aware
HCI and information
logging
www.loggerman.org
Z. Hinbarji, R. Albatal, N. O’Connor and C. Gurrin (2016) LoggerMan,
a comprehensive logging and visualisation tool to capture computer
usage. In: 22st International Conference on MultiMedia Modelling
(MMM 2016), 4-6 Jan, 2016, Miami, FL

22. Using Sony Digital Paper or a digital pen
(EchoPen, LiveScribe), you can create and
annotate digital PDFs
Writing Content
Reading Content

23. Rich multi-modal sensing of life experiences
Biometrics
Locations
Speech
Hearing
Vision
Thought/Attention
Activities
Information Needs
Media Consumption
Writing Content
Reading Content
Media Creation
Device Interactions
Productivity
Social Network Postings
Environmental Data
Profile in other’s lifelogs

24. Guided by the idea that you can never
capture what has already passed

25. It still fits on my custom server (about 1TB per year)

26. ACTIVITIES
Using sensors to
understand detailed
activities of daily life of the
individual
HEALTH
New sensors and apps
allow us to know the
health status of the
individual in next
generation EHRs
COMMUNICATION
Understand and index the
entire communication
history of the individual
LIFE EXPERIENCE
Lifelogging tools allow
us to experience life as
the individual
experiences it
Augmented
Human
Augmented Privacy
Augmented Cognition
Augmented Performance
Augmented Wellness

27. So… What do first generation human ledgers look like?

28. Visual Diary (DCU - 2006)
A Doherty, C Ó Conaire, M Blighe,A.F. Smeaton, N.E. O’Connor (2008) Combining image descriptors to effectively retrieve events from visual lifelogs. In: MIR
2008 - ACM International Conference on Multimedia Information Retrieval, 30-31 October,Vancouver, Canada.

29. Life Abstraction
(objects, people, products)

30. Linking Multiple Data Sources - images to stress levels

31. KidsCam (Univ. Otago & DCU) extracted knowledge for
ethnographic study (2015)…
Wearable cameras were given to 200 school children - to understand their exposure to fast-
food advertising.

32. In a Store In the Home
In Daily Life In the School

33. !
!
Population-wide Analytics

34. In Summary:
Individuals are beginning to capture our own human ledgers.
These will be goal-driven with positive benefits for the individual.
Future generations will have a species-level understanding of humanity.
Human ledgers will outlive us… who will be the custodians?
I never mentioned privacy or sousveillance.

35. Thank You
Associate Professor at the School of Computing, Dublin City University, Ireland
Principal Investigator at the Insight Centre for Data Analytics
(@cathal - cathal@gmail.com - http://about.me/cgurrin)
Cathal Gurrin
LifeLogging: Personal Big Data
Cathal Gurrin,Alan F. Smeaton,Aiden R. Doherty
Published: 16 June 2014
Do a google search and download the book from the DCU website.