Lecture in memory of Prof. Aya Cohen at Technion's Faculty of Industrial Engineering & Management, August 4, 2019

1. Behavioral Big Data
& Healthcare
Research
‫של‬ ‫לזכרה‬ ‫הרצאה‬‫כהן‬ ‫אילה‬ ‫פרופ׳‬
‫ולרבים‬ ‫רבות‬ ‫ותרמה‬ ‫שהובילה‬
‫כסטטיסטיקאית‬,‫חוקרת‬,‫מרצה‬,‫מנחה‬,‫ומנטורית‬
‫וניהול‬ ‫תעשיה‬ ‫להנדסת‬ ‫הפקולטה‬,‫הטכניון‬4.8.2019

2. New Data
Landscape
Researchers
2 Examples

3. What is Behavioral Big Data (BBD)
Special type of Big Data
• Behavioral: people’s measurable
“everyday” behavior,
interactions, self-reported
opinions, thoughts, feelings
• Human and social aspects:
Intentions, deception,
emotion, reciprocation,
herding,…
When aware of data collection ->
modified behavior

4. BBD vs. Inanimate Big Data
Human Subjects
• Aware, ongoing
interaction with the data
• Can be harmed by BBD

5. BBD vs.
Physiological
Big Data
• Individual bodies
• Physical measurements
• Medical systems set
data collection timing
• Aware of collection
• Vested interest
• Collection of connected people
• Measurable behaviors
• User generated content
• Often unaware of collection
• Not always in user’s best
interest

6. ‫ביומטריים‬ ‫חיישנים‬ ‫עליך‬ ‫לובש‬ ‫אתה‬ ‫אם‬(‫צמיד‬ ‫כגון‬‫פיטביט‬),
‫למחשב‬ ‫מחוברים‬ ‫הללו‬ ‫והחיישנים‬,‫המחשב‬‫מהו‬ ‫בדיוק‬ ‫ידע‬
‫שלך‬ ‫הלב‬ ‫קצב‬,‫שלך‬ ‫האדרנלין‬ ‫ורמת‬ ‫שלך‬ ‫הדם‬ ‫לחץ‬,‫ועל‬
‫הזה‬ ‫המידע‬ ‫סמך‬‫טוב‬ ‫שלך‬ ‫הרגשי‬ ‫המצב‬ ‫את‬ ‫לזהות‬ ‫יוכל‬
‫אנושי‬ ‫פסיכולוג‬ ‫מכל‬ ‫יותר‬

7. Behavioral
Research
Health
Research

8. Landscape of
new healthcare
BBD

9. Human data from a typical hospital
Patients
Personal info
Medical history
visits, tests, medications,…
Scheduled events
Billing, insurance
Physicians
Scheduled + actual appointments,
procedures, prescriptions,…
Entries of patient info/data
Nurses
Location, work hours,…
Pharmacy staff
Speed of service
Quality of service
Lab staff
Speed of service
Quality of service
Other staff
Finance/accounting
Cleaning
Receptionists
Volunteers
Food court
Data Collection
Technologies
• Medical devices
• HIT systems
“Smart Hospital”
• Cameras
• Sensors
• GPS
• IoT

10. 5 new types of healthcare BBD

11. Interactions between
Patients – doctors/nurses
Doctors – other doctors
Patients – other patients
Patient family – hospital staff
Patients – social network ”friends”
...
New data #1:
Recorded Interactions

12. BBD:
• 90,000 doctor-patient conversations
during clinical visits
• 151 types of medical visits for
different purposes
• conversation sometimes also
including a nurse, or family member

13. New data #2:
Health-related online behavior

14. Online forums Social networksHealth websites
Search engines
Data voluntarily entered by users
personal details, photos, comments, messages, search terms, likes,
payment information, connections with “friends”
Passive footprints
duration on website, pages browsed, sequence, referring website,
Internet browser, operating system, location, IP address

15. New data #3:
Health-related Apps

16. Health-related gaming apps

17. Self-logged BBD on apps
Data voluntarily entered by users
health condition, symptoms, feelings, behaviors
(eating, exercise, sleep, sex, parking…)
Passive footprints
app log times, pages browsed, sequence, location…

18. In addition to logging a menstruation and health diary, users can
join a number of different themed groups including weight loss,
clothing, fitness, relationships, and travel. These groups
look and work much like “message board”-style social network
To date, Meet You has reportedly accumulated two million daily
active users, 1.2 million daily active users of its social network,
and over 800,000 daily posts.
Big and Behavioral:
Every day, women manually log around 1.4 M new data points
including cycle history, ovulation and pregnancy tests results, age,
height, weight, lifestyle statistics about sleep, activity, and nutrition.
In addition, more data comes from wearable devices like Fitbit &
Apple Watch.

20. New data #4:
Health-related behavior from IoT

21. Mobile health apps and wearable devices
that use artificial intelligence to help
diagnose or even treat medical conditions
pose a new regulatory challenge for the
U.S. Food and Drug Administration

22. New data #5:
Health-“unrelated” (implicit) behaviors

23. “Some hospitals are collecting new information
from patients directly, while others have sought
data from companies that sell consumer and
financial information, or federal agencies that
provide statistics on poverty, housing density
and unemployment”

24. “Quantified self”
devices also collect…

25. Subjects underwent a standardized
neurocognitive assessment, then went home
with an app that measured the ways they
touched their phone’s display (swipes, taps,
and keyboard typing)
Mindstrong conducted studies to figure
out whether there might be a systemic
measure of cognitive ability—or
disability—hidden in how we use our
phones.
memory problems… can be spotted by looking at
things including how rapidly you type and what errors
you make (such as how frequently you delete
characters), as well as by how fast you scroll down a
list of contacts

26. PRIVACY:
“while Mindstrong says it protects users’
data, collecting such data at all could be
a scary prospect for many of the people it
aims to help.
Companies may be interested in, say,
including it as part of an employee
wellness plan, but most of us wouldn’t
want our employers anywhere near our
mental health data”

27. This is where it becomes
ethically challenging:
Who’s collecting the data
and for what purpose?
Are users aware of the data collection
and usage?
What are users’ benefits & risks
from sharing their data?

28. What we’ve learned… is that
we need to take a more
proactive role in a broader
view of our responsibility. It’s
not enough to just build tools,
we need to make sure that
they’re used for good
What we’ve learned… is that
we need to take a more
proactive role in a broader
view of our responsibility. It’s
not enough to just build tools,
we need to make sure that
they’re used for good

29. Medical data privacy is regulated.
What about BBD?

30. Recent data privacy regulations is
reshaping the collection & use of BBD

31. Researchers Using Health-Related BBD

32. Research Fields Using Health-Related BBD
Operations Researchers and Industrial Engineers
For: Hospital Management and Operations
(staffing, scheduling,…)
Medical/Healthcare Researchers & Clinicians
For: Improved Medical Treatment
(safety, effectiveness,…)
Information Systems Researchers
For: Improved Design & Use of Medical IS
(value of IS, effectiveness, standardization,…)
Marketing
Advertising
Insurance
Machine Learning
Social Science

33. How Do Researchers Get
Health BBD?
1. Open/Publicly Available Data
Constantly refreshed or single data dump
API, web scraping
Hacked data
2. Partner with Company/Organization
• Both parties interested in research question
• Data purchase
• Personal connections, sabbaticals, internships
• Partnership between school and organization
• Third party (WCAI)
3. Crowdsourcing
4. China (!)

34. Research Using New Health BBD: Challenges
Behavioral
Big Data
Researcher
Human
Subjects
Research
Question
Scientific vs.
Clinical vs.
Commercial
Explain
vs.
Predict
Different (conflicting) Goals:
Unit of analysis vs.
Unit of measurement
Under/over-coverage
New risks (privacy, liability,
security, HIPAA compliance)
New ethical challenges:
Generalization Challenges:
Acquire + analyze data
Users (self-selection,
spill-over, knowledge of
allocation, network)
Company algorithms
Data contaminated by:New modes of connection &
information (social networks,
forums, IoT, Apps)
Overall vs.
Individual
effect
Technical expertise
larger distance
Old Q, new data: Operationalize new variables
New Q: Lack of literature

35. 2
Examples

36. Two examples of high-profile studies
using new health BBD
Emotional contagion in
social networks
Kramer et al. (PNAS, 2014)
Detecting influenza epidemics
using search engine query data
Ginsberg et al. (Nature, 2009)

37. Example #1

38. • No Ethics Board Review (IRB)
“[The work] was consistent with Facebook’s Data
Use Policy, to which all users agree prior to
creating an account on Facebook, constituting
informed consent for this research.”
• PNAS editorial Expression of Concern
• Varied response from public, academia, press,
ethicists, corporates
Where do data scientists get ethics training?

39. Example #2
• “Up-to-date influenza estimates may enable
public health officials and health professional to
better respond to seasonal epidemics”
• BBD: automated search results for 50M
keywords on Google.com (2003-2007). For each
query: {query text, IP address}
• Fit 450M different models, correlating each
query text with CDC data; Combined 45 queries
with highest correlation

40. Researchers: epidemiologists + data science academics
Dalton et al. )2016(, “Flutracking weekly online community
survey of influenza-like illness annual report, 2015”
Communicable diseases intelligence quarterly report
Challenge: Acquire data

41. • Algorithm detects “flu” or “winter”?
• Persistent over-estimation
• Performs worse than lagged CDC
3-week-old data
• Never released 45 terms used
• Lazer et al. recommend combining/
calibrating GFT with CDC data
But most importantly…

42. Changes made by Google’s search
algorithm to display potential
diagnoses + recommend search for
treatment (more advertising)
-> increased search

43. This type of BBD research is still popular

44. Uses Google searches to measure sensitive
behaviors/opinions/thoughts on
racism, self-induced abortion, depression,
child abuse, hateful mobs, the science of
humor, sexual preference, anxiety, son
preference, and sexual insecurity, among
many other topics.

46. New healthcare BBD offers new research opportunities

47. … and new challenges
Behavioral
Big Data
Researcher
Human
Subjects
Research
Question
Scientific vs.
Clinical vs.
Commercial
Explain
vs.
Predict
Different (conflicting) Goals:
Unit of analysis vs.
Unit of measurement
Under/over-coverage
New risks (privacy, liability,
security, HIPAA compliance)
New ethical challenges:
Generalization Challenges:
Acquire + analyze data
Users (self-selection,
spill-over, knowledge of
allocation, network)
Company algorithms
Data contaminated by:New modes of connection &
information (social networks,
forums, IoT, Apps)
Overall vs.
Individual
effect
Technical expertise
Old Q, new data: Operationalize new variables
New Q: Lack of literature

48. Behavioral Big Data
& Healthcare
Research
‫של‬ ‫לזכרה‬ ‫הרצאה‬‫כהן‬ ‫אילה‬ ‫פרופ׳‬
‫ולרבים‬ ‫רבות‬ ‫ותרמה‬ ‫שהובילה‬
‫כסטטיסטיקאית‬,‫חוקרת‬,‫מרצה‬,‫מנחה‬,‫ומנטורית‬
‫וניהול‬ ‫תעשיה‬ ‫להנדסת‬ ‫הפקולטה‬,‫הטכניון‬4.8.2019

49. • Greene, Shmueli, Ray & Fell (2019), Adjusting to the GDPR: The Impact on Data Scientists and
Behavioral Researchers, Big Data, forthcoming
• Shmueli (2017), Research Dilemmas With Behavioral Big Data, Big Data, vol 5 issue 2, pp. 98-119
• Shmueli (2017), Analyzing Behavioral Big Data: Methodological, Practical, Ethical, and Moral Issues,
with discussion and rejoinder, Quality Engineering, vol 29 no 1, pp. 57-74 and 88-90.