4. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
5. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
6. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
7. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
8. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
9. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
When would you like to be reminded?
Alright, remind you to do the laundry
at 12:00pm on Sunday, is that right?
Great, I’ll remind you!
Sunday at noon.
Remind me to do the laundry.
Yes.
11. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4
Research questions
1. Is there a body of common tasks that underlie the reminder
creation process?
12. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4
Research questions
1. Is there a body of common tasks that underlie the reminder
creation process?
2. Can we identify patterns in the times at which people create
reminders, and, via notification times, when the associated
tasks are to be executed?
13. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4
Research questions
1. Is there a body of common tasks that underlie the reminder
creation process?
2. Can we identify patterns in the times at which people create
reminders, and, via notification times, when the associated
tasks are to be executed?
3. Can we predict when certain tasks are most likely to happen?
14. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5
Is there a body of common tasks that
underlie the reminder creation process?
15. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5
Is there a body of common tasks that
underlie the reminder creation process?
• Common reminder structure:
• “Remind me to <verb> <object>”
• take out laundry
• call mom
• pay bills
16. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5
Is there a body of common tasks that
underlie the reminder creation process?
• Common reminder structure:
• “Remind me to <verb> <object>”
• take out laundry
• call mom
• pay bills
• Identify frequent verb/object-pairs in 2-month sample
(~950k reminders)
17. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5
Is there a body of common tasks that
underlie the reminder creation process?
• Common reminder structure:
• “Remind me to <verb> <object>”
• take out laundry
• call mom
• pay bills
• Identify frequent verb/object-pairs in 2-month sample
(~950k reminders)
• Manually group them into tasks
18. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 6
~60% of reminders start with one of
these (52) verbs
take
eat
have
send
email
text
call
tell
set
make
schedule
turn on
turn off
check
leave
come
be
go to
stop by
take out
feed
clean
wash
charge
do
write
change
cancel
order
renew
book
mail
submit
fill out
print
pay
wake
set
19. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 7
Manual labeling
Ò Dimensions;
Ò Interruption vs. continuation of user activity
Ò Context of task (home, work, on the go)
Ò (Expected) duration of task
Ò Impact on user “availability”
Ò …
20. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8
common tasks that underlie the reminder
creation process
21. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8
Go somewhere Switch context Run errand
common tasks that underlie the reminder
creation process
22. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8
Go somewhere Switch context Run errand
common tasks that underlie the reminder
creation process
Go somewhere
Chores
Switch context Run errand
Recurring Standalone
23. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8
Go somewhere Switch context Run errand
common tasks that underlie the reminder
creation process
Go somewhere
Chores
Switch context Run errand
Recurring Standalone
Go somewhere
Chores
Communicate
Switch context Run errand
Recurring Standalone
General Coordinate
24. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8
Go somewhere Switch context Run errand
common tasks that underlie the reminder
creation process
Go somewhere
Chores
Switch context Run errand
Recurring Standalone
Go somewhere
Chores
Communicate
Switch context Run errand
Recurring Standalone
General Coordinate
Go somewhere
Chores
Communicate
Manage Ongoing
External Process
Manage Ongoing
User Activity
Eat/Consume
Switch context Run errand
Recurring Standalone
General Coordinate
Start
Prepare Stop
25. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8
Go somewhere Switch context Run errand
common tasks that underlie the reminder
creation process
Go somewhere
Chores
Switch context Run errand
Recurring Standalone
Go somewhere
Chores
Communicate
Switch context Run errand
Recurring Standalone
General Coordinate
Go somewhere
Chores
Communicate
Manage Ongoing
External Process
Manage Ongoing
User Activity
Eat/Consume
Switch context Run errand
Recurring Standalone
General Coordinate
Start
Prepare Stop
26. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9
RQ2: Can we identify patterns in the
reminder creation process?
27. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9
RQ2: Can we identify patterns in the
reminder creation process?
Ò Creation time (CT)
Ò When user creates the reminder (remembers task)
28. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9
RQ2: Can we identify patterns in the
reminder creation process?
Ò Creation time (CT)
Ò When user creates the reminder (remembers task)
Ò Notification time (NT)
Ò When reminder is set to trigger (executes task)
29. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9
RQ2: Can we identify patterns in the
reminder creation process?
Ò Creation time (CT)
Ò When user creates the reminder (remembers task)
Ò Notification time (NT)
Ò When reminder is set to trigger (executes task)
Ò Time delta (NT – CT)
Ò How far task is planned in advance
30. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9
RQ2: Can we identify patterns in the
reminder creation process?
Ò Creation time (CT)
Ò When user creates the reminder (remembers task)
Ò Notification time (NT)
Ò When reminder is set to trigger (executes task)
Ò Time delta (NT – CT)
Ò How far task is planned in advance
Ò Text/task description
31. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 10
00:00am – 04:00am
04:00am – 08:00am
08:00am – 12:00pm
12:00pm – 4:00pm
4:00pm – 8:00pm
8:00pm – 0:00am
Sun Mon Tue Wed Thu Fri Sat
32. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 11
RQ2: Can we identify patterns in the
reminder creation process?
1. Aggregate
2. Per task type
3. By Creation Time and Terms (in task description)
33. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12
Creation times Notification times
1: Aggregate patterns (across all tasks)
34. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12
Creation times Notification times
1: Aggregate patterns (across all tasks)
35. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12
Creation times Notification times
1: Aggregate patterns (across all tasks)
36. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12
Creation times Notification times
1: Aggregate patterns (across all tasks)
37. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13
Findings: Aggregate patterns
38. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13
Findings: Aggregate patterns
Ò Creation/Notification
Ò People create reminders in evenings (end of day), w/
notifications set in mornings (start of day).
39. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13
Findings: Aggregate patterns
Ò Creation/Notification
Ò People create reminders in evenings (end of day), w/
notifications set in mornings (start of day).
Ò (not shown) Time deltas
Ò Most reminders are for “short-term tasks”
41. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15
Allrem
inders
Chore
(recurring)
Chore
(standalone)
Com
m
.(coordinate)
Com
m
.(general)
Eat/Consum
e
Go
(sw
itch
context)Go
(errand)
M
anage
activity
M
anage
process
Delay(hours)
72
60
48
36
24
12
0
Time deltas per task type
Time deltas per task type
42. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15
Allrem
inders
Chore
(recurring)
Chore
(standalone)
Com
m
.(coordinate)
Com
m
.(general)
Eat/Consum
e
Go
(sw
itch
context)Go
(errand)
M
anage
activity
M
anage
process
Delay(hours)
72
60
48
36
24
12
0
Time deltas per task type
Time deltas per task type
43. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15
Allrem
inders
Chore
(recurring)
Chore
(standalone)
Com
m
.(coordinate)
Com
m
.(general)
Eat/Consum
e
Go
(sw
itch
context)Go
(errand)
M
anage
activity
M
anage
process
Delay(hours)
72
60
48
36
24
12
0
Time deltas per task type
Time deltas per task type
44. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15
Allrem
inders
Chore
(recurring)
Chore
(standalone)
Com
m
.(coordinate)
Com
m
.(general)
Eat/Consum
e
Go
(sw
itch
context)Go
(errand)
M
anage
activity
M
anage
process
Delay(hours)
72
60
48
36
24
12
0
Time deltas per task type
Time deltas per task type
45. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16
Findings: per task type patterns
46. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16
Findings: per task type patterns
Ò Time deltas
Ò Differ between task type.
47. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16
Findings: per task type patterns
Ò Time deltas
Ò Differ between task type.
Ò (not shown) Creation/Notification
Ò broadly two types of tasks:
Ò Created/set to notify during office hours
Ò e.g., communicate, go somewhere
Ò Created/set to notify outside of office hours
Ò e.g., manage ongoing process, chores
48. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
Patterns 3: By creation time
49. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
50. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
51. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
52. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
53. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
54. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
55. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17
What is the relation between
creation and notification time?
Patterns 3: By creation time
56. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18
“call”
Patterns 3: By terms
57. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18
“call”
Patterns 3: By terms
58. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18
“call”
Patterns 3: By terms
59. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18
“call”
Patterns 3: By terms
64. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21
RQ2: Can we identify patterns in the
reminder creation process?
65. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21
RQ2: Can we identify patterns in the
reminder creation process?
Ò On average people tend to set plans in the evening.
66. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21
RQ2: Can we identify patterns in the
reminder creation process?
Ò On average people tend to set plans in the evening.
Ò Most tasks that drive reminder setting are for short-term
tasks to be executed in the next 24 hours.
67. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21
RQ2: Can we identify patterns in the
reminder creation process?
Ò On average people tend to set plans in the evening.
Ò Most tasks that drive reminder setting are for short-term
tasks to be executed in the next 24 hours.
Ò Patterns differ between task types (suggesting the
distinctions are meaningful).
68. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21
RQ2: Can we identify patterns in the
reminder creation process?
Ò On average people tend to set plans in the evening.
Ò Most tasks that drive reminder setting are for short-term
tasks to be executed in the next 24 hours.
Ò Patterns differ between task types (suggesting the
distinctions are meaningful).
Ò Terms and creation times are informative w.r.t.
notification time.
69. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 22
RQ3: Can we predict when certain tasks
are most likely to happen?
Ò Motivation:
Ò See if patterns are informative.
Ò Applications:
Ò improve resource scheduling,
Ò detect collisions
Ò tailored advertising,
Ò developing systems to automatically terminate
ongoing tasks/allocate time for task completion
70. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 23
Findings
Ò Predict notification day with ~70% accuracy.
Ò Creation time provides the most information.
Ò Terms provides significant additional information.
71. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 24
In summary
1.Studied frequent reminders, developed task type
taxonomy.
2.Shown reminders’ temporal patterns
3.Demonstrated a direction in harnessing patterns;
predicting notification times.
72. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 25
Future work
Ò User studies?
Ò assumptions w.r.t. “wrapping up day, planning ahead”
Ò validate/check task type taxonomy
Ò Classify task types for improved predictions?
Ò More sophisticated predictive models
Ò Look at additional reminder types (e.g., location-based)
73. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 26
Thanks!
Ò Questions?
d.p.graus@uva.nl
@dvdgrs
www.graus.co