像 Whoscall 這種用戶遍及全球、下載量逾兩千萬的手機軟體，整體服務監控及示警是怎麼做的？此演講會針對數個服務環節，探討我們運用開源軟體及付費服務的實務經驗。

1.
Architect @ Gogolook
How Realtime Monitoring
Works in…

3. ★ Instant Caller
Identification
Whoscall identifies
background information of
incoming unknown calls in
seconds through tags reported
by other users, Internet search
results, and our
comprehensive global
database.
Instant Caller Identification

4. Incoming Call Dialogue
Incoming Call DialogueFraud Call
Business
Corporation
Restaurant

5. ★ Database with
over 700 Million
Phone Numbers
Whoscall boasts an online database
with over 700 million phone numbers.
The database of Whoscall covers
yellow pages, spammers,
telemarketers, costumer
services...,etc. with numerous
community tags contributed by users
and comments based on real users’
experiences.
Database & Number Details
0287871XXX
1111 Job Bank
1111 – Job bank of trust
No.1, Lane 35, An Ho Road, Taipei City, Taiwan
http://www.1111.com

6. 3 of every 5
strangers’ calls
can be identified.
Over 500M phone calls
are identified
every month.
3,000 spammer
numbers
are reported
by Whoscall users
every month in Taiwan.
Number Identification
– 2015.03 – 2015.03

7. Market
United States
Brazil
Saudi Arabia
India
Thailand
Indonesia
Malaysia
Taiwan
Japan
South Korea
Hong Kong
South
Korea
Taiwan Hong
Kong
Japan India
Top 5 countries of Whoscall users

8. Join us in creating a contact network of trust

9.
10

10. ❶
11

11. 12

12. 12

13. 13

14. 14

15. 15

16. ❷
16

17. 17

18. 18

19. •
•
•
•
19
👎

20. •
•
•
•
19
•
•
•
•
👍👎

21. 20
Consequence
Likelihood
Risk register

22. 20
Consequence
Likelihood
service
outage
app
crash
Risk register

23. 20
Consequence
Likelihood
service
outage
app
crash
db
outage
Risk register

24. 20
Consequence
Likelihood
service
outage
app
crash
db
outage
malfunction
Risk register

25. 20
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
malfunction
Risk register

26. 20
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

27. 21
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
short term
detection &
recovery

28. 21
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
long term
diagnosis
short term
detection &
recovery

29. 22
API servers DB servers

30. 22
Load balancer API servers DB servers

31. CDN
22
Load balancer API servers DB servers

32. CDN
Virtual Private Cloud
22
Load balancer API servers DB servers

33. 23
VPC
CloudFront ELB API servers MongoDB

34. 24
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

35. 24
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Service levels:
• availability
• response time
• error
Risk register

36. 25
CloudFront
CloudWatch
Black Box

37. 25
CloudFront
CloudWatch
Black Box

38. 25
CloudFront
CloudWatch
Black Box

39. 26

40. 27

41. 27

42. 27

43. 28
CloudFront
CloudWatch
Black Box
Crashlytics

44. 28
CloudFront
CloudWatch
Black Box
Crashlytics

45. 29
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

46. 29
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
Service levels:
• R/W lock
• size
• load
• speed

47. 30
VPC
CloudFront ELB API servers MongoDB

48. 30
CloudWatch
VPC
CloudFront ELB API servers MongoDB

49. 30
CloudWatch
VPC
CloudFront ELB API servers MongoDB

50. 31
VPC
CloudFront ELB API servers MongoDB

51. 31
VPC
CloudFront ELB API servers MongoDB
Cloud Manager

52. 32

53. 33
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

54. 33
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
Diagnosis:
• instance metrics
• app metrics
• resource

55. 34
VPC
CloudFront ELB API servers MongoDB

56. 34
CloudWatch
VPC
CloudFront ELB API servers MongoDB

57. 34
CloudWatch
VPC
CloudFront ELB API servers MongoDB

58. 35

59. 35
memory space?
disk space?

60. 36

61. 36

62. 36
mnemonic?

63. 37
CloudWatch
CloudFront ELB API servers MongoDB
StatsD for long-term metrics

64. 37
CloudWatch
CloudFront ELB API servers MongoDB
StatsD for long-term metrics

65. 37
CloudWatch
CloudFront ELB API servers MongoDB
StatsD
StatsD for long-term metrics

66. 38
@ Taipei.py — Feb 26, 2015
Centralized logging and monitoring in Fluentd
http://www.slideshare.net/suitingtseng/fluentd-49952996
For more details:

67. 39
http://www.oreilly.com/webops-perf/free/lightweight-systems.csp
For more details:

68. 39
http://www.oreilly.com/webops-perf/free/lightweight-systems.csp
For more details:

69. 39
http://www.oreilly.com/webops-perf/free/lightweight-systems.csp
For more details:

70. 40
CloudWatch
CloudFront ELB API servers MongoDB
StatsD
Application-specific metrics
• gauges
• counters
• histograms
• meters & timers

71. 41

72. 42
Metrics

73. 42
http://jcconf.tw/2014/manage-
servers-on-the-cloud-with-
opensource-tools.html
Metrics

74. 43
CloudWatch
CloudFront ELB API servers MongoDB
low-level metrics
profiling

75. 43
CloudWatch
CloudFront ELB API servers MongoDB
low-level metrics
profiling

76. 44

77. 45

78. 46

79. 47

80. 48

81. 49
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

82. 49
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
Diagnosis:
• logging
• patterns
• drill down analysis

83. 50
CloudFront ELB API servers MongoDB
CloudWatch

84. 50
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3

85. 50
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3

86. 50
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3

87. 50
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3

88. 50
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3
StatsD

89. 50
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3
StatsD
BigQuery

90. 51
http://www.slideshare.net/tw_dsconf/elasticsearch-kibana
Aug 23, 2015

91. 52
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

92. 52
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
Diagnosis:
• logging
• context
• aggregation

93. 53
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3
error logs

94. 53
CloudFront ELB API servers MongoDB
CloudWatch
log in
S3
error logs

95. 54

96. 55
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

97. 56
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
StatsD
BigQuery

98. 57
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
StatsD
BigQuery

99. 58
Taipei.py — Feb 26, 2015
Centralized logging and monitoring in Fluentd
http://www.slideshare.net/suitingtseng/fluentd-49952996
For more details:

100. 58
Taipei.py — Feb 26, 2015
Centralized logging and monitoring in Fluentd
http://www.slideshare.net/suitingtseng/fluentd-49952996
For more details:
We’ve built an
unified logging mechanism…

101. 59
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
StatSD
BigQuery

102. 59
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
StatSD
BigQuery
How about
unified monitoring alerts ?

103. 60

104. •
•
•
•
60

105. •
•
•
•
60

106. 61
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
BigQuery

107. 61
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
BigQuery

108. 62
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
BigQuery

109. 63

110. 64

111. 64

112. 65

113. 66

114. 66

115. 67

116. •
•
•
•
68
•
•
•
•
👍👎

117. 69
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register

118. 70
Consequence
Likelihood
service
outage
app
crash
db
outage
poor
performance
capacity
shortage
internal
errors
malfunction
Risk register
long term
diagnosis
short term
detection &
recovery

119. 71
CloudFront ELB API servers MongoDB
Cloud Manager
CloudWatch
log in
S3
BigQuery

120. 72

121. 73