[Review] RAVEN: Perception-aware Optimization of Power Consumption for Mobile Games

1. RAVEN: Perception-aware Optimization
of Power Consumption for Mobile Games
Published in MobiCom’17
Chanyou Hwang et al.
1
Gabin An, Jinhan Kim
2018.10.31

2. Introduction
2

3. Background
● A mobile GPU constitutes a major portion of power consumption on the
devices.
● Visually appealing mobile games => High energy cost
GPU Battery
3

4. Motivation 1
● No matter the states, the game apps
maintain high frame rates.
4
In-Game Lobby
Crossy Road
(길건너친구들)

5. Motivation 2
● Large portions of adjacent frames show strong similarity.
5

6. Goal
● To reduce power consumption of mobile game playing without
degrading user experiences.
6

7. Goal
● To reduce power consumption of mobile game playing without
degrading user experiences.
7
Recap.

8. Challenges
1. The perceptual similarity should be measured efficiently.
2. The degradation of user experiences should be minimal.
3. The system should be transparent to the apps and avoid low-level
system optimization (since mobile gaming apps and GPU drivers are closed source).
8
RAVEN
RAVEN is a DC comics character
whose ability is ‘sense and alter.’

9. Methodology
9

10. RAVEN: System Overview
10

11. RAVEN: System Overview
11

12. RAVEN: System Overview
12
abstraction

13. RAVEN: System Overview
● Three major components for scaling game-frame rendering
○ F-Tracker (FrameDiff Tracker)
○ R-Regulator (Rate Regulator)
○ R-Injector (Rate Injector)
● Small virtual display
○ Low resolution to reduce overhead
13

14. RAVEN: Virtual Display (1)
Mobile gameplay (1920 x 1080)
Mobile gameplay (80 x 45)
Virtual Display
14
piggyback

15. RAVEN: F-Tracker (2)
Virtual Display
F-Tracker
Down-scaled pixels
Current Frame
(fN
)
Recently Rendered Frame
(fN-k
)
Calculate
Y-Diff 15

16. Determine how many
frames will be skipped
(No-Skipping | 1-Skipping | 3-Skipping)
RAVEN: R-Regulator (3)
R-Regulator
predict the similarity
scores with future frames
Y-Diff(fN-k
, fN
)
16
EPS: Estimated Perceptual Similarity

17. RAVEN: R-Injector (4)
R-Injector
Inject a delay into the
rendering loop of game app
The # of frames to be skipped
eglSwapBuffer()
Frames with delay
SurfaceFlinger
17

18. How can we calculate similarity between frames?
● SSIM [1]
○ Existing method to measure user's perception of visual changes
■ SSIM > 0.97
● A strong level of similarity between two images in mobile games [2]
■ Computationally expensive 😭
● YUV Color Space
○ Color encoding system like RGB
○ Leverage Y (luminance) values to approximate SSIM efficiently
[1] Wang, Zhou, et al. "Image quality assessment: from error visibility to structural similarity." IEEE transactions on image processing 13.4 (2004): 600-612.
[2] Cuervo, E., Wolman, A., Cox, L. P., Lebeck, K., Razeen, A., Saroiu, S., & Musuvathi, M. (2015, May). Kahawai: High-quality mobile gaming using gpu offload. In
Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (pp. 121-135). ACM.
18

19. Y-Diff
● Y−Diff is the difference in Y values of two images in the YUV color space.
○ luminance (Y): brightness
○ chrominance (UV): color
● Computation cost: O(N)
○ N = the number of pixels
19
UV space
(Y′ value = 0.5)

20. Would it be reasonable to use Y-Diff?
the relation between Y−Diff and SSIM
20

21. Would it be reasonable to use Y-Diff?
the relation between Y−Diff and SSIM
21
Pearson’s correlation coefficient of -0.926

22. Would it be reasonable to use Y-Diff?
the relation between Y−Diff and SSIM
22
Pearson’s correlation coefficient of -0.926
SSIM and Y−Diff values are strongly correlated
and show a linear relationship.

23. Can we predict similarity of future frames?
● X-axis: Similarity between previous-current frames
● Y-axis: Similarity between current-future frames
23

24. Can we predict similarity of future frames?
● X-axis: Similarity between previous-current frames
● Y-axis: Similarity between current-future frames
24
Pearson’s correlation
coefficient of 0.93

25. Can we predict similarity of future frames?
● X-axis: Similarity between previous-current frames
● Y-axis: Similarity between current-future frames
25
Pearson’s correlation
coefficient of 0.93
The similarity with a future frame can be predicted
from the similarity with a previous one.

26. Perceptual Similarity Prediction
Y-Diff between current
frames and recently
rendered frame
26
● EPS estimates SSIM using Y-Diff (regression)

27. Perceptual Similarity Prediction
Constant
Set depending on app
and index k of previous frame (1, 2, 4)
27
● EPS estimates SSIM using Y-Diff (regression)
Constant

28. Perceptual Similarity Prediction
28
● EPS estimates SSIM using Y-Diff (regression)
Average of recent Y-Diff
scores over window size w
(reflecting past history)
w = 1, 2, 5, and 10
Best

29. SSIM vs. EPS
29
Candy Crush Soda Saga Archery Master 3D

30. SSIM vs. EPS
30
Candy Crush Soda Saga Archery Master 3D
The predictions were more accurate within
the potential range of threshold values (> 0.97)

31. Determining the Scaling Factors (1)
31
1-Skipping
3-Skipping

32. Determining the Scaling Factors (2)
● False transition
○ Skipping of a frame which should have not been skipped
○ EPS > τ1
or τ3
, but SSIM < 0.97
● Increasing the τ1
and τ3
reduces the false transition ratio.
● For example,
○ when τ3
=0.97, the ratio of false transition is 27% for 3-Skipping
○ when τ3
=0.9945, the ratio of false transition is 2.7% for 3-Skipping
32
below the strong level of similaritya skip occur

33. Cloning the Primary Display: Virtual Display
● A virtual display could be allocated and maintained by SurfaceFlinger of
Android.
● The overhead of a virtual display is primarily proportional to the
resolution of the display.
33

34. Cloning the Primary Display: Why
a 0.9989 Pearson’s correlation
34

35. Cloning the Primary Display: Why
35

36. Cloning the Primary Display: Why
36
The Y−Diff results with 80×45 and 1920×1080 are nearly
identical with the selected game-play data, scoring a 0.9989
Pearson’s correlation coefficient

37. Recap.
37

38. RAVEN: Virtual Display (1)
Mobile gameplay (1920 x 1080)
Mobile gameplay (80 x 45)
Virtual Display
38
piggyback

39. RAVEN: F-Tracker (2)
Virtual Display
F-Tracker
Down-scaled pixels
Current Frame
(fN
)
Recently Rendered Frame
(fN-k
)
Calculate
Y-Diff 39

40. Determine how many
frames will be skipped
(No-Skipping | 1-Skipping | 3-Skipping)
RAVEN: R-Regulator (3)
R-Regulator
predict the similarity
scores with future frames
Y-Diff
40
EPS: Estimated Perceptual Similarity

41. RAVEN: R-Injector (4)
R-Injector
Inject a delay into the
rendering loop of game app
The # of frames to be skipped
eglSwapBuffer()
Frames with delay
SurfaceFlinger
41

42. Experiment
42

43. - Two different settings for the evaluation of RAVEN
- PAS : τ1
= 0.9975, τ3
= 0.9993 (more rigorous)
- PAS++: τ1
= 0.975, τ3
= 0.9983 (more generous)
- The lower the thresholds, the more the skips.
Evaluation
1-Skipping
3-Skipping
43

44. Four Different Settings to Compare
● PAS
● PAS++
● 30-FPS (without RAVEN)
● 60-FPS (without RAVEN)
44

45. Game Apps used for Experiments
13 commercial game apps with various
graphical characteristics
A. Static
○ Simple graphic effects
ex) puzzles and board games
B. Dynamic
○ Continuous graphical changes
C. Hybrid
○ Clear separation between the input and
the response phase
45
Solitaire (Static)
Cookie Run
(Dynamic)
Angry Birds
(Hybrid)

46. Baseline of Video Quality
● Perceptual similarity w.r.t. the original sequence of frames
○ SSIM
○ VMAF [3]: A state-of-the-art video quality metric
46
[3] Li, Z., Aaron, A., Katsavounidis, I., Moorthy, A., & Manohara, M. (2016). Toward a practical perceptual video quality metric. The Netflix Tech
Blog, 6.

47. Result
47

48. Result (1): Comparison of PAS and PAS++
48

49. Result (1): Comparison of PAS and PAS++
Summary
➔ PAS++ skipped more frames than PAS.
➔ In 7 out of the 12 cases, PAS++ skipped as many
frames as 30-FPS setting.
49

50. Result (2):
Video
Quality
Scores
50

51. Result (2):
Video
Quality
Scores
Summary
● PAS and PAS++ achieved high video-quality scores.
○ PAS > PAS++ for any cases
● PAS and PAS++ produced better results than 30-FPS,
especially for the games in the Dynamic group.
51

52. Result (3): Energy Saving
on Nexus 5X
52

53. Result (3): Energy Saving
Summary
➔ PAS does not save substantial amount of energy in the
dynamic games. Except those games, PAS saves at
least half the amount of energy saved by 30-FPS.
➔ PAS++ saves at least 10% energy consumption.
on Nexus 5X
53

54. User Study
● 12 participants (7 males and 5 females, in ages from 20 to 30)
● Assessment Protocol
○ Double Stimulus Impairment Scale (DSIS)
○ Double Stimulus Continuous Quality Scale (DSCQS)
54

55. User Study: Results
55

56. Conclusion
● RAVEN: a new, on-the-fly perception-aware rate scaling technique
○ A light-weight frame comparison technique using Y-Diff
○ A low resolution virtual display
● Up-to 35% of total device energy saving
○ w/o any substantial user experience degradation
56

57. Discussion
● Online-learning of the coefficients in the EPS regression equation
○ As RAVEN learns coefficients ( dfs and ) using recorded
gameplay video in prior, it would be better to learn them while
users are playing game.
● Application to other domains
○ Watching a video on Youtube or Netflix
○ It consumes GPU as much as gameplay.
57

58. 58
Thanks!

59. Appendix
59

60. Background: Graphics Processing on Android
60

61. SurfaceFlinger and Framebuffer
61

62. SSIM
62