Exploring the Future Potential of AI-Enabled Smartphone Processors
Improve test automation operation
1. Improve test automation operation
June 30th, 2020
Sadaaki, Emura
Leisure Product Dept.
Rakuten, Inc.
2. 2
Who am I
Name : Sadaaki Emura (nickname M)
Join in Rakuten : 2007
Career : Embedded engineer (2000-2002)
Web engineer (2002-2015)
Product manager (2015-2016)
QA (2016~)
Role: TestAutomation Engineer Lead
Birthplace : Kanazawa-city
Hobby: jog , climbing , horse racing
20. 20
Auto recover system
Test Result is Failure
Predict reason
Temporary
unstable
Re run
Yes
Test result output
• Console error message by Ranorex
• Error page screenshot
21. 21
Auto recover system
Test Result is Failure
Predict reason
Temporary
unstable
Re run
Yes
Use data
• Console error message
• Message in screenshot
• Screenshot
• Previous prediction data
UseTechnology (Python)
• Tesseract-OCR
• Deep Neural Network with Keras
22. 22
Auto recover system
Test Result is Failure
Predict reason
Temporary
unstable
Re run
Yes
Input test report
Get message from
screenshot
Text base
prediction
Image base
prediction
No result
Output prediction
In progress
Feedback
23. 23
Auto recover system
Test Result is Failure
Predict reason
Temporary
unstable
Re run
Yes
• Reduce waste time operation
• Not waste test resource and time
24. 24
Auto recover system benefit
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Oct-19 Nov-19 Dec-19 Jan-20 Feb-20 Mar-20
auto recovery manual recovery
Ex.
Date : March-20
Temporary unstable Ratio : 75%
Operation time without system: 100 hours
100 * 0.65 = 65 hours saved
65%
25. 25
Summary
• Test automation operation takes time and includes boring task
• Let’s automate some of operation
• Auto recover system is not perfect yet, keep improvement