27. • Mostly done manually done
• Functional Automation can help
@BagmarAnand
Visual Testing
28. • Business / Revenue loss
• Brand / Credibility loss
• You loose your users!
@BagmarAnand
If Visual Testing is NOT done …..
29. • Mostly done manually done
• tedious
• very error prone
• impossible to scale and repeat – especially in days of CD!
• Functional Automation may help – a little
@BagmarAnand
Challenges of Visual Testing
38. First, Let’s Provide Initial Coverage
18 Lines of
Code
21 Locators
and Labels
60 Minutes
40
Navigatio
n
Button
Click
Validatio
n
39. NEW
41
Real Bug Escapes
Text Changes
Caught
New Feature
Missed
Visual Bug
Escapes
Broken Locators
Next, We Test a New Build
2 Late
Stage Bugs
2 Changes
Caught
2 False
Positives
Baseline New Candidate
New Feature
Missed
40. Test Results Maintenance Action Time
Functional Bug Write Another Validation 15 mins
Visual Bug Try Writing New Validations, Then Give Up 15 mins
Placeholder Text Differences Update Existing Validations 10 mins
New Terms & Conditions
Feature
Write Another Validation 5 mins
Changed Locators Update Locators 15 mins
Total Time Spent 60 mins
Finally, We Have to Maintain our Tests
NEW
41. Plus Those 21 Locators and Labels That Might
Change Without any Notice!
Build and Maintenance Summary
43
Build & Maintain:
120 Minutes
Bug Detection:
40%
False Positive Rate:
40%
42. Login Page to Test
Remember Our
Simple Example?
Let’s Try Again.
44
43. 45
Let’s Provide Initial Coverage Using Visual AI
2 Minutes to Create
Vs. 60 Minutes
1 Locator
Vs. 21 Locators
and Labels
44. 46
Visual AI Handles the New Build
NEW
Functional Bug
Found
Text Changes
Found
Visual Bug Found
New Feature
Found
Original New Build
No Broken
Locators
All Bugs Found
Text Changes and
New Feature
Detected
0 Broken Locators
45. Test Results Visual AI Maintenance Action Legacy Visual AI
Functional Bug None; Found Automatically 15 mins 0 mins
Visual Bug None; Found Automatically 15 mins 0 mins
Placeholder Text Differences “Thumbs Up” for Each Difference 10 mins 40 secs
New Terms & Conditions Feature “Thumbs Up” for Difference 5 mins 20 secs
Changed Locators Locators No Longer Used 15 mins 0 mins
Total Time Spent 60 mins 1 min
Visual AI Streamlines Test Maintenance
NEW
46. Legacy Test Automation
Visual AI Delivers Results
48
Visual AI
40x Faster
21x More Stable
2.5x Increase in
Bug Detection
Eliminate False
Positives!
Build & Maintain:
3 Minutes
Locators:
1
Build & Maintain:
120 Minutes
Locators and Labels:
21
Bug Detection:
100%
Bug Detection:
40%
False Positive Rate:
0%
False Positive Rate:
40%
47. 49
Legacy Test
Stack
Plus Applitools Visual
AI
Test
Frameworks
Selenium, Appium,
Cypress, Manual
Testing, Visual
DOM/Diff Tools, Etc.
80% Less Code
40x Faster
21x More Stable
Testing &
DevOps
Infrastructure
Jira, GitHub, BitBucket,
Slack Jenkins, Travis,
Circle, Azure, Sauce,
Perfecto, Browserstack
30x Faster
50% $$ Savings
Early Stage
Bug
Detection
Catch 2.5x More Bugs
No False Positives!
Fix Bugs In Minutes (Not
Hours)
Incredibly Accurate, Efficient Test Automation
49. Test Pyramid … includes UX & NFRs
Unit (xUnit / JavaScript)
Component
Integration
View
Web Service
UI
P
e
r
f
o
r
m
a
n
c
e
S
e
c
u
r
i
t
y
Accessibility
UX
@BagmarAnand
Manual / Exploratory