SlideShare a Scribd company logo
1 of 35
Download to read offline
MAD-LAB
Mobile Automation Devices LAB
Anand Bagmar
Software Quality Evangelist
About Me
@BagmarAnand
Blog – essenceoftesting
about.me/anand.bagmar
What do you expect
from this session?
Context
Ecosystem
Platforms
• Android (> v4.4.2)
• iPhones (> v5)
• Desktop / WAP
Regions
• Indonesia
• Malaysia
• India
• Middle-East
• ...
Dynamic Content
Search
Download
Offers
ADs
What this means ... Reality!
• Unique installs (Android) - > 1.75M
• Unique # of devices (Android) - > 13K
What this means ... Product!
• One product, One experience
• Unique / Regional / Local content
• User detection & offers based on origin
• Region
• Carrier network
• ...
What this means ... Testing!
• Video Quality, Dynamic Content, Download Expiry
• Simulating Carriers / Networks
• Consumer focus!
• Experience
• Device / OS combinations
Business Requirements
• Provide visibility into -
• What is Tested
• Overall Quality of Product
APIs
Payment
partners
CMS
....
....
Architecture
APIs
Payment
partners
CMS
....
....
Where are these?
How would you
approach
Automation for such
a product?
What Metrics make
sense for
determining Quality?
Automation
Strategy
Automation Criteria
• E2E tests should focus on Business Rules & User Scenarios
• Reports should reflect the above
• Understood by all Business & Technical stakeholders
• Generated automatically
• Should run frequently
• Every new change in the product-under-test
• On a variety of device / OS combinations
What to Automate?
Validate what was working, continues to work!
• Identify user journeys / scenarios
• Prioritize it!
Run Automation against...
Usage patterns from Analytics / PlayStore / App Store
• Identify devices that matter
• Identify devices that are “automation-friendly”
Automation Tech Stack
• Cucumber-JVM à For specifying Business Rules
• Appium à To allow integration with Android / iOS
• Gradle / Groovy à As build tool scripting language
• Jenkins à Continuous Integration Server
• TTA - Test Trend Analyzer à Trend & Failure Analysis
MVP / POC
Scale
&
Repeat
Experiment, Learn &
Evolve
Experiments
• Run tests on Emulators
• Use Cloud-based services (Local Vs Remote)
• SauceLabs
• TestObject
• Amazon Device Farm
• pCloudy
• Devices
Some Experiments didn’t work
• Emulators
• Video Player
• Location
• Cloud-based services
• Location
• Tech Stack
• Devices
MAD-LAB
MAD-LAB – Mobile Automation Devices LAB
Innovations in MAD-LAB
• Device management (selection, cleanup, app install and
uninstall)
• Parallel test execution (at Cucumber scenario level) -
maximising device utilisation)
• Appium server management
• Adb utilities (periodic ADB server disconnects)
• Custom reporting using cucumber-reports
• Video recording of each scenario and embedding in the
custom reports
• Analytics Automation
Conclusion
Learnings (so far)
Blue-print is important!
• Stepping-stone approach
Experiment!
• Know your objective(s)
• Fail-fast!
Make the Product Testable!!
References
• Sample code
• https://github.com/anandbagmar/cucumber-jvm-appium-infra
• Blog posts
• https://essenceoftesting.blogspot.com/search/label/madlab
• TTA (Test Trend Analyzer)
• https://essenceoftesting.blogspot.com/search/label/tta
Thank you!
@BagmarAnand
Blog – essenceoftesting
about.me/anand.bagmar

More Related Content

More from Anand Bagmar

More from Anand Bagmar (20)

Design Patterns in Automation
Design Patterns in AutomationDesign Patterns in Automation
Design Patterns in Automation
 
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective KeynoteChange Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
 
Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)
 
Next Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AINext Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AI
 
The Best Test Automation Framework is...
The Best Test Automation Framework is...The Best Test Automation Framework is...
The Best Test Automation Framework is...
 
Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021
 
Getting started with Appium 2.0
Getting started with Appium 2.0Getting started with Appium 2.0
Getting started with Appium 2.0
 
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
 
Selenium Deep Dive
Selenium Deep DiveSelenium Deep Dive
Selenium Deep Dive
 
Does your functional automation really add value?
Does your functional automation really add value?Does your functional automation really add value?
Does your functional automation really add value?
 
Measuring Coverage From E2E Tests
Measuring Coverage From E2E TestsMeasuring Coverage From E2E Tests
Measuring Coverage From E2E Tests
 
Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020
 
Visual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA SymposiumVisual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA Symposium
 
Collaboration - A Taboo!
Collaboration - A Taboo!Collaboration - A Taboo!
Collaboration - A Taboo!
 
Visual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation PyramidVisual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation Pyramid
 
Measuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback LoopMeasuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback Loop
 
What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?
 
Sharing (less) Pain of using Protractor & WebDriver
Sharing (less) Pain of using Protractor & WebDriverSharing (less) Pain of using Protractor & WebDriver
Sharing (less) Pain of using Protractor & WebDriver
 
Sharing the pain using Protractor
Sharing the pain using ProtractorSharing the pain using Protractor
Sharing the pain using Protractor
 
Test Data - Food for your Test Automation Framework
Test Data - Food for your Test Automation FrameworkTest Data - Food for your Test Automation Framework
Test Data - Food for your Test Automation Framework
 

Recently uploaded

TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
mohitmore19
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
VishalKumarJha10
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
Health
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
VictorSzoltysek
 

Recently uploaded (20)

TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
 
HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.comHR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.com
 
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerHow To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
 
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
 
A Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxA Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docx
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS LiveVip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
 

Build your own MAD-LAB - for Mobile Test Automation