1. Agile Testing at eBay
Dominik Dary
Quality Engineering Europe
eBay International AG
User Group: Softwaretest und Qualitätssicherung
September 2012
2. Facts about eBay
eBay Inc. Facts: Platform Facts:
104 million
active buyers and sellers
Founded in 1995
worldwide
68 billion USD
Based in San Jose, California transaction volume 2011
2000 USD
transaction volume every
27770 Employees worldwide
second
2 billion page views
each day
3. Quality Engineering Team Europe
We are responsible for the software
quality of 12 European sites
We are testing the Web, Mobile
Web, Mobile Native Apps and Win32
We use a combination of End-to-
End test automation together with
manual exploratory testing
4. Who I am
Dominik Dary
Staff Software Engineer in Test
E-mail: ddary@ebay.com
5. Agenda
Project Setup
Testing in an Agile World
Automated and Manual Testing
Working in a cross functional team
6. The Setup
DISTRIBUTED
(Zürich, London, Berlin, Chennai, Pune, San Jose … and some more)
OUTSOURCED
(Development, Scrum Master, Product Owner)
AGILE
TEAM
7. Software Development + Testing = Scrum?
Development Q Development QA Development QA Development
A
Dev Q Dev Q Dev Q Dev Q Dev Q Dev Q Dev Q Dev Q
A A A A A A A A
Automation is a key enabler for proper SCRUM:
• Regression runs at any time
• Eliminate Mini-Waterfall
• Eliminate test slippage into next Sprint
8. Flip the testing triangle
Manual
Tests
Manual Tests
E2E
Tests
E2E Automation
Tests Integration Tests
Unit
Tests Unit Tests
Inspiration: http://patrickwilsonwelsh.com/?p=32
9. Test Automation?
To get early and instant Feedback
We automate the regression tests
It is our safety net to save time & this
is reinvested into manual testing
10. Our Guide for End-to-End Test Automation
3. Test Implementation
2. Modeling of the Biz Domain Layer
1. Design the Automated Test with Test Aspects
11. 1. Design the Automated Tests with Test Aspects
Use Test Aspects** Advantages
• Test aspects briefly describe • Clear overview for all team
“why” and “what” has to be members about test coverage
tested, without describing • Simplicity helps you to do
“how” quick reviews and
• Enables for early testing walkthroughs with your product
because tests can be written at owner
the same time the user story is • Enables you to identify manual
created and automated test duplicates
• Test aspects are structurally • Traceability between test
derived cases and specification
**Swiss Testing Day Presentation 2010: “Mit Testaspekten früh, intelligent und redundanzfrei testen” Melanie Späth
Image Source: http://www.masternewmedia.org/images/social_software_impact_individual_organizations.jpg
13. 2. Modeling of the Biz Domain Layer
Tests
• Modularization and
reuse of components:
Flow Objects maintainable &
Biz
Domain extendable e2e tests
Layer
Page Objects
• Crucial to have in
Biz Domain Layer
depth knowledge of
WebDriver
the domain
Browser • Enable data driven
testing by generic test
cases
14. Pages and Flows
e-Commerce shopping site like eBay
Select
View Order
Review Payment
Category Summary
Provider
Search View View
Update
Item Category Orderlist
View Delete
Product
View Item Shopping Cart
Page Page Checkout Page Success Page
16. The importance of parallel Test Execution
• Instant Feedback is
essential
• 12 European Sites
• Multiple Products
• Web
• Win32
• Mobile Native Apps
• Mobile Web
• Multiple OS
• Windows
• Linux
• iOS
• Android
Picture Source: http://www.flickr.com/photos/ippei-janine/483345027/
18. Manual testing is very important
Automated tests only do what you have
implemented: at the same speed, same
order, with exactly the same mouse clicks
Manual testing allows you to go off the
beaten path
Test automation in combination with
exploratory testing combine the best of
both worlds
19. Speak the same language as the developers
Use the same programming language as
development
Developers can write or maintain tests if
needed
Enables for a cross functional team where
everyone can work on nearly every task
20. Applying Agile Principles and Values
Provide Continuous Feedback
Deliver Value to the Customer
Have Courage
Practice Continuous Improvement
Self Organize
Focus on People
Enjoy
21. Conclusion
Applying Agile Principles and Values
Open source is your best friend
Flip the testing triangle
Treat automation as a software project
Work with the right people
Whole Team approach
Call out shapes can be found on pg. 56 and 57Arrows can be found on pg. 59
Instant Feedback is the key enabler for AGILE TESTING.Automated tests are early feedback for the whole teamQuickly detect destabilizing changes in new buildsMinimize the execution time of your test suite (max 1hour) Expose regression bugs as quickly as possibleFindings bugs early is saving money
Talk hereaboutwhywefocus on parallel execution.Weare asmallteamWehavetotest multiple appsfor multiple sites on different platformsWewantedtoadaptthelearningsfromthe web to mobile‚Build in Scalability“ veryimportant
Image size: 9” (width) x 5.17” (height)Right click and select “Change picture” to replace image
