All new features at eBay Europe are developed using SCRUM. One key success factor for those projects is to have a reliable end-to-end test automation safety net. This presentation illustrates how in addition to a robust automation toolset it is essential to have an integrated approach to test automation design: Test Aspects - Test Aspects are used to do the functional design of the end-to-end automation test cases. Since this is done upfront, the tester is able to focus on the what rather than the how. Modeling of the Biz Domain Layer - The Biz Domain Layer is an abstraction layer above the user interface that is implemented in the test code. This layer is divided into pages and flows which are then used in the tests. Test Implementation - Tests are written in Java, stored in SVN and are executed using the WebDriver Grid (Selenium2). Tests execution can be triggered by all team members via a continuous integration server (Hudson). Lean Test Automation – it is important to retain and maintain the quality of the tests. Key success factors for this are: Code Reviews, Software Craftsmanship, Test Aspect Reviews and the “Definition of Done”. Following an integrated approach to test automation ensures high efficiency, low overhead and easier maintenance.

1. Implementing Test
Automation in Agile Projects
Michael Palotas & Dominik Dary
Quality Engineering Europe
eBay International
Dutch Testing Conference 2012
April 2012

2. eBay Inc. Facts
Founded in 1995
Based in San Jose, California
27770 Employees worldwide

3. Platform Facts
104 million
active buyers and sellers worldwide
68 Billion USD
transaction volume 2011
2000 USD
transaction volume every second

4. Data Facts
9 petabytes of data
In our Hadoop and Teradata clusters
2 billion page views
each day
75 billion database calls
each day

5. Mobile Facts
70,000,000 downloads
of eBay Inc’s apps
5 billion USD
transacted 2011 on mobile devices
1 million items
are listed with using mobile apps every week

6. Who we are
Michael Palotas
Head of Quality Engineering Europe
E-mail: mpalotas@ebay.com
Dominik Dary
Staff Software Engineer in Test
E-mail: ddary@ebay.com

7. Test Automation?
To get early and instant Feedback
We automate the regression tests
It is our safety net to save time &
this is reinvested into manual testing

8. Flip the testing triangle
Manual
Tests
Manual Tests
E2E
Tests
E2E Automation
Tests Integration Tests
Unit
Tests Unit Tests
Inspiration: http://patrickwilsonwelsh.com/?p=32

9. Our guide through this Presentation
4. Lean Test Automation
3. Test Implementation
2. Modeling of the Biz Domain Layer
1. Design the Automated Test with Test Aspects

10. 1. Design the Automated Tests with Test Aspects
Use Test Aspects** Advantages
• Test aspects briefly describe • Clear overview for all team
“why” and “what” has to be members about test coverage
tested, without describing • Simplicity helps you to do
“how” quick reviews and
• Enables for early testing walkthroughs with your product
because tests can be written at owner
the same time the user story is • Enables you to identify manual
created and automated test duplicates
• Test aspects are structurally • Traceability between test
derived cases and specification
**Swiss Testing Day Presentation 2010: “Mit Testaspekten früh, intelligent und redundanzfrei testen” Melanie Späth
Image Source: http://www.masternewmedia.org/images/social_software_impact_individual_organizations.jpg

11. Test Aspect Example

12. 2. Modeling of the Biz Domain Layer
Tests
• Modularization and
reuse of components:
Flow Objects maintainable &
Biz
Domain extendable e2e tests
Page Objects Layer
• Crucial to have in
Biz Domain Layer
depth knowledge of
WebDriver
the domain
Browser • Enable data driven
testing by generic test
cases

13. Pages and Flows
e-Commerce shopping site like eBay
Select
View Order
Review Payment
Category Summary
Provider
Search View View
Update
Item Category Orderlist
View Delete
Product
View Item Shopping Cart
Page Page Checkout Page Success Page

14. 3. Test Implementation using Selenium
Tests
Flow Objects
Objects
Page

15. Technical High Level Overview

16. Instant Feedback - Selenium Grid
Sequential Execution Parallel Execution
Test Test Test Test
Parallel Execution
Parallel Execution
Test Test Test Test
Test Test Test Test Test
Test Test Test Test
1 2 3 … 4500
Execution Time Execution Time
Execution time: Execution time:
7000 minutes (~116 hours = ~5 days) 60 minutes
by using 150 Virtual Machines /
Browsers using the WebDriver GRID
Footnote about the size of the Tests @eBay Europe:
- 4500 end-to-end regression test cases
- 2-3 minutes execution time for each

17. 4. Lean Test Automation
Technical Debt Lean Test Automation
• Activities that result in • Eliminate Waste Definition of Done
technical debt include • Build Quality In Reviews
• Postponing
documentation and
• Create Knowledge Document the essentials
writing tests • Defer Commitment Don’t build a framework
• attending to TODO • Deliver Fast
comments and
• tackling compiler and • Respect People Long Leash
static code analysis • Optimize the Whole
warnings
• Knowledge that isn't
shared around the
organization
• Code that is too confusing
to be modified easily

18. Conclusion
Be clear why you want to automate
Open source is your best friend
Flip the testing triangle
Treat automation as a software project
Work with the right people
Stay lean
Test automation is much more than
just your automation toolset!

19. Thank You!
Questions or Comments?