Effective and recommended alternative for automated regression testing - HTTPS (REST) automation in SoapUI

1. Do not automate GUI
Reinvention of wheel (HTTP performance testing) like an effective tool
for regression testing
Atila Inovecký

2. Something about pyramids
WHY to do API Testing

3. Abraham Maslow – Human Needs
Don't reinvent a wheel

4. Gojko Adzic - Quality Criteria
Don't reinvent a wheel

5. Mike Cohn (2009)– Automated regression testing
Don't reinvent a wheel

6. API testing in 2001
• Kaner, Bach, Pettichord: Lessons Learned in Software Testing – Lesson 132:
Automate tests using programming interfaces
“…When we look at many product testing effort we’ve observed, we conclude
that there is a strong correlation between the availability of programming
interfaces for testing and the development of powerful automated test
suites. Programming interfaces include APIs, command-line interfaces, COM
interfaces, HTTP and more. …If you’re serious about automation, you’ll learn
or get someone who knows this to help..
…Focus your attention on where automation can help the most. Sometimes,
you can effectively automate the GUI; other times you can’t. Don’t let
people’s preconceptions of what automation has to look like limit you.”
Don't reinvent a wheel

7. Ice Cream Cone Antipattern
Don't reinvent a wheel

8. Reasons for antipattern(traditional approach)
• Automate everything!
• Just QA is responsible for quality
• Few testing rounds – no Continuous Integration
• WHAT TO TEST(Business Functionality)=HOW TO TEST (GUI)?
• “This small hotfix won’t break anything”
• Production delivery 1 per year
• DIDN’T FAIL YET
• Trying to find a lot bugs
Don't reinvent a wheel

9. Test find bugs, not automation
• Value of the regression test depend on value of the covered
functionality
• Count of found bugs say nothig about valueeffectivity of automated
test
• Most of regression bugs is caused by unexpected impact of backend
integration not GUI
• Automation is not medicine for wrong communication
Don't reinvent a wheel

10. Conclusions
• Unit testing is the core of regression testing
• API testing is not the FUTURE but it is really just a reinvented wheel
• Future of automated GUI testing is (Exploratory tests, …)???
• GUI testing vs API testing
Don't reinvent a wheel

11. Something about REST API
How to test API through HTTP (REST)
Don't reinvent a wheel

12. REST – Representational State Transfer
• REST or RESTful web services are one way of providing
interoperability between computer systems on the internet. REST-
compliant web services allow requesting systems to access and
manipulate textual representations of web resources using a uniform
and predefined set of stateless operations.
Don't reinvent a wheel

13. Layered architecture of RESTful interactions
Don't reinvent a wheel

14. RESTful URLs
• URLs intended to improve the usability and accessibility of
a website or web service by being immediately and intuitively
meaningful to non-expert users.
Don't reinvent a wheel
Non-semantic URL Semantic URL
http://example.com/index.php?page=name http://example.com/name
http://example.com/index.php?page=consulting/mar
keting
http://example.com/consulting/marketing
http://example.com/products?category=2&pid=25 http://example.com/products/2/25
http://example.com/cgi-
bin/feed.cgi?feed=news&frm=rss
http://example.com/news.rss
http://example.com/services/index.jsp?category=lega
l&id=patents
http://example.com/services/legal/patents

15. Relationships between URL and HTTP methods
Don't reinvent a wheel
Uniform Resource Locator (URL) GET POST
Collection, such as
http://mgs.com/resources/
List the URIs and perhaps other
details of the collection's
members.
Create a new entry in the
collection. The new entry's URI is
assigned automatically and is
usually returned by the operation.
Element, such as
http://mgs.com/resources/item17
Retrieve a representation of the
addressed member of the
collection, expressed in an
appropriate Internet media type.
Not generally used. Treat the
addressed member as a collection
in its own right and create a new
entry within it.

16. HTTP API Design
Don't reinvent a wheel

17. Important HTTP Request Headers
Don't reinvent a wheel
Header field
name
Description Example
Cache-Control
Used to specify directives that must be obeyed by all
caching mechanisms along the request-response chain.
Cache-Control: no-cache
Content-Type
The MIME type of the body of the request (used with
POST and PUT requests).
Content-Type: application/x-www-form-urlencoded
Content-Type: application/json
Content-Type: multipart/form-data
Cookie
An HTTP cookie previously sent by the server with Set-
Cookie.
Cookie: $Version=1; Skin=new;
Host
The domain name of the server (for virtual hosting), and
the TCP port number on which the server is listening. The
port number may be omitted if the port is the standard
port for the service requested.
Host: qa16-ats.mgspdtesting.com

18. Important HTTP Status Codes
Don't reinvent a wheel
Status Code Description
200
Ok
Standard response for successful HTTP requests. The actual response will depend on the request method
used. In a GET request, the response will contain an entity corresponding to the requested resource. In a
POST request, the response will contain an entity describing or containing the result of the action.
302 Found Common way of performing URL redirection.
400 Bad Request The server cannot or will not process the request due to an apparent client error
403 Forbidden
The request was a valid request, but the server is refusing to respond to it. The user might be logged in but
does not have the necessary permissions for the resource.
404
Not Found
The request was a valid request, but the server is refusing to respond to it. The user might be logged in but
does not have the necessary permissions for the resource. Usually means that the resource already didn’t
exist.
501 Internal Server
Error
Indicates that the request could not be processed because of conflict in the request, such as an edit
conflict between multiple simultaneous updates.

19. Open Source Tools for API Testing
• Performance testing tools
• SoapUI (SoapUI NG Pro), Jmeter(SmartMeter, Blazemeter)
• Web Service testing tool
• Postman, DHC, HTTPRequester, Wizdler, WebInject, Http Master, …
• Automation Frameworks
• FitNesse, Cucumber, Robot Framework, …
Don't reinvent a wheel

20. SoapUI
• Open-source web service testing application:
• for service-oriented architectures (SOA)
• for representational state transfers (REST).
• Covers:
• web service inspection, invoking, development, simulation and mocking,
• functional testing,
• load and compliance testing.
• A commercial version, SoapUI Pro (Api Ready), which mainly focuses
on features designed to enhance productivity.
Don't reinvent a wheel

21. SoapUI Open Source– pros and cons
• Pros
• support HTTP(S), REST, SOAP, JDBC, AMF
• IDE -Groovy -> command line, visible SoapUI API
• User forums, tradition
• Easy to learn, easy to use
• Cons
• Team work support
• No recording
• Weak user documentation
• Test history, test debugging, compare results
• Buggy
Don't reinvent a wheel

22. Conclusions
• Web application automated testing is possible through HTTP(S)
• Web application automated testing is effective through REST
• REST API popularity is growing
Don't reinvent a wheel

23. Something about fiddling
How to capture HTTPS communication
Don't reinvent a wheel

24. Fiddler
• Captures HTTP and HTTPS traffic and logs it for the user to
review (the latter by implementing man-in-the-middle interception
using self-signed certificates).
• https://www.telerik.com/download/fiddler
Don't reinvent a wheel

25. HTTPS Traffic settings
• Capture - F12 or Click on left bottom corner
• Select captured process by
• Preview - Tab Inspector
• Request – Top part
• Response – Bottom part
Don't reinvent a wheel

26. Icon help
Don't reinvent a wheel

27. Fiddler - Inspector
• Request Inspectors
• Headers—Shows request headers and status.
• Webforms – Shows parameters visualized as table from QueryString and Body
• Raw—Shows the “real” request header and body
• Response Inspectors
• Headers—Shows response headers and status.
• SyntaxView—Shows the response body with HTML syntax
• WebView – Shows the browser output
• Raw – Shows “real” response header and body
Don't reinvent a wheel

28. Fiddler - Settings
ToolsTelerik Fiddler OptionsHTTPS Filters settings
Don't reinvent a wheel

29. Something about soap with UI
REST testing in SoapUI Open Source
Don't reinvent a wheel

30. Initial settings
• FilePreferences –
• Proxy Settings -> Manual 127.0.0.1:8888
• UI Settings – Size of Req Response Message to Show – 1000000
• FileImport project (FileCreate Empty Project)
Don't reinvent a wheel

31. Don't reinvent a wheel

32. What you get for 500$ in SoapUI NG Pro
• a
• FileImport project (FileCreate Empty Project)
Don't reinvent a wheel

33. Useful links
 REST 101: The Beginner's Guide to Using and Testing RESTful APIs
 SoapUI - Getting Started with REST Testing
 SoapUI - Functional Testing
 SoapUI – Scripting & Properties
 SoapUI – API Docs
Don't reinvent a wheel

34. Reference List
• Gojko Adzic, David Evans - Fifty Quick Ideas To Improve Your Tests
• Mike Cohn - Succeeding with Agile: Software Development Using
Scrum
• Lisa Crispin, Janet Gregory - Agile Testing: A Practical Guide for Testers
and Agile Teams
• Dorothy Graham, Mark Fewster - Experiences of Test Automation:
Case Studies of Software Test Automation
• Cem Kaner, James Bach, Bret Pettichord - Lessons Learned in Software
Testing: A Context-Driven Approach
Don't reinvent a wheel

35. Don't reinvent a wheel