Presentation slides from a talk on Ciklum/Divio joint event on 14.11.2015

1. Python tools for testing web
services over HTTP
Mykhailo Kolesnyk, back-end developer at Divio AG.

2. Agenda
area of interest
libraries
CLI tools
involving non-developers

3. Intro
– What are we interested in?

4. Intro
– What are we interested in?
– Anything that looks like an service, can fail
under certain conditions and speaks HTTP.

5. Intro
– What are we interested in?
– Anything that looks like an service, can fail
under certain conditions and speaks HTTP.
web service REST API
document-oriented database
sepecialised search engine interface
remote storage interface
maybe even your fridge? RTFM!

6. Standing on the shoulders of
the giants
Low-level protocols implement most of the
complexity involved in networking operations
if we operate on HTTP (application) level.
Communication flow in HTTP is really
straightforward and easy to understand, test
and debug.

7. HTTP is fairly simple

8. HTTP is fairly simple
GET / HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: xkcd.com
User-Agent: HTTPie/0.9.2
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 56
Cache-Control: public, max-age=300
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 2762
Content-Type: text/html; charset=utf-8
Date: Wed, 11 Nov 2015 22:01:41 GMT
ETag: "4112135422"
Expires: Wed, 11 Nov 2015 05:05:40 GMT
Last-Modified: Wed, 11 Nov 2015 05:00:05 GMT
Server: lighttpd/1.4.28

9. Vary: Accept-Encoding
HTTP librar(y/ies) in Python
So, there must be some good Python library to
work with this stuff, right?
There should be one – and preferably only one
– obvious way to do it.

10. HTTP librar(y/ies) in Python
So, there must be some good Python library to
work with this stuff, right?
There should be one – and preferably only one
– obvious way to do it.
httplib (http.client in Python3)

11. HTTP librar(y/ies) in Python
So, there must be some good Python library to
work with this stuff, right?
There should be one – and preferably only one
– obvious way to do it.
httplib (http.client in Python3)
httplib2

12. HTTP librar(y/ies) in Python
So, there must be some good Python library to
work with this stuff, right?
There should be one – and preferably only one
– obvious way to do it.
httplib (http.client in Python3)
httplib2
urllib2

13. HTTP librar(y/ies) in Python
So, there must be some good Python library to
work with this stuff, right?
There should be one – and preferably only one
– obvious way to do it.
httplib (http.client in Python3)
httplib2
urllib2
urllib3

14. HTTP librar(y/ies) in Python
So, there must be some good Python library to
work with this stuff, right?
There should be one – and preferably only one
– obvious way to do it.
httplib (http.client in Python3)
httplib2
urllib2
urllib3
requests!

15. requests + purl + jpath = <3
Lots of utilities are being written on top of
requests these days.
Tools like furl/purl/jpath make parsing and
manipulating data way easier.

16. Before coding often comes
debugging
Debugging and exploring
external/blackboxed/undocumented HTTP service
might be unavoidable.
CLI tools can be of some use during
testing/debugging.
auth (penetration) testing
edge cases (payload size, wrong encoding)

17. Required knowledge
Minimal requrements:
HTTP basics
Cookies/sessions
Auth types
POST payload formats
UNIX shell
pipes
I/O redirection
JSON

18. HTTPie: a CLI, cURL-like tool
for humans
Written in Python, modular, based on
requests -> easy to extend!
And here are some basic examples...

19. HTTPie
Configurable output

20. HTTPie
Basic auth
$ http -a user1:password1 example.org

21. HTTPie
Session
Refer to the session by its name, and the
previously used headers will automatically be
set:
$ http --session=session1 example.org
$ http -a user1:password1 --session=session1 example.org X-Foo:Bar

22. HTTPie
JSON
$ http PUT api.example.com/person/1
name=John
age:=29 married:=false
hobbies:='["http", "pies"]' # Raw JSON
description=@about-john.txt # Embed text file
bookmarks:=@bookmarks.json # Embed JSON file

23. HTTPie
Enter multiline data
$ cat | http POST example.com/todos Content-Type:text/plain
- buy milk
- call parents
^D

24. HTTPie
Check response status and override
timeout
#!/bin/bash
if http --check-status --timeout=1.5 HEAD example.com/health-check &> /dev/null
echo 'OK!'
else
case $? in
2) echo 'Request timed out!' ;;
3) echo 'Unexpected HTTP 3xx Redirection!' ;;
4) echo 'HTTP 4xx Client Error!' ;;
5) echo 'HTTP 5xx Server Error!' ;;
*) echo 'Other Error!' ;;
esac
fi

25. HTTPie
Streamed response
Send each new tweet (JSON object)
mentioning "Apple" to another server as soon
as it arrives from the Twitter streaming API.
$ http --stream -f -a TWITTER-NAME
https://stream.twitter.com/1/statuses/filter.json track=Apple
| while read tweet; do echo "$tweet" | http POST example.org/tweets ;

26. Responsibility
– Who has to write integration (or system)
tests for an API?

27. Responsibility
– Who has to write integration (or system)
tests for an API?
– Maybe, the developer himself?
He'll be more capable fixing his own stuff, for
example.

28. Involving others
What if testing requires more resources than
you can afford?
Сonsumer of your API migth need precise and
up-to date docs sooner than you'll be able to
deliver them.

29. Involving others
What if testing requires more resources than
you can afford?
Сonsumer of your API migth need precise and
up-to date docs sooner than you'll be able to
deliver them.
– Rewrite everything in Haskel. Broader your
audience and give this people tools they can
use. Simple tools.

30. Behaviour Driven
Development
What if we start writing exact specifications
according to real-world requirements?
What if we reuse those specs to be our test cases
that run automatically?
BDD definition usually includes two or three of
the following buzz-words:
TDD
ATDD
DDD

31. OOAD
Take parts of BDD toolkit
and use them in weird way
BDD here looks like a replacement or a good
addition to the integration tests.
But it still allows you to go through the very
natural cycle:
you define behaviour
behaviour defines tests
tests define development direction

32. requirements_tester.txt

33. requirements_tester.txt
{"JSON": "JavaScript Object
Notation"}

34. requirements_tester.txt
{"JSON": "JavaScript Object
Notation"}

35. requirements_tester.txt
{"JSON": "JavaScript Object
Notation"}
Gherkin language - a business readable, DSL
that allows to describe software behaviour
without diving into implementation details.

36. Yet another API accessible
through HTTP
Let's imagine we are building some API.
And we are minimalists - so want it to use BasicAuth.
And we are purists - so we want proper HTTP status
codes, content headers, etc.
How can the authentication test look like?

37. Authentication test example
in Gherkin
yet_another_api.feature
Feature: Yet another API
As an API client
I want to be able to do some really useful fluffy awesome colorful stuff
Background: Set server name, and auth headers
Given I am using server "$SERVER"
And I set base URL to "$URL"
And I set "Accept" header to "application/json"
And I set "Content-Type" header to "application/json"
And I set BasicAuth username to "user@example.com" and password to "password
Scenario: Ensure account exists
When I make a GET request to "account"
Then the response status should be 200

38. Behave
Behaviour-driven development, Python style.

39. Steps implementation
examples
@behave.given('I set BasicAuth username to "{username}" and password to "{passw
@dereference_step_parameters_and_data
def set_basic_auth_headers(context, username, password):
context.auth = (username, password)
@behave.when('I make a GET request to "{url_path_segment}"')
@dereference_step_parameters_and_data
def get_request(context, url_path_segment):
url = append_path(context.server, url_path_segment)
context.response = requests.get(
url, headers=context.headers, auth=context.auth)
@behave.then('the response status should be one of "{statuses}"')
@dereference_step_parameters_and_data
def response_status_in(context, statuses):
ensure(context.response.status_code).is_in(
[int(s) for s in statuses.split(',')])

40. Your custom behave module
code layout
yourproject/
behave/
yet_another_api.feature # the only file for humans, everything else is
__init__.py
environment.py # setup, helpers
steps/
__init__.py # steps implementation

41. environment.py
from behave_http.environment import before_scenario
APP_ROOT = os.path.normpath(
os.path.abspath(os.path.join(os.path.dirname(__file__), '..'
parse_config_file(os.path.join(APP_ROOT, "settings.py"))
default_env = {
'SERVER': 'http://localhost:8081',
'API': 'api',
[...]
}
credentials = {
'password': 'featuretester',
'email': 'featuretester@example.com'
}
def _recreate_account(db, auth, params , [...]):
[...]

42. environment.py (continued)
def before_all(context):
for k,v in default_env.iteritems():
os.environ.setdefault(k, v)
app_url = URL(os.environ['SERVER'])
api_url = app_url.add_path_segment(os.environ['API'])
api_auth = (credentials['email'], credentials['password'])
# DB connection setup goes here
_recreate_account(db, auth, params , ...)
_add_account_data(api_url, api_auth)
def after_tag(context, tag):
"""Clean up database after destructive operation"""
if tag == 'modifies':
before_all(context)

43. Example tag usage
[...]
@modifies
Scenario: Delete account, ensure it does not exist
When I make a DELETE request to "account"
Then the response status should be 200
When I make a GET request to "account"
Then the response status should be 401
@wip
Scenario: Create account
[...]

44. Running tests
# behave -q -f progress3
HTTP requests # features/http.feature
Test getting context variable ......
Test GET request ......
Test POST request by checking we get the same JSON payload back .......
Test JSON path expection .......
Test JSON array length calculation .......
[...]
Test GET polling with checking for value that eventually succeeds ......
Test Basic Auth .......
1 feature passed, 0 failed, 0 skipped
17 scenarios passed, 0 failed, 0 skipped
121 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.516s

45. Failing tests
[...]
Test JSON array length calculation ......F
--------------------------------------------------------------------------------
FAILURE in step 'the JSON array length at path "array" should be 3'
(features/http.feature:55):
Assertion Failed: Expected [1, 2, 3, 4] to have length 3
--------------------------------------------------------------------------------
[...]
Failing scenarios:
features/http.feature:49 Test JSON array length calculation
0 features passed, 1 failed, 0 skipped
16 scenarios passed, 1 failed, 0 skipped
120 steps passed, 1 failed, 0 skipped, 0 undefined
Took 0m0.313s

46. Behave-HTTP
Implements basic reusable Behave steps for
testing HTTP services

47. Tools
Libraries and utilities
requests
purl, furl
jpath
HTTPie
Behave
Behave-HTTP

48. Resources
Specs
HTTP 1.1
Gherkin
Videos
"Cutting Off the Internet: Testing Applications that
Use Requests", Ian Cordasco
"Doing BDD with Python and PyCharm", Ilya
Kazakevich