In a world where things get more and more connected to services there is a dire need to understand the principals off HTTP protocols. It is simply not enough to 'just return' some data and expect the the client knows what to do. Proper clients, wether it is a 'thing' or 'software' can intelligently communicate with servers. And servers should take the responsibility to give clear answers.... or let the client know that it is not fully clear what the client requested. This talk is a brief overview on how client and server communicate with each other concerning caching, content-negociation and the methods provided. Its all about what is going on inside the HEADs when client and server are TALKING http. Then there is Web::Machine, based on Basho's work. It brings structure in the whole decision tree of what is happening with all the Request Header Fileds. A nice work from, but arguably, it has some weak spots. And of course a great new module for Dancer2 will be introduced: Dancer2::Plugin::HTTP::Auth::Extensible, the first of a series that will make life easy when developing REST api's with Dancer

1. Talking Heads
REST api’s don’t need to be your “psycho-killer”

2. Dancer 2
A Lightweight WEB Framework
… that does nothing wrong

3. Dancer 2
A Lightweight WEB Framework
… that does nothing you don’t want

4. Dancer 2
A Lightweight WEB Framework
… does it do what it should do?

5. RESTful API’s
Dancer 2
HTTP protocols and RFC’s

6. Overview
• HTTP 1.1 Protocol
• Let’s Dance
• Caching
• Conditional Requests
• Content Negotiation
• Authorization

7. HTTP/1.1 Protocol
• Requests
• Responses
• Request Methods
• Header Fields
• Status codes

8. HTTP 1.1 Protocol
Request
GET /some_resource HTTP/1.1
Host: server.tst
User-Agent: insane_client/0.1
Accept: text/html; q=0.8, image/gif; q=0.3
Accept-Charset: iso8509-1
Content-Length: 123
Here can come some content, for what ever
purpose

9. HTTP 1.1 Protocol
Request
GET /some_resource HTTP/1.1
Host: server.tst
User-Agent: insane_client/0.1
Accept: text/html; q=0.8, image/gif; q=0.3
Accept-Charset: iso8509-1
Content-Length: 123
Here can come some content, for what ever
purpose

10. HTTP 1.1 Protocol
Request
GET /some_resource HTTP/1.1
Host: server.tst
User-Agent: insane_client/0.1
Accept: text/html; q=0.8, image/gif; q=0.3
Accept-Charset: iso8509-1
Content-Length: 123
Here can come some content, for what ever
purpose

11. HTTP 1.1 Protocol
Request
GET /some_resource HTTP/1.1
Host: server.tst
User-Agent: insane_client/0.1
Accept: text/html; q=0.8, image/gif; q=0.3
Accept-Charset: iso8509-1
Content-Length: 123
Here can come some content, for what ever
purpose

12. HTTP 1.1 Protocol
Request
GET /some_resource HTTP/1.1
Host: server.tst
User-Agent: insane_client/0.1
Accept: text/html; q=0.8, image/gif; q=0.3
Accept-Charset: iso8509-1
Content-Length: 123
Here can come some content, for what ever
purpose

13. HTTP 1.1 Protocol
Response
HTTP/1.1 200 OK
Server: dancer/2.0
Date: Tue, 14 Oct 2014 12:34:56 GMT
Content-Type: text/plain
Charset: iso8509-1
Content-Length: 456
You asked for something, here it is!

14. HTTP 1.1 Protocol
Response
HTTP/1.1 200 OK
Server: dancer/2.0
Date: Tue, 14 Oct 2014 12:34:56 GMT
Content-Type: text/plain
Charset: iso8509-1
Content-Length: 456
You asked for something, here it is!

15. HTTP 1.1 Protocol
Response
HTTP/1.1 200 OK
Server: dancer/2.0
Date: Tue, 14 Oct 2014 12:34:56 GMT
Content-Type: text/plain
Charset: iso8509-1
Content-Length: 456
You asked for something, here it is!

16. HTTP 1.1 Protocol
Response
HTTP/1.1 200 OK
Server: dancer/2.0
Date: Tue, 14 Oct 2014 12:34:56 GMT
Content-Type: text/plain
Charset: iso8509-1
Content-Length: 456
You asked for something, here it is!

17. HTTP 1.1 Protocol
Response
HTTP/1.1 200 OK
Server: dancer/2.0
Date: Tue, 14 Oct 2014 12:34:56 GMT
Content-Type: text/plain
Charset: iso8509-1
Content-Length: 456
You asked for something, here it is!

18. HTTP 1.1 Protocol
Request Methodes
• GET
• POST
• PUT
• DELETE
• HEAD
• OPTIONS
• PATCH

19. HTTP 1.1 Protocol
Header Fields
• Host
• User-Agent
• If-Modified-Since
• Content-Type
• Content-Length
• Location
• Server
• Last-Modification
• Content-Type
• Content-Length

20. HTTP 1.1 Protocol
Status Codes
• 1—— Informational
• 2—— Succes
• 3—— Redirection
• 4—— Client Error
• 5—— Server

21. Status: 404
Not Found

22. Status: 404
Not Found

23. Status: 500
Internal Server Error

24. HTTP 1.1 Protocol
Status 1—— Informational
• 100 Continue

25. HTTP 1.1 Protocol
Status 2—— Succes
• 200 OK
• 201 Created
• 202 Accepted
• 204 No Content

26. HTTP 1.1 Protocol
Status 3—— Redirection
• 300 Multiple Choices
• 301 Moved Permanently
• 304 Not Modified

27. HTTP 1.1 Protocol
Status 4—— Client Error
• 400 Bad Request
• 401 Unauthorized
• 403 Forbidden
• 404 Not Found
• 405 Method Not Allowed
• 407 Not Acceptable
• 410 Gone
• 412 Precondition Failed

28. HTTP 1.1 Protocol
Status 5—— Server Error
• 500 Internal Server Error
• 501 Not Implemented
• 503 Service Unavailable

29. It takes Two
Dancer 2 & LWP

30. Client Server
Let’s Dance
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
template ’user.tt’, $user_info;
};
dance;
1;

31. Client Server
Let’s Dance
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
template ’user.tt’, $user_info;
};
dance;
1;

32. Client Server
Let’s Dance
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
template ’user.tt’, $user_info;
};
dance;
1;

33. Client Server
Let’s Dance
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
template ’user.tt’, $user_info;
};
dance;
1;

34. Client Server
Let’s Dance
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
template ’user.tt’, $user_info;
};
dance;
1;

35. Client Server
Let’s Dance
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
template ’user.tt’, $user_info;
};
dance;
1;

36. Client Server
LWP::UserAgent
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $response = $agent->get(’/user/999-999’);

37. Caching
Temporary Storage
• Reduce cost
• Improve Responsivness

38. Caching
Temporary Storage
• Client Side
• Public Proxy
• Server Side

39. Caching
If-Modified-Since
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
GET => ’/user/999-999-999’);
$request->header(
If-Modified-Since => Tue, 14 Oct 2014 12:34:56
GMT );
my $response = $agent->request($request);

40. Caching
Response
• Response Status: 304 Not Modified
• Last-Modified: Tue, 07 Oct 2014 12:34:56 GMT
• Age: 3600
• Expire: Tue, 21 Oct 2014 12:34:56 GMT
• Cache-Control: public | private | max-age | no-cache

41. Conditional Requests
• Stateless
• No Resource Locking
• Only execute request if not modified since
last
• Only execute request if the resource is still
the same

42. Conditional Requests
If-Unmodified-Since
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
DELETE => ’/user/999-999-999’);
$request->header(
If-Unmodified-Since => ’Tue, 14 Oct 2014
12:34:56 GMT’);
my $response = $agent->request($request);

43. Conditional Requests
If-Match
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
DELETE => ’/user/999-999-999’);
$request->header(
If-Match =>
’a23dfe4532dab7b21a83d3e0f4c2a6f1’ );
my $response = $agent->request($request);

44. Conditional Requests
Response
• Response Status: 412 Precondition Failed
• Response Status: 428 Precondition Required
• Last-Modified: Tue, 07 Oct 2014 12:34:56 GMT
• ETag: a23dfe4532dab7b21a83d3e0f4c2a6f1

45. Content Negotiation
• the same resource
• another representation
• the same URL
• client can let the server know what type is preferred
• server will try to deliver in requested content
• caches need to know that this is another variant
• the same URL

46. Content Negotiation
Resources & Representation
• Uniform Resouse Identifier
• Does not say anything about representation:
• Charset
• Encoding
• Language
• Format

47. Content Negotiation
Accept & friends
• Accept: text/html, text/plain, image/png
• Accept-Language: nl, en, fr
• Accept-Charset: iso_
• Accept-Encoding: gzip

48. Content Negotiation
Accepting Preferences
• The client can have some nice preferences
• it might like some representation above the other
• it might not like anything else
• The server can only deliver some representations
• it might deliver something it prefers
• it might give a list of options

49. Content Negotiation
Mutable Serializer
use Dancer2;
use Dancer2::Plugin::Mutable::Serializer;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
return $user_info;
};
1;

50. Content Negotiation
Mutable Serializer
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
GET => ’/user/999-999-999’);
$request->header(
Accept => ’application/json’ );
my $response = $agent->request($request);

51. Content Negotiation
Mutable Serializer
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
GET => ’/user/999-999-999’);
$request->header(
Accept => ’application/json’ );
$request->header(
Content-Type => ’application/xml’ );
my $response = $agent->request($request);

52. Content Negotiation
Let’s do things differently
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)->find(id=>params(’id’));
if (grep $_ eq ’text/html’, header(’Accept’)) {
template ’user.tt’, $user_info; }
if (grep $_ eq ’application/json’, header(’Accept’’)) {
return to_json $user_info; }
if (grep $_ eq ’application/xml’, header(’Accept’’)) {
return to_xml $user_info; }
};
1;

53. Content Negotiation
Let’s do things differently
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
GET => ’/user/999-999-999’);
$request->header(
Accept => ’application/json’ );
my $response = $agent->request($request);

54. Content Negotiation
Let’s do things differently
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)->find(id=>params(’id’));
if (grep $_ eq ’text/html’, header(’Accept’)) {
template ’user.tt’, $user_info; }
if (grep $_ eq ’application/json’, header(’Accept’’)) {
return to_json $user_info; }
if (grep $_ eq ’application/xml’, header(’Accept’’)) {
return to_xml $user_info; }
};
1;

55. Content Negotiation
Let’s do things differently
use LWP;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(
GET => ’/user/999-999-999’);
$request->header(
Accept => ’application/xml; q=0.1, text/html’ );
my $response = $agent->request($request);

56. Content Negotiation
Let’s do things differently
use Dancer2;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)->find(id=>params(’id’));
if (grep $_ eq ’text/html’, header(’Accept’)) {
template ’user.tt’, $user_info; }
if (grep $_ eq ’application/json’, header(’Accept’’)) {
return to_json $user_info; }
if (grep $_ eq ’application/xml’, header(’Accept’’)) {
return to_xml $user_info; }
};
1;

57. Content Negotiation
Let’s get Messy
Accept: text/plain; q=0.3, text/html; q=0.5, */*; q=0.0
text/plain
q=0.3
text/html
q=0.5
*/*
q=0.0
«I’m fine with plain-txt, but like html more… but
if it’s anything else, I DON’T WANT THAT»

58. Content Negotiation
Let’s get Messy
use Dancer2;
use Dancer2::Plugin::HTTP::ContentNegotiation;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)->find(id=>params(’id’));
http_choose_accept (
’text/html’
=> sub {template ’user.tt’, $user_info},
’application/json’
=> sub {to_json $user_info},
’application/xml’
=> sub {to_xml $user_info},
);
};

59. Content Negotiation
Let’s get Messy
use Dancer2;
use Dancer2::Plugin::HTTP::ContentNegotiation;
get ’/user/:id’ => {
my $user_info
= resultset(’users’)-
>find(id=>params(’id’));
http_choose_accept (
[’image/png’, ’image/jpg’, ’image/gif’]
=> sub {magick(http_accept->minor)},
);
};

60. Content Negotiation
Resources & Representation
• Status 300: Multiple Choices
• format at the end of the URL
• language at the beginning of the URL
• Status 406: Not Acceptable
• Vary: Accept, Aceept-Language . . .

61. Auth
Resource Access
• Stateless
• Submit credentials with every request
• Authentication
• Username & Password
• Authentication Scheme
• Authorisation
• Rolebased access

62. Auth
Usual WEB handling
1. Attempt to acces some page
2. Not Authorised ?
3. Go to /login
4. Send credentials
5. Authenticated Now ?
6. Setup session cookie
7. Go back to original requested page

63. Auth
REST api
1. Attempt to acces some page
2. Not Authenticated ?
3. Status: 401 “Not Authorized”
4. Resend same request including credentials
5. Authorised ?
• Continue processing
• Status: 403 “Forbidden”

64. Auth
HTTP Auth::Extensible
use Dancer2;
use Dancer2::Plugin::HTTP::Auth::Extensible;
get '/realm' => http_require_authentication sub {
"You are logged in using realm: " . http_realm
};
get '/vodka' => http_require_role HardDrinker =>
sub {
"Only hard drinkers get vodka";
};

65. Auth
HTTP Auth::Extensible
use Dancer2;
use Dancer2::Plugin::HTTP::Auth::Extensible;
get '/realm' => http_require_authentication sub {
"You are logged in using realm: " . http_realm
};
get '/vodka' => http_require_role HardDrinker =>
sub {
"Only hard drinkers get vodka";
};

66. Auth
HTTP Auth::Extensible
use Dancer2;
use Dancer2::Plugin::HTTP::Auth::Extensible;
get '/realm' => http_require_authentication sub {
"You are logged in using realm: " . http_realm
};
get '/vodka' => http_require_role HardDrinker =>
sub {
"Only hard drinkers get vodka";
};

67. Auth
HTTP Auth::Extensible
plugins:
'HTTP::Auth::Extensible':
realms:
example:
provider: Config
users:
- user: ‘beerdrinker'
pass: ‘password'
name: 'Beer drinker’
roles:
- BeerDrinker

68. Auth
Resource Access
• Status 401: Unauthorized
• You should return a WWW-Authenticate
also
• HTTP Request Header Field: Authorize
• Status 403: Forbidden

69. Dancer2::Plugin::HTTP
Family
• Dancer2::Plugin::HTTP::ContentNegotiation
• Dancer2::Plugin::HTTP::Auth::Extensible
• Dancer2::Plugin::HTTP::Conditional
• Dancer2::Plugin::HTTP::Cache

70. Dancer2::Plugin::HTTP
HTTP::Header::ActionPack
• Authentication / Authorisation
• MIME-types
• DateTime conversion

71. Dancer2::Plugin::HTTP
HTTP::Header::ActionPack
• Authentication / Authorisation
• MIME-types
• DateTime conversion

72. Net::WebMachine
HTTP::Header::ActionPack
• Basho schema
• Ruby implementation
• Stevan Little / Dave Rolsky

73. Talking Heads
http://github/THEMA-MEDIA/
th.J.v.Hoesel@THEMA-MEDIA.nl