RubyKaigi 2016 http://rubykaigi.org/2016/presentations/tkawa.html

1. Web Clients for Ruby
and
What they should be in the future
Toru Kawamura
@tkawa
RubyKaigi 2016

2. @tkawa
Toru Kawamura
• RESTafarian
inspired byYoheiYamamoto (@yohei)
• Technology Assistance Programmer at
SonicGarden Inc.
Programmer at PlayLife Inc.
(on the side)
• Co-organizer of Sendagaya.rb
(Regional Rubyist Community & Every Monday Meetup)
https://sendagayarb.doorkeeper.jp/
• Facilitator of RESTful-towa (“What is
RESTful”) Workshop
(Monthly, next 2016-09-13 in Omotesando)
https://rubychildren.doorkeeper.jp/

3. I’m going to talk about
• Human-driven client written in Ruby that accesses a Web API
• A part in server-side app that accesses a Web API is also a client
• The idea of “Web Client” gem
• The thoughts and the findings from creating this gem

4. I don’t want a client that is…
• Rigid because of being tightly coupled
• Hard to reuse because of too much dedication

5. I want a client that is…
• Adaptable to change because of being decoupled
• Easy to reuse because of versatility

6. I want a client that is…
• Adaptable to change because of being decoupled
• Easy to reuse because of versatility

7. HYPERMEDIA:THEMISSINGELEMENT
to Building Adaptable Web APIs in Rails
RubyKaigi 2014
–Johnny Appleseed

8. HYPERMEDIA:THEMISSINGELEMENT
Many clients are built from
human-readable documentation
GET /v1/statuses?id=#{id} GET /v1/statuses?id=#{id}

9. HYPERMEDIA:THEMISSINGELEMENT
GET /v2/statuses/#{id} GET /v1/statuses?id=#{id}
×Need to rewrite code

10. HYPERMEDIA:THEMISSINGELEMENT
{
uber: {
version: "1.0",
data: [{
url: "http://www.ishuran.dev/notes/1",
name: "Article",
data: [
{
name: "articleBody",
value: "First note's text"
},
{
name: "datePublished",
value: null
},
{
name: "dateCreated",
value: "2014-09-11T12:00:31+09:00"
},
{
name: "dateModified",
value: "2014-09-11T12:00:31+09:00"
},
{
name: "isPartOf",
rel: "collection",
url: "/notes"
},
{
• API changes should be reflected in clients
• It is good to split up explanations of the API and
embed them into each API response
• A lot of assumptions about the API make a tight
coupling
Because of Coupling

11. HYPERMEDIA:THEMISSINGELEMENT Decoupling in a example:
FizzBuzzaaS
• by Stephen Mizell
http://fizzbuzzaas.herokuapp.com/
http://smizell.com/weblog/2014/solving-fizzbuzz-with-hypermedia
• Server knows how to calculate
FizzBuzz for given number (<= 100)
• Server knows what the next
FizzBuzz will be
• Client wants all FizzBuzz from one to
the last in orderhttp://sef.kloninger.com/posts/
201205fizzbuzz-for-
managers.html

12. HYPERMEDIA:THEMISSINGELEMENT
Coupled client
• Every URL and parameter is hardcoded
• Duplicates the server logic such as counting
up
"/v2/fizzbuzz/#{i}"
(1..1000)
(1..100).each do |i|
answer = HTTP.get("/v1/fizzbuzz?number=#{i}")
puts answer
end

13. HYPERMEDIA:THEMISSINGELEMENT
Decoupled client
• No hardcoded URLs
• Client doesn’t break when changing
URLs / the restriction
root = HTTP.get_root
answer = root.link('first').follow
puts answer
while answer.link('next').present?
answer = answer.link('next').follow
puts answer
end Link ‘next’ is the key

14. I want a client that is…
• Adaptable to change because of being decoupled
• Easy to reuse because of versatility

15. HTTP Clients for Ruby
• Standard equipment libraries
• net/http
• open-uri

16. http://bit.ly/RubyHTTPClients

17. HTTP Clients for Ruby
• Feature comparison by nahi
• 「大江戸HTTPクライアント絵巻」Oedo
RubyKaigi 01 (2011-04-10)
• http://regional.rubykaigi.org/oedo01/
• “net/http has various derivatives and alternatives
because of old-style API and simple structure”

18. Web API is easy to use
• We can use one right away with net/http or other HTTP client
• HTTP has the uniform interface
• We can also use it with web browser or curl
• That’s why Web API becomes popular
• → Do you really use net/http or other HTTP client in your app?

19. There are so many gems dedicated to each Web API
• google-api-client, aws-sdk, octokit, twitter, koala, … (looks similar inside)
• Pros
• The gem provides classes corresponding to data types of Web API
• The gem can support detailed specs dedicated to the Web API
• Using classes and method calls, you can write a code with less thinking of Web API
• Cons
• The way of use differs depending on the gem
• You have to read a gem’s documentation instead of API’s

20. There are so many gems dedicated to each Web API
• What if you are on the side of providing a gem?
• You have to re-design an interface different from the Web API
• You have a lot of trouble creating multi-language library if you need
• Some client library reproduce the same class/method structure as in server-side
• It has CRUD mappings in HTTP communication
• But I think it would be better for such a complex API to use RPC
• Web API should be easy for everyone to use!

21. What makes us produce so many dedicated gems?
• Difference between JSON structure of each Web API
• Handling dedicated error, more detailed than 4xx
• Gap between calling API once and performing a function

22. Gap between calling API once and performing a function
• We want to perform a function provided by Web API, rather than just call it
• Fetch current data, then update old one if it exists
• Fetch the past 1000 records using the API that returns 100 records
limited at once
• In human-driven client, they rarely accomplish their goal in single API call
• A Client app is made up of many functions (or microservices)

23. Gap between calling API once and performing a function
• How does a client decide what API to call next?
• allow the user to choose or choose by itself from options
• The options are hardcoded in a gem
• The gem defines some classes and methods, which are statically
mapped on APIs
• The options should depend on what “state” the client is in

24. State management
• HTTP client doesn’t have a state
• App have a state
• What screen the app is in now
• What screen the app came from
• What does the app show/select now
• In a classic web app, an app state is
represented by the current URL

25. State transition on Web API
• App have a state for deciding what API
to call next
• It is better for HTTP client to have such
a state
• and get close to web browser that
makes state transition in a way to
follow a link
• It depends on the app how faithful the
screen reflects the transition ”RESTful Web APIs” p.11 Figure 1-7

26. I want a client that is…
Adaptable to change because of being decoupled
Easy to reuse because of versatility
Capable of state management
= Web Clients*
* definition in this talk

27. Consider in terms of
implementation layers for client & server

28. Framework
App Server
HTTP Client
Client App
Web API App
Request Response

29. Rack
Web API App
Framework
App Server
HTTP Client
Client App
Request Response

30. • Rack provides an interface between
web server and ruby app
• An object based on Rack interface is
called “Rack App”
• Web app built on Sinatra/Rails is also a
Rack App
rack_app = Proc.new do |env|
[
'200',
{'Content-Type' => 'text/html'},
['A barebones rack app.']
]
end
Rack::Handler::Puma.run rack_app
by Christian Neukirchen

31. Rack App requirements
• An object that responds to the call method,
• Taking the env hash as an argument,
• Returning an array with three elements:
• HTTP status code
• Hash of response headers
• Array filled with response body
rack_app = Proc.new do |env|
[
'200',
{'Content-Type' => 'text/html'},
['A barebones rack app.']
]
end
Rack::Handler::Puma.run rack_app

32. Rack Middleware
• Between the server and the framework, Rack Middleware can customize the
request/response and process data to your applications needs
• Rack::URLMap, to route to multiple applications inside the same process
• Rack::CommonLogger, for creating Apache-style logfiles
• Rack::Static, for serving static files in specific directories
• Rack::Reloader, Rack::ContentLength, Rack::Auth::Basic, Rack::MethodOverride, …

33. Rack Middleware requirements
• A class that takes the other Rack App,
then instantiates a wrapped Rack App class FooMiddleware
def initialize(app)
@app = app
end
def call(env)
# do something in request
res = @app.call(env)
# do something in response
res
end
end
http://docs.pylonsproject.org/projects/pylons-
webframework/en/latest/concepts.html#wsgi-middleware

34. Rack Middleware structure
wrapped_app = Rack::Builder.new do
use Rack::ContentLength
use Rack::CommonLogger
use FooMiddleware
run rack_app
end.to_app
Rack::Handler::Puma.run wrapped_app
Rack::ContentLength
Rack::CommonLogger
FooMiddleware
rack_app

35. Rack
Rack Middleware
Framework
Web API App
App Server
HTTP Client
Client App
Request Response

37. Faraday
• Faraday is an HTTP client library that provides a common interface
over many adapters (such as net/http)
• and embraces the concept of Rack Middleware when processing
the request/response cycle
by Rick Olson, Zack Hobson

38. Faraday Middleware
• Mechanism for customizing a request/response like Rack Middleware
• url_encoded, to encode parameters into x-www-form-urlencoded in request
• authorization, to add an auth token to request header
• json(ParseJson), for converting JSON of response body into Hash
• follow_redirects
• http_cache
• rack-compatible, to use a rack middleware as a faraday middleware(experimental)

39. Faraday Middleware requirements
• Very similar to Rack Middleware
• processing response in on_complete
block
class BarMiddleware
def initialize(app)
@app = app
end
def call(env)
# do something in request
@app.call(env).on_complete do |res_env|
# do something in response
end
end
end

40. Faraday::Request::Authorization
FaradayMiddleware::ParseJson
BarMiddleware
Faraday::Adapter::NetHttp
Faraday Middleware structure
conn = Faraday.new('https://api.github.com') do |b|
b.request :authorization
b.response :json
b.use BarMiddleware
b.adapter Faraday.default_adapter
end
res = conn.get('/')
Adapter corresponds
to Rack App

41. Rack
Rack Middleware
Framework
Web App / Web API
App Server
Faraday
Faraday Middleware
Adapter
Client App
Request Response

42. Rack
Rack Middleware
Framework
Web App / Web API
App Server
Faraday
Faraday Middleware
Adapter
Client App
Request Response
Build a gem not as a whole but as a Faraday Middleware
• Reusable
• Respect a common interface

43. Implemented Middleware
• https://github.com/tkawa/faraday-hypermedia
• faraday-navigation
• faraday-link-extractor

44. faraday-navigation
• Allow us to go back/forward using a history like a common web
browse
• Allow us to follow a link
• And fill in parameters of URL just like an HTML form field

45. • Link Header from RFC 5988 (Web Linking)
• Link-Template Header from Internet-Draft
(draft-nottingham-link-template-01; expired)
• URITemplate from RFC 6570
Link/Link-Template Header
Link: <https://api.github.com/users/tkawa/repos?page=2>; rel="next"
Link-Template: <https://api.github.com/search{?q}>; rel="search”

46. faraday-link-extractor
• Extract links in each kind of Web API and translate them into Link/
Link-Template header
• LinkExtractorCJ (Collection+JSON)
• LinkExtractorGithub (GitHub)

47. Extract Links into Header
(in the case of GitHub)
{
"login": "tkawa",
"id": 562433,
"url": "https://api.github.com/users/tkawa",
"followers_url": "https://api.github.com/users/tkawa/followers",
"following_url": "https://api.github.com/users/tkawa/following{/other_user}",
...
}
Link: <https://api.github.com/users/tkawa>; rel="self",
<https://api.github.com/users/tkawa/followers>; rel="followers"
Link-Template: <https://api.github.com/users/tkawa/following{/other_user}>;
rel="following”
url/*_url treated as a link

48. history = Faraday::Hypermedia::History.new
conn = Faraday.new(url: 'https://api.github.com') do |b|
b.use :navigation, history
b.request :authorization, ‘bearer', token
b.response :json
b.response :link_github
b.adapter Faraday.default_adapter
end
res = conn.get('/'); history.pp_current_links
res = conn.get('navigation:link?rel=current_user')
res = conn.get('navigation:link?rel=repos')
res = conn.get('navigation:link(2)?rel=item')
res = conn.get('navigation:back')
res = conn.get('navigation:link?title=hypermicrodata')
history.fill_in_template_params(number: 1)
res = conn.get('navigation:link?rel=pulls')
⭐
⭐

49. Demo
/
current_user
/user
/users/tkawa/repos
/repos/tkawa/activerecord-endoscope
/repos/tkawa/hypermicrodata
/repos/tkawa/hypermicrodata/pulls/1
repos
item#2 back
title=hypermedia
pulls
https://asciinema.org/a/85363

50. • Make it decoupling
• Tight-coupling over a boundary between client and server makes it hard to change
• Taking advantage of Ruby, dynamic processing lead to decoupling
• Enable to Reuse
• Clip the app/domain-specific part
• Designing along with standards including RFC, we can use general-purpose library
• Build single-function component based on combinable interface such as Faraday
Middleware
Conclusion