Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Consistent RESTful APIs in a high-performance environment

This is one of two presentations given by LinkedIn engineers at Java One 2009.

This presentation was given by Brandon Duncan, Director of Engineering, and Yegor Borovikov, Software Architect at LinkedIn.

For more information, check out

  • Login to see the comments

Building Consistent RESTful APIs in a high-performance environment

  1. 1. Building Consistent RESTful APIs in a High- Performance Environment Yegor Borovikov, Software Architect Brandon Duncan, Director of Engineering LinkedIn Corporation
  2. 2. Topics We’ll Cover > Examples of RESTful APIs  What’s missing?  Variety versus Uniformity > Domain Model as Foundation  Provides Uniformity  Allows Flexibility > Examples of LinkedIn APIs > Using Incentives to Scale  Some LinkedIn production APIs metrics > Q&A 2
  3. 3. Examples of RESTful APIs What if you want to get a person’s profile? > Use one of these…{guid}/profile{guid}{guid}/@self <?xml version="1.0" encoding="UTF-8"?> <person> <id>111222</id> <first-name>Gertrude</first-name> <last-name>Stein</last-name> <headline>Author of Tender Buttons</headline> <connections total="76"> … </person> 3
  4. 4. Examples of RESTful APIs What’s Missing? > Ability to get exactly what you need (variety)  If you need more, may require multiple API calls (if they exist)  If you need less, resources are wasted > Consistency of responses (uniformity)  “Same” object returned by different APIs may have different structure  Once in production, hard to get consistent later 4
  5. 5. Examples of RESTful APIs Multiple Calls to Get What You Need > Want to get user’s friend’s profile? Do this…  <connections yahoo:start="0" yahoo:count="1" yahoo:total="1"> <connection yahoo:uri=" view=usercard"> <guid>456</guid> <contactId>4</contactId> </connection> </connections> 5
  6. 6. Examples of RESTful APIs Multiple Calls to Get What You Need > … then make second call to get friend’s profile: <profile yahoo:uri=""> <guid>456</guid> <birthdate>3/3</birthdate> <created>2008-08-4T17:13:56Z</created> ... </profile>  Latent, redundant data  Optimization requires stickiness 6
  7. 7. Typical Solution Variety versus Uniformity > Solution: introduce another call > Desire for variety of responses undermines uniformity of requests > Leads to RPC-like REST APIs > Many APIs + Great Documentation = Lots of Reading + Lack of Automation 7
  8. 8. Domain Model as Foundation Sample Domain Model /people : Person[] // collection of Person resources /id : string // primary key /name : string /email : string // unique key /photo : url /best-friend : Person /friends : Person[] /jobs : Job[] // collection of Job resources /company : Company /title : string /start-date : date /end-date : date … /companies : Company[] /name : string /ceo : Person … 8
  9. 9. Domain Model as Foundation Follow request URL to navigate through your model > To get a person’s profile: /people[/id=123] <person uri=“urn:linkedin:v2:people/123” key=“123”> /id <id>123</id> /name <name>Reid Hoffman</name> /email /photo <email></email> /best-friend <best-friend uri=“urn:linkedin:v2:people/456”/> /friends … /jobs /company </person> /title /start-date /end-date …  Conventional URL in request /companies /name  Default representation in response /ceo … 9
  10. 10. Domain Model as Foundation Fine-grained Request > What if you only need certain fields (e.g., name and photo)?,photo) <person> <name>Reid Hoffman</name> /people[/id=123] <photo></photo> /id </person> /name /email /photo /best-friend /friends /jobs /company /title /start-date /end-date … 10
  11. 11. Domain Model as Foundation Fine-grained Request > To get names and photos of one’s friends and their best friends: …/v2/people/456/friends:(name,photo,best-friend: (name,photo)) /people[/id=456] /id <friends total=“66” start=“0”> /name <friend uri=“urn:linkedin:v2:people/123” key=“123”> /email /photo <name>Reid Hoffman</name> /best-friend <photo></photo> /friends <best-friend uri=“urn:linkedin:v2:people/456” key=“456”> /123 /id <name>Brandon Duncan</name> /name <photo></photo> /email /photo </best-friend> /best-friend </friend> /name <friend>…</friend> /photo /jobs </friends> … 11
  12. 12. Domain Model as Foundation Fine-grained Request > Allows client to construct custom calls > Better than digging for the closest matching API: > Allows optimization on the backend 12
  13. 13. Domain Model as Foundation Benefits > Provides a frame for both request and response semantics > Still allows for flexible syntax  Requests – path, query params, matrix params…  Responses – JSON, XML, POJOs, protobuff… > Helps to unify and automate many development tasks on both ends  Request / response creation, parsing, marshalling  Code (and documentation) generation  Discovery services 13
  14. 14. Examples of LinkedIn APIs HTTP GET - Read …/people/ …/people/123/friends;sort=name:(name,jobs;sort=start-date) …/people:(id,name,photo)?name=page&company=google …/people::(123,456) …/people::(123,456):(name,photo) 14
  15. 15. Examples of LinkedIn APIs HTTP PUT - Update > Set the user’s name: /people[/id=123] /id /name PUT /email <name>Reid Hoffmann</name> /photo /best-friend … > Update the user’s profile - change name and best- friend and remove photo: PUT <person> /people[/id=123] <name>Reid Hoffman</name> /id /name <best-friend uri=“urn:linkedin:v2:people/999”/> /email <photo xsi:nil=“true”/> /photo </person> /best-friend … 15
  16. 16. Examples of LinkedIn APIs HTTP POST - Create > Add a friend /people[/id=123] /id POST /name /email <friend uri=“urn:linkedin:v2:people/888”/> /photo /best-friend 201 Created /friends /456 Location: /888 … 16
  17. 17. Examples of LinkedIn APIs HTTP DELETE - Remove > Remove a friend DELETE > Delete a company DELETE > Delete two companies DELETE,665) 17
  18. 18. Use Standard Headers > Content-Type > Last-Modified > Accept > Vary > Authorization > Cache-Control > Content-MD5 > Location > Warning 18
  19. 19. Incentive System > Multiple ways to get at the same data > Partner can ask for exactly what they need > Associate cost with resources, system of accounting creates incentives for partners > Throttling by resource rather than API 19
  20. 20. Real-World Example Xobni Toolbar > Xobni makes ~20 million profile API calls per week > Default representation is ~2k on average > Using in-line filter brings average to ~1.5k  25% reduction in response size  ~11000 Mbits savings per day  11k Mbits out of LinkedIn datacenter  11k Mbits into Xobni datacenter  Saves both companies money 20
  21. 21. Yegor Borovikov Brandon Duncan 21