A simple introduction about HTTP2 and gRPC

1. gRPC
Jee-Arr-Pee-See

2. gRPC introduction
➤ what is gRPC
➤ why we us gRPC
➤ HTTP 1.1/ HTTP2
➤ TCP Lazy Start / Heartbeat
➤ Protobuf
➤ how gRPC works
➤ the good part
➤ the bad part
➤ what we do next

3. What is gRPC
➤ gRPC is a modern, open source remote procedure call (RPC) framework that can run
anywhere. It enables client and server applications to communicate transparently,
and makes it easier to build connected systems.
Python Client
gRPC Server
Java Client
gRPC with GO!

4. why we us gRPC - HTTP
➤ We try to use Protobuf and HTTP
➤ HTTP is easy for everyone!
➤ We need to do a lot to make HTTP better performace
➤ We need to think about time, package size and safety
➤ Transport need to trunked
Web Proxy Jedi
Account
NOSQL
DB
200ms
200ms
200ms
100ms
10ms

5. why we us gRPC - How about thrift?
➤ Thrift is based on raw socket
➤ This is super fast!
➤ But hard for develop
➤ We need to look into the connection
➤ Not very good for mobile
App Server
sorry? we lose connection?
OK! We create a new one! I response! But no FIN!
I have to WAIT!

6. why we us gRPC - gRPC
➤ IT IS HTTP but version 2
➤ Better performance than HTTP1
➤ It using Protobuf which is binary
➤ HTTP2 Transport layer using binary
➤ Persistent Connection

7. What's problem with HTTP1
➤ textual
➤ ordered and blocking
➤ large size of HEADER - most are cookies
➤ Persistent Connection may not work for every system
➤ multi connection - TCP Slow Start
➤ can't push from server - web socket is not for everywhere

8. HTTP1 - textual
https://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/draft-lafon-rfc2616bis-latest.html#message.types
HEADER
BODY
CRLF

9. HTTP1 - textual
Cookie: A=1
HEADER-1: HELLO
nr
This is the body of http request or response.
What da luo li ji?

10. HTTP1 - ordered and blocking
Browser
or
APP
Server
give me html
give me css
give me js
That why we put js at the end of the content, it will block
and why we make icon together and file small...

11. HTTP1 - ordered and blocking
Browser
or
APP
Server
give me more ...
Limit of TCP connection
may block heartbeat
Slow Start of TCP

12. HTTP1 - Large size of HEADER
Cookie: AWSELB:
AD910F851AAED57C481223E777E0EB4FC465E931AB9639541C1EC1B1D5723D2290FDEAD3216B4F9B93C1B30CD47F245B24175467919E3B654E480345440B279CA95626A3C8;
Hm_lpvt_5e1b9d2c20eff9c9278a43f0870f4593: 1458451373; Hm_lvt_5e1b9d2c20eff9c9278a43f0870f4593: 1458451373; __ads_session: k34 wk5FrsQjw5w0ACgA
Response: 200 OK
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:X-Requested-With, content-type, accept, origin, authorization, x-csrftoken
Access-Control-Allow-Methods:GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Origin:*
Connection:keep-alive
Content-Encoding:gzip
Content-Language:en-US
Content-Length:6833
Lots of content make things bad

13. HTTP1 - Persistent Connection
➤ Persistent Connection need heartbeat but...
➤ large size content may block heartbeat
BODY❤️
This is a connection!
➤ So we need to make body small
➤ Make them fail fast
BODY❤️BODY❤️
Read more...

14. HTTP1 - TCP Slow Start
➤ Yes! But we could use multi connection to solve this problem, but ...
➤ TCP Slow Start...
End Point End Point
let me send you 1
It's ok send me 2
let me send you 2
It's ok send me 4

15. HTTP1 - TCP Slow Start
➤ If there is a problem it will make less data
➤ So multi connection with large data is not so good hmmm...
End Point End Point
let me send you 4
It's ok send me 8
let me send you 8
no! my buffer is full, send me 4!

16. HTTP1 - Can't push from server
➤ Web socket is good
➤ Different browser has different support
➤ What if the network is so bad?
➤ What about the APP?

17. HTTP2
➤ binary
➤ multiplexed
➤ one connection for parallelism
➤ make data smaller
➤ uses header compression to reduce overhead
➤ server push

18. HTTP2 - binary, so this THE frame
https://http2.github.io/http2-spec/
LENGTH (24)
Stream Identifier
Type (8) Flags (8)
Frame payload
R

19. HTTP2 - DATA FARME
PAD LENGTH (8)
Stream Identifier
Frame payload
➤ DATA frames (type=0x0) convey arbitrary, variable-length sequences of octets associated with a stream. One
or more DATA frames are used, for instance, to carry HTTP request or response payloads.

20. HTTP2 - HEADER FARME
PAD LENGTH (8)
Stream Dependency ? (31)
Weight (8)
➤ The HEADERS frame (type=0x1) is used to open a stream (Section 5.1), and additionally
carries a header block fragment.
E
Header Block Fragment (*)
Padding (*)

21. HTTP2 - CONTINUATION FARME
Header Block Fragment (*)
➤ The CONTINUATION frame (type=0x9) is used to continue a sequence of header block
fragments .

22. HTTP2 - Stream
➤ A "stream" is an independent, bidirectional sequence of frames exchanged between the client
and server within an HTTP/2 connection
➤ A single HTTP/2 connection can contain multiple concurrently open streams, with either
endpoint interleaving frames from multiple streams.
➤ Streams can be established and used unilaterally or shared by either the client or server.
➤ Streams can be closed by either endpoint.
➤ The order in which frames are sent on a stream is significant.
➤ Streams are identified by an integer. Stream identifiers are assigned to streams by the endpoint
initiating the stream.

23. HTTP2 - Stream Prioritization
➤ Once an HTTP message can be split into many individual frames, and we allow for frames from multiple streams to be
multiplexed.
➤ Each stream may be assigned an integer weight between 1 and 256
➤ Each stream may be given an explicit dependency on another stream
Stream 1
Stream 2
Stream 3
Stream 4
F
F
F
F

24. HTTP2 - Stream Prioritization
➤ So...
*
A 12 B 4
*
D 1
C 8
*
E 8 F 8
D 4
B 4 A 12
root stream root stream root stream

25. CONTINUATIONCONTINUATIONCONTINUATION
END_HEADERS
HTTP2 - Header Compression and Decompression
➤ The CONTINUATION frame (type=0x9) is used to continue a sequence of header block fragments .
➤ a single HEADERS or PUSH_PROMISE frame, with the END_HEADERS flag set
➤ a HEADERS or PUSH_PROMISE frame with the END_HEADERS flag cleared and one or more
CONTINUATION frames, where the last CONTINUATION frame has the END_HEADERS flag set.
HEADERS
END_HEADERS
PUSH_PROMISE
END_HEADERS
HEADERS

26. HTTP2 - Server Push
➤ of course
➤ but we did not do that :)

27. so we choose Jee-Arr-Pee-See
-DaGuGuJi

28. gRPC
➤ based on HTTP2
➤ based on Protobuf

29. Protobuf 3
➤ Protobuf is binary
➤ Support format to JSON, XML and Thrift
➤ Support multi Programming Language
C++
C#
Java
GO
Python

30. Protobuf 3
➤ Simple and Easy
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
}
Person john =
Person.newBuilder()
.setId(1234)
.setName("Guojing")
.setEmail("g@vipkid.com")
.build();
output = new
FileOutputStream(args[0]);
john.writeTo(output);

31. Show me the gRPC!
-Big Family

32. gRPC
➤ gRPC support multi Programming language too
➤ Python, PHP, Ruby ... are based on gRPC C Core
➤ gRPC-Java
➤ gRPC-GO
➤ simple RPC and Stream RPC!

33. gRPC
➤ scripting language is easy for develop
➤ but lower performance than C/C++/Java/GO
➤ but language performace is not what we consider these days
➤ best choice is GO
➤ run gRPC-Java on JVM - not a web project

34. gRPC
➤ run as a service, not part of web project
gRPC server gRPC server gRPC server
Proxy like tomcat or flask
FE

35. gRPC
➤ run as a service, if part of web project, we may go die
FE
tomcat or flask
gRPC Server
tomcat or flask
gRPC Server
tomcat or flask
gRPC Server
I don't know which one is dead with
what protocol

36. gRPC
➤ deploy will be easier if we ...
gRPC server
Proxy like tomcat or flask
FE
Proxy like tomcat or flask
nginx
gRPC server gRPC server gRPC server gRPC server
One gRPC Server is down
web is still working!

37. How gRPC works
➤ Server
➤ Stub
➤ Channel
➤ CompletionQueue

38. How gRPC works - C Core
➤ Server
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
def serve():
server = route_guide_pb2.beta_create_RouteGuide_server(RouteGuideServicer())
server.add_insecure_port('[::]:50051')
server.start()

39. How gRPC works - C Core - Server
➤ Server is just Start multi TCP Server Listener
➤ Server Start ---> for each tcp server start
➤ Tcp Server waiting for stream and frame
➤ Check frame type then call the Frame parser
➤ Run callback

40. How gRPC works - C Core - Channel
➤ Channel is connection maintainer
➤ Channel init status is IDLE
➤ If a client call, Channel status will be READY
➤ We could subscribe a channel with callback
➤ Also unsubscribe channel
➤ Channel could reconnect automatically
Channel
ConnectionConnection

41. How gRPC works - C Core - Stub
➤ Stub is just a function wrapper
➤ It create a Call with a Ticket
Channel
Call With Ticket

42. How gRPC works - C Core - Channel
➤ Channel is complicate
➤ Channel has it stack
➤ Channel has filter
➤ Every Call to Channel need to copy Channel filter to Call element filter

43. How gRPC works - C Core - CompletionQueue
➤ When server receive stream
➤ After parse to object and then call the callback function
➤ It may enqueue
➤ The Programming language layer (Python) will create a thread to get the element
from the queue with registered callback

44. How gRPC works - C Core - CompletionQueue
Completion Queue
Main Thread
Sub Thread
EventEventEvent

45. gRPC - The good part
➤ Based on HTTP2
➤ Fast
➤ We could use Thread Pool
➤ From Google and someone is working with gRPC
this is important because we don't want a frame work no on maintain

46. gRPC - The bad part
➤ We need to know everything is not good enough
➤ So still bug

47. gRPC - The bad part
➤ Stream stack overflow then abort() ...
➤ Because a channel has max request number
➤ Server may not create such threads > 1000
➤ Install monit to restart when it down
➤ We have haproxy to make thread and request number in control

48. gRPC - The bad part
➤ Persistent Connection will (maybe) not load balanced
➤ Because IT IS Persistent Connection

49. gRPC - The bad part
Proxy
gRPC gRPC
at least one machine is IDLE

50. gRPC - The bad part
Proxy
gRPC gRPC

51. gRPC - The bad part
Proxy
gRPC gRPC
we need zookeeper or consul

52. gRPC - The bad part
Proxy
gRPC gRPC
Channel subscribe callback

53. gRPC - The bad part
➤ We change service, then we need to rebuild in both client
➤ We have lots of end points, how to maintain?
➤ That it is, we must be smart.

54. gRPC - What problem we have now
DEVELOP
with
UNIT TEST
CI
➤ We will have a lot service
➤ We need to have UNIT TEST and CI
➤ We need to make everyone start a project easy
docker
vagrant
Mac OS
Linux

55. gRPC - What problem we have now
Service
Test
+
Smoke
Test
+
End
Test
➤ QA team to make things faster
➤ Service deploy need to be fast
➤ We can't test every part of the service
➤ This need to be auto

56. gRPC - What problem we have now
Deploy on
THE
VERSION
that
passed
CI
and
Test
➤ SA Team need to
➤ Deploy fast
➤ Rollback fast
➤ Pre-Release
➤ A/B Test

57. gRPC - What problem we have now
After
Release
➤ Log !
➤ Performance
➤ PM need to check the result
➤ Exception Collection
➤ Hot fix! (add smoke test)

58. gRPC - We need to have a public develop team to
VIPKID
Toolkit
➤ We need to have a public develop team to:
➤ SDK
➤ Toolkit
➤ Automatically system for whole thing like - (test
request in gitlab CI)

59. We are a team
➤ We need to be faster
➤ We need to think more and different
➤ We don't blame A person
➤ We make things better

60. May the force be with you

61. FAQ

62. Thanks