gRPC is best suited for microservice communication. gRPC is fast, clear and powerful. It is an excellent alternative to address the verbose client problem when architecting a microservice infrastructure. But the legacy environment is always a big hurdle for changes. You must support existing clients that only understand RESTful HTTP API. In other cases, you need to provide RESTful APIs to the outside world. This session suggests solutions to resolve these problems. The session covers: - Why the team chose gRPC as the inter-service communication protocol while moving from a monolith to microservices and the challenges they faced. - How they leveraged Istio to support RESTful APIs using gRPC servers without additional development. - How they set up CI/CD to deliver API changes (including legacy API) using Helm and Spinnaker. - What they have learned through it and future improvements. gRPC는 마이크로서비스 커뮤니케이션에 가장 적합합니다. gRPC는 빠르고 명확하고 강력합니다. 이는 마이크로서비스 인프라를 설계할 때 복잡한 클라이언트의 문제를 해결하는데 있어 훌륭한 대안입니다. 하지만 기존 레거시 환경은 항상 변화의 큰 장애물입니다. RESTful HTTP API만을 이해하는 기존 클라이언트를 지원해야 합니다. 다른 경우, RESTful API를 외부에 제공해야 합니다. 본 세션에서는 이러한 문제를 해결할 솔루션을 제안합니다. 이 세션에서 다루는 내용: - 팀이 모놀리스에서 마이크로서비스로 전환하면서 서비스 간 커뮤니케이션 프로토콜로 gPRC를 선택한 이유 및 직면했던 난관들 - 추가 개발 없이 gRPC 서버를 이용해 RESTful API를 지원하기 위해 이스티오를 활용한 방법 - 헬름 및 스피네이커를 사용해 API 변경 사항 (레거시 API 포함)을 전달하기 위해 CI/CD를 설정하는 방법 - 이를 통해 배운 것과 앞으로 개선할 점

2. Running gRPC Services
for Serving Legacy RESTful API on
Kubernetes

3. Who We Are
Sungwon Lee / Whale
Buzzvil Chief Architect
Hoseong Hwang / Liam
Buzzvil DevOps Lead

4. Why?
Why did we choose gRPC?
Why do we still need to support REST API?

5. Why microservices architecture?
There are so many
domains and stakeholders
in ad-tech industry.

6. Why gRPC?
Performance matters
Multiple services increases network latency
Ad request should be done within 100ms
API-first Approach
Need to support polyglot
IDL(Interface Definition Language) required

7. gRPC is great, but..
We still need to support legacy RESTful JSON API
There are partners
using the legacy APIs
REST APIs:
mostly beloved
API protocol
B2B business

8. Support both gRPC/REST protocols
Build a server for transcoding?
gRPC ServerREST JSON
Transcoding
server
Client
Expensive maintenance cost
1. Parse JSON request
2. Transform JSON to gRPC
Request
3. Send request to gRPC Server
4. Transcode gRPC response to
JSON format
5. Send a response to the client
It seems familiar!

9. Istio Service Mesh
Moving to Microservices
Micoservices grow in size and complexity
Difficult to understand and manage
Service Mesh
Detach network logic from business logic
Monolythic Microservice
Microservice Proxy
Microservice Proxy

10. Istio Service Mesh

12. Let’s Try It !
Setup Protobuf for Custom Routes
Setup Istio/EnvoyFilter for gRPC JSON transcoder

13. Setup Protocol Buffers
service CalendarApi {
rpc ListEvents(ListEventsRequest) returns (ListEventsResponse);
// ...
}
The easiest way: Do nothing.
Transcoder will handle it automatically.

14. Setup Protocol Buffers (Cont.)
package buzzvil.calendar.v1;
service CalendarApi {
rpc ListEvents(ListEventsRequest) returns (ListEventsResponse);
}
$ curl -X POST https://host.here/buzzvil.calendar.v1.CalendarApi/ListEvents
=
POST /<package>.<service>/<method>

15. Setup Protocol Buffers (Cont.)
package buzzvil.calendar.v1;
service CalendarApi {
rpc ListEvents(ListEventsRequest) returns (ListEventsResponse) {
option (google.api.http) = { get: "/v1/events" };
}
}
Custom routes: google.api.http annotation
$ curl -X GET https://host.here/v1/events

16. Setup Protocol Buffers (Cont.)
Parameters
service CalendarApi {
rpc UpdateEvent(UpdateEventRequest)
returns (Event) {
option (google.api.http) = {
put: "/v1/events/{event.id}",
body: "event"
};
}
}
message Event {
int64 id = 1;
string name = 2;
}
message UpdateEventRequest {
Event event = 1;
}

17. Setup Envoy Proxy
Let’s try Istio /
EnvoyFilter to setup
envoy sidecar proxy

18. Setup Envoy Proxy (Cont.)
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: example-transcoder
namespace: example-namespace
spec:
workloadLabels:
...
filters:
- listenerMatch:
listenerType: SIDECAR_INBOUND
filterName: envoy.grpc_json_transcoder
filterType: HTTP
filterConfig:
proto_descriptor: “path/to/bin”
match_incoming_request_route: True
auto_mapping: False
services:
- buzzvil.calendar.v1.CalendarApi
filterName: envoy.grpc_json_transcoder
match_incoming_request_route: True
Services:
- buzzvil.calendar.v1.CalendarApi
Apply grpc_json_transcoder
filter to the services
Use the specified route

19. Setup Envoy Proxy (Cont.)
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: example-transcoder
namespace: example-namespace
spec:
workloadLabels:
...
filters:
- listenerMatch:
listenerType: SIDECAR_INBOUND
filterName: envoy.grpc_json_transcoder
filterType: HTTP
filterConfig:
proto_descriptor: “path/to/bin”
match_incoming_request_route: True
auto_mapping: False
services:
- buzzvil.calendar.v1.CalendarApi
Proto Descriptor
Envoy has to know the proto descriptor of
your gRPC service in order to the
transcoding.
$ protoc -I$(GOOGLEAPIS_DIR) -I.
--include_imports
--include_source_info
--descriptor_set_out=proto.pb
test/proto/bookstore.proto
proto_descriptor: “path/to/bin”

20. Setup Envoy Proxy (Cont.)
Proto Descriptor Path
proto_descriptor: “generated/file/path/proto.pb”
proto_descriptor_bin: Cr15ChVnb29nbGUvYXBpL2h0dHAucHJvdG8SCm...
Proto Descriptor Bin
$ cat proto.pb | openssl base64 -A
Encode proto descriptor using base64 encoding,
and set the value in yaml file

21. Start the Service
$ kubectl apply -f
blog.yaml

22. Setup Gateway
$ kubectl apply -f
blog-gateway.yaml

23. Setup Transcoder
$ kubectl apply -f
blog-transcoder.yaml

24. Handle gRPC Request

25. Handle JSON Request

26. Deploying service using transcoding
Protocol Buffer Artifact Pipeline
Releasing new API version using Helm

27. API Changes Over Time
message Post {
int32 id = 1;
string title = 2;
string body = 3;
repeated string tags = 4;
google.protobuf.Timestamp created_at = 5;
}
Service spec changes over time
● Additional field to message, new RPC
● New package version
● Beware of backward incompatible changes!(renaming, changing type, removing field, …)
message Post {
int32 id = 1;
string title = 2;
string body = 3;
repeated string tags = 4;
google.protobuf.Timestamp created_at = 5;
string featured_image_url = 6;
}

28. Deploying EnvoyFilter
proto_descriptor should be updated whenever
protocol buffer is changed.
● proto_descriptor: expect *.pb descriptor
to exist on file system → volumeMount to
istio-proxy(sidecar) container
● proto_descriptor_bin: embed
base64-encoded *.pb descriptor content
directly into EnvoyFilter
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: blog-transcoder
namespace: blog
spec:
workloadLabels:
...
filters:
- listenerMatch:
listenerType: SIDECAR_INBOUND
filterName: envoy.grpc_json_transcoder
filterType: HTTP
filterConfig:
proto_descriptor: “blog.pb”
match_incoming_request_route: True
auto_mapping: False
services:
- blog.BlogService

29. Deploying EnvoyFilter (Cont.)
● proto_descriptor
○ Ensuring *.pb descriptor file is mounted to istio-proxy container is not easy
○ Automatic sidecar injection from istio-sidecar-injector is not dynamic
enough(mounted volume contains all pb descriptors)
○ Hard to control deployment timing
● proto_descriptor_bin
○ Lack of readability
○ No volume dependency
○ New transcoding configuration is applied as soon as EnvoyFilter is updated

30. Protocol Buffer Artifact Pipeline
Language-specific gRPC Artifacts should be created whenever API is updated
$ protoc -I (GOOGLE_APIS_DIR)
--python_out=..
--go_out=..
--js_out=..
./blog.proto
blog.proto
Private repository
Private pypi server
Private npm registry
+ rubygems, maven, ...Lint, Breaking Changes warning, ...
comment.
proto
user.proto

31. ├── Jenkinsfile
├── Makefile
└── packages
├── blog
│ ├── blog.proto
│ └── package.json
└── user
├── user.proto
└── package.json
Mono Repository Approach
● Create CI pipeline that generates & publishes language-specific artifacts
● Common CI checks(lint, breaking changes warning) run for changed proto files
● Reduced boilerplate!
● Use lerna(node package) to bump versions of each API
Generating protobuf descriptor artifact:
protoc --include_imports
--include_source_info
--descriptor_set_out=blog.pb

32. Helm as Configuration Management Tool
Helm
● Already using helm for managing cluster-level services
(ingress controller, EFK, prometheus, grafana, …)
● Hosting private registry is easy(chartmuseum, object storage, ..)
● Easy CI pipeline setup
● CRDs can also be handled(VirtualService, EnvoyFilter, ...)

33. Helm Chart for gRPC Transcoding
apiVersion: apps/v1
kind: Deployment
metadata:
...
spec:
...
containers:
- name: {{ .Chart.Name }}
...
ports:
- name: grpc
containerPort: 9000
protocol: TCP
apiVersion: v1
kind: Service
metadata:
...
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: grpc
protocol: TCP
name: grpc
selector:
{{- include "blog.matchLabels" . | nindent 4 }}
Istio looks for port named grpc or any name prefixed with grpc-

34. Helm Chart for gRPC Transcoding
New Helm release will update
proto_descriptor_bin
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: {{ include "blog.fullname" . }}-transcoder
namespace: {{ .Release.Namespace }}
spec:
workloadLabels:
{{- include "blog.matchLabels" . | nindent 4 }}
filters:
- listenerMatch:
listenerType: SIDECAR_INBOUND
filterName: envoy.grpc_json_transcoder
filterType: HTTP
filterConfig:
proto_descriptor_bin: {{ .Values.protoDescriptorBin }}
match_incoming_request_route: True
auto_mapping: False
services:
- blog.v1.Blog
print_options:
always_print_primitive_fields: True
preserve_proto_field_names: True

35. Releasing New API Version
Using Helm CLI:
$ helm upgrade RELEASE -f values.prod.yaml -f
proto_descriptor.yaml --set image.tag=IMAGE_TAG repo/blog
Helm Chart
values.prod.yaml proto_descriptor.yaml
image:
tag: dkr.registry/blog:954ade7
secret:
DATABASE_URL: mysql://xx:yy@db:3306
WEB_CONCURRENCY: 5
proto_descriptor_bin: Cr15ChVnb29n
bGUvYXBpL2h0dHAucHJvdG...
*We internally use Spinnaker to deploy helm chart based services

36. ● 503 status code is returned when requested to non-mapped path
● kubectl port-forward didn’t work for testing out REST APIs
● Arbitrary(non-JSON) message can be also supported by using
google.api.HttpBody
● grpc-status, grpc-message headers in response are useful when
debugging
Pitfalls & Tips

37. Conclusion
● It takes initial effort to build pipeline, but after then it becomes easy to
develop any gRPC service that supports JSON transcoding
● You can develop gRPC services while allowing other teams time to transition
● API-First approach becomes standard throughout the organization
● Over time, initial setup cost pays off

38. Resources
Sample Repo - https://github.com/Buzzvil/grpc-json-transcoding-example/
● https://medium.com/@ssowonny/grpc를-쓰면-rest가-공짜-19e3a6bed4a9
● https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/
grpc_json_transcoder_filter
● https://blog.envoyproxy.io/envoy-and-grpc-web-a-fresh-new-alternative-to-res
t-6504ce7eb880
● https://medium.com/google-cloud/grpc-transcoding-e101cc53d51d
● https://blog.jdriven.com/2018/11/transcoding-grpc-to-http-json-using-envoy/
● https://medium.com/building-ibotta/building-a-scaleable-protocol-buffers-grp
c-artifact-pipeline-5265c5118c9d

39. Thank You!
Sungwon Lee (Whale)
whale.lee@buzzvil.com
GitHub & Twitter @ssowonny
Hoseong Hwang (Liam)
liam.hwang@buzzvil.com
GitHub & Twitter @thefron