ApacheCon NA - Apache Camel K: connect your Knative serverless applications with everything else
1. Apache Camel K
+ Knative
Connect your Knative serverless applications with everything else
2. Who we are...
Andrea Tarocchi:
● Open source enthusiast
● Apache Camel committer
● Software Engineer at Red Hat
● Twitter: @valdar
● Blog: http://blog.valdar.it/
Nicola Ferraro
● Open source enthusiast
● Apache Camel PMC Member
● Principal Software Engineer at Red Hat
● Twitter: @ni_ferraro
● Blog: https://www.nicolaferraro.me
3. Agenda
● What we talk about when we talk about “serverless”
● Knative basics
● Camel K basics
● Knative + Camel K
● DEMO
● Next steps
5. Serverless: what does it mean?
No! It does not mean:
“No servers”
And no! It does not
mean:
“The servers of
someone else”
6. “Serverless computing refers to the concept of building and running applications
that do not require server management. It describes a finer-grained deployment
model where applications, bundled as one or more functions, are uploaded to a
platform and then executed, scaled, and billed in response to the exact demand
needed at the moment.”
https://www.cncf.io/blog/2018/02/14/cncf-takes-first-step-towards-serverless-computing/
Serverless: what it does (really) mean
8. How to do serverless?
The future is
serverless!!!!
Current state of
enterprise
applications
How??? Is this
transition really
possible?
No,
it isn’t!!
9. How to do serverless?
The future is
serverless!!!!
Current state of
enterprise
applications
You better use
both!
12. Knative Serving
● Configurations represent the ‘floating
HEAD’ of a history of Revisions
● Revisions represent immutable snapshot
of code and configuration
● Routes configure ingress over a collection
of Revisions and/or Configurations
● Services (nope, not K8s services) are
top-level controllers that manage a set of
Routes and Configurations to implement a
network service
Service
(my-application)
Route
(name) Configuration
Revision
Revision
Revision
manages
Routes
traffic to
records
history of
10%
90%
13. Building blocks for event-based serverless applications.
ChannelProducers
https://cloudevents.io/
Kafka, In-memory, ...
Consumers
Subscription
Subscription
Subscription
Subscription
Mmh? Isn’t that like JMS?Knative Eventing
16. How?
// Message from a bot transformed and ingested into a kafka
topic, then sent to an API
from(“telegram:bots/bot-id”)
.transform()...
.to(“kafka:topic”)
from(“kafka:topic”)
.to(“http:my-host/api/path”)
1. Create a integration file (Java, Groovy, Kotlin, JS, XML…)
$ kamel run integration.groovy
2. Run it
3. It runs on Kubernetes
Camel DSL, based on
EIPs...
Well… the “kamel” CLI is
optional!
17. Architecture
Dev Environment Cloud
kamel CLI
Camel K
Operator
“Integration”
Custom
Resource
Running
Pod
Live
updates!
Fast redeploy!
Less than 1 second!
Tailored for cloud-native development experience
18. “Integration” Custom Resource
kind: Integration
apiVersion: camel.apache.org/v1alpha1
metadata:
name: my-integration
spec:
sources:
- name: source.groovy
content: |-
from(“telegram:...”)
.transform()...
.to(“kafka:...”)
from(“telegram:bots/bot-id”)
.transform()...
.to(“kafka:topic”);
from(“kafka:topic”)
.to(“http:my-host/api/path”);
This is what people
care about: Camel DSL
Camel K Operator
1. Choose a runtime
2. Scaffold a project
3. Add boilerplate
4. Add dependencies
5. Create container image
6. Create Kubernetes
resources for deployment
kamel CLI
or others ...
20. Classic scenario:
“When someone uploads a file on Dropbox, I want to upload that file also to a FTP location, publish some file details on a Slack channel and
also send a confirmation email to a specific user”
Our web-based integration platform (Syndesis).
Target: Citizen Integrators
Features:
● Multiple connectors built from Camel
components
● Few clicks to define a integration
● Graphical data mapping capabilities
● Design, expose or consume REST API
● Integrated with Apicur.io “Apicurito” for API
design
● Integrated with 3-scale for API management
22. kind: Integration
apiVersion: camel.apache.org/v1alpha1
metadata:
name: my-integration
spec:
sources:
- name: source.groovy
content: |-
from(“knative:channel/a”)
.to(“http:my-host/api/path”)
from(“knative:channel/a”)
.to(“http:my-host/api/path”);
Camel K Operator
Knative
profile?
kind: Service
apiVersion: serving.knative.dev/v1alpha1
yes
kind: Deployment
# standard one
no
Knative Profile
23. 300+ components!
Kubernetes Namespace
There’s no container if no one needs it!
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
System 1
System 2
Knative
Service
What does it mean?
24. Kubernetes Namespace
A container is created only when needed!
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Request Pod
What does it mean?
27. Same model for different purposes
Channel
Channel
Channel
Ext
System
Event
Source
EIP
Integration
Function
300+ components!
Camel K in Knative Eventing
29. DEMO
Context:
Legacy system simulated by file component. The file is picked up and, depending on the content, splitted
and sent to a Telegram chat or just discarded.
The Telegram response service should scale out (due to an artificially inserted timeout).
Code is here: https://github.com/nicolaferraro/knative-legacy-demo
Legacy
System
Shared Volume
File File
ProcessLegacy In
OutDev-null
TelegramQuarkus
Split + CBR
Knative
Channel
Camel K
Integration
Native Image
(it just logs)
Telegram
Servers
31. What is Quarkus?
Supersonic, subatomic Kubernetes native Java!
Why do we need Quarkus in Camel?
● Java sucks in containers
● Serverless functions need fast startup
A new subproject of Apache Camel: https://github.com/apache/camel-quarkus
To make all Camel
components Quarkus
compatible (extensions).
32. 32
CONTAINER ORCHESTRATION
Host Host Host
Traditional Java Stack
NodeJS Go Go Go
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
Traditional Java Stack
Traditional Java Stack
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
RSS
Heap Metaspace
Off
Heap
Increased density
Heap is only
a fraction of
the memory
used by the
JVM CPU
memory
ParallelGCThreads
ConcGCThreads
CICompilerCount
ForkJoinPoolJVM
Ergonomics
The hidden truth about Java + Containers
33. GraalVM landscape - Camel K edition
33
JVM CI
Sulong (LLVM)
Truffle
Graal Compiler
Substrate VM
Java HotSpot VM
Graal JS used to support for
toutes written in JavaScript
for both JVM and Native
mode.
34. 34
Yet to be optimized
Memory Startup time (Camel
context)
Camel K
35. Push based endpoints
Pull based endpoint
from(“slack:#yourRoom”)
.to(“log:info”)
Pod
Get message
Continuously!Long
running...
Slack Server
PodCamel K
Setup webhook
Push message
Scaling
from zero!
Push based endpoint
from(“webhook:slack:#yourRoom”)
.to(“log:info”)
Slack Server
36. Current developments
● Quarkus
● Push based endpoints
...
● Support Knative Eventing Broker-Trigger model
● Continue Knative CamelSources
● Tekton pipelines
● BDD Testing
Knative contrib:
https://github.com/knative/eventing-contrib/tree/master/camel/source
Knative
sources can
now use 300+
Camel
components!
37. We love contributions!
● Project repo: https://github.com/apache/camel-k
● list of tasks to start from
(pick anyone with “easy fix - newcomers” label on them)