apidays LIVE Australia 2020 - Events are Cool Again! by Nelson Petracek

www.linkedin.com/in/npetrace
npetrace@tibco.com
Events are Cool Again!
Building the Modern Application
Presented by: Nelson Petracek, CTO
September, 2020

4
Built with
Microservices &
Functions
● Does one thing really
well.
● Is stateless or relies
on external state.
● Orchestrates across
distributed
microservices &
functions.
Event-Driven
● Sense and react to
critical business
moments.
● Event-first design
pattern.
● Native support for
event-first messaging
& streaming (e.g.
Kafka).
Embraces
AI/ML
● Services are
becoming
predictive and
prescriptive.
Deployable
Anywhere
● Embracing
Serverless/FaaS,
Containers/PaaS
and devices.
Exposed
via Open APIs
● CRUD/Resource
based (REST).
● Query, Consumer
APIs (GraphQL).
● Command based
(gRPC).
● Asynchronous,
Streaming
(AsyncAPI).
The Modern Application
Change the Game with Cloud Native, “Anywhere”, Event Driven Microservices and APIs.
© Copyright 2000-2020 TIBCO Software Inc.

5
Extended With Evolving Patterns & Technologies
“By 2022, event-sourced, real-time situational awareness will be a required
characteristic for 70% of new digital business solutions. And 80% of new
business ecosystems will require support for event processing.” *
* Source: Gartner; Top 10 Strategic Technology Trends for 2018: Event-Driven Model (Mar 2018)
IoT / Edge
Computing
Serverless / FaaS Service Meshes
Inclusion of
Analytics
(Batch +
Streaming)
APIs / AsyncAPIs
Saga Design
Patterns, Event
Sourcing, CQRS
”Let It Crash”
Scalable /
Queryable Event
Stores
Event-Driven APIs
for Decision
Management
”Anywhere &
Everywhere”

6© Copyright 2000-2020 TIBCO Software Inc.
The Modern Application: Powered By Events
Traditional data processing simply takes too long!
• Data is collected from a variety of sources,
and placed in a persistent store.
– Relational database.
– NoSQL store.
– Big Data environment.
• Analytical processes are executed against
the stored data to detect opportunities or
threats.
• Actions are identified, delivered, and
executed across various business channels.Store
Analyze
Act

The Modern Application: Powered By Events
Traditional data processing simply takes too long!
• Introduces too much “decision latency” into
the business.
• Responses are delivered “after-the-fact”.
• Maximum value of the identified situation is
lost.
– Cross-sell / up-sell opportunities are lost,
impending equipment failure is missed, business
processes are slow to respond and lack timely
context.
• Decisions are made on old and stale data.Store
Analyze
Act

8
ü In essence, events represent the way the world works.
• Not only ordered streams, but true “event clouds”.
ü Events represent a “change of state” in key business
data elements.
• IoT sensor reading, customer interaction, purchase transaction, patient
admission, passenger check-in, …
ü Events may be found in any vertical or business.
• Government, healthcare, transportation & logistics, retail, finance,
manufacturing, telecommunications, …
What Is An “Event”?
Events provide value to the business.

9
What Is An “Event”?
Events provide value to the business.
ü Events can come from any place, at any time, in any order.
• Sometimes they don’t come at all, which is also important!
ü Organizations must be able to handle events as a core
component of their architecture.
• Treat events with the same importance as data.
• Microservices do NOT automatically mean “request-reply over HTTP”.
ü Events are a mandatory component of today’s enterprise
architecture.
• Facilitate scaling & simplify fault tolerance.
• Increase location independence & allow one to move towards loose
coupling.
• Reduce decision latency, handle events when they have the most value.
• Eliminate expensive “polling for answers” which may never come.

10
Retail –
Personalized Offers
Travel –
Passenger Experience
Insurance –Dynamic
Pricing
Healthcare – Improved
Patient Outcomes
Manufacturing –
Predictive Maintenance
Banking - Fraud
Detection/Mitigation
Retail –
Real-time Inventory
Management
Telecom –
Service Quality
Management
Use events to drive decisions across every vertical, for a variety of use cases.
The Modern Application: Powered by Events

11
Event driven pattern compared to typical request-response.
Request-
Response
Request
App
Reply
Back-end
ServiceAPI ServiceAPI

12
Request-
Response
Request
App
Reply
Back-end
Issues:
• Client driven.
• Typically synchronous.
• No support for streaming data or server-side pushes.
• Communications are point-to-point.
• Scaling to large numbers of endpoints (e.g. IoT).
• Fixed message exchange pattern.
• Potential for tightly coupled dependency chains.
• Often utilizes shared state concurrency.
• No separation of commands and queries.

13
Request-
Response
Request
App
Reply
Back-end
Event Driven
Pattern
Pub
Sub
Microservice
Microservice
Microservice
Microservice
Microservice Microservice
Event Event Event
Event Trigger
Trigger Trigger
App 2
MicroserviceApp 1
App 3
Pub
APIAPI
API
API
API API
API
Pub
Sub
Pub
Sub

14
Characteristics:
• Client or server driven.
• Synchronous or asynchronous, non-blocking, location-independent.
• Support for streaming data and server-side pushes.
• Communications are point-to-point, many-to-one, one-to-many.
• Scales to large numbers of endpoints (e.g. IoT).
• Flexible message exchange patterns.
• Loosely coupled dependency chains.
• Separation of commands, events, state, and queries.
• “Guess, apologize, compensate”

The Modern Application: Enabled with APIs
Change the game by adding event driven microservices and APIs.
Customer
Device
Shopper
Tracking
Shopping
Cart
Promotion
s
Product
Catalog
Shipping Payment
PricingMicro
Gateway
Micro
Gateway
Micro
Gateway
Micro
Gateway
Micro
Gateway
Micro
Gateway
1
2
3
4
5
6
7Add to cart
(Pub / Sub)
Start Tracking
(Pub / Sub)
Check Promos
(Request / Reply)
Get Pricing
(Request / Reply)
Price Query
(Request / Reply)
Ship Quote
(Pub / Sub)
Execute Pymt
(Pub / Sub)
Micro
Gateway

16
The Modern Application: Event Contracts
It all starts with the event contract.
https://github.com/project-flogo/asyncapi
https://www.asyncapi.com/
• AsyncAPI is to events as OpenAPI
(Swagger) is to REST, or Schemas are to
GraphQL.
• Utilize AsyncAPI to build protocol-
agnostic event contracts to:
• facilitate code/documentation
generation
• support the creation of standardized
event interfaces
• improve event governance
• TIBCO is a Platinum Sponsor of the
AsyncAPI initiative.

The Modern Application: Event Patterns
Some patterns to consider when building the modern application using events.
• Event Sourcing
• Persist the events that lead up to a particular state, not the state itself.
• Easy to recreate a state at any point in time.
• Other services (e.g. Big Data, streaming) can subscribe to event store changes.
• CQRS
• Command Query Responsibility Segregation
• Services that handle updates are different from those that handle queries.
• Utilize a “view store” to handle queries.
• Events & Sagas
• Use sagas to coordinate how a transaction is processed across many microservices.
• Control sequence, compensate for errors, & trigger via events in a loosely coupled manner.
• Can be centralized (“orchestrated”) or purely event driven (“choreographed”).
• Commands (an action that starts processing, triggers an event) vs Events.

Using the guidelines of a Reactive Application.
Responsive Resilient
Message Driven Elastic
Reactive
Applications
* http://www.reactivemanifesto.org/
• Coherent approach to systems architecture.
• Reactive systems are flexible, loosely
coupled, and scalable.
• Use asynchronous message passing to
establish a boundary between components.
• To ensure responsiveness, the system must be
elastic.
• Failures must be met with elegance.

APIGateway
Microservice Inventory
Service
Validation
Rules
Choreography
Microservice Inventory
Service
Validation
Rules
Choreography
Clients
Event
Store
View Store Aggregate Store
state changes
change events
view queries
Event Processing…
Big Data Store…
Using event driven microservices and event sourcing with CQRS.

Batch
APIs
Adapters /
Channels
Integration
Messaging
Event Ingest
Transformation
Aggregation
Enrichment
Filtering
Event
Preprocessing
Process
Management
Analytics
(Real Time)
Applications
& APIs
Analytics / DW
Reporting
Event
Outcomes
• Transform
• Deep ML
• Analytics
• Data Lake
• …
Event Analytics &
Processing
Real-Time
Index / SearchNormalization
• RT Analytics
• Contextual Rules
• Windowing
• Patterns
• …
Using event pipelines across microservices to support end-to-end processing.

Using events (or the lack of events) in a contextual, declarative, temporal fashion.
EntityState&
Relationships
In Memory, IMDG/DB, GraphDB, …
Data Stream
Contextual, Reactive State (Rules & AI/ML)
• Event processing is not just about
applying mathematical
operations on time-ordered,
unbounded data.
• Business runs on rules and
algorithms applied to context.
• Logic is not always sequential, and
may not be ordered.
• Often more important to
detect when things “do not”
happen.
• Manage “reactive state” in
addition to events.

23
The Modern Application: Building An EDA
Putting the layers together to build and maintain an event-driven architecture.
Event Catalog
API
Architect
Kafka
Source Code
Control
GitHub, etc.
Event Decision Mgmt
Event Processing
Pulsar MQTT
Event Patterns
Container Management
Developer
Messaging
Microservices Microservices Serverless Functions
Runtime Event /
Correlation System
of Record
Streams AI/ML
AWS, Azure, GCP, Private/Hybrid Cloud
CI/CD
API Management & Gateway (SaaS and/or Local)
Internal or External Service Mesh, Microservices

Ultralight Edge & Platform
Microservices Framework
10-50x lighter than
Java, .NET or Node.js
100% Open Source Stack
for all things event-driven
Common core for all event-
driven capabilities
Deploy as serverless
functions, containers or to
IoT edge devices
Machine Learning &
Data Transformations
The Modern Application: Project Flogo™
Open source, developer friendly framework for building the next generation of digital apps.

• Start with AsyncAPI and use
CLI to generate the Flogo
application.
• Run as a microservice, fully
containerizable.
• Lightweight, Golang footprint
(much lighter than Java).
• Leverages the complete open
source Flogo ecosystem,
including hundreds of triggers
and the ability to combine Flogo
Streams with Flows and Rules.
• Process and read/write data
from/to a Kafka cluster.
Flogo Kafka App
{
"ref": "#aggregate",
"settings": {
"function": "sum",
"windowType": "timeTumbling",
"windowSize": "5000"
}
}
Flogo Kafka App
{
"ref": "#aggregate",
"settings": {
"function": "sum",
"windowType": "timeTumbling",
"windowSize": "5000"
}
}
Open Source, fully
supported by TIBCO
TIBCO Messaging
Bridge
The Modern Application: Flogo™ + Kafka
Open source, developer friendly framework for building the next generation of modern apps.

• Same support as with Apache
Kafka.
• But…also run Flogo streaming
applications as native Pulsar
functions, within the Pulsar
environment itself.
• Reduces system complexity,
improves performance and
reduces latency, and utilizes
Apache BookKeeper for stateful
support.
• Same lightweight framework,
and leverages the entire Flogo
ecosystem.
The Modern Application: Flogo™ + Pulsar
Open source, developer friendly framework for building the next generation of modern apps.
https://hackernoon.com/event-driven-functions-with-apache-pulsar-and-project-flogo-wag3y4u

Putting It All Together: An Example

Many architectures are
designed to support request-
response, but also require
loose coupling
No easy way to manage
interaction between
events and microservices
No standard
specifications for event-
driven, inter-service
communication
Roadblocks to Adopting Events

Microgateways apply
protection and mediation,
via APIs, between events
and microservices
APIs create
interoperability between
loosely coupled services
An event-driven API
specification has become
the defacto standard
Most architectures are designed
to support request-response,
but also require loose coupling
No easy way to manage
interaction between
events and microservices
No standard specifications
for event-driven, inter-
service communication
Event-Driven APIs Solve Our
Biggest Challenges

https://web.cvent.com/event/ebcb07b0-b1b0-4e01-80c1-06c259a0c49a/summary

10000+ customers 4000+ employees
Global footprint
Based in Palo Alto,
California, and present in
30+ countries
20+ years of
innovation
Founded in 1997, has fueled
digital ever since

apidays LIVE Australia 2020 - Events are Cool Again! by Nelson Petracek

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to apidays LIVE Australia 2020 - Events are Cool Again! by Nelson Petracek

Similar to apidays LIVE Australia 2020 - Events are Cool Again! by Nelson Petracek (20)

More from apidays

More from apidays (20)

Recently uploaded

Recently uploaded (20)

apidays LIVE Australia 2020 - Events are Cool Again! by Nelson Petracek