The document discusses best practices for integration strategies including using an integration platform, designing integrations, and implementing resiliency patterns. It recommends having an integration platform to provide features like batch processing, loose coupling, reuse, governance, and security. When designing integrations, questions about data, users, transactions, orchestrations, and future needs should be considered. Common resiliency patterns discussed are timeouts, circuit breakers, bulkheads, retries, and idempotency.
4. Agenda
• About a Integration platform
• Buying/building guide
• Objections of an integration platform
• How to kick start
• Practices
• Bulkhead and Circuit-breaker in depths
5. About your
Integration
Platform
Don’t have an
integration
platform yet
You own one,
but it is
unorganized
and there is
no control
Third party
apps are very
unresponsive,
reliability is at
stake
Dev to Deploy
time is huge
Where do you fit ?
6. Batch Processing
Loose coupling
Micro services & Reuse
Governance
Analytics
Security
Integrations
Platform –
Features
What features should the platform provide
Points to consider while buying or
building an integration platform
7. Integrations
Platform –
Key objectives
Reliable
Available
Intuitive APIs
Least manual interventions
Consumer Friendly errors & exceptions
Ease of Monitoring & Reporting
Enable Self Service
Low maintenance costsWhat are the customers confidence factors
that the platform should abide to get a
great NPS J
API or integration platform users = customers
9. Design Questions
• Is there an existing integration, if yes, how
does it work and what are the problems
• What data needs to be integrated
• Who are the users (systems or
applications)
• Transaction count: daily-monthly-yearly.
• What kind of orchestrations are required,
if any
• Personas and use-cases
• Manual tasks
• Fallbacks
• Future scaling needs
• Data security or confidentiality
Ask these questions when there is a
need of integrations
#non-technical #birds-eye view
11. Some more questions
• Data
• Data transfer format
• Rest (json/xml), graph, SOAP, File, Email, Queue,
manual feed
• Single / multiple payload
• Synchronous / Asynchronous responses
• Performance
• Counts of requests
• Size of the data payload
• What is the system max parallel request limit
• System transactions per second
• Response times
• System resilience
• Security
• Service authentication
• Is the data confidential/federal
• Error
• How to react to errors, when & how to retry
• Analytics and Reporting
#technical #Worm’s-eye view
Ask these questions to the 3rd party
application technical team and analysts
12. Object level considerations - decisions
• IDs or GUIDs
• Rule of thumb - In Hub-Spoke integration model, the spoke systems should use the id/ GUID created by the master. This reduces a lot
of maintenance problems that occur otherwise.
• Date-Time
• These should be in GMT always and the consuming systems or the client should convert them as needed. Use standard date time
format- ISO 8601
• APIs
• Export self-explanatory API objects
• Huge flat structures should be always categorized in groups or nested objects, This makes the object look like a DOM/tree.
• Version APIs
• Paginate bulk web service calls. This solves performance issues.
• Data Mapping
• Define the role of the middleware. There are many misconceptions about what a middleware should do, correct those.
• Outline roles of each system, so as to make sure there is a unified mapping strategy.
• Avoid patch work mappings like using contains/ substring based decision making on strings.
#technical #discussions #before the design
13. • Hardcode only if there is a technical limitation.
• Always remember that complex data mapping often leads to a lot of serialization and de-serialization, in turn causing performance
bottlenecks.
• Orchestration
• Answer the following questions. These help the consumers to know what is happening under the water
• When to orchestrate
• How to report the results back to the consuming systems. These updates could be in a form of status changes which should be
reported
• What happens when a part of a complete transaction set fails. Is the transaction type - ACID and what is the rollback policy.
• Retries
• Discuss and decide if the transaction should be retried and under what conditions. Usually the retry logic should be the same for a
complete integration unless there is an exception
• Error Reporting
• This is a very crucial aspect and a good integration is one which is designed for failures.
• List all the known failures based on integration type and decide the action to take. Some failures might need immediate attention while
some can be retried for a certain period.
Object level considerations - decisions
#technical #discussions #before the design
14. Analysis Design Develop
Test &
Performance
Manage Checkups
Important Phases in Integration
Building an integration is same as building a car - invest more on the design, so as worry less on the maintenance
15. • Analysis
o Understand the roadmap of the integrating app
o Identify Primary and secondary needs
• Design
o Reusable Patterns
• FTP
• Schedulers
• PUB-SUB
• Errors
o Always think of performance
• Throughput
• Latency
Analysis
Design
Develop
Test
Perform
ance
Manage
&
checkups
Best Practices
16. o Secure
o Reliable
o Flexible
o Error handling
o Entity and relationship
o Data exchange patterns
• Synchronous vs asynchronous
o Data in Chunks
• Paginate
o API data transfer should be in JSON. XML only if the
consumer has a limitation.
Best Practices Analysis
Design
Develop
Test
Perform
ance
Manage
&
checkups
17. • Develop
o Handle all known cases
o Mock the service first – consumers will not be surprised later
o Write unit tests
o Usable
• Example - Document usage on the api console, avoid multiple
documents
o Follow Http web service guidelines:
• POST - create a record or resource
• PUT – update a record or resource
• GET - read a record or resource(single or multiple)
• DELETE - remove a record or resource(single or multiple)
• A Response should be simple and meaningful, avoid
duplicate responses
• Http status 200 OK is more than enough for a consumer to know that
is request was successful
Best Practices Analysis
Design
Develop
Test
Perform
ance
Manage
&
checkups418 I'm a teapot
18. o Coding conventions
• Java for internal code
• Restful APIs
§ Naming conventions
o Redesign if necessary
o Ensure loose coupling for reusability
o Code Reviews
• Test & Performance
o Automate
o Load test with JMeter, Nodejs Simulators
Best Practices Analysis
Design
Develop
Test
Perform
ance
Manage
&
checkups
19. • Manage
oIdentify Risks – Impacting Systems
oHandle changes with versioning
• Regular checkups and adjustments
oNode Capacity
oFrequency
Best Practices Analysis
Design
Develop
Test
Perform
ance
Manage
&
checkups
22. Attributes
• IdentifierName BestPhotoCompanyCktBreaker
• List of exception that would trip the circuit
Exceptions
java.util.concurrent.TimeoutException,
java.net.SocketException
• Window of time in ms to count the requestsRolling Window 100
• CountThreshold 4
• Ms time to close the circuit againTrip Timeout 7000
23. Response Timeout
Trip Timeout 7000
ms
Rolling Window
100 ms
Timed out requests
Circuit Open - All requests
are being rejected
Circuit
breaker
in action
