Accelerating Time-To-Market with Continuous Delivery at Tech in Asia, PDC 2023 Jakarta TIA PDC 2023 16/May/2023

1. Accelerating
Time-to-Market
with Continuous
Delivery
Manoj Awasthi
CTO of JULO
16/05/2023 at Tech In Asia, PDC 2023

2. Hello, nama saya
Manoj Awasthi!
● CTO at JULO
● Earlier, more than 6 years at
Tokopedia
● Last 7 years in Indonesia

3. What is JULO?
An innovative financial technology company that was
established in 2016 and have revolutionized access to financial
products for unbanked and underbanked Indonesians
Adrianus Hitijahubessy
Chief Executive Officer
JULO’s Purpose
Empower Indonesians to be able to
achieve a stable financial condition so
that their quality of life increases and
support financial inclusion in Indonesia.

4. JULO Milestones
2016 2018 2020 2022
2017 2019 2021
Founded
Winner of the
Indonesia Fintech
Festival
Dec 2016
Registered at OJK
#1 Customer
Launched Android
Application
1 million app
downloads
Series A Funding
Winner of the UN
Fintech Challenge
ISO 27001 Data
Security
Certification
(April 2019)
50 Inclusive
Fintech Awards
National coverage
credit distribution
(one of the first
fintech players)
OJK Fully
Licensed (May
2020)
Cumulative Loan
Disbursement
IDR 3 Trillion
Strategic
Collaboration
with Grab
10 million app
downloads
Series B Funding
1 million
customers in all
provinces of
Indonesia

5. Worksho
p
outline
Overview of CI/CD &
Success stories of CI/CD
adoption
Interactive segment
Q n A

6. What is CI/CD

7. COMEDY, IMPROVE AND
DELIVERY

8. THE CIRCUS OF DEVELOPMENT

9. THE COMEDY OF ERRORS
Software development is not always
smooth sailing with bugs, typos, and
unexpected issues.
But fear not, our friend CI/CD is here to
rescue us ensuring our code is always ready
for the stage.

10. Little Bit of History
Requirements
Gathering
Design
Development
Testing
Deployment

11. Fast forward to era
with CI/CD!
Requirements
Gathering
Design
Development
• CI/CD *
Testing
Deployment
• CD *

12. CONTINUOUS INTEGRATION: THE IMPROV ACT
● Frequent code merges into a shared repository
with CI server automatically building and testing
the changes
● If tests pass, the code is merged to shared repo
● If tests fail, the developers are notified and they
can address the issue
Just like improv comedy, where actors build the scene together,
Continuous Integration brings devs together to integrate code frequently

13. … AND WHAT KIND OF TESTING IS DONE AT CI
STAGE
01 Code Quality Tests
02 Unit Tests
03 Integration Tests
04 Security Tests
05 UI Tests and more.

14. BENEFITS THAT CI BRINGS
Early Detection of Errors Reduced Risk of Integration
Issues
Faster Feedback Loops
Increased confidence in
the code

15. CONTINUOUS DEPLOYMENT: THE SHOW MUST GO ON
● Delivery: Releasing the CI tested code to a non
production environment for an elaborate testing
● Deployment: Automatic release to production
once the code passes all automated tests. Needs
a high level of confidence and monitoring.
Continuous delivery and deployment ensure a seamless production by
automating the release process.

16. BENEFITS THAT CD BRINGS
Faster time to market Reduced Risk
Improved Quality Faster Feedback loop

17. CI/CD Tooling

18. KEY CONCEPTS IN CI/CD
Overall structure
of the flow
PIPELINE/
WORKFLOW
JOBS/TASKS
STEPS/RUN/S
CRIPTS
KEYS/CONFIGS
Incremental jobs
within the pipeline
Individual scripts
or commands
Inserted at run time
e.g. credentials or
environment variables

19. 2016
Deployment code: Ansible
● Git checkout release branch
● Install python dependencies
● Run DB migration via Django
migration
● Restart services
Deployment:
● Non-prod: running Ansible
manually from remote VM
(jumpbox) via bash script
● Prod: manually running of
Ansible from dev’s laptop
2017
Introduced CircleCI
Running Ansible automatically from
remote VM (jumpbox) via bash script
Introduced flake8 (linting) static
analysis tool and jenkins replacing
remote server
2020
Introduced Sonarqube
For vulnerabilities scanning in the
code and establishing secure code
practices
Jan 2023
CI/CD JOURNEY AT JULO

20. CI/CD PIPELINE AT JULO
From 1x week (2017) to 2x week (2019) to 5x a week (2022+)
Impact on deployment frequency

21. SUCCESS STORIES FROM THE BIG TECHS
• Reduced the time to
deploy from several days
to minutes
• Able to release new
features and updates
more frequently
• Improved user
satisfaction and
engagement
• Reduced the time to
market for new features
from months to weeks
Source:
https://www.simform.com/blog/etsy-devops-
case-study/
Source:
https://www.infoq.com/presentations/vogels
-amazon-platform/
• Reduced time for changes
to production, reducing
downtime and improving
the user experience
Source:
https://netflixtechblog.com/how-we-build-
code-at-netflix-
c5d9bd727f15?gi=a8b9580d4791

22. SUCCESS STORIES FROM THE BIG TECHS
• Reduced the time
required to release new
versions of the Windows
operating system from
years to just a few
months
• Improved reliability of
Google services, reduce
downtime and the risk of
issues making it to
production
Source:
https://azure.microsoft.com/en-
us/solutions/devops/devops-at-microsoft/
Source:
https://sre.google/workbook/canarying-
releases/
• Helped release new
features and updates
more frequently which
further helped iterate
quickly and respond to
user feedback more
effectively.
Source:
https://engineering.atspotify.com/2014/03/s
potify-engineering-culture-part-1/

23. BUILD SUCCESS RATE
% of successful builds
BUILD TIME
Average time to finish
the build
TEST COVERAGE
% tests covered in
automation
DEPLOYMENT
FREQUENCY
Frequency of the
deployments
CHANGE LEAD
TIME
Time taken between code
commit and deployment in
production
CHANGE
FAILURE RATE
% of deployed changes
that fail in production
METRICS: HOW TO MEASURE SUCCESS

24. … AND WHAT KIND OF TESTING IS DONE AT CI
STAGE
01 Code Quality Tests
02 Unit Tests
03 Integration Tests
04 Security Tests
05 UI Tests and more.
Do you see potential caveats here?

25. SOME PRACTICAL TIPS
● Automate all the “best practices” you want the developers to follow
● Parallelize the tests or scripts in CI as much as possible. Time more important than
space.
● Divide the tests in CI/CD into blocking and non blocking tests. Run non-blocking in a
separate pipeline at a less frequent interval to optimise regular CI/CD pipeline.
● Leverage the CI/CD infrastructure for more complex delivery automation.
● Build a culture to commit often.
● Build a culture to deploy often (frequent; small deployments)
● Build a culture to prioritise fixing broken builds.
● Keep evolving.

26. Interactive Segment

27. LET’S LEARN FROM EACH OTHER
OPEN QUESTIONS FOR DISCUSSION
● What stage of a startup should CI/CD be
introduced? Is it valuable to introduce
right when starting up?
● Any challenges you observed with CI/CD
in your organisation or elsewhere?
● Any best practices you came across?

28. QnA