First overview of the deployment of Smart City Platform, Powered by FIWARE solutions following the recommendation of the FIWARE DevOps lesson learns. We introduce the concepts and the requirements to explain why we have adopted this approach based on Docker and Docker Compose and the reason behind the Orchestration of services, applied in this presentation into Docker Swarm. Finally, we provide the reason, why should be use the Infrastructure as Code (IaC) with Terraform and Ansible.
1. How to deploy a Smart City Platform
(Terraform + Ansible + Docker Swarm)
Fernando López
FIWARE Cloud & Platform Senior Expert
fernando.lopez@fiware.org
@flopezaguilar
https://www.slideshare.net/flopezaguilar
https://github.com/flopezag
2. About me
▪ More than 35 years of programming experience (more than 12 programming languages)
▪ More than 10 years working with OpenStack and AWS
▪ Web Development, Message Queues, Functional Programming, Big Data and Data
Engineering
▪ Developer, Team Leader, QA Manager, Project & Product Manager
▪ DevOps(Secs) activities with more than 6 years
▪ Now Cloud and IoT Platform Senior Expert in FF
▪ Principal Cloud Architect in FIWARE Lab
▪ Evangelist of FIWARE Technology and TSC Member of FIWARE Technology
▪ Love Coding, CI & CD and AI/ML
1
3. Why this presentation
▪ I am here to spread the awareness of FIWARE Lab & OpenStack
▪ I want you to use Docker Swarm (or K8S/K3S for other session…)
▪ I want you to love Terraform
▪ I want you to feel my passion with Ansible
▪ I want to show you how to deploy and scale a Smart Solution using all the above
(in 40 minutes… )
2
4. 3
In this talk, we will see how to deploy
a Smart City platform…
5. Hands up
▪ Who’s using Docker (… yet)?
▪ Who’s using Docker Swarm or Kubernetes?
▪ Who’s using Terraform?
▪ Who’s using Ansible?
▪ Who’s not played with any of them, and would love to?
4
9. Architecture solution of a Smart City Platform (beta…)
8
▪ IoT Agent to transform IoT
Devices to NGSI-[v2, LD]
protocol.
▪ Orion Context Broker ->
Keystone Broker Message
▪ Data persistency into historical
Database using Cygnus
▪ PEP Proxy support with Wilma
▪ Identity Management with
Keyrock.
10. Architectural issues or the Nightmare of the deployments
9
▪ Manual deployment.
▪ Security, security, security….
▪ Manual provision of infrastructure
▪ Manual configuration of
infrastructure
▪ HA, scaling, replication
▪ Persistency of data in ephemeral
environments
14. Containers manually allocated on multiple nodes
▪ Non-linear resources usage
▪ No service discovery, hardcoded configurations
▪ Manual reaction to failures… OMG!!!
▪ Storage on a node? on several?
▪ Manually integrate to network storage?
▪ Security …
▪ How is the deployment process?
13
20. How nodes work
19 Source: https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/
21. How services work
20 Source: https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/
swarm manager
3 Orion CB
replicas
service orion.1 orion:latest
orion.2 orion:latest
orion.3 orion:latest
swarm worker 1
swarm worker 2
swarm worker 3
• Image
▪ Exposing external ports
▪ Overlay network for connecting to
other services
▪ CPU and memory limits and
reservations
▪ Update policy
▪ Number of replicas
▪ Global services
28. Infrastructure as Code
▪ Rotate machines in and out of service
• Create process to create new servers quickly
• Use the code to destroy and replace them
whenever it is needed
▪ How
• Base images (cloud provider)
• Infrastructure Management Tool (Terraform)
• Configuration Management Tool (Ansible)
27
33. Ansible
▪ Idempotent machine setup
▪ Define how things should be
▪ Support parallel execution
▪ Only needed python and ssh connectivity
▪ The same config is used for dev, test, and
production.
▪ Ensures consistency and a known working
config.
▪ Apps and environments are provisioned on
demand.
▪ Reduces the potential for human error
32
37. Problems and barriers
▪ Environment provisioning
▪ Manual testing
▪ Manual deployments
▪ Planning in a DevOps environment
▪ DevOps and suppliers
▪ DevOps and governance
▪ No integrated tools architecture
▪ Manual releases
▪ No service virtualization
▪ Large releases
▪ Inconsistent environments
36
▪ Limited transparency
▪ Manual processes
▪ Collaboration between development
and operations
▪ No DevOps vision or strategy
▪ No production-like environments
▪ Waste in existing processes
▪ No standard SCM repository
▪ …
38. Problems and barriers
▪ There are overlapping between some of them (e.g. manual operations).
▪ Some challenges focus on people, some on process, and some on technology.
▪ Others focus on the enterprise rather than a team.
▪ Majority of the issues are DevOps related operations
• Automate where appropriate
• Automated deployments
• Automated testing
• Standardized DevOps ecosystem
37