Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to .Net Microservices with Event Sourcing, CQRS, Docker and... Windows Server 2016! Best practices with microservices (or just services)
Similar to .Net Microservices with Event Sourcing, CQRS, Docker and... Windows Server 2016! Best practices with microservices (or just services) (20)
Recently uploaded
Recently uploaded (20)
.Net Microservices with Event Sourcing, CQRS, Docker and... Windows Server 2016! Best practices with microservices (or just services)
- 1. http://DOTNETMALAGA.es // 30th Mar 2016
- 2. • What is a microservice • Good things about microservices you should be using now • DIY in .NET • Containers, VMs, self-hosted, … • Demo • “Microsoft Microservices” • Azure Service Fabric • Demo
- 3. About Sequel We are leading software providers for the insurance market Heavily investing on R&D (& new offices in Málaga ) 55 people in Málaga, 6 scrum teams 140 people in total Currently hiring in Málaga – @ndsrf for more info Graphic designers | .NET Architects / AngularJs devs | Testers / Automation www.sequel.com
- 4. Disclaimer This is not a “this is what I did” presentation, but rather a “this is what I have seen” and open for discussion presentation. O:-)
- 5. What is a microservice? Let’s build our definition! link
- 6. Our definition of microservices (Created live in the actual presentation - brainstorming session 30/03/2016)
- 7. Principles of microservices • Autonomous service (decentralise!) • Code + State (high cohesion) • Contracts between them (hide implementation) • Versioned • Consistent and available between failures • Culture of automation • Focus on delivery (products not projects) • Team collaboration • Team size “pizza size” • Lightweight components • Accesible by any device • Language & platform transparency • Smart endpoints / dump pipes • Unit of deployment • Implementation of a bound context (DDD) • Separated code repositories • Inmutable infrastructure • HTTP (REST – HATEOAS) vs others • Self-healing • Automatic upgrades / rollback
- 9. Good technical practices for microservices • Discovery, Configuration & Versioning • Security • Resilience & autohealing • Monitoring & Logging • Composition • Testing • Inter service communication • Data consistency • Service orchestration
- 10. Y A X B Service Discovery + Configuration • Problem: Hard-code service locations in 100s of microservices? • Problem: The service discovery needs to be redundant and reliable (this is hard!) • Discovery services rely on distributed DBs can be used to store configuration • Nice to have: Discovery + health check + auto healing • HATEOAS Options in .NET • Microphone (.NET) + Consul (in Production: master on Linux only!)
- 11. Service Discovery (Microphone example)
- 12. Versioning • Problem: Not all teams should deliver at the same pace • Try to avoid dependencies less management & more happiness Concepts • You still need to deal with old clients of your service relying on previous version • Do not only distribute a client (distributed monolith!), just rely on contracts (& provide example implementations if you wish) • Semantic Romantic Versioning + Minimum required version concept • URL • Headers (Accept header) • API generation / documentation Swagger, apiary, …
- 13. Security (1/2) • Problem: much greater attack surface • Problem: Secure all calls slow / hassle? Concepts • Protect just Internet boundary? What happens in a intruder gets in the network? • Secure communication channels? Services trust each other? • Tamper proofing, replay protection, principal authorization, weakest link, least privilege • API gateway pattern (not just for security but…) • Rate limiting for external APIs (X-Rate-Limit-… headers (Twitter)) Security checklist http://www.grahamlea.com/2015/07/microservices-security-questions/
- 14. Security (2/2) • Authentication, Authorisation, Delegation • Problem: Once the principal is authenticated, it should only have access to certain resources, and we should do that in a scalable way Concepts • Authorisation server (provides tokens and jwt signed with its PK) • Resource server (each of the services) • Delegation OAuth 2 protocol • OpenID protocol (JWT – Json Web Token, a signed json doc) to achieve statelessness • Authentication Leave that to the OAuth 2 server (token) • Authorisation All microservices should consume JWTs (identity)
- 15. Resilience • Problem: If one service is not available, everything crashes Techniques: retry N times, retry forever, retry and wait, circuit breaker, post execution steps Circuit breaker: custom fallback, fail silent, fail fast http://techblog.netflix.com/2011/12/making-netflix-api-more-resilient.html Autohealing (Consul) Options in .NET • Polly
- 16. • Problem: You don´t want to have to check N log files, just 1 • Logging framework vs Logging service • Correlate events • Log deployments / Orchestration (more on this later) Options in .NET • SeriLog + ELK • App Insights (Azure) • Runscope for API monitoring (cloud) • Raygun for error logging (cloud) • Newrelic (cloud)
- 17. Composition • Problem: Once we have all the small bits and pieces, how do we integrate them to provide higher level functionality Options • Monolithic UI to integrate everything • API Gateway • AJAX/Javascript • IFrames • Server Side Includes / Edge Side Includes • Specific services like Netflix Zuul (Java) or Compoxure (NodeJS)
- 18. • Avoid at all costs committing to features that require more than one team to be done in the same place at the same time in order to make an integrated delivery • Other teams to do pull requests with tests (you might not know how your service is really used!) Options in .NET • Unit testing • Contract testing PACT.NET • Mock services
- 19. Inter-service communication • Problem: As we increase the number of services, the number of calls between them also increases • Tools/applications: RestSharp, Akka.Net, RabbitMQ (.NET library available), NATS (.NET lib available), HAProxy (load balance) • Message formats: XML (text), JSON (text), Google Protobuf (binary) One to One One to many Synchronous Request / response - Asynchronous Notification Request / async response Publish / Subscribe Publish / Async responses
- 20. Y A X B Data consistency • Problem: Each microservice is made of code + state • Problem: Distributed transactions are really hard Polyglot persistence Options • Single database • Another microservice to deal with domain data access • Event sourcing + pub/sub • Two phase commit • Replication • Eventual consistency • Business rules
- 21. Orchestration • Problem: How do we deal with 100s of services? Start, stop, move, upgrade… • A nice option: Containers • Linux containers (Mono / ASP.NET 5) • Windows containers Options for Windows (@ 30th March 2016) • Docker Compose + Swarm • Rancher • Kubernetes • Mesos + Marathon • Azure Container Service (Marathon) • Azure service fabric • Akka.NET
- 22. Orchestration Rancher (Linux containers) Marathon (Linux containers)
- 23. Example – Microcafe app CQRS, microservices CQRS Event sourcing 3 services: front-end + read + write data writes in event store data reads from redis (aggregates) message bus with rabbitmq
- 24. Microcafe app by Richard Banks https://github.com/rbanks54/microcafe Microservices: • n-Front-End services • n-Read Services • n-Write Services • Event Store cluster • N-Redis (1 x read service) • RabbitMQ cluster • HAProxy cluster • Consul cluster Diagram by Gabriel Schenker - https://lostechies.com/gabrielschenker/2015/04/07/cqrs-revisited/
- 25. Microcafe app by Richard Banks https://github.com/rbanks54/microcafe Deal with eventual consistency, baby
- 26. Docker on Windows Server Core 2016 TP4
- 27. Service Fabric Microsoft’s take on microservices
- 28. What is Service Fabric? A PaaS for building and running large scale microservice based applications and services © Microsoft
- 29. Service Fabric programming models Stateful vs. Stateless Actors vs. Services
- 30. Service Fabric
- 31. “my” Key Takeaways There is no “right way” for doing microservices Most concepts could be applied to almost any architecture Orchestration is hard Service Fabric is nice… but it is “the Microsoft way” Windows Containers are not ready yet (@ 27th March 2016) Luckily we have OPEN SOURCE (Mono, ASP.NET 5 and… Linux :-) )
Editor's Notes
- http://returnsmart.blogspot.com.es/2015/07/building-microservices-in-net-ian.html https://kf6jna-bn1306.files.1drv.com/y3m-LBl7nt0CQnAZ4MZ081p-oRhqlTAa_gG7TwSJ9VcZQJp7rQxZvCSeZTJsKCtXoh3bqpoWXsetSYqz_oIsbaC4qLOPP1SXxw3JyEJCU7Udr3YR0ouMy6zVOf_S6hKEbpIe1MQDTLBl7U5lZN2kyBqAf7og-BX7PRj6A0dv4sH_Go/Applying%20Microservices%20on%20the%20Microsoft%20Platform%20-%20BASTA%202015%20-%20Ricardo%20Niepel.pdf?psid=1
- HATEOAS, an abbreviation for Hypermedia as the Engine of Application State, is a constraint of the REST application architecture that distinguishes it from most other network application architectures. The principle is that a client interacts with a network application entirely through hypermedia provided dynamically by application servers. A REST client needs no prior knowledge about how to interact with any particular application or server beyond a generic understanding of hypermedia. By contrast, in some service-oriented architectures(SOA), clients and servers interact through a fixed interface shared through documentation or an interface description language (IDL). The HATEOAS constraint decouples client and server in a way that allows the server functionality to evolve independently. For example [2] here is a GET request to fetch an Account resource, requesting details in an XML representation: GET /account/12345 HTTP/1.1 Host: somebank.org Accept: application/xml ... Here is the response: HTTP/1.1 200 OK Content-Type: application/xml Content-Length: ... <?xml version="1.0"?> <account> <account_number>12345</account_number> <balance currency="usd">100.00</balance> <link rel="deposit" href="http://somebank.org/account/12345/deposit" /> <link rel="withdraw" href="http://somebank.org/account/12345/withdraw" /> <link rel="transfer" href="http://somebank.org/account/12345/transfer" /> <link rel="close" href="http://somebank.org/account/12345/close" /> </account> Note the response contains 4 possible follow-up links - to make a deposit, a withdrawal, a transfer or to close the account. Some time later the account information is retrieved again, but now the account is overdrawn: HTTP/1.1 200 OK Content-Type: application/xml Content-Length: ... <?xml version="1.0"?> <account> <account_number>12345</account_number> <balance currency="usd">-25.00</balance> <link rel="deposit" href="http://somebank.org/account/12345/deposit" /> </account> Now only one link is available: to deposit more money. In its current state, the other links are not available. Hence the term Engine of Application State. What actions are possible vary as the state of the resource varies.
- SEMANTIC VERSIONING Given a version number MAJOR.MINOR.PATCH, increment the: MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards-compatible manner, and PATCH version when you make backwards-compatible bug fixes. Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
- Tamper proofing – no one can sabotage the system by changing messages Replay protection – if I capture a message, and I replay it, does it work? Weakest link – a system is as secure as the weakest of its links Least privilege – a principal should have the least of the privileges to do something
- Delegation – client has no idea about the credentials of the user
- Custom fallback - in some cases a service’s client library provides a fallback method we can invoke, or in other cases we can use locally available data on an API server (eg, a cookie or local JVM cache) to generate a fallback response Fail silent - in this case the fallback method simply returns a null value, which is useful if the data provided by the service being invoked is optional for the response that will be sent back to the requesting client Fail fast - used in cases where the data is required or there’s no good fallback and results in a client getting a 5xx response. This can negatively affect the device UX, which is not ideal, but it keeps API servers healthy and allows the system to recover quickly when the failing service becomes available again.
- http://www.infoq.com/articles/microservices-intro
- https://auth0.com/blog/2015/11/07/introduction-to-microservices-part-4-dependencies/
- Actors – virtual actors, can be called using a proxy State - It includes the data structures and variables that the service reads and writes to do work. Service Fabric provides the infrastructure to ensure that this state is co-located with the code that manipulates the data, and highly available and fault tolerant in the event of a failure. ActorId actorId = ActorId.NewId(); string applicationName = "fabric:/CalculatorActorApp"; ICalculatorActor calculatorActor = ActorProxy.Create<ICalculatorActor>(actorId, applicationName); double result = calculatorActor.AddAsync(2, 3).Result; RunAsync