apidays LIVE Paris - Bring the API culture to DevOps teams by Christophe Bourcier & Raphael Manfredi

1. 12/08/20API Days - Bring the API Culture to DevOps Teams1 ⓒ Copyright Entique Consulting 2020 Bring the API Culture to DevOps Teams Christophe Bourcier <c.bourcier@ahead-technology.com> Raphaël Manfredi <ram@entique.fr>

2. 12/08/20API Days - Bring the API Culture to DevOps Teams2 ⓒ Copyright Entique Consulting 2020 Agenda  Exploring API types  Programs viewed as an API  Why DevOps loves API  Examples of API usage (screenshots)

3. 12/08/20API Days - Bring the API Culture to DevOps Teams3 ⓒ Copyright Entique Consulting 2020 Definitions An Application Programming Interface (API) is a computing interface that defines interactions between multiple software intermediaries: the kinds of calls or requests that can be made, how to make them, the data formats that should be used, the conventions to follow, etc. – From Wikipedia: https://en.wikipedia.org/wiki/API An interface is a shared boundary across which two or more separate components of a computer system exchange information. The exchange can be between software, computer hardware, peripheral devices, humans, and combinations of these. – From Wikipedia: https://en.wikipedia.org/wiki/Interface_(computing) A DevOps Team is a cross-functional value chain, made of people sharing a common business objective, whose aim is to regularly bring value increments to customers, at a sustainable steady pace, as quickly as possible and without sacrificing quality nor stability. – Our definition, for the purpose of this talk

4. 12/08/20API Days - Bring the API Culture to DevOps Teams4 ⓒ Copyright Entique Consulting 2020 A Word (or Two) on Caution  This presentation is not deeply technical but rather an overview!  As such, it makes simplifications:  All our technical information is correct,  However we conveniently hide low-level details when possible.  We want you to feel some of the power behind these concepts!  And why they matter for your business and your teams...

5. 12/08/20API Days - Bring the API Culture to DevOps Teams5 ⓒ Copyright Entique Consulting 2020 Exploring API Types

6. 12/08/20API Days - Bring the API Culture to DevOps Teams6 ⓒ Copyright Entique Consulting 2020 Partitioning Interfaces There are three interface types between software components:  Between a (user) program and the underlying operating system: system calls,  Within a program between one “module” and another: library calls, • We will not make any distinction here between a function call, a procedure call, a feature call, a method call, etc. but name everything a function call. • A function takes zero or more parameters, processes them and then maybe returns a result. • These functions have little to do with mathematical functions. :-) In particular, they can have side effects!  And across programs, we can perform remote calls. • By remote here, we mean that one program invokes an action in another one. • These calls can be done locally (on the same system or machine), or remotely (across a network), we do not care: we equally call them “remote” calls.

7. 12/08/20API Days - Bring the API Culture to DevOps Teams7 ⓒ Copyright Entique Consulting 2020 System Calls

8. 12/08/20API Days - Bring the API Culture to DevOps Teams8 ⓒ Copyright Entique Consulting 2020 At The User / Kernel Frontier User space Kernel space write(2, buffer, len); ; assembly wrapper for system call PUSH len PUSH buffer PUSH 2 SYSCALL #25 ; for instance, CPU trap POP val RET val ; return value sys_write(u, 2, buffer, len); ; kernel trap entry point PUSH u ; user process context LD A, syscalls[25] CALL A ; calls sys_write() RETK ; return to user space CPU in « user » mode CPU in « privileged » mode

9. 12/08/20API Days - Bring the API Culture to DevOps Teams9 ⓒ Copyright Entique Consulting 2020 Benefits  The separation of concerns and memory isolation are enforced by the hardware barrier between the User and Kernel CPU modes.  The system, as managed by the kernel, can maintain its whole integrity by providing its “user services” through advertised system calls.  Each system call has a well-defined contract:  input is carefully validated by the kernel upon entry,  the semantic of the call is precisely known,  output is tightly controlled, to avoid information leakage from kernel to user space. The only access to the system is through system calls.

10. 12/08/20API Days - Bring the API Culture to DevOps Teams10 ⓒ Copyright Entique Consulting 2020 Architectural / Design Choices There is not one unique API, even for “standard” operations:  An API reflects higher level concerns such as global system architecture.  An API is meant to be used by programmers (also known as developers)!  Being at the boundary of software components, an API hides its implementation to clients and exposes only an interface, as stable as possible through time. This is called information hiding.  It offers concepts / abstractions to clients that they can interact with through the API and never directly. This is called encapsulation. Examples of system concepts:  Memory  Files  Processes  Users

11. 12/08/20API Days - Bring the API Culture to DevOps Teams11 ⓒ Copyright Entique Consulting 2020 Example: Process Creation On Windows: On UNIX systems: BOOL CreateProcessW( LPCWSTR lpApplicationName, LPWSTR lpCommandLine, LPSECURITY_ATTRIBUTES lpProcessAttributes, LPSECURITY_ATTRIBUTES lpThreadAttributes, BOOL bInheritHandles, DWORD dwCreationFlags, LPVOID lpEnvironment, LPCWSTR lpCurrentDirectory, LPSTARTUPINFOW lpStartupInfo, LPPROCESS_INFORMATION lpProcessInformation ); Signature 10 parameters Return type Function name pid_t fork(void); No parameter Different signatures, different semantics, demonstrating different architectural choices. Child process number

12. 12/08/20API Days - Bring the API Culture to DevOps Teams12 ⓒ Copyright Entique Consulting 2020 Library Calls

13. 12/08/20API Days - Bring the API Culture to DevOps Teams13 ⓒ Copyright Entique Consulting 2020 Libraries / Classes Define an API p = malloc(64); User side Library side char *malloc(size_t sz) { /* Allocate sz bytes */ … return ptr; } … r = Regex.compile(”a.*b+”) s = r.match(”xacdbb”) if s /= Void then matched = s.string end Class sideUser side match(s: STRING): MATCH_INFO do … Result := … … end

14. 12/08/20API Days - Bring the API Culture to DevOps Teams14 ⓒ Copyright Entique Consulting 2020 Same Concepts We have the same exact concepts as with system calls, but with two important differences…  There is no hardware-enforced barrier to protect the implementation: all the functions lie within the same process address space.  Only the implementation language semantics can properly isolate the internals and prevent users from (accidentally) bypassing our interface or corrupting data (thereby creating malfunctions).

15. 12/08/20API Days - Bring the API Culture to DevOps Teams15 ⓒ Copyright Entique Consulting 2020 Benefits Libraries / Classes offer additional benefits, besides encapsulation and information hiding:  Reusability: only a few people will ever write a Quicksort routine or a regular expression engine!  Maintainability: code we do not write is maintained by other people!  Readability: once an API is well-known, everyone is able to understand what code using this API is doing, without having to look at each function.  Correctness (hopefully!) and performance: library routines are likely to be more widely used than company code, hence be more tested and more subject to optimization (through the selection of well-behaving algorithms). Libraries also open the door to possible security problems !

16. 12/08/20API Days - Bring the API Culture to DevOps Teams16 ⓒ Copyright Entique Consulting 2020 Remote Calls

17. 12/08/20API Days - Bring the API Culture to DevOps Teams17 ⓒ Copyright Entique Consulting 2020 Another Level of Complexity The ability to use remote calls brings more power but also creates more complexity in our systems and integration of the various pieces:  More moving pieces means more opportunities for breakage.  Distribution of business processes and data leads to new limitations (for instance the CAP Theorem),  Dependencies between systems can become hard to sever / change through time.  Interoperability concerns may cause technology lock-ups.  Clients may need to authenticate with servers before invoking calls to them.  Furthermore, when the network is involved: • Security concerns are likely to pop-up (traffic eavesdropping, man-in-the-middle attacks, etc.), • Additional failure points are created: a network is not reliable!

18. 12/08/20API Days - Bring the API Culture to DevOps Teams18 ⓒ Copyright Entique Consulting 2020 Remote Call Sequence x = f(a, b) b a f() b a f() f(a, b) ResultResult Calling side Called side Serialization Deserialization SerializationDeserialization Transport We pay 2 transport latencies + serialization & deserialization costs

19. 12/08/20API Days - Bring the API Culture to DevOps Teams19 ⓒ Copyright Entique Consulting 2020 Remote Call Concerns (1/3) x = f(a, b) b a f() b a f() f(a, b) ResultResult Error Error Fault We also need to prepare for transport errors and remote execution faults … and protocol errors! Error handling

20. 12/08/20API Days - Bring the API Culture to DevOps Teams20 ⓒ Copyright Entique Consulting 2020 Remote Call Concerns (2/3) x = f(a, b) b a f() b a f() f(a, b) ResultResult How do we manage changes in the API (addition of arguments, new type)? … and know about the problem! cc A Result v2

21. 12/08/20API Days - Bring the API Culture to DevOps Teams21 ⓒ Copyright Entique Consulting 2020 Remote Call Concerns (3/3) x = f(a, b) b a f() b a f() f(a, b) ResultResult Can we allow different languages to interact remotely (think serialization)? … and where do we find the server? Language #1 : java Language #2 : C

22. 12/08/20API Days - Bring the API Culture to DevOps Teams22 ⓒ Copyright Entique Consulting 2020 Remote Call Benefits Despite the problems we mentioned, there are also some benefits:  Because the calling and called parties are in different processes, we have a process- enforced barrier protecting the implementation (comparable to the protection we have for system calls).  We are able to share (offer) the implementation of the function and its resources (code, memory, etc.) across several clients. This is called the client-server paradigm (caller => client, callee => server).  We can conceivably scale the processing of our function on the server-side by distributing incoming requests to various processors (load balancing).  We can make our remote calls asynchronous, allowing parallelism and exploitation of remote CPU time. Benefits must outweigh the costs, or we would never do remote calls !

23. 12/08/20API Days - Bring the API Culture to DevOps Teams23 ⓒ Copyright Entique Consulting 2020 Another Subclass of Remote Calls There is an important subclass of remote calls… … one which alleviates most of the concerns we detailed earlier:  Client language independence: allows clients to interact with servers even when they are not implemented in the same language.  Some API signature flexibility, allowing additional (unknown) parameters to be sent (which will be naturally ignored by the server) or specifying optional parameters (that do not need to be sent).  Simplifies the ability for clients to find the server to send their requests to.  Can also avoid service overloading by allowing some response caching. We will name this class of remote calls “ Web Calls “.

24. 12/08/20API Days - Bring the API Culture to DevOps Teams24 ⓒ Copyright Entique Consulting 2020 Web Calls

25. 12/08/20API Days - Bring the API Culture to DevOps Teams25 ⓒ Copyright Entique Consulting 2020 Web Call – Our Definition A Web Call is a Remote Call bearing the following characteristics:  The exchange protocol between the client and the server is HTTP (Hyper Text Transfer Protocol).  Both parties can leverage the caching semantics defined by HTTP: servers can mark some replies cacheable, clients may request validation of a cached value (to determine whether it is still consistent), etc.  The client and server negotiate an appropriate format for delivering the reply payload. This is called content-type negotiation.  The client sends its function parameters using named arguments.  The server could be a facade (API Gateway) and it should not matter for the client.

26. 12/08/20API Days - Bring the API Culture to DevOps Teams26 ⓒ Copyright Entique Consulting 2020 Useful Architectural Properties for Web Calls: REST A Web Call is further called “RESTful” when it follows these REST (Representational State Transfer) architectural properties:  The server is stateless, hence all the function context must be supplied by the client: the server does not remember any client state.  The client manipulates resources through well-defined HTTP Methods: • GET to retrieve a representation of the resource. Idempotent, no side effect, “read-only”. • POST to create a new resource. Not idempotent, and has side effects (resource allocation). • PUT to replace a resource with a new representation. This is idempotent! • PATCH to update parts of the resource representation (no need to supply the whole representation). • DELETE to destroy a resource.  Resources are uniform, represented via a URI (Uniform Resource Identifier).  Client and server rely on HATEOAS (Hypermedia As The Engine Of Application State) to drive their interactions besides the initial URI entry point.

27. 12/08/20API Days - Bring the API Culture to DevOps Teams27 ⓒ Copyright Entique Consulting 2020 Important Points About REST Web Calls Contrary to other attempts to use HTTP for Remote Calls:  We can leverage the existing web infrastructure and semantics to scale: • We use HTTP to really transport Hyper Text thanks to HATEOAS. • We can leverage the HTTP caches to prevent GET overloading on the server and perform time- decoupling between services.  We are at liberty to select a suitable extensible representation for our resources: • Using XML (eXtensible Markup Language) or JSON (JavaScript Object Notation) as serialization formats allows easy inspection of our API inputs and outputs by humans… • … whilst still preserving our interface flexibility to add optional parameters or returned values.  We do not need custom middleware (ORB = Object Request Broker) to distribute our application web calls.  We can easily use OAuth 2.0 (RFC-6750) tokens for credentials allowing access to protected resources.

28. 12/08/20API Days - Bring the API Culture to DevOps Teams28 ⓒ Copyright Entique Consulting 2020 Our API Pyramidal Taxonomy System Calls Library Calls Remote Calls Network Web Calls REST Calls Local SOAP RMI RPC RPC : Remote Procedure Call RMI : Remote Method Invocation SOAP: Simple Object Access Protocol

29. 12/08/20API Days - Bring the API Culture to DevOps Teams29 ⓒ Copyright Entique Consulting 2020 Programs Viewed as an API

30. 12/08/20API Days - Bring the API Culture to DevOps Teams30 ⓒ Copyright Entique Consulting 2020 Introducing the Command Line Interface A Command-Line Interface (CLI) processes commands to a computer program in the form of lines of text. The program which handles the interface is called a command-line interpreter or command-line processor. Operating systems implement a command-line interface in a shell for interactive access to operating system functions or services. – From Wikipedia: https://en.wikipedia.org/wiki/Command-line_interface We can view this from another angle...  A command to the shell is just a high-level function call: • It takes parameters: the given arguments to the command and its possible options. • It executes the function • It outputs a result.  Our shell command can call one or several lower-level functions to create its output.

31. 12/08/20API Days - Bring the API Culture to DevOps Teams31 ⓒ Copyright Entique Consulting 2020 Example: The “date” Command Sat Feb 6 16:19:22 CET 2020 $ date -d ’next week + 2 months’ Sun Dec 13 16:19:08 CET 2020 $ date -d ’next week’ Mostly a wrapper of two library calls : localtime() strftime() $ date Sun Dec 6 16:19:00 CET 2020 Command prompt Command Command output Cool, we can also parse text ! And more symbolic expressions !

32. 12/08/20API Days - Bring the API Culture to DevOps Teams32 ⓒ Copyright Entique Consulting 2020 Launching a Web Call... $ echo $MM_API https://gitlab.ram.loc:2443/api/v4 $ curl -s "$MM_API/teams/name/days" -H "Authorization: Bearer $MM_TOKEN" { "id": "6dtpusyuytg8fcsazkkafmqgtw", "create_at": 1605955699434, "update_at": 1605955699434, "delete_at": 0, "display_name": "api-days", "name": "days", "description": "", "email": "ram@entique.fr", "type": "O", "company_name": "", "allowed_domains": "", "invite_id": "j3uxp54zy3ga7easarduessjje", "allow_open_invite": false, "scheme_id": null, "group_constrained": null } This is the URL of my own Mattermost running at home. Environment variable holding my OAuth 2.0 access token for this Mattermost instance. I will keep this private and not show it… I will also skip this -H option in further examples. This is JSON output, supplying us with the Mattermost information about a team « days » created on the server. We learn its display name « api-days » and its internal ID.

33. 12/08/20API Days - Bring the API Culture to DevOps Teams33 ⓒ Copyright Entique Consulting 2020 Going Further With our Web Calls  The command curl -s “$MM_API/teams/name/days” lets us trigger the GET method on the URI with the parameter “days” given at the end: https://gitlab.ram.loc:2443/api/v4/teams/name/days  Mattermost returns us the representation for that resource.  If all we needed was, say, the ID of the team “days” we could write: $ curl -s "$MM_API/teams/name/days" | jq '.id' | sed -e 's/"//g' 6dtpusyuytg8fcsazkkafmqgtw  We could request special « slash » commands defined for that team : $ ID=6dtpusyuytg8fcsazkkafmqgtw $ curl -s "$MM_API/commands?team_id=$ID&custom_only=1" | jq '.[].trigger' "sh" "lab2" "lab1"

34. 12/08/20API Days - Bring the API Culture to DevOps Teams34 ⓒ Copyright Entique Consulting 2020 Writing a New Command If getting to know the special “slash” commands that were defined by a given team, and witness what they are doing, is a repeated task that needs to be performed, we can even write a new program: $ mm-commands Usage: mm-commands [-dh] [--raw] team_name -d/--debug : enable debug messages -h/--help : show this help message --raw : use raw JSON output $ mm-commands days /lab1 Perform common operations on GitLab project: API Days / Scratch project https://gitlab.ram.loc/api/v4/projects/2/services/mattermost_slash_commands/trigger /lab2 Testing POST to an echo service http://netlab.ram.loc:22080/echo /sh Run shell commands http://netlab.ram.loc:22080/run Very small program : 100 lines, about 300 words and 2258 bytes total! Note that the output lists the 3 commands in alphabetical order… We are extracting additional info from the representation: hooked URLs and descriptions.

35. 12/08/20API Days - Bring the API Culture to DevOps Teams35 ⓒ Copyright Entique Consulting 2020 Example: The /lab2 Command /lab2 arg1 arg2 arg3 "arg4 with space" /lab2 Testing POST to an echo service http://netlab.ram.loc:22080/echo We have :  The command arguments in the text parameter, URL-encoded.  The URL to respond back to, if we want to POST back some interesting reply instead of just echo-ing the input request body, as we are doing here : it is given in the response_url parameter, URL-encoded : https://gitlab.ram.loc:2443/hooks/commands/wpje89…  The access token for the reply to this command if we want to use reponse_url We have the basis for a remote integration at our programming fingertips.

36. 12/08/20API Days - Bring the API Culture to DevOps Teams36 ⓒ Copyright Entique Consulting 2020 What Does This Buy Us?  We can rely on APIs to compose new functions that in turn will become pieces on top of which we can compose more. (That’s programming!)  Enterprise-grade software is usually offering APIs, allowing us to automate processing instead of having to do everything through a graphical interface!  And by treating this new automation as code we can:  Document the process: code is living documentation that can also be executed!  Version-control it, and therefore review it, tracing its changes.  Improve it over time, as needed.  Share it to save the lives of others :-)

37. 12/08/20API Days - Bring the API Culture to DevOps Teams37 ⓒ Copyright Entique Consulting 2020 Why DevOps Loves API

38. 12/08/20API Days - Bring the API Culture to DevOps Teams38 ⓒ Copyright Entique Consulting 2020 DevOps in a Nutshell  “Create customer value permanently”  “Extend agile software development practices to production”  “Deliver reliable software applications faster”  “You build it, you run it!” – Werner Vogel, CTO Amazon DevOps is about fostering collaboration among the complete value chain to continuously enhance business value!

39. 12/08/20API Days - Bring the API Culture to DevOps Teams39 ⓒ Copyright Entique Consulting 2020 The Three Ways of DevOps 1 Flow  The flow of work from Dev to Ops must be as smooth and steady as possible.  Each step in our production pipeline must be triggered when the previous one has completed successfully with minimum overhead. 2 Feedback  This is really about having a reverse flow from Ops to Dev, as quickly and efficiently as possible: short feedback loops.  We want to access information easily for a quick diagnosis and remediation. 3 Experimentation & Learning  Exploring new tools, new technologies, new ways of doing our work.  Sharing our discoveries, helping others practice.

40. 12/08/20API Days - Bring the API Culture to DevOps Teams40 ⓒ Copyright Entique Consulting 2020 Improving Flow with APIs

41. 12/08/20Grenoble EM - Master Big Data 202041 ⓒ Copyright Entique Consulting 2020 The DevOps Pipeline Project Team Source Code Mgmt Build & Unit Tests Acceptance Tests UATs Delivery Check-in Trigger Feedback Check-in Trigger Feedback Trigger Feedback Check-in Trigger Feedback Trigger Feedback Trigger Feedback Approbation Feedback Source: Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation Execution of the pipeline stages must be automatic, meaning tools communicate via APIs for triggers, feedback delivery, metrics collection, etc.

42. 12/08/20API Days - Bring the API Culture to DevOps Teams42 ⓒ Copyright Entique Consulting 2020 Actions Within the DevOps Pipeline During the pipeline execution, many steps can rely on an API:  The triggering of the pipeline itself (web hook, which invokes an URL via HTTP)  The provisioning of the infrastructure to perform the various steps of the pipeline  The registration of artifacts into an artifact repository  The collection of metrics  The execution of tests against the application  The preparation of a Change Request in ITSM systems  Etc. Relying on a manual step within the pipeline is likely going to create a constraint and hurt the flow !

44. 12/08/20API Days - Bring the API Culture to DevOps Teams44 ⓒ Copyright Entique Consulting 2020 Accelerating Notifications & Problem Solving Whenever something “unexpected” happens in our pipeline or in production, it needs to be addressed. Quickly. Urgently!  Speed of notification, targeting the right group is of the essence. And e-mail is not the answer here (being an asynchronous medium). We need some kind of ChatOps.  Make information available for diagnosis. We need an automated access to logs, maybe even proactive pushing of context to a ChatOps platform.  Incident management needs to trigger real-time notifications to Ops, and maybe Dev as well. Did we say ChatOps?  Information exchanged during incident management needs to be captured and fed back to the ITSM systems. Yes, API again, to collect the data (from ChatOps) and automatically feed it back to the proper incident log. APIs let machines collaborate (with humans) !

45. 12/08/20API Days - Bring the API Culture to DevOps Teams45 ⓒ Copyright Entique Consulting 2020 Monitoring is Also (Source of) Feedback Applications / Systems in production need monitoring. Right?  Monitoring is not just doing health checks.  Applications / Systems should offer APIs to help information gathering.  Collected information can also be given to “Machine Learning” (via some APIs) in order to: • Identify failure patterns (to repair before it is too late), leading to incident avoidance. • Assess our overall application end-to-end performance, find where bottlenecks are. • Identify security attacks: unusual traffic patterns or errors.  Embedding monitoring APIs in applications is different from having applications log! • For instance the Self-Monitoring, Analysis and Reporting Technology (SMART) in disks. • SMART also offers actions, not just monitoring (e.g. “launch long test”).

47. 12/08/20API Days - Bring the API Culture to DevOps Teams47 ⓒ Copyright Entique Consulting 2020 Of Course There are APIs! As far as learning is concerned:  If you want to learn, there is a famous human-friendly Web API available at https://www.wikipedia.org/  If you want a machine to learn, there is the Keras API to train tensors. And for experimenting, well…  We are surrounded by APIs :-)  You can even invent new APIs

49. 12/08/20API Days - Bring the API Culture to DevOps Teams49 ⓒ Copyright Entique Consulting 2020 From a Developer Standpoint Developers spend their (professional) life with APIs.  We have seen system calls, library calls, remote calls (along with the subclass of web calls) and we can throw in some CLI as well with the shell. They are all part of every program a developer can think of.  Existing APIs are combined together to create new programs. They are the basic building block.  Developers can also architect and design APIs, just like they architect and design software: with object-oriented programming, functional programming, imperative programming, we practice this art (yes, there is an artistic dimension to it).  Developers can also create facade API to offer a higher-level or more convenient set of concepts / abstractions than the underlying API, or simply to offer a common API on top of different APIs (for portability or ease of implementation switching). Developers love API by design.

50. 12/08/20API Days - Bring the API Culture to DevOps Teams50 ⓒ Copyright Entique Consulting 2020 From an Ops Standpoint Ops spend their (professional) life ensuring systems work as they should.  They do not necessarily know about API, although most already use CLI.  As they collaborate more and more with Developers, early on, during system architecture, they can nonetheless influence on API they could get in the developed systems to monitor them.  Modern systems are designed to be “Cloud-native” with a “micro-service” architecture, both of which are deeply connected to APIs! More opportunities to assist Developers before they release to production something unmanageable.  As Infrastructure as Code, immutable infrastructure and containerization approaches emerge, API are going to invade the world of Ops as well.  Operation automation is on the rise, with approaches such as Google SRE (where we put software engineers in charge of operations) and ChatOps. Operations are about to become API lovers.

51. 12/08/20API Days - Bring the API Culture to DevOps Teams51 ⓒ Copyright Entique Consulting 2020 Conclusion  One of DevOps tenets is efficiency of the value chain: to be able to create more value, we want to focus on doing more things that create value, and reduce waste.  APIs allow creation of programs, i.e. of automation. That brings us:  Speed of execution  Coherence of execution  Documentation of our execution (the program being a working specification). APIs are everywhere, and this is just the beginning !