Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 41

MariaDB, MySQL and Ansible: automating database infrastructures

0

Share

Download to read offline

Automating database infrastructures is important. Here you can find some hints and good practices, as well as a general explanation of Ansible.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

MariaDB, MySQL and Ansible: automating database infrastructures

  1. 1. MariaDB, MySQL and Ansible: automating database infrastructures Federico Razzoli
  2. 2. $ whoami Hi, I’m Federico Razzoli from Vettabase Ltd Database consultant, open source supporter, long time MariaDB and MySQL user ● vettabase.com ● Federico-Razzoli.com
  3. 3. This talk ● General Ansible concepts ● Opinionated code examples, automating MariaDB/MySQL ● Good and bad practices
  4. 4. Ansible Overview
  5. 5. What is Ansible ● A tool for deployment and configuration automation ● Infrastructure as code ● Idempotent ● Simple, non-centralised
  6. 6. Ansible Concepts ● Inventory ● Module ● Playbook ● Role ● Play ● Variable ● Facts
  7. 7. Inventories
  8. 8. Inventory example [pmm] pmm-1 [mariadb_main] mariadb-main-1 mariadb-main-2 mariadb-main-3 [mariadb_main:vars] pmm_server_host=pmm-1 pmm_server_port=80
  9. 9. Ansible hosts [pmm] pmm-1 ansible_host=123.123.123.123 [mariadb_main] mariadb-main-1 ansible_host=123.0.0.1 mariadb-main-2 ansible_host=123.0.0.2 mariadb-main-3 ansible_host=123.0.0.3 [mariadb_main:vars] pmm_server_host=123.123.123.123 pmm_server_port=80 ● Sometimes we cannot use meaningful hostnames
  10. 10. Hierarchical groups [mariadb_master] mariadb-master-1 mariadb-master-2 [mariadb_replica] mariadb-replica-1 mariadb-replica-2 [mariadb:children] mariadb_master mariadb_replica ● We have different groups for masters and replicas, and a supergroup that includes them both
  11. 11. Groups goals ● You assign each group some roles ● You can deploy one or more group
  12. 12. Multiple inventories production_db: [mariadb_master] mariadb-master-1 ansible_host=111.0.0.1 mariadb-master-2 ansible_host=111.0.0.2 staging_db: [mariadb_master] mariadb-master-1 ansible_host=222.0.0.1 mariadb-master-2 ansible_host=222.0.0.2
  13. 13. Plays
  14. 14. Play example --- - hosts: mariadb_main roles: - linux-for-mysql - mariadb - hosts: mysql_main roles: - linux-for-mysql - mysql ● You assign roles to groups (or individual hosts)
  15. 15. Applying all roles to a certain host Call Ansible specifying an inventory and a play: % ansible-playbook -i staging -l mariadb_main main.yml
  16. 16. Play example Here is where it can be convenient to have multiple inventories: % ansible-playbook -i staging -l mariadb_main main.yml % ansible-playbook -i production -l mariadb_main main.yml
  17. 17. Roles, variables and templates
  18. 18. Example - name: Create script dir file: name: "{{ script_dir }}" owner: root group: root mode: 755 state: directory ● This is a task, but the term is misleading ● A role is a list of tasks ● The name appears on screen ● "file" is the module ● script_dir is a variable
  19. 19. Defining variables script_dir: /opt/scripts base_dir: /usr/local/mysql ... ● /host_vars/<host_name>.yml ● /group_vars/<group_name>.yml ● /roles/<role_name>/defaults/main.yml
  20. 20. Notes on variables ● The same variable can be used by multiple roles ● Roles from Ansible Galaxy can’t use this opportunity ● This is a good reason to make your own roles ● Document what each variable does in README.md
  21. 21. Copying a configuration file - name: Copy my.cnf copy: src: ./files/my.cnf dest: "/etc/mysql/{{ inventory_hostname }}.cnf" ● my.cnf is copied to /etc/mysql
  22. 22. Using a template - name: Copy my.cnf template: src: ./templates/my.cnf .j2 dest: /etc/mysql/my.cnf ● This time my.cnf is a Jinja 2 template
  23. 23. Template example [mysqld] user = mysql datadir = {{ mysql_data_dir }} innodb_buffer_pool_size = {{ innodb_buffer_pool_size }} ● This time my.cnf is a Jinja 2 template
  24. 24. Problems ● If we want to set a variable, we should be able to add it in one place ○ host_vars or group_vars or role's defaults ● We can make a template more dynamic
  25. 25. Dynamic template example In group_vars we create a list of dictionaries: (arrays of objects, if you prefer) group_mysql_variables: - { name: 'innodb_buffer_pool_size', value: '50G' } - { name: 'innodb_log_file_size', value: '50G' } In host_vars we optionally create another list: host_mysql_variables: - { name: 'innodb_buffer_pool_size', value: '80G' }
  26. 26. Dynamic template example We loop over both the lists in the template: {% for var in group_mysql_variables %} {{ var.name }} = {{ var.value }} {% endfor %} {% if host_mysql_variables is defined %} {% for var in host_mysql_variables %} {{ var.name }} = {{ var.value }} {% endfor %} {% endif %}
  27. 27. Notes ● We cannot use the same list in group_vars and host_vars, because the one in host_vars would overwrite the whole list ● Some variables may appear twice in the configuration file ● This is fine: the last occurrence of a variable will override the previous one
  28. 28. Another use for lists Hosts of a cluster can be a list too: # print the IPs separated by commas wsrep_cluster_address = gcomm://{{ private_ips|join(',') }} # first node is the donor wsrep_sst_donor = {{ cluster_hosts[0].node_name }} # Only the first node imports a backup when the cluster is created - name: Import backup ... when: cluster_hosts[0] == hostvars[inventory_hostname]['ansible_default_ipv4']['address']
  29. 29. Validate variables Validate variables for complex roles: - name: Validate cluster size assert: that: cluster_hosts|length > 0 fail_msg: cluster_hosts must contain at least 1 element success_msg: "cluster_hosts size: {{ cluster_hosts|length }}" - name: Validate cluster IPs assert: that: "'{{ item.public_ip }}'|ipaddr" fail_msg: "{{ item.public_ip }} is not a valid IP" success_msg: "{{ item.public_ip }}: OK" with_items: "{{ cluster_hosts }}"
  30. 30. Tags and similar features
  31. 31. Conditional tasks ● Certain tasks should only be run for certain servers: - name: Make server read only mysql_variables: variable: read_only value: 1 when: is_replica is sameas true ● Sometimes this avoids the need to create multiple roles which are very similar ● Instead of creating mariadb and mariadb_replica roles, we may create mariadb only, with some conditional tasks ○ Sometimes it makes sense, sometimes it doesn't
  32. 32. Conditional tasks ● We can group optional tasks into separate files: - name: Make server read only include: replica.yml when: is_replica is sameas true
  33. 33. Idempotency of Tasks ● Most modules are idempotent ● This means that you can run a task like this multiple times safely: - name: Start MariaDB service: state: started ● If the service is already running, nothing happens
  34. 34. Idempotency of Tasks ● Sometimes you make non-idempotent tasks on purpose: - Copy my.cnf ... - name: Restart MariaDB service: state: restarted ● If the service is already running, nothing happens
  35. 35. Idempotency of Tasks ● Sometimes you'd like a task to be idempotent, but it can't be: - name: Run a system command shell: > rm "{{ mysql_data_dir }}"/* - name: Run a script shell: > "{{ mysql_script_dir }}"/my_script.py - name: Run some SQL mysql_query: query: CREATE OR REPLACE TABLE db.table ( ... ); ● Ansible doesn't understand system commands, scripts or queries, so it has not way to check if they were already run. It runs them every time.
  36. 36. Tags ● Ansible supports tags ● Each task can be assigned a list of tags: - name: (Re)create users ... - name: Update timezone info tags: [ tzinfo-update ] ... - name: Do something else ... ansible-playbook -i production_mysql --tag tzinfo-update production.yml
  37. 37. Tags ● Typical tasks I want to have: - name: Do something - name: Copy my.cnf tags: [ mysql-config- static-update ] - name: Restart MySQL tags: [ mysql-config- static-update ] - name: Set variables at runtime tags: [ mysql-config- dynamic-update ] - name: Do something else
  38. 38. Tags ● So there is a way to selectively run certain tasks ● But what if we want to selectively exclude certain tasks? ● There is not built-in way, but we can do something like this: - name: Do something - name: Restart MySQL service: state: restarted when: {{ mysql_restart }} is defined and {{ mysql_restart }} is sameas true - name: Do something else mething else ansible-playbook -i production_mysql --tag mysql-config-dynamic-update -e 'mysql_no_restart=1' production.yml
  39. 39. Thanks for attending! Question time :-)
  40. 40. xxxxx ● Xx ● yy
  41. 41. Example CREATE OR REPLACE TABLE ticket ( id INT PRIMARY KEY NOT NULL AUTO_INCREMENT, state ENUM('OPEN', 'VERIFIED', 'FIXED', 'INVALID') NOT NULL DEFAULT 'OPEN', summary VARCHAR(200) NOT NULL, description TEXT NOT NULL ) ENGINE InnoDB ; ● We want to start to track changes to bugs over time

×