Pgsql Provision
Playbook Overview
After completing the [infrastructure initialization](. /infra-provision/), users can pgsql.yml
to complete the initialization of the database cluster.
First in [Pigsty configuration file](… /… //config/pgsql/) and then apply the changes to the real environment by executing pgsql.yml
.
. /pgsql.yml # Execute the database cluster initialization operation on all the machines in the manifest (danger!)
. /pgsql.yml -l pg-test # Perform database cluster initialization on the machines under the pg-test manifest (recommended!)
. /pgsql.yml -l pg-meta,pg-test # Initialize both pg-meta and pg-test clusters at the same time
. /pgsql.yml -l 10.10.10.11 # Initialize the database instance on the machine 10.10.10.11
Caution
-
Using
pgsql.yml
without parameters is convenient, but it is a high-risk operation in production environmentsIt is strongly recommended that you add the
-l
parameter to the execution to limit the range of objects for which the command can be executed. -
Users can treat metanodes as normal nodes reuse, i.e. define and create PostgreSQL databases on metanodes.
In the default sandbox environment, executing
. /pgsql.yml
will complete the initialization ofpg-meta
andpg-test
at the same time. -
Separately When performing initialization for a cluster slave, the user must make sure that the master library has been initialized, but not the master and its slaves at the same time.
Protection mechanism
pgsql.yml
provides a protection mechanism determined by the configuration parameter pg_exists_action
. When there is a running instance of PostgreSQL on the target machine before executing the script, Pigsty will act according to the configuration abort|clean|skip
of pg_exists_action
.
abort
: recommended to be set as the default configuration to abort script execution if an existing instance is encountered to avoid accidental library deletion.clean
: recommended to be used in local sandbox environment, to clear existing database in case of existing instances.skip
: Execute subsequent logic directly on the existing database cluster.- You can use
. /pgsql.yml -e pg_exists_action=clean
to override the configuration file option and force wipe the existing instance
The pg_disable_purge`'' option provides double protection; if enabled,
pg_exists_action'' is forced to be set to ``abort
' and will not wipe out running database instances under any circumstances.
dcs_exists_action` and
dcs_disable_purge` have the same effect as the above two options, but for DCS (Consul Agent) instances.
Selective execution
Users have the option to execute a subset of scripts through ansible’s tagging mechanism.
As an example, if you want to execute only the service initialization part, you can do so with the following command
./pgsql.yml --tags=service
Frequently used subsets
./pgsql.yml --tags=infra # 完成基础设施的初始化,包括机器节点初始化与DCS部署
./pgsql.yml --tags=node # 完成机器节点的初始化
./pgsql.yml --tags=dcs # 完成DCS:consul/etcd的初始化
./pgsql.yml --tags=dcs -e dcs_exists_action # 完成consul/etcd的初始化,抹除已有的consul agent
./pgsql.yml --tags=pgsql # 完成数据库与监控的部署
./pgsql.yml --tags=postgres # 完成数据库部署
./pgsql.yml --tags=monitor # 完成监控的部署
./pgsql.yml --tags=service # 完成负载均衡的部署,包括Haproxy与VIP
./pgsql.yml --tags=haproxy_config,haproxy_reload # 修改Haproxy配置并应用。
Description
pgsql.yml
does the following, among others.
- Initialize the database node infrastructure (
node
) - Initialize the DCS Agent (or DCS Server if it is a meta-node) service (
consul
) - Install, deploy, and initialize PostgreSQL, Pgbouncer, Patroni (
postgres
) - Installation of PostgreSQL monitoring system (
monitor
) - Installing and deploying Haproxy and VIP, exposing services to the outside world (
service
)
Please refer to Tasks for the precise label about tasks
#!/usr/bin/env ansible-playbook
---
#==============================================================#
# File : pgsql.yml
# Mtime : 2020-05-12
# Mtime : 2021-03-15
# Desc : initialize pigsty cluster
# Path : pgsql.yml
# Copyright (C) 2018-2021 Ruohang Feng
#==============================================================#
#------------------------------------------------------------------------------
# init node and database
#------------------------------------------------------------------------------
- name: Pgsql Initialization
become: yes
hosts: all
gather_facts: no
roles:
- role: node # init node
tags: [infra, node]
- role: consul # init consul
tags: [infra, dcs]
- role: postgres # init postgres
tags: [pgsql, postgres]
- role: monitor # init monitor system
tags: [pgsql, monitor]
- role: service # init service
tags: [service]
...
Tasks
list all available tasks with following commands
./pgsql.yml --list-tasks
默认任务如下:
playbook: ./pgsql.yml
play #1 (all): Pgsql Initialization TAGS: []
tasks:
node : Update node hostname TAGS: [infra, node, node_name]
node : Add new hostname to /etc/hosts TAGS: [infra, node, node_name]
node : Write static dns records TAGS: [infra, node, node_dns]
node : Get old nameservers TAGS: [infra, node, node_resolv]
node : Truncate resolv file TAGS: [infra, node, node_resolv]
node : Write resolv options TAGS: [infra, node, node_resolv]
node : Add new nameservers TAGS: [infra, node, node_resolv]
node : Append old nameservers TAGS: [infra, node, node_resolv]
node : Node configure disable firewall TAGS: [infra, node, node_firewall]
node : Node disable selinux by default TAGS: [infra, node, node_firewall]
node : Backup existing repos TAGS: [infra, node, node_repo]
node : Install upstream repo TAGS: [infra, node, node_repo]
node : Install local repo TAGS: [infra, node, node_repo]
node : Install node basic packages TAGS: [infra, node, node_pkgs]
node : Install node extra packages TAGS: [infra, node, node_pkgs]
node : Install meta specific packages TAGS: [infra, node, node_pkgs]
node : Install node basic packages TAGS: [infra, node, node_pkgs]
node : Install node extra packages TAGS: [infra, node, node_pkgs]
node : Install meta specific packages TAGS: [infra, node, node_pkgs]
node : Node configure disable numa TAGS: [infra, node, node_feature]
node : Node configure disable swap TAGS: [infra, node, node_feature]
node : Node configure unmount swap TAGS: [infra, node, node_feature]
node : Node setup static network TAGS: [infra, node, node_feature]
node : Node configure disable firewall TAGS: [infra, node, node_feature]
node : Node configure disk prefetch TAGS: [infra, node, node_feature]
node : Enable linux kernel modules TAGS: [infra, node, node_kernel]
node : Enable kernel module on reboot TAGS: [infra, node, node_kernel]
node : Get config parameter page count TAGS: [infra, node, node_tuned]
node : Get config parameter page size TAGS: [infra, node, node_tuned]
node : Tune shmmax and shmall via mem TAGS: [infra, node, node_tuned]
node : Create tuned profiles TAGS: [infra, node, node_tuned]
node : Render tuned profiles TAGS: [infra, node, node_tuned]
node : Active tuned profile TAGS: [infra, node, node_tuned]
node : Change additional sysctl params TAGS: [infra, node, node_tuned]
node : Copy default user bash profile TAGS: [infra, node, node_profile]
node : Setup node default pam ulimits TAGS: [infra, node, node_ulimit]
node : Create os user group admin TAGS: [infra, node, node_admin]
node : Create os user admin TAGS: [infra, node, node_admin]
node : Grant admin group nopass sudo TAGS: [infra, node, node_admin]
node : Add no host checking to ssh config TAGS: [infra, node, node_admin]
node : Add admin ssh no host checking TAGS: [infra, node, node_admin]
node : Fetch all admin public keys TAGS: [infra, node, node_admin]
node : Exchange all admin ssh keys TAGS: [infra, node, node_admin]
node : Install public keys TAGS: [infra, node, node_admin]
node : Install ntp package TAGS: [infra, node, ntp_install]
node : Install chrony package TAGS: [infra, node, ntp_install]
node : Setup default node timezone TAGS: [infra, node, ntp_config]
node : Copy the ntp.conf file TAGS: [infra, node, ntp_config]
node : Copy the chrony.conf template TAGS: [infra, node, ntp_config]
node : Launch ntpd service TAGS: [infra, node, ntp_launch]
node : Launch chronyd service TAGS: [infra, node, ntp_launch]
consul : Check for existing consul TAGS: [consul_check, dcs, infra]
consul : Consul exists flag fact set TAGS: [consul_check, dcs, infra]
consul : Abort due to consul exists TAGS: [consul_check, dcs, infra]
consul : Clean existing consul instance TAGS: [consul_clean, dcs, infra]
consul : Stop any running consul instance TAGS: [consul_clean, dcs, infra]
consul : Remove existing consul dir TAGS: [consul_clean, dcs, infra]
consul : Recreate consul dir TAGS: [consul_clean, dcs, infra]
consul : Make sure consul is installed TAGS: [consul_install, dcs, infra]
consul : Make sure consul dir exists TAGS: [consul_config, dcs, infra]
consul : Get dcs server node names TAGS: [consul_config, dcs, infra]
consul : Get dcs node name from var TAGS: [consul_config, dcs, infra]
consul : Get dcs node name from var TAGS: [consul_config, dcs, infra]
consul : Fetch hostname as dcs node name TAGS: [consul_config, dcs, infra]
consul : Get dcs name from hostname TAGS: [consul_config, dcs, infra]
consul : Copy /etc/consul.d/consul.json TAGS: [consul_config, dcs, infra]
consul : Copy consul agent service TAGS: [consul_config, dcs, infra]
consul : Get dcs bootstrap expect quroum TAGS: [consul_server, dcs, infra]
consul : Copy consul server service unit TAGS: [consul_server, dcs, infra]
consul : Launch consul server service TAGS: [consul_server, dcs, infra]
consul : Wait for consul server online TAGS: [consul_server, dcs, infra]
consul : Launch consul agent service TAGS: [consul_agent, dcs, infra]
consul : Wait for consul agent online TAGS: [consul_agent, dcs, infra]
postgres : Create os group postgres TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Make sure dcs group exists TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Create dbsu {{ pg_dbsu }} TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Grant dbsu nopass sudo TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Grant dbsu all sudo TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Grant dbsu limited sudo TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Config patroni watchdog support TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Add dbsu ssh no host checking TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Fetch dbsu public keys TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Exchange dbsu ssh keys TAGS: [instal, pg_dbsu, pgsql, postgres]
postgres : Install offical pgdg yum repo TAGS: [instal, pg_install, pgsql, postgres]
postgres : Install pg packages TAGS: [instal, pg_install, pgsql, postgres]
postgres : Install pg extensions TAGS: [instal, pg_install, pgsql, postgres]
postgres : Link /usr/pgsql to current version TAGS: [instal, pg_install, pgsql, postgres]
postgres : Add pg bin dir to profile path TAGS: [instal, pg_install, pgsql, postgres]
postgres : Fix directory ownership TAGS: [instal, pg_install, pgsql, postgres]
postgres : Remove default postgres service TAGS: [instal, pg_install, pgsql, postgres]
postgres : Check necessary variables exists TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Fetch variables via pg_cluster TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Set cluster basic facts for hosts TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Assert cluster primary singleton TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Setup cluster primary ip address TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Setup repl upstream for primary TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Setup repl upstream for replicas TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Debug print instance summary TAGS: [always, pg_preflight, pgsql, postgres, preflight]
postgres : Check for existing postgres instance TAGS: [pg_check, pgsql, postgres, prepare]
postgres : Set fact whether pg port is open TAGS: [pg_check, pgsql, postgres, prepare]
postgres : Abort due to existing postgres instance TAGS: [pg_check, pgsql, postgres, prepare]
postgres : Clean existing postgres instance TAGS: [pg_check, pgsql, postgres, prepare]
postgres : Shutdown existing postgres service TAGS: [pg_clean, pgsql, postgres, prepare]
postgres : Remove registerd consul service TAGS: [pg_clean, pgsql, postgres, prepare]
postgres : Remove postgres metadata in consul TAGS: [pg_clean, pgsql, postgres, prepare]
postgres : Remove existing postgres data TAGS: [pg_clean, pgsql, postgres, prepare]
postgres : Make sure main and backup dir exists TAGS: [pg_dir, pgsql, postgres, prepare]
postgres : Create postgres directory structure TAGS: [pg_dir, pgsql, postgres, prepare]
postgres : Create pgbouncer directory structure TAGS: [pg_dir, pgsql, postgres, prepare]
postgres : Create links from pgbkup to pgroot TAGS: [pg_dir, pgsql, postgres, prepare]
postgres : Create links from current cluster TAGS: [pg_dir, pgsql, postgres, prepare]
postgres : Copy pg_cluster to /pg/meta/cluster TAGS: [pg_meta, pgsql, postgres, prepare]
postgres : Copy pg_version to /pg/meta/version TAGS: [pg_meta, pgsql, postgres, prepare]
postgres : Copy pg_instance to /pg/meta/instance TAGS: [pg_meta, pgsql, postgres, prepare]
postgres : Copy pg_seq to /pg/meta/sequence TAGS: [pg_meta, pgsql, postgres, prepare]
postgres : Copy pg_role to /pg/meta/role TAGS: [pg_meta, pgsql, postgres, prepare]
postgres : Copy postgres scripts to /pg/bin/ TAGS: [pg_scripts, pgsql, postgres, prepare]
postgres : Copy alias profile to /etc/profile.d TAGS: [pg_scripts, pgsql, postgres, prepare]
postgres : Copy psqlrc to postgres home TAGS: [pg_scripts, pgsql, postgres, prepare]
postgres : Setup hostname to pg instance name TAGS: [pg_hostname, pgsql, postgres, prepare]
postgres : Copy consul node-meta definition TAGS: [pg_nodemeta, pgsql, postgres, prepare]
postgres : Restart consul to load new node-meta TAGS: [pg_nodemeta, pgsql, postgres, prepare]
postgres : Config patroni watchdog support TAGS: [pg_watchdog, pgsql, postgres, prepare]
postgres : Get config parameter page count TAGS: [pg_config, pgsql, postgres]
postgres : Get config parameter page size TAGS: [pg_config, pgsql, postgres]
postgres : Tune shared buffer and work mem TAGS: [pg_config, pgsql, postgres]
postgres : Hanlde small size mem occasion TAGS: [pg_config, pgsql, postgres]
postgres : Calculate postgres mem params TAGS: [pg_config, pgsql, postgres]
postgres : create patroni config dir TAGS: [pg_config, pgsql, postgres]
postgres : use predefined patroni template TAGS: [pg_config, pgsql, postgres]
postgres : Render default /pg/conf/patroni.yml TAGS: [pg_config, pgsql, postgres]
postgres : Link /pg/conf/patroni to /pg/bin/ TAGS: [pg_config, pgsql, postgres]
postgres : Link /pg/bin/patroni.yml to /etc/patroni/ TAGS: [pg_config, pgsql, postgres]
postgres : Config patroni watchdog support TAGS: [pg_config, pgsql, postgres]
postgres : Copy patroni systemd service file TAGS: [pg_config, pgsql, postgres]
postgres : create patroni systemd drop-in dir TAGS: [pg_config, pgsql, postgres]
postgres : Copy postgres systemd service file TAGS: [pg_config, pgsql, postgres]
postgres : Drop-In consul dependency for patroni TAGS: [pg_config, pgsql, postgres]
postgres : Render default initdb scripts TAGS: [pg_config, pgsql, postgres]
postgres : Launch patroni on primary instance TAGS: [pg_primary, pgsql, postgres]
postgres : Wait for patroni primary online TAGS: [pg_primary, pgsql, postgres]
postgres : Wait for postgres primary online TAGS: [pg_primary, pgsql, postgres]
postgres : Check primary postgres service ready TAGS: [pg_primary, pgsql, postgres]
postgres : Check replication connectivity to primary TAGS: [pg_primary, pgsql, postgres]
postgres : Render init roles sql TAGS: [pg_init, pg_init_role, pgsql, postgres]
postgres : Render init template sql TAGS: [pg_init, pg_init_tmpl, pgsql, postgres]
postgres : Render default pg-init scripts TAGS: [pg_init, pg_init_main, pgsql, postgres]
postgres : Execute initialization scripts TAGS: [pg_init, pg_init_exec, pgsql, postgres]
postgres : Check primary instance ready TAGS: [pg_init, pg_init_exec, pgsql, postgres]
postgres : Add dbsu password to pgpass if exists TAGS: [pg_pass, pgsql, postgres]
postgres : Add system user to pgpass TAGS: [pg_pass, pgsql, postgres]
postgres : Check replication connectivity to primary TAGS: [pg_replica, pgsql, postgres]
postgres : Launch patroni on replica instances TAGS: [pg_replica, pgsql, postgres]
postgres : Wait for patroni replica online TAGS: [pg_replica, pgsql, postgres]
postgres : Wait for postgres replica online TAGS: [pg_replica, pgsql, postgres]
postgres : Check replica postgres service ready TAGS: [pg_replica, pgsql, postgres]
postgres : Render hba rules TAGS: [pg_hba, pgsql, postgres]
postgres : Reload hba rules TAGS: [pg_hba, pgsql, postgres]
postgres : Pause patroni TAGS: [pg_patroni, pgsql, postgres]
postgres : Stop patroni on replica instance TAGS: [pg_patroni, pgsql, postgres]
postgres : Stop patroni on primary instance TAGS: [pg_patroni, pgsql, postgres]
postgres : Launch raw postgres on primary TAGS: [pg_patroni, pgsql, postgres]
postgres : Launch raw postgres on primary TAGS: [pg_patroni, pgsql, postgres]
postgres : Wait for postgres online TAGS: [pg_patroni, pgsql, postgres]
postgres : Check pgbouncer is installed TAGS: [pgbouncer, pgbouncer_check, pgsql, postgres]
postgres : Stop existing pgbouncer service TAGS: [pgbouncer, pgbouncer_clean, pgsql, postgres]
postgres : Remove existing pgbouncer dirs TAGS: [pgbouncer, pgbouncer_clean, pgsql, postgres]
postgres : Recreate dirs with owner postgres TAGS: [pgbouncer, pgbouncer_clean, pgsql, postgres]
postgres : Copy /etc/pgbouncer/pgbouncer.ini TAGS: [pgbouncer, pgbouncer_config, pgbouncer_ini, pgsql, postgres]
postgres : Copy /etc/pgbouncer/pgb_hba.conf TAGS: [pgbouncer, pgbouncer_config, pgbouncer_hba, pgsql, postgres]
postgres : Touch userlist and database list TAGS: [pgbouncer, pgbouncer_config, pgsql, postgres]
postgres : Add default users to pgbouncer TAGS: [pgbouncer, pgbouncer_config, pgsql, postgres]
postgres : Copy pgbouncer systemd service TAGS: [pgbouncer, pgbouncer_launch, pgsql, postgres]
postgres : Launch pgbouncer pool service TAGS: [pgbouncer, pgbouncer_launch, pgsql, postgres]
postgres : Wait for pgbouncer service online TAGS: [pgbouncer, pgbouncer_launch, pgsql, postgres]
postgres : Check pgbouncer service is ready TAGS: [pgbouncer, pgbouncer_launch, pgsql, postgres]
include_tasks TAGS: [pg_user, pgsql, postgres]
include_tasks TAGS: [pg_db, pgsql, postgres]
postgres : Reload pgbouncer to add db and users TAGS: [pgbouncer_reload, pgsql, postgres]
postgres : Copy pg service definition to consul TAGS: [pg_register, pgsql, postgres, register]
postgres : Reload postgres consul service TAGS: [pg_register, pgsql, postgres, register]
postgres : Render grafana datasource definition TAGS: [pg_grafana, pgsql, postgres, register]
postgres : Register datasource to grafana TAGS: [pg_grafana, pgsql, postgres, register]
monitor : Install exporter yum repo TAGS: [exporter_install, exporter_yum_install, monitor, pgsql]
monitor : Install node_exporter and pg_exporter TAGS: [exporter_install, exporter_yum_install, monitor, pgsql]
monitor : Copy node_exporter binary TAGS: [exporter_binary_install, exporter_install, monitor, pgsql]
monitor : Copy pg_exporter binary TAGS: [exporter_binary_install, exporter_install, monitor, pgsql]
monitor : Create /etc/pg_exporter conf dir TAGS: [monitor, pg_exporter, pgsql]
monitor : Copy default pg_exporter.yaml TAGS: [monitor, pg_exporter, pgsql]
monitor : Config /etc/default/pg_exporter TAGS: [monitor, pg_exporter, pgsql]
monitor : Config pg_exporter service unit TAGS: [monitor, pg_exporter, pgsql]
monitor : Launch pg_exporter systemd service TAGS: [monitor, pg_exporter, pgsql]
monitor : Wait for pg_exporter service online TAGS: [monitor, pg_exporter, pgsql]
monitor : Register pg-exporter consul service TAGS: [monitor, pg_exporter_register, pgsql]
monitor : Reload pg-exporter consul service TAGS: [monitor, pg_exporter_register, pgsql]
monitor : Config pgbouncer_exporter opts TAGS: [monitor, pgbouncer_exporter, pgsql]
monitor : Config pgbouncer_exporter service TAGS: [monitor, pgbouncer_exporter, pgsql]
monitor : Launch pgbouncer_exporter service TAGS: [monitor, pgbouncer_exporter, pgsql]
monitor : Wait for pgbouncer_exporter online TAGS: [monitor, pgbouncer_exporter, pgsql]
monitor : Register pgb-exporter consul service TAGS: [monitor, node_exporter_register, pgsql]
monitor : Reload pgb-exporter consul service TAGS: [monitor, node_exporter_register, pgsql]
monitor : Copy node_exporter systemd service TAGS: [monitor, node_exporter, pgsql]
monitor : Config default node_exporter options TAGS: [monitor, node_exporter, pgsql]
monitor : Launch node_exporter service unit TAGS: [monitor, node_exporter, pgsql]
monitor : Wait for node_exporter online TAGS: [monitor, node_exporter, pgsql]
monitor : Register node-exporter service to consul TAGS: [monitor, node_exporter_register, pgsql]
monitor : Reload node-exporter consul service TAGS: [monitor, node_exporter_register, pgsql]
service : Make sure haproxy is installed TAGS: [haproxy_install, service]
service : Create haproxy directory TAGS: [haproxy_install, service]
service : Copy haproxy systemd service file TAGS: [haproxy_install, haproxy_unit, service]
service : Fetch postgres cluster memberships TAGS: [haproxy_config, service]
service : Templating /etc/haproxy/haproxy.cfg TAGS: [haproxy_config, service]
service : Launch haproxy load balancer service TAGS: [haproxy_launch, haproxy_restart, service]
service : Wait for haproxy load balancer online TAGS: [haproxy_launch, service]
service : Reload haproxy load balancer service TAGS: [haproxy_reload, service]
service : Copy haproxy exporter definition TAGS: [haproxy_register, service]
service : Copy haproxy service definition TAGS: [haproxy_register, service]
service : Reload haproxy consul service TAGS: [haproxy_register, service]
service : Make sure vip-manager is installed TAGS: [service, vip_l2_install]
service : Copy vip-manager systemd service file TAGS: [service, vip_l2_install]
service : create vip-manager systemd drop-in dir TAGS: [service, vip_l2_install]
service : create vip-manager systemd drop-in file TAGS: [service, vip_l2_install]
service : Templating /etc/default/vip-manager.yml TAGS: [service, vip_l2_config, vip_manager_config]
service : Launch vip-manager TAGS: [service, vip_l2_reload]
service : Fetch postgres cluster memberships TAGS: [service, vip_l4_config]
service : Render L4 VIP configs TAGS: [service, vip_l4_config]
include_tasks TAGS: [service, vip_l4_reload]