實戰：ansible自動化部署nginx+keepalived+mysql負載均衡集群

一、目的

使用ansible自動化部署nginx+keepalived+mysql負載均衡集群。

二、拓撲規劃

三、詳細步驟

• 1、環境的搭建

        (1)安裝ansible，同時配置私鑰免密碼進行通信

[root@localhost ~]# ssh-keygen  -t rsa #-t表示使用的加密類型，其中rsa1表示version1版本，rsa、dsa、ecdsa的加密對于的是version2版本
Generating public/private rsa key pair.
#這里詢問你要把生成的密鑰文件保存在哪里，默認是在家目錄下的.ssh文件夾中，回車保存默認目錄
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
#這里是對密鑰文件加密，不輸入則表示不加密
Enter passphrase (empty for no passphrase): 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
04:9f:cb:9c:9d:1e:47:d7:e1:d4:c1:87:71:c3:a4:22 root@localhost.localdomain
The key's randomart image is:
+--[ RSA 2048]----+
|      .       =O+|
|       o .    ===|
|        +E .....o|
|       + +.o..   |
|        S + .    |
|         . o     |
|          .      |
|                 |
|                 |
+-----------------+

–

        (2)查看已經成功生成了一對密鑰

[root@localhost ~]# ls /root/.ssh
id_rsa  id_rsa.pub#其中id_rsa為私鑰，id_rsa.pub為公鑰

–

        (3)在生成完密鑰對之后將公鑰上傳給服務器對應用戶的家目錄

[root@localhost ~]# ssh-copy-id -i .ssh/id_rsa.pub root@10.1.252.215
[root@localhost ~]# ssh-copy-id -i .ssh/id_rsa.pub root@10.1.252.235
[root@localhost ~]# ssh-copy-id -i .ssh/id_rsa.pub 10.1.253.107
[root@localhost ~]# ssh-copy-id -i .ssh/id_rsa.pub root@10.1.249.75
[root@localhost ~]# ssh-copy-id -i .ssh/id_rsa.pub root@10.1.249.75

·

        (4)編輯ansible的hosts文件，定義后所有的主機

[19:05 root@centos6.8/etc/ansible]# cat hosts 
# This is the default ansible 'hosts' file.
#
# It should live in /etc/ansible/hosts
#
#   - Comments begin with the '#' character
#   - Blank lines are ignored
#   - Groups of hosts are delimited by [header] elements
#   - You can enter hostnames or ip addresses
#   - A hostname/ip can be a member of multiple groups

# Ex 1: Ungrouped hosts, specify before any group headers.
[web]
10.1.252.215
10.1.252.235
[nginx]
10.1.253.107 state=MASTER priority=100
10.1.249.75  state=BACKUP priority=90
[mysql]
10.1.252.36

·

        OK，環境已經搭配好，所有主機同步下時間：

[19:34 root@centos6.8/etc/ansible]# ansible all -a 'ntpdate 10.1.0.1'
10.1.252.215 | success | rc=0 >>
 3 Nov 19:34:30 ntpdate[38293]: adjust time server 10.1.0.1 offset -0.003936 sec

10.1.252.36 | success | rc=0 >>
 3 Nov 19:34:30 ntpdate[3291]: adjust time server 10.1.0.1 offset 0.200434 sec

10.1.252.235 | success | rc=0 >>
 3 Nov 19:34:36 ntpdate[38723]: adjust time server 10.1.0.1 offset -0.001469 sec

10.1.253.107 | success | rc=0 >>
 3 Nov 19:34:37 ntpdate[7161]: adjust time server 10.1.0.1 offset -0.001905 sec

10.1.249.75 | success | rc=0 >>
 3 Nov 19:34:37 ntpdate[4951]: adjust time server 10.1.0.1 offset 0.018952 sec

• 2、下面來進行ansible的roles和playbook的定義：
  在/etc/ansible/roles目錄下創建相關的角色目錄：

[19:56 root@centos6.8/etc/ansible/roles]# mkdir -pv {mysql,web,nginx}/{files,tasks,templates,variables,handlers,meta,defult}
mkdir: created directory `mysql'
mkdir: created directory `mysql/files'
mkdir: created directory `mysql/tasks'
mkdir: created directory `mysql/templates'
mkdir: created directory `mysql/variables'
mkdir: created directory `mysql/handlers'
mkdir: created directory `mysql/meta'
mkdir: created directory `mysql/default'
mkdir: created directory `web'
mkdir: created directory `web/files'
mkdir: created directory `web/tasks'
mkdir: created directory `web/templates'
mkdir: created directory `web/variables'
mkdir: created directory `web/handlers'
mkdir: created directory `web/meta'
mkdir: created directory `web/default'
mkdir: created directory `nginx'
mkdir: created directory `nginx/files'
mkdir: created directory `nginx/tasks'
mkdir: created directory `nginx/templates'
mkdir: created directory `nginx/variables'
mkdir: created directory `nginx/handlers'
mkdir: created directory `nginx/meta'
mkdir: created directory `nginx/default'
[19:58 root@centos6.8/etc/ansible/roles]# tree
.
├── mysql
│   ├── default
│   ├── files
│   ├── handlers
│   ├── meta
│   ├── tasks
│   ├── templates
│   └── variables
├── nginx
│   ├── default
│   ├── files
│   ├── handlers
│   ├── meta
│   ├── tasks
│   ├── templates
│   └── variables
└── web
    ├── default
    ├── files
    ├── handlers
    ├── meta
    ├── tasks
    ├── templates
    └── variables

24 directories, 0 files

說明：
files/：存儲由copy或script等模塊調用的文件；
tasks/：此目錄中至少應該有一個名為main.yml的文件，用于定義各task；其它的文件需要由main.yml進行“包含”調用；
handlers/：此目錄中至少應該有一個名為main.yml的文件，用于定義各handler；其它的文件需要由main.yml進行“包含”調用；
vars/：此目錄中至少應該有一個名為main.yml的文件，用于定義各variable；其它的文件需要由main.yml進行“包含”調用；
templates/：存儲由template模塊調用的模板文本；
meta/：此目錄中至少應該有一個名為main.yml的文件，定義當前角色的特殊設定及其依賴關系；其它的文件需要由main.yml進行“包含”調用；
default/：此目錄中至少應該有一個名為main.yml的文件，用于設定默認變量；

下面就是添加playbook了，首先設置web：

• 3、設置web的playbook

  (1)Tasks：任務

[17:27 root@centos6.8/etc/ansible]# cat roles/web/tasks/main.yml 
- name: install web pakgs
  yum: name={{ item }}
  with_items:
  - httpd
  - php
  - php-mysql
- name: config the web
  copy: src=httpd.conf dest=/etc/httpd/conf/httpd.conf
  notify: reload the service
- name: install wordpress
  copy: src=wordpress dest=/var/www/html/wordpress/
- name: restart the service
  service: name=httpd state=started

        (2)添加觸發器:handlers

[20:26 root@centos6.8/etc/ansible]# cat roles/web/handlers/main.yml 
- name: relaod the service
  service: name=httpd state=restarted

        (3)添加需要的file：

[18:45 root@centos6.8/etc/ansible]# ll roles/web/files/
total 40
-rw-r--r--. 1 root   root      34419 Nov  2 20:23 httpd.conf  #主要是配置httpd的默認配置，要事先準備好
drwxr-xr-x. 5 nobody nfsnobody  4096 Nov  3 14:00 wordpress   #wordpres的安裝程序，注意這里的配置文件已經更改了后面的連接數據庫
[18:58 root@centos6.8/etc/ansible]#vim  roles/web/files/wordpress/wp-config.php  #修改數據庫信息，信息已經在后面的mysql劇本已經定義好了
/** WordPress數據庫的名稱 */
define('DB_NAME', 'wp');

/** MySQL數據庫用戶名 */
define('DB_USER', 'wpuser'); 

/** MySQL數據庫密碼 */
define('DB_PASSWORD', 'wppass');

/** MySQL主機 */
define('DB_HOST', '10.1.252.109');

/** 創建數據表時默認的文字編碼 */
define('DB_CHARSET', 'utf8');

        (4)添加主劇本：

[20:28 root@centos6.8/etc/ansible]# ll web.yml 
-rw-r--r--. 1 root root 51 Nov  2 20:22 web.yml
[20:28 root@centos6.8/etc/ansible]# pwd
/etc/ansible
[20:28 root@centos6.8/etc/ansible]# cat web.yml 
- hosts: web
  remote_user: root
  roles:
  - web     

        檢查語法沒有問題：

[20:25 root@centos6.8/etc/ansible]# ansible-playbook --syntax-check web.yml 

playbook: web.yml

• 4、下面來部署前端的nginx調度起和keepalived配置：

  (1)添加task任務：

[19:34 root@centos6.8/etc/ansible]# cat roles/nginx/tasks/main.yml 
- name: install keepalived
  yum: name=keepalived
- name: copy nginx
  copy: src=nginx-1.10.0-1.el7.ngx.x86_64.rpm dest=/tmp/nginx.rpm
- name: install nginx
  yum: name=/tmp/nginx.rpm
- name: delete the nginx pkg
  shell: "rm -f /tmp/nginx.rpm"
- name: config nginx&keepalived
  template: src=keepalived.j2 dest=/etc/keepalived/keepalived.conf
  notify: reload the service
- name: config nginx&keepalived
  template: src=nginx.j2 dest=/etc/nginx/nginx.conf
  notify: reload the service
- name: start the service
  service: name={{ item }} state=started 
  with_items:
  - keepalived
  - nginx

        (2)添加handlers：

[21:28 root@centos6.8/etc/ansible]#cat roles/nginx/handlers/mainx.yml 
- name: reload the service
  service: name={{ item }} state=restarted
  with_items:
  - nginx
  - keepalived

        (3)在hosts列表中復制變量：

[19:38 root@centos6.8/etc/ansible]# cat hosts 
# This is the default ansible 'hosts' file.
#
# It should live in /etc/ansible/hosts
#
#   - Comments begin with the '#' character
#   - Blank lines are ignored
#   - Groups of hosts are delimited by [header] elements
#   - You can enter hostnames or ip addresses
#   - A hostname/ip can be a member of multiple groups

# Ex 1: Ungrouped hosts, specify before any group headers.

[web]
10.1.252.215
10.1.252.235
[nginx]
10.1.253.107 state=MASTER priority=100
10.1.249.75  state=BACKUP priority=90
[mysql]
10.1.252.36

        (4)提供files文件：nginx的安裝程序

[19:39 root@centos6.8/etc/ansible]# ll roles/nginx/files/
total 644
-rw-r--r--. 1 root root 655648 Apr 29  2016 nginx-1.10.0-1.el7.ngx.x86_64.rpm

        (5)在template中使用了變量：
nginx配置：

[21:33 root@centos6.8/etc/ansible]# cat roles/nginx/templates/nginx.j2 

user  nginx;
worker_processes  {{ ansible_processor_vcpus  }}; #使用變量，進程數為cpu數量：

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;

    upstream web {
    server 10.1.24.113;
    server 10.1.24.114;
 }
    location / {
    proxy_pass http://web;  
}

}

keepalived配置：

[21:33 root@centos6.8/etc/ansible]# cat roles/nginx/templates/keepalived.j2 
! Configuration File for keepalived

global_defs {
   notification_email {
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state {{ state }}  #使用變量，變量在hosts中已經定義了
    interface eno16777746
    virtual_router_id 55
    priority {{ priority }} #使用變量，變量在hosts中已經定義了
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 232332
    }
    virtual_ipaddress {
        10.1.24.222
    }
}

        (6)添加主劇本：

[21:30 root@centos6.8/etc/ansible]# cat nginx.yml 
- hosts: nginx
  remote_user: root
  roles:
  - nginx

        (7)語法檢測沒有問題

[21:34 root@centos6.8/etc/ansible]#ansible-playbook --syntax-check nginxx.yml 

playbook: nginx.yml

• 5、設置mysql：
  (1)設置mysql的task：

[18:46 root@centos6.8/etc/ansible]# cat roles/mysql/tasks/main.yml 
- name: install mysql
  yum: name=mysql-server
- name: copy sql file
  copy: src=mysql.sql dest=/tmp/mysql.sql
- name: start mysql service
  service: name=mysqld state=started
- name: config mysql
  shell: "mysql < /tmp/mysql.sql"

        (2)設置files文件

[18:47 root@centos6.8/etc/ansible]# ll roles/mysql/files/
total 4
-rw-r--r--. 1 root root 78 Nov  3 15:41 mysql.sql
[19:39 root@centos6.8/etc/ansible]# cat roles/mysql/files/mysql.sql #提供數據庫的sql腳本，創建wordpres的用戶和數據庫
CREATE DATABASE wp;
GRANT ALL ON wp.* TO 'wpuser'@'%' IDENTIFIED BY 'wppass';

        (3)添加主劇本：

[18:48 root@centos6.8/etc/ansible]# cat mysql.yml 
- hosts: mysql
  remote_user: root
  roles:
  - mysql

        (4)語法檢查沒有問題：

[18:49 root@centos6.8/etc/ansible]# ansible-playbook --syntax-check mysql.yml 

playbook: mysql.yml

        mysql的ansible配置已經完成

• 6、至此，web、nginx+keepalived、mysql的配置都已經完成，下面來依次執行劇本
  (1)來看下整個目錄的結構：

[19:05 root@centos6.8/etc/ansible]# tree -L 4
.
├── ansible.cfg
├── ansible.cfg.bak
├── hosts
├── mysql.yml
├── nginx.yml
├── roles
│   ├── mysql
│   │   ├── default
│   │   ├── files
│   │   │   └── mysql.sql
│   │   ├── handlers
│   │   ├── meta
│   │   ├── tasks
│   │   │   └── main.yml
│   │   ├── templates
│   │   └── variables
│   ├── nginx
│   │   ├── default
│   │   ├── files
│   │   │   └── nginx-1.10.0-1.el7.ngx.x86_64.rpm
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── meta
│   │   ├── tasks
│   │   │   └── main.yml
│   │   ├── templates
│   │   │   ├── keepalived.j2
│   │   │   └── nginx.j2
│   │   └── variables
│   └── web
│       ├── default
│       ├── files
│       │   ├── httpd.conf
│       │   └── wordpress
│       ├── handlers
│       │   └── main.yml
│       ├── meta
│       ├── tasks
│       │   └── main.yml
│       ├── templates
│       └── variables
├── test.yaml
└── web.yml

26 directories, 17 files

        (2)執行web的playbook：

[17:15 root@centos6.8/etc/ansible]# ansible-playbook  web.yml 

PLAY [web] ******************************************************************** 

GATHERING FACTS *************************************************************** 
ok: [10.1.252.235]
ok: [10.1.252.215]

TASK: [web | install web pakgs] *********************************************** 
ok: [10.1.252.215] => (item=httpd,php,php-mysql)
ok: [10.1.252.235] => (item=httpd,php,php-mysql)

TASK: [web | config the web] ************************************************** 
ok: [10.1.252.215]
ok: [10.1.252.235]

TASK: [web | install wordpress] *********************************************** 
changed: [10.1.252.235]
changed: [10.1.252.215]

TASK: [web | restart the service] ********************************************* 
ok: [10.1.252.215]
ok: [10.1.252.235]

PLAY RECAP ******************************************************************** 
10.1.252.215               : ok=5    changed=1    unreachable=0    failed=0   
10.1.252.235               : ok=5    changed=1    unreachable=0    failed=0         

        (3)執行mysql的playbook：

[18:52 root@centos6.8/etc/ansible]# ansible-playbook  mysql.yml 

PLAY [mysql] ****************************************************************** 

GATHERING FACTS *************************************************************** 
ok: [10.1.252.36]

TASK: [mysql | install mysql] ************************************************* 
ok: [10.1.252.36]

TASK: [mysql | copy sql file] ************************************************* 
ok: [10.1.252.36]

TASK: [mysql | start mysql service] ******************************************* 
ok: [10.1.252.36]

TASK: [mysql | config mysql] ************************************************** 
skipping: [10.1.252.36]
ok: [10.1.252.36]

PLAY RECAP ******************************************************************** 
10.1.252.36                : ok=4    changed=0    unreachable=0    failed=0 

        (4)執行nginx的playbook：

[18:53 root@centos6.8/etc/ansible]# ansible-playbook nginx.yml 

PLAY [nginx] ****************************************************************** 

GATHERING FACTS *************************************************************** 
ok: [10.1.249.75]
ok: [10.1.253.107]

TASK: [nginx | install keepalived] ******************************************** 
ok: [10.1.253.107]
ok: [10.1.249.75]

TASK: [nginx | copy nginx] **************************************************** 
changed: [10.1.249.75]
changed: [10.1.253.107]

TASK: [nginx | install nginx] ************************************************* 
ok: [10.1.249.75]
ok: [10.1.253.107]

TASK: [nginx | delete the nginx pkg] ****************************************** 
changed: [10.1.249.75]
changed: [10.1.253.107]

TASK: [nginx | config nginx&keepalived] *************************************** 
ok: [10.1.249.75]
ok: [10.1.253.107]

TASK: [nginx | config nginx&keepalived] *************************************** 
ok: [10.1.249.75]
ok: [10.1.253.107]

TASK: [nginx | start the service] ********************************************* 
ok: [10.1.253.107] => (item=keepalived)
ok: [10.1.249.75] => (item=keepalived)
ok: [10.1.253.107] => (item=nginx)
ok: [10.1.249.75] => (item=nginx)

PLAY RECAP ******************************************************************** 
10.1.249.75                : ok=8    changed=2    unreachable=0    failed=0   
10.1.253.107               : ok=8    changed=2    unreachable=0    failed=0   

        執行完成后，下面就是驗證效果的時候：

        激動的時刻，成功了！

中間本人遇到的問題：在每個task中的name任務中不能有過多的任務，必須要分為多個步驟進行，如此例中的copy nginx然后再yum nginx，不能寫在同一個name中，否則會報錯！此外，copy的模塊拷貝目錄時注意最后的“/” 問題。

        OK，更多文章請關注我的博客。