keepalived雙主模型的實現

一、簡介

 keepalived 簡介:是服務器高可用的一個重要軟件，它的核心組件有vrrp ，stack， checker ，ipvs， warpper， watch dog 

它是vrrp協議的實現，原生設計目的為高可用ipvs服務；keepalived能夠通過配置文件中定義生成ipvs規則

并能夠對RS的健康狀態進行檢測；vrrp_script,vrrp_track; 

雙主模型的實現

簡介:雙方模型(主/備，備/主)的這里的意思是，一個keepalived配置中，一個虛擬IP地址為主，另一個為備。而在另一個

keepalived的配置中，與其它主機則恰恰相反，一個虛擬IP地址為備，另一個為主

以下為此次雙主模型實現的拓撲

二、HA Cluster配置的前提；

(1)要點:各節點之間的時間秘需要同步: 

# ntpdate 172.16.0.1 (注此:處可自己在網上找個時間同步服務器)

(2)確保iptables及selinux不會阻礙:

# iptables -F && setenforce 0

(3)各節點之間可通過主機名互相通信(對keepalived并非必須):

# vim /etc/hosts 

172.16.250.140 kpl1

172.16.250.158 kpl2 兩臺keepalived主機都要修改 

(4)各節點之間root用戶可以基于密鑰認證的ssh通信

# ssh-copy-id -i 172.16.250.140

# ssh-copy-id -i 172.16.250.158

三、dr集群配置

此處使用dr集群類型

首先將dr模型配置好 

編輯setkp.sh 

#!/bin/bash 

vip=172.16.26.126

vip2=172.16.26.127

mask=255.255.255.255

interface=’lo:0′

interface2=’lo:1′

eth=’eno16777736:0′ 

case $1 in 

start) 

echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

ifconfig $interface $vip netmask $mask broadcast $vip up 

ifconfig $interface2 $vip2 netmask $mask broadcast $vip2 up 

route add -host $vip dev $interface 

route add -host $vip2 dev $interface2

;; 

dstart) 

ifconfig $eth $vip/32 netmask $mask broadcast $vip up

;;

dstop)

ifconfig $eth down

;;

stop)

ifconfig $interface down 

ifconfig $interface2 down

echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore

echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce

;; 

status) 

ifconfig 

cat /proc/sys/net/ipv4/conf/all/arp_ignore

cat /proc/sys/net/ipv4/conf/lo/arp_ignore

cat /proc/sys/net/ipv4/conf/all/arp_announce

cat /proc/sys/net/ipv4/conf/lo/arp_announce

;; 

*)

echo “Usage: $(basename $0) {dstart|dstop|start|stop}”

exit 1 

esac

RS1、RS2主機中都執行執行此腳本 

# sh setkp.sh start 

# sh setkp.sh status 可用來查看當前網絡的配置狀態 

RS1、RS2主機中安裝httpd并啟動 

# yum -y install httpd && systemctl start httpd  

RS1 # echo “<h1>RS1</h1>” > /usr/share/nginx/html/index.html 

RS2 # echo “<h1>RS2</h1>” > /usr/share/nginx/html/index.html

四、keepalived配置

keepalived,keepalived2 中安裝nginx(httpd和nginx都可以,此處用來做為keepalived的本身的提供sorry server)

當dr集群的兩臺節點都停止時，會由keepalived本身來提供一個頁面

# yum -y install nginx keepalived ipvsadm && systemctl start nginx 

# echo “<h1>sorry server keepalived 1 </h1>” > /usr/share/nginx/html/index.html 

# echo “<h1>sorry server keepalived 2 </h2>” > /usr/share/nginx/html/index.html 

在一臺keepalived主機上進行測試

# curl 172.16.251.232

<h1>RS1</h2>

# curl 172.16.250.159

<h1>RS2</h1>

keepalived,keepalived2 中在 /etc/keepalived/目錄下編輯一個腳本來用來在主備變化，或服務down掉時發郵件給系統用戶 

# vim kmail.sh 

#!/bin/bash

contact=’root@localhost’

notify(){

mailsubject=”$(hostname) to be $1:vip floating”

mailbody=”$(date +’%F %T’):vrrp transition, $(hostname) change to be $1″

echo $mailbody | mail -s “$mailsubject” $contact

}

case $1 in 

master )

notify master

;;

backup )

notify backup 

;;

fault )

notify fault 

;;

*)

echo “Usage:$(basename $0) {master |backup|fault}”

;;

esac

#chmod +x kmail.sh 

注:keepalived的兩臺主機的配置基本相同，當配置模型為主/備模型的時候，主備之間需要修改三個指令 

分別為 router_id kpl1 在keepalived2上時需要修改為 router_id kpl2 

  state MASTER 在keepalived2上時需要修改為 state BACKUP 

  priority 100 在keepalived2上時需要修改為  priority 90 (此處的值比主服務器的小便可以)

以下主/備的配置示例

先把備的服務器啟動

# tcpdump -i eno33554984 -nn host 224.0.61.61

可使用tcpdump 抓取組播地址 來查看其通過組播方式傳遞的心跳信息

以下的配置文件便是keepavlied雙主模型的實現 

對防火墻打標記 

keepalived 1

# iptables -t mangle -A PREROUTING -d 172.16.26.126 -p tcp –dport 80 -j MARK –set-mark 3 

# iptables -t mangle -A PREROUTING -d 172.16.26.127 -p tcp –dport 80 -j MARK –set-mark 3

# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

  notification_email {

root@localhost

}

  notification_email_from Alexandre.Cassen@firewall.loc

  smtp_server 127.0.0.1

  smtp_connect_timeout 30

  router_id kpl1

  vrrp_mcast_group4 224.0.61.61

}

vrrp_instance VI_1 {

state MASTER

interface eno33554984

virtual_router_id 55

priority 100

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kNsRQ

}

virtual_ipaddress {

172.16.26.126 dev eno33554984 label eno33554984:0    

}

}

vrrp_instance VI_2 {

state BACKUP

interface eno33554984

virtual_router_id 66

priority 90

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kfsRQ

}

virtual_ipaddress {

172.16.26.127 dev eno33554984 label eno33554984:1    

}

}

virtual_server fwmark 3 {

delay_loop 2

lb_algo wrr 

lb_kind DR

nat_mask 255.255.0.0

protocol TCP

sorry_server 127.0.0.1 80

real_server 172.16.251.232 80 {

weight 3

HTTP_GET {

url { 

 path /

 status_code 200  

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

real_server 172.16.250.159 80 {

weight 1

HTTP_GET {

url { 

 path /

 status_code 200  

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

keepalived 2 

# iptables -t mangle -A PREROUTING -d 172.16.26.126 -p tcp –dport 80 -j MARK –set-mark 3

# iptables -t mangle -A PREROUTING -d 172.16.26.127 -p tcp –dport 80 -j MARK –set-mark 3

# vim /etc/keepavlied/keepalived.conf 

! Configuration File for keepalived

global_defs {

  notification_email {

root@localhost

}

  notification_email_from Alexandre.Cassen@firewall.loc

  smtp_server 127.0.0.1

  smtp_connect_timeout 30

  router_id kpl2

  vrrp_mcast_group4 224.0.61.61

}

vrrp_instance VI_1 {

state BACKUP

interface eno33554984

virtual_router_id 55

priority 90

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kNsRQ

}

virtual_ipaddress {

172.16.26.126 dev eno33554984 label eno33554984:0    

}

}

vrrp_instance VI_2 {

state MASTER

interface eno33554984

virtual_router_id 66

priority 100

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kfsRQ

}

virtual_ipaddress {

172.16.26.127 dev eno33554984 label eno33554984:1    

}

}

virtual_server fwmark 3  {

delay_loop 2

lb_algo wrr

lb_kind DR

nat_mask 255.255.0.0

protocol TCP

sorry_server 127.0.0.1 80

real_server 172.16.251.232 80 {

weight 3

HTTP_GET {

url {

 path /

 status_code 200

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

real_server 172.16.250.159 80 {

weight 1

HTTP_GET {

url {

 path /

 status_code 200

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

五、測試

此時可以測試其訪問

當一個keepavlied停止時 

當RS2停止時 

當RS1，RS2都停止時

   

六、keepavlied配置指令說明 

虛擬路由器段

state MASTER：當前節點在虛擬路由器中的初始狀態；

interface ETHERCARD: vrrp實際工作的網卡接口

virtual_route_id 51 :虛擬路由器ID，范圍0-255；

priority 100 :當前物理節點在此虛擬路由器中的優先級；

advert_int 1:每隔多久發送心跳(通行的時間間隔)

auth_type PASS ：選擇認證機制 

auth_pass 1111 ：密碼 八位有效

virtual_ipaddress ：定義虛擬IP 

track_interface ： 定義要監控的接口

notify_master <STRING> | <QUOTED-STRING> ：當前節點變為主節點時用STRING腳本通告

notfy_backup<STRING> | <QUOTED-STRING> : 當前節點變為主節時用 STRING腳本通告

notify_fault<STRING> | <QUOTED-STRING> : 當前節點上不了線時用STRING腳本通告

notify<STRING> | <QUOTED-STRING> : 如果三種狀態用一個腳本來實現用STRING腳本通告 

虛擬服務段

lb_algo rr | wrr|lc|lblc|sh|dh :定義負載均衡調度算法

delay_loop<INT>：：定義服務輪詢時間間隔

bl_kind NAT |DR |TUN ：集群的類型

persistence_time_out<INT> ：持久連接時長

protocol TCP : 服務協議

sorry_server<IPADDR><PORT>：所有RS均故障時，提供sorry server的服務器；

real_server<IPADDR><PORT>:

weight<INT>：權重

notify_up<STRING>|<QUOTED-STRING> ： 節點上線通知腳本

notify_down <STRING>|<QUOTED-STRNG>：節點離線通知腳本；

#HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK ：支持的所以健康狀態的檢測方式

url:健康狀態檢測時請求的資源的URL 

delay_before_retry<INT> ：兩次嘗試之間的時間間隔 

connect_timeoute<STRING>：連接的超時時長

connect_ip<IP ADDRESS>：向此處指定的地址發測試請求

connect_port<PORT>：向此處指定的PORT發測試請求

bindto<IP ADDRESS>：指定測試請求報文的源IP 

bind_port<PORT>： 指定測試請求報文的源PORT