HA Cluster
集群類型:LB、HA、HP
系統可用性的公式:A=MTBF/(MTBF+MTTR)
(0,1)
幾個9: 99%, …, 99.999%
建議使用3個9的系統可用性
如何降低MTTR:冗余(redundant)
active/passive
active –> HEARTBEAT –> passive
(1) passive node的數量?
備用節點可以有多個,HEARTAEAT信息建議使用多播域
3個周期沒收到HEARTAEAT信息,則進行資源切換
(2) 資源切換?
ip/服務的切換
·shared storage:
NAS:文件共享服務器
SAN:存儲區域網絡,塊級別的共享
·Network partition:<網絡分區>
隔離設備:
node:STONITH
資源:fence
·quorum:
with quorum:> total/2 可以占有資源
without quorum: <= total/2 必須釋放資源
·HA Service:
nginx service:ip/nginx
·TWO nodes Cluster?
輔助設備:ping node, quorum disk
HA Cluster實現方案:
vrrp協議的實現:簡單來說就是心跳信息,基于節點優先級實現
keepalived
ais:完備HA集群;上述的quorum投票機制,反而是ais的機制
heartbeat
corosync
keepalived:
·vrrp協議:Virtual Redundant Routing Protocol
術語:
虛擬路由器:Virtual Router
虛擬路由器標識:VRID(0-255)
物理路由器:
master:主設備
backup:備用設備
priority:優先級(0-224)
VIP:Virtual IP
VMAC:Virutal MAC (00-00-5e-00-01-VRID)
通告:心跳,優先級等;周期性
搶占式,非搶占式:
搶占式:當主設備恢復后,把資源搶占過來
非搶占式:當主設備恢復后,不搶占資料,而是等待下一次選舉
安全工作:
認證:
無認證
簡單字符認證:推薦簡單字符認證
MD5
工作模式:
主/備:單虛擬路徑器
主/主:主/備(虛擬路徑器1),備/主(虛擬路徑器2)
·keepalived
vrrp協議的軟件實現,原生設計的目的為了高可用ipvs服務:
vrrp協議完成地址流動
為vip地址所在的節點生成ipvs規則(在配置文件中預先定義)
為ipvs集群的各RS做健康狀態檢測
基于腳本調用接口通過執行腳本完成腳本中定義的功能,進而影響集群事務
web服務器的高可用,基本都是基于keepalive實現
組件:
核心組件:
vrrp stack
ipvs wrapper
checkers
控制組件:配置文件分析器
IO復用器
內存管理組件
·HA Cluster的配置前提:
(1)各節點時間必須同步
ntp, chrony
(2)確保iptables及selinux不會成為阻礙
(3)各節點之間可通過主機名互相通信(對KA并非必須)
建議使用/etc/hosts文件實現
(4)各節點之間的root用戶可以基于密鑰認證的ssh服務完成互相通信(并非必須)
keepalived安裝配置:
CentOS 6.4+版本以后,keepalived被放置在base倉庫中了
程序環境:
配置文件:/etc/keepalived/keepalived.conf
主程序:/usr/sbin/keepalived
Unit File:keepalived.service
配置文件組件部分:
TOP HIERACHY
GLOBAL CONFIGURATION
Global definitions
Static routes/addresses
VRRPD CONFIGURATION
VRRP synchronization group(s)
VRRP instance(s)
LVS CONFIGURATION
Virtual server group(s)
Virtual server(s)
配置語法:
·Global definitions:
vrrp_mcast_group4 224.0.100.19:定義keepalived廣播的地址
·配置虛擬路由器:
vrrp_instance <STRING> {
…
}
專用參數:
state MASTER|BACKUP:當前節點在此虛擬路由器上的初始狀態;只能有一個是MASTER,余下的都應該為BACKUP
interface IFACE_NAME:綁定為當前虛擬路由器使用的物理接口
virtual_router_id VRID:當前虛擬路由器的惟一標識,范圍是0-255
priority 100:當前主機在此虛擬路徑器中的優先級;范圍1-254
advert_int 1:vrrp通告的時間間隔
authentication {
auth_type AH|PASS
auth_pass <PASSWORD> #PASS認證的字符串不能超過8個,超過8個只取前8個
}
virtual_ipaddress {
<IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
192.168.200.17/24 dev eth1
192.168.200.18/24 dev eth2 label eth2:1
}
track_interface {
eth0
eth1
…
}
配置要監控的網絡接口,一旦接口出現故障,則轉為FAULT狀態
nopreempt:定義工作模式為非搶占模式
若某vrrp_instance定義為nopreempt,而所有關于此vrrp_instance的主機的state定義都只能是BACKUP模式
默認是preempt搶占模式
preempt_delay 300:搶占式模式下,節點上線后觸發新選舉操作的延遲時長
定義通知腳本:
notify_master <STRING>|<QUOTED-STRING>:當前節點成為主節點時觸發的腳本
notify_backup <STRING>|<QUOTED-STRING>:當前節點轉為備節點時觸發的腳本
notify_fault <STRING>|<QUOTED-STRING>:當前節點轉為“失敗”狀態時觸發的腳本
notify <STRING>|<QUOTED-STRING>:通用格式的通知觸發機制,一個腳本可完成以上三種狀態的轉換時的通知,不可和上面3個同時使用
單主機模式:
主機1:
[root@node1 keepalived]# yum -y install httpd keepalived [root@node1 keepalived]# vim /var/www/html/index.html keepalived1 [root@node1 keepalived]# service httpd start [root@node1 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 18 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } [root@node1 keepalived]# service keepalived start [root@node1 keepalived]# cat notify.sh #!/bin/bash # contact='root@localhost' notify() { mailsubject="$(hostname) to be $1, vip floating" mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1" echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master) notify master ;; backup) notify backup ;; fault) notify fault ;; *) echo "Usage: $(basename $0) {master|backup|fault}" exit 1 ;; esac
主機2:
[root@node2 keepalived]# yum -y install httpd keepalived [root@node2 keepalived]# vim /var/www/html/index.html keepalived2 [root@node2 keepalived]# service httpd start [root@node2 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 18 priority 98 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } [root@node2 keepalived]# service keepalived start
請求結果:
[root@node3 ~]# curl 10.1.43.100 keepalived1
雙主機模式:
主機1:
[root@node1 keepalived]# yum -y install httpd keepalived [root@node1 keepalived]# vim /var/www/html/index.html keepalived1 [root@node1 keepalived]# service httpd start [root@node1 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 18 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 19 priority 98 advert_int 2 authentication { auth_type PASS auth_pass 3a732491 } virtual_ipaddress { 10.1.43.200/16 dev eth0 } } [root@node1 keepalived]# service keepalived start
主機2:
[root@node2 keepalived]# yum -y install httpd keepalived [root@node2 keepalived]# vim /var/www/html/index.html keepalived2 [root@node2 keepalived]# service httpd start [root@node2 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 18 priority 98 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 19 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 3a732491 } virtual_ipaddress { 10.1.43.200/16 dev eth0 } } [root@node2 keepalived]# service keepalived start
請求結果:
[root@node3 ~]# curl 10.1.43.100 keepalived1 [root@node3 ~]# curl 10.1.43.200 keepalived2
·虛擬服務器:
配置參數:
virtual_server IP port |
virtual_server fwmark int
{
…
real_server {
…
}
…
}
常用參數:
delay_loop <INT>:服務輪詢的時間間隔;對后端服務器進行健康檢測的時長
lb_algo rr|wrr|lc|wlc|lblc|sh|dh:定義調度方法
lb_kind NAT|DR|TUN:集群的類型
persistence_timeout <INT>:持久連接時長
protocol TCP:服務協議,僅支持TCP
sorry_server <IPADDR> <PORT>:備用服務器地址
real_server <IPADDR> <PORT>
{
weight <INT>
notify_up <STRING>|<QUOTED-STRING>
notify_down <STRING>|<QUOTED-STRING>
HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK { … }:定義當前主機的健康狀態檢測方法
}
HTTP_GET|SSL_GET {
url {
path <URL_PATH>:定義要監控的URL
status_code <INT>:判斷上述檢測機制為健康狀態的響應碼
digest <STRING>:判斷上述檢測機制為健康狀態的響應的內容的校驗碼
}
nb_get_retry <INT>:重試次數
delay_before_retry <INT>:重試之前的延遲時長
connect_ip <IP ADDRESS>:向當前RS的哪個IP地址發起健康狀態檢測請求
connect_port <PORT>:向當前RS的哪個PORT發起健康狀態檢測請求
bindto <IP ADDRESS>:發出健康狀態檢測請求時使用的源地址
bind_port <PORT>:發出健康狀態檢測請求時使用的源端口
connect_timeout <INTEGER>:連接請求的超時時長
}
TCP_CHECK {
connect_ip <IP ADDRESS>:向當前RS的哪個IP地址發起健康狀態檢測請求
connect_port <PORT>:向當前RS的哪個PORT發起健康狀態檢測請求
bindto <IP ADDRESS>:發出健康狀態檢測請求時使用的源地址
bind_port <PORT>:發出健康狀態檢測請求時使用的源端口
connect_timeout <INTEGER>:連接請求的超時時長
}
keepalived+lvs來調度httpd服務:
拓撲結構:
實驗環境:
node1主機: 10.1.43.1 提供keepalived服務
node3主機: 10.1.43.2 提供keepalived服務
node4主機: 10.1.43.101 提供httpd服務
node5主機: 10.1.43.102 提供httpd服務
主機1:
[root@node1 keepalived]# yum -y install httpd keepalived [root@node1 keepalived]# vim /var/www/html/index.html keepalived1 [root@node1 keepalived]# service httpd start [root@node1 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 18 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 19 priority 98 advert_int 2 authentication { auth_type PASS auth_pass 3a732491 } virtual_ipaddress { 10.1.43.200/16 dev eth0 } } virtual_server 10.1.43.100 80 { delay_loop 3 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 10.1.43.101 8080 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 10.1.43.102 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 10.1.43.200 80 { delay_loop 3 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 10.1.43.101 8080 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 10.1.43.102 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } [root@node1 keepalived]# service keepalived start
主機2:
[root@node2 keepalived]# yum -y install httpd keepalived [root@node2 keepalived]# vim /var/www/html/index.html keepalived2 [root@node2 keepalived]# service httpd start [root@node2 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 18 priority 98 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 19 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 3a732491 } virtual_ipaddress { 10.1.43.200/16 dev eth0 } } virtual_server 10.1.43.100 80 { delay_loop 3 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 10.1.43.101 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 10.1.43.102 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 10.1.43.200 80 { delay_loop 3 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 10.1.43.101 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 10.1.43.102 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } [root@node2 keepalived]# service keepalived start
主機4、5上運行此腳本即可,并配置上httpd服務
[root@node4 ~]# cat set.sh #!/bin/bash # vip1=10.1.43.100 vip2=10.1.43.200 ifcfg1=lo:1 ifcfg2=lo:2 netmask=255.255.255.255 case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $ifcfg1 $vip1 netmask $netmask broadcast $vip1 up ifconfig $ifcfg2 $vip2 netmask $netmask broadcast $vip2 up route add -host $vip1 dev $ifcfg1 route add -host $vip2 dev $ifcfg2 ;; stop) ifconfig $ifcfg1 down ifconfig $ifcfg1 down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; esac [root@node4 ~]# yum -y install httpd [root@node4 ~]# vim /var/www/html <h1>RS1 CentOS7</h1> [root@node4 ~]# service httpd start [root@node5 ~]# vim /var/www/html <h1>RS2 www.gm.com</h1>
訪問結果:
[root@node3 ~]# curl 10.1.43.100 <h1>RS1 CentOS7</h1> [root@node3 ~]# curl 10.1.43.100 <h1>RS2 www.gm.com</h1> [root@node3 ~]# curl 10.1.43.200 <h1>RS2 www.gm.com</h1> [root@node3 ~]# curl 10.1.43.200 <h1>RS1 CentOS7</h1>
·keepalived能調用外部的輔助腳本進行資源監控,并根據監控的結果狀態能實現優先動態調整
分兩步:(1) 先定義一個腳本;(2) 調用此腳本;
vrrp_script <SCRIPT_NAME> {
script ""
interval INT
weight -INT
}
track_script {
SCRIPT_NAME_1
SCRIPT_NAME_2
…
} #追蹤在vrrp_instance中追蹤,即調用腳本在vrrp_instance
keepalived+nginx來調度后端的http服務:
實驗拓撲:
實驗環境:
node1主機: 10.1.43.1 提供keepalived服務+nginx服務
node3主機: 10.1.43.2 提供keepalived服務+nginx服務
node4主機: 10.1.43.101 提供httpd服務
node5主機: 10.1.43.102 提供httpd服務
主機1:
[root@node1 keepalived]# yum -y install keepalived [root@node1 keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode1 vrrp_mcast_group4 224.0.43.200 } vrrp_script chk_nginx { script "pidof nginx" interval 1 weight -5 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 18 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } track_script { chk_nginx } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 19 priority 98 advert_int 2 authentication { auth_type PASS auth_pass 3a732491 } track_script { chk_nginx } virtual_ipaddress { 10.1.43.200/16 dev eth0 } } [root@node1 keepalived]# rpm -ivh nginx-1.10.0-1.el6.ngx.x86_64.rpm #此包需要去nginx官網下載 nginx的配置: 在/etc/nginx/nginx.conf中httpd段中配置 upstream gm { server 10.1.43.101; server 10.1.43.102; } 在/etc/nginx/conf.d/default.conf的server中配置: location / { root /usr/share/nginx/html; proxy_pass http://gm; index index.html index.htm; }
主機2:
[root@node2 keepalived]# yum -y install keepalived [root@node2 keepalived]# cat keepalived.conf.nginx ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id inode2 vrrp_mcast_group4 224.0.43.200 } vrrp_script chk_nginx { script "pidof nginx" interval 1 weight -5 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 18 priority 98 nopreempt advert_int 2 authentication { auth_type PASS auth_pass 9a735491 } virtual_ipaddress { 10.1.43.100/16 dev eth0 } track_script { chk_nginx } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 19 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 3a732491 } track_script { chk_nginx } virtual_ipaddress { 10.1.43.200/16 dev eth0 } } [root@node2 keepalived]# rpm -ivh nginx-1.10.0-1.el6.ngx.x86_64.rpm nginx的配置: 在/etc/nginx/nginx.conf中httpd段中配置 upstream gm { server 10.1.43.101; server 10.1.43.102; } 在/etc/nginx/conf.d/default.conf的server中配置: location / { root /usr/share/nginx/html; proxy_pass http://gm; index index.html index.htm; }
主機4、5的配置同前文keepalived+lvs調度httpd服務
訪問結果:
[root@node3 ~]# curl 10.1.43.100 <h1>RS1 CentOS7</h1> [root@node3 ~]# curl 10.1.43.100 <h1>RS2 www.gm.com</h1> [root@node3 ~]# curl 10.1.43.200 <h1>RS2 www.gm.com</h1> [root@node3 ~]# curl 10.1.43.200 <h1>RS1 CentOS7</h1>
原創文章,作者:megedugao,如若轉載,請注明出處:http://www.www58058.com/56576