ldirectord 結合ipvsadm 配置nat,dr模型

ldirectord 結合ipvsadm 配置nat,dr模型 

一、nat模型

1、 drector # wget ftp://172.16.0.1/pub/Sources/7.x86_64/crmsh/ldirectord-3.9.6-0rc1.1.1.x86_64.rpm

# yum -y install nginx (同時用于做為sorry主機)

# yum -y install ldirectord-3.9.6-0rc1.1.1.x86_64.rpm

# echo “sorry, the service is down for maintenance, is recovering” > /usr/share/nginx/html/index.html 

主機為兩塊網卡， 一個是橋接，一個僅主機 (其中僅主機的配置靜態地址為192.16.0.5) 

開啟核心轉發功能 

# echo 1 > /proc/sys/net/ipv4/ip_forward 

nod1 # yum -y install httpd 

# echo “<h1>RS1</h1>” > /var/www/html/index.html

主機為一塊網卡，僅主機（配置靜態地址為192.16.0.2）

網關指向 192.16.0.5  

# route add default gw 192.16.0.5 

node2   # yum -y install nginx 

# echo “<h1>RS2</h1>” > /var/www/html/index.html 

主機為一塊網卡，僅主機（配置靜態地址為192.16.0.3）

網關指向 192.16.0.5  

# route add default gw 192.16.0.5 

directory 此時:可以在drector主機上進行測試，

# curl 192.16.0.2 返回RS1 

# curl 192.16.0.3 返回RS2 

directory 使用ipvsadm指定調度算法及調度主機；

# ipvsadm -A -t 172.16.250.89:80 -wrr 

# ipvsadm -a -t 172.16.250.89:80 -r 192.16.0.2:80 -m -w 3 

# ipvsadm -a -t 172.16.250.89:80 -r 192.16.0.3:80 -m -w 1

在別的主機中測試結果 

# for i in {1..4} ; do curl 172.16.250.89; done 

<h1>RS1</h1>

<h1>RS2</h2>

<h1>RS1</h1>

<h1>RS1</h1>

# 將規則保存 ipvsadm -S > /etc/sysconfig/ipvsadm 

2、 directory 配置ldirectord 

# cp /usr/share/doc/ldirectord-3.9.6/ldirectord.cf /etc/ha.d/ldirectord.cf 

# vim /etc/ha.d/ldirectord.cf 

checktimeout=3

checkinterval=1

fallback=127.0.0.1:80

autoreload=yes

logfile=”/var/log/ldirectord.log”

quiescent=no

virtual=172.16.250.89:80 

real=192.16.0.2:80 masq 1

real=192.16.0.3:80 masq 3

fallback=127.0.0.1:80 masq

service=http

scheduler=wrr

protocol=tcp

checktype=negotiate

checkport=80

注：virtual:172.16.250.89:80 ，后面需要指定端口，否則protocol=tcp指定啟動時會報錯 

real=192.16.0.2:80 masq  因為上面配置的為nat模型，所以此處使用masq 

fallback=172.0.0.1:80 此處便是nginx的sorry server,但所有結點都停掉時，會向用戶提供一個sorry server 

# systemctl start ldirectord 

# ipvsadm -Ln 查看結點 

在別的主機中測試 

#for i in {1..4} ; do curl 172.16.250.89; done 

<h1>RS2</h2>

<h1>RS1</h1>

<h1>RS2</h2>

<h1>RS2</h2>

node1,node2 將兩個結點手動停掉

# systemctl stop httpd  在測試主機中會返回sorry信息

二 、 dr 模型

1、 directory ,node1 ,node2 三臺主機都是一塊網塊， 并且網卡都為橋接，且node1,nod2，不需要指定網關

編寫腳本

#vim setkp.sh 

#!/bin/bash 

vip=172.16.252.166

mask=255.255.255.255

interface=’lo:0′

eth=’eno16777736:0′ 

case $1 in 

start) 

echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

ifconfig $interface $vip netmask $mask broadcast $vip up 

route add -host $vip dev $interface 

;; 

dstart) 

ifconfig $eth $vip/32 netmask $mask broadcast $vip up

;;

dstop)

ifconfig $eth down

;;

stop)

ifconfig $interface down 

echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore

echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce

;; 

status) 

ifconfig 

cat /proc/sys/net/ipv4/conf/all/arp_ignore

cat /proc/sys/net/ipv4/conf/lo/arp_ignore

cat /proc/sys/net/ipv4/conf/all/arp_announce

cat /proc/sys/net/ipv4/conf/lo/arp_announce

;; 

*)

echo “Usage: $(basename $0) {dstart|dstop|start|stop}”

exit 1 

esac

在director主機中執行 

# sh setkp.sh dstart 

# sh setkp.sh status 查看狀態

# scp setkp.sh 172.16.251.232:/root

# scp setkp.sh 172.16.251.191:/root 

# ipvsadm -A -t 172.16.252.166:http -s wrr

# ipvsadm -a -t 172.16.252.166:http -r 172.16.251.232:http -g -w 1

# ipvsadm -a -t 172.16.252.166:http -r 172.16.251.191:http -g -w 3

# systemctl start nginx 

# echo “Sorry Page” > /usr/share/nginx/html/index.html 

在node1主機中執行

# sh setkp.sh start 

# sh setkp.sh status 

# systemctl start httpd 

echo “<h1>NODE1</h1>” > /var/www/html/index.html 

  在node2主機中執行 

# sh setkp.sh start

# sh setkp.sh status 

# systemctl start httpd 

echo “<h2>NODE2</h2>” > /var/www/html/index.html  

在其它主機中進行測試

#for i in {1..4} ; do curl 172.16.252.166; done 

<h1>RS1</h2>

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

2、 ldirectord配置

# cat /etc/ha.d/ldirectord.cf | grep -v  “^[[:space:]]*#” | grep -v “^[[:space:]]*$” 

# vim /etc/ha.d/ldirectord.cf 

checktimeout=3

checkinterval=1

fallback=127.0.0.1:80

autoreload=yes

logfile=”/var/log/ldirectord.log”

quiescent=no

virtual=172.16.252.166:80

real=172.16.251.191:80 gate 1

real=172.16.251.232:80 gate 3

fallback=127.0.0.1:80 gate

service=http

scheduler=wrr

protocol=tcp

checktype=negotiate

checkport=80

# systemctl start ldirectord 

在其它主機中進行測試 

# for i in {1..4} ; do curl 172.16.252.166; done 

<h1>RS1</h2>

<h1>RS1</h2>

<h1>RS2</h2>

<h1>RS1</h2>

當主機所有結點都停止服務時 （node1,node2）

# systemctl stop httpd 

# for i in {1..4} ; do curl 172.16.252.166; done 

Sorry Page

Sorry Page

Sorry Page

Sorry Page

 3 、借助防火墻標記來分類報文,而后標記定義集群服務，這樣不同的服務可以使用一個集群進行調度,并啟用持久連接 

將兩臺node結點啟動 

# systemctl start httpd 

在director主機中配置 

# iptables -t mangle -A PREROUTING -d 172.16.252.166 -p tcp -m multiport –dport 80,443 -j MARK –set-mark 10 為端口打標記 

# ipvsadm -A -f 10 -s rr -p 360

# ipvsadm -a -f 10 -r 172.16.251.191:0 -g -w 1

# ipvsadm -a -f 10 -r 172.16.251.232:0 -g -w 1

在其它主機中進行測試 

# for i in {1..5} ; do curl 172.16.252.166; done 

<h1>RS1</h2>

<h1>RS1</h2>

<h1>RS1</h2>

<h1>RS1</h2>

<h1>RS1</h2>

在兩臺node結點上建立https服務

nod1 # mkdir /etc/httpd/cacert 

# cd /etc/httpd/cacert 

# (umask 066;openssl genrsa -out httpd.key 1024)

# openssl req -new -key httpd.key  -out httpd.crt -days 7200

# scp httpd.csr 172.16.252.162:/root

在director主機中生成自簽證書

# echo 01 > /etc/pki/CA/serial

# touch /etc/pki/CA/index.txt 

# cd /etc/pki/CA/

# (umask 077; openssl genrsa -out /etc/pki/CA/private/cakey.pem 2048)

# openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -days 7300 -out /etc/pki/CA/cacert.pem

# openssl ca -in /root/httpd.csr  -out /tmp/httpd.crt

# scp /tmp/httpd.crt  172.16.251.232:/root

# scp /etc/pki/CA/cacert.pem  172.16.250.69:/root

nod2    # mkdir /etc/httpd/cacert

nod1    # cd /etc/httpd/cacert/ && scp * 172.16.251.191:/etc/httpd/cacert/ 

nod1,node1 # vim /etc/httpd/conf.d/ssl.conf  

修改 : SSLCertificateFile /etc/httpd/cacert/httpd.crt

  SSLCertificateKeyFile /etc/httpd/cacert/httpd.key 

# systemctl restart httpd

在其它主機中進行測試:

# vim /etc/hosts 

加入 : 172.16.252.166  www.rj.com 

測試https與http的持久連接 

# for i in {1..4} ;do curl –cacert /root/cacert.pem https://www.rj.com && curl http://www.rj.com ; done 

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

<h1>RS2</h2>

在 ldirectord 中實現 

driector   # vim /etc/ha.d/ldirectord.cf 

checktimeout=3

checkinterval=1

fallback=127.0.0.1:80

autoreload=yes

logfile=”/var/log/ldirectord.log”

quiescent=no

virtual=10

real=172.16.251.191:80 gate 1

real=172.16.251.232:80 gate 3

fallback=127.0.0.1:80 gate

service=http

scheduler=wrr

checktype=negotiate

checkport=80

# systemctl start ldirectord

# ipvsadm -Ln 

在其它主機中進行測試:

# for i in {1..4} ;do curl –cacert /root/cacert.pem https://www.rj.com && curl http://www.rj.com ; done 

<h1>RS1</h2>

<h1>RS2</h2>

<h1>RS1</h2>

<h1>RS1</h2>

<h1>RS1</h2>

<h1>RS2</h2>

<h1>RS1</h2>

<h1>RS1</h2>