基于 heartbeat v2 crm 實現HA高可用性的 LAMP+wordpress
基本環境設置
OS version: CentOS release 6.7 (Final) Soft version: heartbeat-2.1.4-11.el5.x86_64.rpm heartbeat-devel-2.1.4-11.el5.x86_64.rpm heartbeat-pils-2.1.4-9.el5.x86_64.rpm heartbeat-stonith-2.1.4-11.el5.x86_64.rpm libltdl3-1.5.26-alt11.x86_64.rpm 高可用節點:node 1 192.168.2.216/24 node1.test.com VIP :node 3 192.168.2.218/24
關閉iptables,selinux
一、修改主機名
[root@node1 ~]# more /etc/hosts hosts hosts.allow hosts.deny [root@node1 ~]# more /etc/hosts 127.0.0.1 localhost node1 node1.test.com localhost4 localhost4.localdomain4 ::1 localhost node1 node1.test.com localhost6 localhost6.localdomain6 [root@node1 ~]# more /etc/sysconfig/network NETWORKING=yes HOSTNAME=node1
二、安裝apahce
[root@node1 ~]# yum install -y httpd [root@node1 ~]# vim /var/www/html/index.html [root@node1 ~]# service httpd restart Stopping httpd: [ OK ] Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using node1.test.com for ServerName [ OK ] [root@node1 ~]# curl -I http://192.168.2.216 HTTP/1.1 200 OK Date: Sat, 13 Aug 2016 16:32:38 GMT Server: Apache/2.2.15 (CentOS) Last-Modified: Sat, 13 Aug 2016 16:30:22 GMT ETag: "260270-f-539f683fbfea1" Accept-Ranges: bytes Content-Length: 15 Connection: close Content-Type: text/html; charset=UTF-8
三、SSH免密鑰登錄
[root@node1 ~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: f0:33:40:f4:f1:83:3d:7d:1e:5b:67:a2:a8:08:44:b3 root@node1.test.com The key's randomart image is: +--[ RSA 2048]----+ | o.o . | | . + . = . | | E o o = . + +| | . + + + *.| | . S . . o | | . . + | | . . | | | | | +-----------------+ [root@node1 ~]# ssh-copy-id 192.168.2.217 The authenticity of host '192.168.2.217 (192.168.2.217)' can't be established. RSA key fingerprint is bc:71:16:97:d8:ce:a0:3e:8b:4a:b9:da:ca:bd:dd:07. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.2.217' (RSA) to the list of known hosts. root@192.168.2.217's password: Now try logging into the machine, with "ssh '192.168.2.217'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [root@node1 ~]# ssh root@192.168.2.217 Last login: Sun Aug 14 00:24:50 2016 from 192.168.2.106 [root@node2 ~]#
四、安裝heartbeat
[root@node1 ~]# wget http://mirrors.aliyun.com/epel/6/x86_64/epel-release-6-8.noarch.rpm
[root@node1 ~]# rpm -ivh epel-release-6-8.noarch.rpm
warning: epel-release-6-8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]
1:epel-release ########################################### [100%]
#直接安裝rpm heartbeat會報依賴錯誤，這里我們先解決依賴關系
[root@node1 ~]#yum -y install perl-TimeDate net-snmp-libs libnet PyXML
#再安裝就可以了
[root@node1 heartbeat]# rpm -ivh *.rpm --nodeps --force
warning: heartbeat-2.1.4-11.el5.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 217521f6: NOKEY
warning: heartbeat-pils-2.1.4-9.el5.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 192a7d7d: NOKEY
Preparing... ########################################### [100%]
1:heartbeat-pils ########################################### [ 25%]
2:heartbeat-stonith ########################################### [ 50%]
3:heartbeat warning: /etc/ha.d/shellfuncs created as /etc/ha.d/shellfuncs.rpmnew
########################################### [ 75%]
4:heartbeat-devel ########################################### [100%]
[root@node1 heartbeat]# ll /etc/ha.d/
total 24
-rwxr-xr-x 1 root root 745 Mar 21 2010 harc
drwxr-xr-x 2 root root 4096 Aug 14 17:14 rc.d
-rw-r--r-- 1 root root 692 Mar 21 2010 README.config
drwxr-xr-x 2 root root 4096 Aug 14 17:14 resource.d
-rw-r--r-- 1 root root 7864 Mar 21 2010 shellfuncs
[root@node1 ha.d]# cp -r /usr/share/doc/heartbeat-2.1.4/ /etc/ha.d/
[root@node1 ha.d]# ll
total 28
-rwxr-xr-x 1 root root 745 Mar 21 2010 harc
drwxr-xr-x 2 root root 4096 Aug 14 04:24 heartbeat-2.1.4
drwxr-xr-x 2 root root 4096 Aug 14 04:20 rc.d
-rw-r--r-- 1 root root 692 Mar 21 2010 README.config
drwxr-xr-x 2 root root 4096 Aug 14 04:20 resource.d
-rw-r--r-- 1 root root 7864 Mar 21 2010 shellfuncs
[root@node1 ha.d]# cp ./heartbeat-2.1.4/{ha.cf,authkeys,haresources} ./
[root@node1 ha.d]# chmod 600 authkeys
/etc/ha.d目錄下：
#ha.cf :主配置文件，定義各節點上的heartbeat HA集群的基本屬性；
#authkeys :集群內節點間彼此傳遞消息時使用加密算法及密鑰；
#haresources :為heartbeat v1的提供資源管理器配置接口: v1版本專用的配置接口
[root@node1 ha.d]# ll
total 52
-rw------- 1 root root 666 Aug 14 04:27 authkeys
-rw-r--r-- 1 root root 10770 Aug 14 04:29 ha.cf
-rwxr-xr-x 1 root root 745 Mar 21 2010 harc
-rw-r--r-- 1 root root 5970 Aug 14 04:54 haresources
drwxr-xr-x 2 root root 4096 Aug 14 04:24 heartbeat-2.1.4
drwxr-xr-x 2 root root 4096 Aug 14 22:39 rc.d
-rw-r--r-- 1 root root 692 Mar 21 2010 README.config
drwxr-xr-x 2 root root 4096 Aug 14 22:39 resource.d
-rw-r--r-- 1 root root 7864 Mar 21 2010 shellfuncs
這里要注意的是：如何在網絡中我們期望的節點集群成為我們所需要的節點，在集群中信息不能隨便傳遞，而心跳節點是基于組播地址傳遞的，如果別人也裝了heartbeat也連接到這個組播地址上來，這都不安全，基于這種情況，我們各節點這間信息傳遞是需要認證的，這種認證基于HMAC
配置heartbeat，它的配置文件在/etc/ha.d/的目錄下，但是安裝完程序之后這個目錄下沒有這個配置文件，只有/usr/share/doc/heartbeat-2.1.4/目錄下有ha.cf的主配置文件樣本，復制到/etc下修改配置文件即可使用；還有一個authkeys的認證文件，這個文件就是我們各節點認證時所保存的認證密碼和認證機制，所以這個文件的權限至關重要，必須是600，否則啟動不了服務；第三個haresources，定義資源時需要資源管理器來讀取這個文件，所以這個也得有；
[root@node1 ha.d]# vim authkeys auth 2 2 sha1 be45617e827e7a7d
[root@node1 ha.d]# vim ha.cf debugfile /var/log/ha-debug keepalive 2 deadtime 30 warntime 10 udpport 694 mcast eth0 225.10.0.1 694 1 0 auto_failback on node node1.test.com node node2.test.com crm on ping 192.168.2.217 compression bz2 compression_threshold 2 #logfile /var/log/ha-log #日志文件，正常日志信息記錄到哪去的 #keepalive 2 #每隔2s發送一次心跳信息的，單位是秒，毫秒用ms #deadtime 30 #隔多長時間探測到對方不在線就kill掉的時間間隔 #warntime 10 #警告時間 #udpport 694 #mcast eth0 225.10.0.1 694 1 0 #定義組播地址 #auto_failback on #開啟故障轉回功能 #node node1.test.com #定義兩個節點 #node node2.test.com #crm on #啟用crm功能 #ping 192.168.2.218 #ping節點 #compression bz2 #壓縮格式 #compression_threshold 2 #表示小于2K時不壓縮傳輸
[root@node1 ha.d]# scp -p ha.cf haresources authkeys node2:/etc/ha.d/ The authenticity of host 'node2 (192.168.2.217)' can't be established. RSA key fingerprint is bc:71:16:97:d8:ce:a0:3e:8b:4a:b9:da:ca:bd:dd:07. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2' (RSA) to the list of known hosts. ha.cf 100% 11KB 10.5KB/s 00:00 haresources 100% 5970 5.8KB/s 00:00 authkeys 100% 666 0.7KB/s 00:00
在資源管理器的配置文件中定義；/etc/ha.d/haresources，在/etc/ha.d/resource.d下有各種資源類型，當在資源配置文件中定義時就會調用這里的資源類型來運行相應的程序；
[root@node1 ha.d]# node1.test.com 192.168.2.218/24/eth0/192.168.255.255 httpd [root@node1 ha.d]# service heartbeat start [root@node1 ha.d]# service heartbeat start logd is already running Starting High-Availability services: Done.
碰到一些包依賴的錯誤，都是小問題就不一一舉例了，缺什么補什么；
還有就是剛啟動服務，機器就重啟，懵逼了。。。報錯信息如下：
[root@node1 ~]# Message from syslogd@node1 at Aug 15 01:32:04 ... heartbeat: [1439]: EMERG: Rebooting system. Reason: /usr/lib64/heartbeat/cib atu Connection closed by foreign host.
網上有說改hearbeat文件的用戶和組，沒用~~，后注釋ha.cf的#crm on 恢復。
[root@node1 ha.d]# service heartbeat restart Stopping High-Availability services: Done. Waiting to allow resource takeover to complete: Done. Starting High-Availability services: 2016/08/14_18:08:55 INFO: Resource is stopped Done.
檢測運行狀態,所有資源在node1上 ，node2 上只有heartbeat服務
[root@node1 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:46:B1:B7 inet addr:192.168.2.216 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe46:b1b7/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2522 errors:0 dropped:0 overruns:0 frame:0 TX packets:4289 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:320135 (312.6 KiB) TX bytes:2902096 (2.7 MiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:46:B1:B7 inet addr:192.168.2.218 Bcast:10.10.10.254 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) [root@node1 ha.d]# ps -ef |grep httpd root 2635 1 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2640 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2641 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2642 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2643 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2644 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2645 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2646 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd apache 2647 2635 0 18:10 ? 00:00:00 /usr/sbin/httpd root 2900 1592 0 18:12 pts/0 00:00:00 grep httpd [root@node1 ha.d]# pa aux |grep heartbeat -bash: pa: command not found [root@node1 ha.d]# ps aux |grep heartbeat root 2338 0.0 1.4 69608 14364 ? SLs 18:10 0:00 heartbeat: master control process nobody 2341 0.0 0.7 62876 7632 ? SL 18:10 0:00 heartbeat: FIFO reader nobody 2342 0.0 0.7 63004 7760 ? SL 18:10 0:00 heartbeat: write: mcast eth0 nobody 2343 0.0 0.7 63004 7760 ? SL 18:10 0:00 heartbeat: read: mcast eth0 nobody 2344 0.0 0.7 63004 7760 ? SL 18:10 0:00 heartbeat: write: ping 192.168.2.217 nobody 2345 0.0 0.7 63004 7760 ? SL 18:10 0:00 heartbeat: read: ping 192.168.2.217 root 2904 0.0 0.0 103312 876 pts/0 S+ 18:13 0:00 grep heartbeat [root@node2 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:2D:5B:EC inet addr:192.168.2.217 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe2d:5bec/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11766 errors:0 dropped:0 overruns:0 frame:0 TX packets:5494 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3603815 (3.4 MiB) TX bytes:837732 (818.0 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:355 errors:0 dropped:0 overruns:0 frame:0 TX packets:355 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:49398 (48.2 KiB) TX bytes:49398 (48.2 KiB) [root@node2 ha.d]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 3198/httpd [root@node2 ha.d]# ps -ef |grep httpd root 3198 1 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3200 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3201 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3202 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3203 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3204 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3205 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3206 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd apache 3207 3198 0 18:11 ? 00:00:00 /usr/sbin/httpd root 3217 1811 0 18:14 pts/0 00:00:00 grep httpd [root@node2 ha.d]#
測試網頁：
停止node1節點上的heartbeat后
[root@node2 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:2D:5B:EC inet addr:192.168.2.217 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe2d:5bec/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:12689 errors:0 dropped:0 overruns:0 frame:0 TX packets:6338 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3701251 (3.5 MiB) TX bytes:980188 (957.2 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:2D:5B:EC inet addr:192.168.2.218 Bcast:192.168.2.254 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:413 errors:0 dropped:0 overruns:0 frame:0 TX packets:413 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:57576 (56.2 KiB) TX bytes:57576 (56.2 KiB) [root@node2 ha.d]# ps -ef |grep httpd root 4452 1 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4454 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4455 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4456 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4457 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4458 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4459 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4460 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd apache 4461 4452 0 18:21 ? 00:00:00 /usr/sbin/httpd root 4468 1811 0 18:21 pts/0 00:00:00 grep httpd [root@node2 ha.d]# ps -ef |grep heartbeat root 4022 1 0 18:20 ? 00:00:00 heartbeat: master control process nobody 4024 4022 0 18:20 ? 00:00:00 heartbeat: FIFO reader nobody 4025 4022 0 18:20 ? 00:00:00 heartbeat: write: mcast eth0 nobody 4026 4022 0 18:20 ? 00:00:00 heartbeat: read: mcast eth0 nobody 4027 4022 0 18:20 ? 00:00:00 heartbeat: write: ping 192.168.2.216 nobody 4028 4022 0 18:20 ? 00:00:00 heartbeat: read: ping 192.168.2.216 root 4470 1811 0 18:21 pts/0 00:00:00 grep heartbeat
切換到node2上面了：
當node1節點停止后，node2的節點就會頂上去，成為主節點，提供高可用服務，httpd內容現實node1.test.com和node2.test.com只是為了測試，加以區別。
附上node1、node2 ha-debug
[root@node1 ha.d]# tail -f /var/log/ha-debug heartbeat[2338]: 2016/08/14_18:16:23 info: killing HBWRITE process 2342 with signal 15 heartbeat[2338]: 2016/08/14_18:16:23 info: killing HBREAD process 2343 with signal 15 heartbeat[2338]: 2016/08/14_18:16:23 info: killing HBWRITE process 2344 with signal 15 heartbeat[2338]: 2016/08/14_18:16:23 info: killing HBREAD process 2345 with signal 15 heartbeat[2338]: 2016/08/14_18:16:23 info: Core process 2342 exited. 5 remaining heartbeat[2338]: 2016/08/14_18:16:23 info: Core process 2341 exited. 4 remaining heartbeat[2338]: 2016/08/14_18:16:23 info: Core process 2345 exited. 3 remaining heartbeat[2338]: 2016/08/14_18:16:23 info: Core process 2344 exited. 2 remaining heartbeat[2338]: 2016/08/14_18:16:23 info: Core process 2343 exited. 1 remaining heartbeat[2338]: 2016/08/14_18:16:23 info: node1.test.com Heartbeat shutdown complete. [root@node2 ha.d]# tail -f /var/log/ha-debug IPaddr[4294]:2016/08/14_18:21:13 INFO: Success INFO: Success ResourceManager[4196]:2016/08/14_18:21:13 debug: /etc/ha.d/resource.d/IPaddr 192.168.2.218/24/eth0/192.168.2.254 start done. RC=0 ResourceManager[4196]:2016/08/14_18:21:13 info: Running /etc/init.d/httpd start ResourceManager[4196]:2016/08/14_18:21:13 debug: Starting /etc/init.d/httpd start Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using node2.test.com for ServerName [ OK ] ResourceManager[4196]:2016/08/14_18:21:13 debug: /etc/init.d/httpd start done. RC=0 heartbeat[4022]: 2016/08/14_18:21:23 info: Local Resource acquisition completed. (none) heartbeat[4022]: 2016/08/14_18:21:23 info: local resource transition completed.
安裝 heartbeat CRM LAMP WordPress
[root@node1 ha.d]# yum install mysql-server php php-mysql -y [root@node1 ha.d]# vim /var/www/html/info.php [root@node1 ha.d]# service httpd restart Stopping httpd: [ OK ] Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using node1.test.com for ServerName [ OK ]
打卡node1節點httpd，已經配置好php了
測試mysql
[root@node1 ha.d]# mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.1.73 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | test | +--------------------+ 3 rows in set (0.00 sec) mysql>
nfs服務器上：
[root@php-server ~]# mkdir /data/mysql
node1上:
[root@node1 ha.d]# more /etc/my.cnf [mysqld] datadir=/data/ mount -t nfs 192.168.2.216:/data/mysql /data /usr/bin/mysql_install_db --user=mysql --datadir=/data/ socket=/var/lib/mysql/mysql.sock user=mysql # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0 [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid [root@node1 ha.d]# service mysqld start Starting mysqld: [ OK ] [root@node1 ha.d]#mysql mysql> CREATE DATABASE `wordpress` CHARACTER SET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON wordpress.* TO 'wordpress'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;FLUSH PRIVILEGES; Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | test | | wordpress | +--------------------+ 4 rows in set (0.00 sec)
|
安裝 heartbeat-gui [root@node1 ~]# yum -y install pygtk2-libglade xorg-x11-xauth [root@node1 ~]# rpm -ivh heartbeat-gui-2.1.4-12.el6.x86_64.rpm [root@node1 ~]# echo "password" | passwd --stdin hacluster |
后面明天繼續吧
原創文章，作者：zuoyang1990，如若轉載，請注明出處：http://www.www58058.com/34785
