nginx+varnish+httpd小試牛刀

srayban ? 2017-06-27 17:51 ? Linux干貨

簡介: Varnish 是一款高性能且開源的反向代理服務器和 HTTP 加速器，其采用全新的軟件體系機構，和現在的硬件體系緊密配合，與傳統的 squid 相比，varnish 具有性能更高、速度更快、管理更加方便等諸多優點；

此次試驗的目的是讓Nginx做前端訪問負載均衡,varnish代理后端的web服務器,并緩存結果

1.準備機器,做好時間同步,主機名解析

192.168.42.150 node1 [nginx 負載均衡]
192.168.42.151 node2 [varnish1]
192.168.42.152 node3 [varnish2]
192.168.42.153 node4 [web1 httpd]
192.168.42.154 node5 [web2 httpd]

2.這次我們也是從后面的節點做起吧

node5:

(1).安裝httpd

yum install httpd -y

(2).創建網站目錄

mkdir /application/test -pv
chown  -R apache.apache /application/test
cd /application/test
echo "this is test 2 page." >index.html

(3).配置虛擬主機

cd /etc/httpd/conf.d
vim test.conf
<VirtualHost *:80>
        ServerName test.varnish.com
        DocumentRoot "/application/test"
        <Directory "/application/test">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/test.varnish.com_access_log" combined
</VirtualHost>

(4).啟動httpd,并測試一下子

systemctl start httpd
[root@node5 /]# curl 127.0.0.1
this is test 2 page.

node4同上,只需修改 /application/test/index.html

echo "this is test 1 page." >index.html

啟動并測試

3.node3,node2 安裝varnish

node3:

(1).安裝varnish

yum install varnish -y

(2).varnish服務端配置

cd /etc/varnish/
cp default.vcl  default.vcl.back
[root@node3 varnish]# ls
default.vcl  default.vcl.back  secret  varnish.params
[root@node3 varnish]# vim varnish.params 

VARNISH_LISTEN_PORT=80
#VARNISH_STORAGE="malloc,256M"
VARNISH_STORAGE="file,/data/varnish/cache,1G"

(3).創建緩存目錄

mkdir /data/varnish/cache -pv
chown -R varnish.varnish  /data/varnish/cache

(4).啟動varnish

systemctl start varnish
[root@node3 varnish]# ss -tnl
State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port              
LISTEN     0      1024                  *:80                   *:*                  
LISTEN     0      128                   *:22                   *:*                  
LISTEN     0      100           127.0.0.1:25                   *:*                  
LISTEN     0      10            127.0.0.1:6082                 *:*                  
LISTEN     0      1024                 :::80                  :::*                  
LISTEN     0      128                  :::22                  :::*                  
LISTEN     0      100                 ::1:25                  :::*

此時我們去瀏覽器訪問是出錯的,因此我們需要配置default.vcl,指向后端的web

Error 503 Backend fetch failed
Backend fetch failed
Guru Meditation:
XID: 32796
Varnish cache server

(5).配置default.vcl,先配置一臺測試一把

vim default.vcl
backend default {
    .host = "192.168.42.153";
    .port = "80";
}

(6).重新加載配置問件

varnish_reload_vcl

Loading vcl from /etc/varnish/default.vcl
Current running config name is 
Using new config name reload_2017-06-27T09:15:28
VCL compiled.
VCL 'reload_2017-06-27T09:15:28' now active
available       0 boot
active          0 reload_2017-06-27T09:15:28

Done

(7).去瀏覽器刷新一把,ok
(8).因為我們的后端web是有多臺的,因此我們需要把vcl配置成,多臺調度相關常用配置如下:

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
import directors;

#健康狀態檢查公共變量
probe www_probe {
   .url = "/index.html";
   .interval = 1s;
   .timeout = 1s;
   .window = 8;
   .threshold = 5;
}

#web1
backend websrv1 {
    .host = "192.168.42.153";
    .port = "80";
    .probe = www_probe;
}
#web2
backend websrv2 {
    .host = "192.168.42.154";
    .port = "80";
    .probe = www_probe;
}

#初始化
sub vcl_init {
     new websrvs= directors.round_robin();
     websrvs.add_backend(websrv1);
     websrvs.add_backend(websrv2);
}

#允許清除緩存的ip
acl purgers{
    "127.0.0.1";
    "192.168.42.0"/24;    
}

sub vcl_recv {

    #調用輪詢
    set req.backend_hint = websrvs.backend();    

    #對某類資源的請求不檢查緩存 
    if (req.url ~ "(?i)^/(login|admin)") {
        return(pass);
    }
    #將客戶端的主機推送給后端的web,寫入后端web日志中
    if (req.restarts == 0) {
        if (req.http.X-Fowarded-For) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + "," + client.ip;
            } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
    #如果是PURGE就清除緩存
    if (req.method == "PURGE") {
        if (!client.ip ~ purgers) {
            return(synth(405,"Purging not allowed for " + client.ip));
        }
        return(purge);
    }
}

#對于特定類型的資源，例如公開的圖片等，取消其私有標識，
#并強行設定其可以由varnish緩存的時長
sub vcl_backend_response {
    if (beresp.http.cache-control !~ "s-maxage") {
        if (bereq.url ~ "(?i)\.(jpg|jpeg|png|gif|css|js)$") {
            unset beresp.http.Set-Cookie;
            set beresp.ttl = 3600s;
        }
    }
}

#自定義顯示緩存是否命中
sub vcl_deliver {
    if (obj.hits>0) {
        set resp.http.X-Cache = "HIT via " + server.ip;
    } else {
        set resp.http.X-Cache = "MISS via " + server.ip;
    }
}

(10).配置完以后,把配置文件推送到node2節點上

scp default.vcl varnish.params  192.168.42.151:/etc/varnish/

在node2節點上操作:創建緩存目錄

mkdir /data/varnish/cache -pv
chown -R varnish.varnish  /data/varnish/cache

啟動varnish并查看一下子

[root@node2 varnish]# systemctl start varnish
[root@node2 varnish]# ss -tnl
State      Recv-Q Send-Q   Local Address:Port     Peer Address:Port              
LISTEN     0      1024                 *:80                  *:*                  
LISTEN     0      128                  *:22                  *:*                  
LISTEN     0      100          127.0.0.1:25                  *:*                  
LISTEN     0      10           127.0.0.1:6082                *:*                  
LISTEN     0      1024                :::80                 :::*                  
LISTEN     0      128                 :::22                 :::*                  
LISTEN     0      100                ::1:25                 :::*

(11).在node4,node5上多添加幾個虛擬主機

虛擬主機配置如下:

[root@node4 test1]# cat /etc/httpd/conf.d/test.conf 
<VirtualHost *:80>
        ServerName test.varlish.com
        DocumentRoot "/application/test"
        <Directory "/application/test">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access_log" combined
</VirtualHost>

<VirtualHost *:80>
        ServerName test.varlish1.com
        DocumentRoot "/application/test1"
        <Directory "/application/test1">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access1_log" combined
</VirtualHost>

<VirtualHost *:80>
        ServerName test.varlish2.com
        DocumentRoot "/application/test2"
        <Directory "/application/test2">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access2_log" combined
</VirtualHost>
<VirtualHost *:80>
        ServerName test.varlish3.com
        DocumentRoot "/application/test3"
        <Directory "/application/test3">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access3_log" combined
</VirtualHost>
<VirtualHost *:80>
        ServerName test.varlish4.com
        DocumentRoot "/application/test4"
        <Directory "/application/test4">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access4_log" combined
</VirtualHost>

網站目錄如下:

[root@node4 application]# ll /application/
total 0
drwxr-xr-x 2 apache apache 24 Jun 27 10:03 test
drwxr-xr-x 2 apache apache 24 Jun 27 15:21 test1
drwxr-xr-x 2 apache apache 24 Jun 27 15:17 test2
drwxr-xr-x 2 apache apache 24 Jun 27 15:17 test3
drwxr-xr-x 2 apache apache 24 Jun 27 15:18 test4

文件結構如下:

[root@node4 application]# tree /application/
/application/
├── test
│   └── index.html
├── test1
│   └── index.html
├── test2
│   └── index.html
├── test3
│   └── index.html
└── test4
    └── index.html

每個index.html的內容都需要不一樣哦

hosts文件域名解析如下:

192.168.42.153 test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

訪問一下子:

curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

this is test 1 page.
test1 page
test2 page
test3 page
test4 page

以上是我在node4上的操作,而node5只需要把node4的配置文件和網站目錄推送過去稍微改一下即可

scp /etc/hosts 192.168.42.154:/etc/hosts
cd /application
scp  -rp  test1 test2 test3 test4 192.168.42.154:/application/ 
scp /etc/httpd/conf.d/test.conf  192.168.42.154:/etc/httpd/conf.d/

然后去node5操作:

chown -R  apache.apache /application/*
echo "node5  11111111111111111" > /application/test1/index.html
echo "node5  22222222222222222" > /application/test2/index.html
echo "node5  33333333333333333" > /application/test3/index.html
echo "node5  44444444444444444" > /application/test4/index.html

vim /etc/hosts
192.168.42.154 test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
systemctl restart httpd

[root@node5 application]# curl test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 2 page.
node5  11111111111111111
node5  22222222222222222
node5  33333333333333333
node5  44444444444444444

現在我們在node4,node5上都弄了5個虛擬主機,并且訪問正常,接下來就是讓varnish能夠正常訪問那個5個虛擬主機,其實很簡單,直接在varnish機器上 hosts文件添加對應的ip

node3:

vim /etc/hosts
192.168.42.152  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

測試一把: (1).每次訪問需要清除緩存

curl -X PURGE test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

(2).訪問

curl test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

ok,可以負載均衡

node2:

vim /etc/hosts
192.168.42.151  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

測試一把: (1).每次訪問需要清除緩存

curl -X PURGE test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

(2).訪問

curl test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

ok,也是可以負載均衡

4.node1安裝nginx 負載均衡

(1).安裝nginx

yum install  nginx -y

(2).配置nginx 因為我們的后端做了5個域名的虛擬主機,因此我們的負載均衡器也需要做5個一樣的虛擬主機

hosts解析:

vim /etc/hosts
192.168.42.150  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

upsteam配置:

vim /etc/nginx/nginx.conf
upstream testsrvs {
    server 192.168.42.151:80;
    server 192.168.42.152:80;
    least_conn;
}

虛擬主機配置:

vim /etc/nginx/conf.d/test.conf

server {
  listen 80;
  server_name test.varlish.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish1.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish2.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish3.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish4.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

測試一把: 為了能夠測出負載均衡的效果,我們先去node2,node3的兩個varnish兩個節點清除緩存
(1).清除緩存

curl -X PURGE test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

然后在node1中訪問:

第一次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 1 page.
test1 page
node5  22222222222222222
node5  33333333333333333
test4 page

第二次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 1 page.
node5  11111111111111111
node5  22222222222222222
test3 page
test4 page

第三次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 2 page.
node5  11111111111111111
test2 page
test3 page
node5  44444444444444444

第四次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 2 page.
test1 page
test2 page
node5  33333333333333333
node5  44444444444444444

好了,我們經過四次的訪問得到不同的結果,說明負載均衡OK了

5.壓測 ab -n 10000 -c 1000 http://test.varlish.com/

原創文章，作者：srayban，如若轉載，請注明出處：http://www.www58058.com/78690

贊 (0)

0

Gdevops 2017全球敏捷運維峰會【上海站】

上一篇 2017-06-27 17:50

企業面試題

下一篇 2017-06-27 20:51

馬哥教育網絡21期+第五周練習博客

1、顯示/boot/grub/grub.conf中以至少一個空白字符開頭的行； [root@localhost ~]# grep '[^[:space:]]\+' /boot/grub2/grub.cfg 2、顯示/etc/rc.d/rc.sysinit文件中以#開頭，后面跟至少一個空白字符…

Linux干貨 2016-08-08
8.1作業

創建用戶gentoo，附加組為bin和root，默認shell為/bin/csh，注釋信息為“Gentoo Distribution” [root@localhost wang]# useradd -G bin,root -s /bin/csh -c "Gentoo&n…

Linux干貨 2016-08-02
Linux干貨

利用nginx實現基于傳輸層的四層負載均衡

nginx利用ngx_stream_core_module實現四層的負載均衡服務。作為四層負載均衡nginx和lvs的區別在于： lvs工作于內核層，相對來說效率更高，性能更強； nginx工作于用戶空間； lvs不會受到套接字數量的限制； nginx作為四層負載均衡也需要監聽套接字來和客戶端，后臺服務器進行連接，會受到套接字數量限制，不過這個問題可以通過k…

2017-07-03
馬哥教育網絡第21期-第七周課程練習

1、創建一個10G分區，并格式為ext4文件系統； (1) 要求其block大小為2048, 預留空間百分比為2, 卷標為MYDATA, 默認掛載屬性包含acl； (2) 掛載至/data/mydata目錄，要求掛載時禁止程序自動運行，且不更新文件的訪問時間戳； [root@6 ~]# …

Linux干貨 2016-08-29
iptables

Evernote Export 基于本機服務器的iptables：創建、重命名、刪除自定義chain ~]# iptables -N testchain ~]# iptables -nL Chain INPUT (policy ACCEPT) target prot opt source …

Linux干貨 2016-11-20
Linux干貨

shell中的引號用法

SHELL引號在shell里可以通過使用單引號，雙引號，反引號（TAB鍵上方的按鍵），反斜線來轉換某些shell元字符的含義。比如說，我們希望echo命令顯示出$本身字符的意義，而不是變量，我們需要使用某些功能來屏蔽$符號本身賦予的特殊含義，使其還原本身字面的意思。主要講述四個比較特殊常用的符號。 1.反…

2017-06-11

欧美性久久久久