nginx+varnish+httpd小試牛刀

srayban ? 2017-06-27 17:51 ? Linux干貨

簡介: Varnish 是一款高性能且開源的反向代理服務器和 HTTP 加速器，其采用全新的軟件體系機構，和現在的硬件體系緊密配合，與傳統的 squid 相比，varnish 具有性能更高、速度更快、管理更加方便等諸多優點；

此次試驗的目的是讓Nginx做前端訪問負載均衡,varnish代理后端的web服務器,并緩存結果

1.準備機器,做好時間同步,主機名解析

192.168.42.150 node1 [nginx 負載均衡]
192.168.42.151 node2 [varnish1]
192.168.42.152 node3 [varnish2]
192.168.42.153 node4 [web1 httpd]
192.168.42.154 node5 [web2 httpd]

2.這次我們也是從后面的節點做起吧

node5:

(1).安裝httpd

yum install httpd -y

(2).創建網站目錄

mkdir /application/test -pv
chown  -R apache.apache /application/test
cd /application/test
echo "this is test 2 page." >index.html

(3).配置虛擬主機

cd /etc/httpd/conf.d
vim test.conf
<VirtualHost *:80>
        ServerName test.varnish.com
        DocumentRoot "/application/test"
        <Directory "/application/test">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/test.varnish.com_access_log" combined
</VirtualHost>

(4).啟動httpd,并測試一下子

systemctl start httpd
[root@node5 /]# curl 127.0.0.1
this is test 2 page.

node4同上,只需修改 /application/test/index.html

echo "this is test 1 page." >index.html

啟動并測試

3.node3,node2 安裝varnish

node3:

(1).安裝varnish

yum install varnish -y

(2).varnish服務端配置

cd /etc/varnish/
cp default.vcl  default.vcl.back
[root@node3 varnish]# ls
default.vcl  default.vcl.back  secret  varnish.params
[root@node3 varnish]# vim varnish.params 

VARNISH_LISTEN_PORT=80
#VARNISH_STORAGE="malloc,256M"
VARNISH_STORAGE="file,/data/varnish/cache,1G"

(3).創建緩存目錄

mkdir /data/varnish/cache -pv
chown -R varnish.varnish  /data/varnish/cache

(4).啟動varnish

systemctl start varnish
[root@node3 varnish]# ss -tnl
State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port              
LISTEN     0      1024                  *:80                   *:*                  
LISTEN     0      128                   *:22                   *:*                  
LISTEN     0      100           127.0.0.1:25                   *:*                  
LISTEN     0      10            127.0.0.1:6082                 *:*                  
LISTEN     0      1024                 :::80                  :::*                  
LISTEN     0      128                  :::22                  :::*                  
LISTEN     0      100                 ::1:25                  :::*

此時我們去瀏覽器訪問是出錯的,因此我們需要配置default.vcl,指向后端的web

Error 503 Backend fetch failed
Backend fetch failed
Guru Meditation:
XID: 32796
Varnish cache server

(5).配置default.vcl,先配置一臺測試一把

vim default.vcl
backend default {
    .host = "192.168.42.153";
    .port = "80";
}

(6).重新加載配置問件

varnish_reload_vcl

Loading vcl from /etc/varnish/default.vcl
Current running config name is 
Using new config name reload_2017-06-27T09:15:28
VCL compiled.
VCL 'reload_2017-06-27T09:15:28' now active
available       0 boot
active          0 reload_2017-06-27T09:15:28

Done

(7).去瀏覽器刷新一把,ok
(8).因為我們的后端web是有多臺的,因此我們需要把vcl配置成,多臺調度相關常用配置如下:

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
import directors;

#健康狀態檢查公共變量
probe www_probe {
   .url = "/index.html";
   .interval = 1s;
   .timeout = 1s;
   .window = 8;
   .threshold = 5;
}

#web1
backend websrv1 {
    .host = "192.168.42.153";
    .port = "80";
    .probe = www_probe;
}
#web2
backend websrv2 {
    .host = "192.168.42.154";
    .port = "80";
    .probe = www_probe;
}

#初始化
sub vcl_init {
     new websrvs= directors.round_robin();
     websrvs.add_backend(websrv1);
     websrvs.add_backend(websrv2);
}

#允許清除緩存的ip
acl purgers{
    "127.0.0.1";
    "192.168.42.0"/24;    
}

sub vcl_recv {

    #調用輪詢
    set req.backend_hint = websrvs.backend();    

    #對某類資源的請求不檢查緩存 
    if (req.url ~ "(?i)^/(login|admin)") {
        return(pass);
    }
    #將客戶端的主機推送給后端的web,寫入后端web日志中
    if (req.restarts == 0) {
        if (req.http.X-Fowarded-For) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + "," + client.ip;
            } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
    #如果是PURGE就清除緩存
    if (req.method == "PURGE") {
        if (!client.ip ~ purgers) {
            return(synth(405,"Purging not allowed for " + client.ip));
        }
        return(purge);
    }
}

#對于特定類型的資源，例如公開的圖片等，取消其私有標識，
#并強行設定其可以由varnish緩存的時長
sub vcl_backend_response {
    if (beresp.http.cache-control !~ "s-maxage") {
        if (bereq.url ~ "(?i)\.(jpg|jpeg|png|gif|css|js)$") {
            unset beresp.http.Set-Cookie;
            set beresp.ttl = 3600s;
        }
    }
}

#自定義顯示緩存是否命中
sub vcl_deliver {
    if (obj.hits>0) {
        set resp.http.X-Cache = "HIT via " + server.ip;
    } else {
        set resp.http.X-Cache = "MISS via " + server.ip;
    }
}

(10).配置完以后,把配置文件推送到node2節點上

scp default.vcl varnish.params  192.168.42.151:/etc/varnish/

在node2節點上操作:創建緩存目錄

mkdir /data/varnish/cache -pv
chown -R varnish.varnish  /data/varnish/cache

啟動varnish并查看一下子

[root@node2 varnish]# systemctl start varnish
[root@node2 varnish]# ss -tnl
State      Recv-Q Send-Q   Local Address:Port     Peer Address:Port              
LISTEN     0      1024                 *:80                  *:*                  
LISTEN     0      128                  *:22                  *:*                  
LISTEN     0      100          127.0.0.1:25                  *:*                  
LISTEN     0      10           127.0.0.1:6082                *:*                  
LISTEN     0      1024                :::80                 :::*                  
LISTEN     0      128                 :::22                 :::*                  
LISTEN     0      100                ::1:25                 :::*

(11).在node4,node5上多添加幾個虛擬主機

虛擬主機配置如下:

[root@node4 test1]# cat /etc/httpd/conf.d/test.conf 
<VirtualHost *:80>
        ServerName test.varlish.com
        DocumentRoot "/application/test"
        <Directory "/application/test">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access_log" combined
</VirtualHost>

<VirtualHost *:80>
        ServerName test.varlish1.com
        DocumentRoot "/application/test1"
        <Directory "/application/test1">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access1_log" combined
</VirtualHost>

<VirtualHost *:80>
        ServerName test.varlish2.com
        DocumentRoot "/application/test2"
        <Directory "/application/test2">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access2_log" combined
</VirtualHost>
<VirtualHost *:80>
        ServerName test.varlish3.com
        DocumentRoot "/application/test3"
        <Directory "/application/test3">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access3_log" combined
</VirtualHost>
<VirtualHost *:80>
        ServerName test.varlish4.com
        DocumentRoot "/application/test4"
        <Directory "/application/test4">
                Options None
                AllowOverride None
                Require all granted
        </Directory>
        CustomLog "logs/iounix_access4_log" combined
</VirtualHost>

網站目錄如下:

[root@node4 application]# ll /application/
total 0
drwxr-xr-x 2 apache apache 24 Jun 27 10:03 test
drwxr-xr-x 2 apache apache 24 Jun 27 15:21 test1
drwxr-xr-x 2 apache apache 24 Jun 27 15:17 test2
drwxr-xr-x 2 apache apache 24 Jun 27 15:17 test3
drwxr-xr-x 2 apache apache 24 Jun 27 15:18 test4

文件結構如下:

[root@node4 application]# tree /application/
/application/
├── test
│   └── index.html
├── test1
│   └── index.html
├── test2
│   └── index.html
├── test3
│   └── index.html
└── test4
    └── index.html

每個index.html的內容都需要不一樣哦

hosts文件域名解析如下:

192.168.42.153 test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

訪問一下子:

curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

this is test 1 page.
test1 page
test2 page
test3 page
test4 page

以上是我在node4上的操作,而node5只需要把node4的配置文件和網站目錄推送過去稍微改一下即可

scp /etc/hosts 192.168.42.154:/etc/hosts
cd /application
scp  -rp  test1 test2 test3 test4 192.168.42.154:/application/ 
scp /etc/httpd/conf.d/test.conf  192.168.42.154:/etc/httpd/conf.d/

然后去node5操作:

chown -R  apache.apache /application/*
echo "node5  11111111111111111" > /application/test1/index.html
echo "node5  22222222222222222" > /application/test2/index.html
echo "node5  33333333333333333" > /application/test3/index.html
echo "node5  44444444444444444" > /application/test4/index.html

vim /etc/hosts
192.168.42.154 test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
systemctl restart httpd

[root@node5 application]# curl test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 2 page.
node5  11111111111111111
node5  22222222222222222
node5  33333333333333333
node5  44444444444444444

現在我們在node4,node5上都弄了5個虛擬主機,并且訪問正常,接下來就是讓varnish能夠正常訪問那個5個虛擬主機,其實很簡單,直接在varnish機器上 hosts文件添加對應的ip

node3:

vim /etc/hosts
192.168.42.152  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

測試一把: (1).每次訪問需要清除緩存

curl -X PURGE test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

(2).訪問

curl test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

ok,可以負載均衡

node2:

vim /etc/hosts
192.168.42.151  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

測試一把: (1).每次訪問需要清除緩存

curl -X PURGE test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

(2).訪問

curl test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

ok,也是可以負載均衡

4.node1安裝nginx 負載均衡

(1).安裝nginx

yum install  nginx -y

(2).配置nginx 因為我們的后端做了5個域名的虛擬主機,因此我們的負載均衡器也需要做5個一樣的虛擬主機

hosts解析:

vim /etc/hosts
192.168.42.150  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

upsteam配置:

vim /etc/nginx/nginx.conf
upstream testsrvs {
    server 192.168.42.151:80;
    server 192.168.42.152:80;
    least_conn;
}

虛擬主機配置:

vim /etc/nginx/conf.d/test.conf

server {
  listen 80;
  server_name test.varlish.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish1.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish2.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish3.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

server {
  listen 80;
  server_name test.varlish4.com;
  location / {
        proxy_pass http://testsrvs;
        proxy_set_header Host  $host;
        proxy_set_header X-Forwarded-For  $remote_addr;
        add_header X-Via  $server_addr;
        add_header X-Accel $server_name;
  }
}

測試一把: 為了能夠測出負載均衡的效果,我們先去node2,node3的兩個varnish兩個節點清除緩存
(1).清除緩存

curl -X PURGE test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com

然后在node1中訪問:

第一次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 1 page.
test1 page
node5  22222222222222222
node5  33333333333333333
test4 page

第二次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 1 page.
node5  11111111111111111
node5  22222222222222222
test3 page
test4 page

第三次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 2 page.
node5  11111111111111111
test2 page
test3 page
node5  44444444444444444

第四次:

[root@node1 nginx]# curl  test.varlish.com test.varlish1.com test.varlish2.com  test.varlish3.com  test.varlish4.com
this is test 2 page.
test1 page
test2 page
node5  33333333333333333
node5  44444444444444444

好了,我們經過四次的訪問得到不同的結果,說明負載均衡OK了

5.壓測 ab -n 10000 -c 1000 http://test.varlish.com/

原創文章，作者：srayban，如若轉載，請注明出處：http://www.www58058.com/78690

贊 (0)

0

Gdevops 2017全球敏捷運維峰會【上海站】

上一篇 2017-06-27

企業面試題

下一篇 2017-06-27

馬哥教育網絡班21期+第5周課程練習

1.顯示/boot/grub/grub.conf中至少以一個空白字符開頭的行 [root@itop ~]# grep "^[[:space:]]\+" /boot/grub/grub.conf 2.顯示/etc/rc.d/rc.sysinit文件中以#開頭，后面至少一個空白字符，而后又有至少一個非空…

Linux干貨 2016-08-08
netstat

netstat命令 netstat 命令用于顯示各種網絡相關信息，如網絡連接，路由表，接口狀態 (Interface Statistics)，masquerade 連接，多播成員 (Multicast Memberships) 等等 1 常用選項： -a ：(all)顯示所有選項，默認不顯示LISTEN相關 -t ：(tcp)僅顯示tcp相關選項 -u ：(…

Linux干貨 2017-05-07
馬哥教育網絡班22期+第1周課程練習

1.描述計算機的組成及其功能。組成：CPU (運算器+控制器)，存諸器（內存與硬盤），輸入設備，輸出設備。運算器：對數據進行加工處理的部件（包括算述運算與邏輯運算）。控制器：負責從存儲器取出指令，按指令的要求發出控制信號，使各部件協調的，一步步的完成各種操作。 …

Linux干貨 2016-08-22
馬哥教育網絡班20期+第二周課程練習

1、Linux上的文件管理類命令都有哪些，其常用的使用方法及其相關示例演示。答：文件管理類命令有： touch ：創建一個新文件示例：在/tmp/目錄下創建一個1.log文件 touch /tmp/1.log mv：移動文件示例：將/etc/fstab文件移至/tmp/目錄中。 mv /etc/fstab /tmp/…

Linux干貨 2016-06-23
Linux之初見

前言第一次聽到Linux的大名是在進行網絡培訓的時候，隔壁的紅帽子培訓時聽到的。由于這樣，導致我很長一段時間都以為Linux就是紅帽，紅帽就是Linux，當聽到Ubantu的時候，還以為又是另一個全新的，區別于windows和Linux的版本，現在想起來很是羞愧。近期因為…

Linux干貨 2016-02-14
bash腳本編程練習：判斷、循環

1、寫一個腳本，判斷當前系統上所有用戶的shell是否為可登錄shell（即用戶的shell不是/sbin/nologin）；分別這兩類用戶的個數；通過字符串比較來實現； #!/bin/bash # declare -i sumlogin=0 declare -i sumnologin=0 for&nbs…

Linux干貨 2016-11-22

欧美性久久久久