LVS+Keepalived负载均衡配置部署

http://www.ituring.com.cn/article/179806

相关概念

维基百科-LVS百度百科-LVSLVS中文站点LVS中文手册

一、准备工作 1. 服务器、操作系统（当前CentOS6.6 x86_64）、网络环境（服务期间网络互通） ；

# 操作系统：CentOS6.6 x86_64 # 服务器 （LVS主备各1台，WEB服务器3台，图片服务器2台） WEBServer负载虚拟IP：192.168.0.20 IMGServer负载虚拟IP：192.168.0.30 LVS主：192.168.0.11 LVS备：192.168.0.12 WebServer1： 192.168.0.21 WebServer2： 192.168.0.22 WebServer3： 192.168.0.23 IMGServer1： 192.168.0.31 IMGServer2： 192.168.0.32

2. 关闭SELinux(改完需重启服务器)；

# 关闭SELinux sed -i 's#^SELINUX=.*#SELINUX=disabled#' /etc/sysconfig/selinux # 重启服务器 reboot

3. Iptables开启需要端口，如80；

# Iptables 开启相应端口 iptables -A INPUT -p tcp -s 0/0 --dport 80 -j ACCEPT iptables -A OUTPUT -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT

二、LVS主机安装Keepalived，ipvsadm 1. 安装依赖包

yum -y install wget popt popt-devel popt-static libnl libnl-devel kernel-devel; ln -s /usr/src/kernels/$(uname -r)/ /usr/src/linux;

2. 安装 Keepalived，ipvsadm（YUM安装或TarBall安装）

# YUM安装 Keepalived，ipvsadm yum -y install keepalived ipvsadm; # TarBall安装Keepalived wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz; tar xvf keepalived-1.2.15.tar.gz -C /usr/local/src/; cd /usr/local/src/keepalived-1.2.15/; ./configure \ --bindir=/usr/bin \ --sbindir=/usr/sbin \ --sysconf=/etc \ --with-kernel-dir=/usr/src/kernels/$(uname -r)/; make; make install; # TarBall安装ipvsadm wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz; tar xvf ipvsadm-1.26.tar.gz -C /usr/local/src/; cd /usr/local/src/ipvsadm-1.26/; make; make install;

3. 创建ipvsadm配置文件，启动并加入开机启动

/etc/init.d/ipvsadm save; /etc/init.d/ipvsadm start; chkconfig ipvsadm on;

4. 配置并启动keepalived

# 备份keepalived配置文件 cd /etc/keepalived/; [ -f "keepalived.conf" -a ! -f "keepalived.conf.default" ] && mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.default; [ -f "keepalived.conf" ] && mv keepalived.conf keepalived.conf.$(date +%F_%T); # 创建 Keepalived 配置文件（需先修改 virtual_ipaddress，virtual_server，real_server 配置的IP） cat >> /etc/keepalived/keepalived.conf <<KEEPALIVED ! Configuration File for keepalived global_defs { notification_email { mr@mruse.cn } notification_email_from xxx@163.com smtp_server smtp@163.com smtp_connect_timeout 30 router_id LVS_1 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 60 priority 100 advert_int 1 authentication { auth_type PASS auth_pass MrUse } virtual_ipaddress { 192.168.0.20 192.168.0.30 } } virtual_server 192.168.0.21 80 { delay_loop 3 lb_algo rr lb_kind DR persistence_timeout 3 protocol TCP real_server 192.168.0.22 80 { weight 3 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.0.23 80 { weight 3 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } } virtual_server 172.16.100.130 80 { delay_loop 3 lb_algo rr lb_kind DR persistence_timeout 3 protocol TCP real_server 192.168.0.31 80 { weight 3 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.0.32 80 { weight 3 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } } KEEPALIVED # 启动Keepalived并加入开机启动 /etc/init.d/keepalived restart; chkconfig keepalived on;

5. 记录Keepalived日志

# 注释Keepalived默认配置 grep ^KEEPALIVED_OPTIONS /etc/sysconfig/keepalived && sed -i 's#^KEEPALIVED_OPTIONS#\#KEEPALIVED_OPTIONS#g' /etc/sysconfig/keepalived; # 更新keepalived配置 echo 'KEEPALIVED_OPTIONS="-D -d -S 0"' >> /etc/sysconfig/keepalived; /etc/init.d/keepalived restart grep "# CmsTop Keepalived" /etc/rsyslog.conf||cat >> /etc/rsyslog.conf <<SYSLOG # CmsTop Keepalived $(date +%F) # keepalived -D -d -S 0 local0.* /var/log/keepalived.log SYSLOG /etc/init.d/rsyslog restart

6. 为真实服务器 WebServer, IMGServe创建配置启动脚本（需修改IP地址）

[ -f "/etc/init.d/lvsr" ] && mv /etc/init.d/lvsr /etc/init.d/lvsr.$(date +%F_%T) cat >> /etc/init.d/lvsr <<LVSR #!/bin/bash SNS_VIP=172.16.100.120 /etc/rc.d/init.d/functions start() { /sbin/ifconfig lo:0 \$SNS_VIP netmask 255.255.255.255 broadcast \$SNS_VIP up /sbin/route add -host \$SNS_VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "LVS RealServer Start OK" } stop() { /sbin/ifconfig lo:0 down /sbin/route del \$SNS_VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "LVS RealServer Stoped" } case "\$1" in start) start; ;; stop) stop; ;; restart) stop; start; ;; status) /sbin/ifconfig|tail -5 ;; *) echo "Usage: \$0 {start|stop|restart|status}" exit 1 esac exit 0 LVSR # 修改依赖文件及启动脚本权限并启动 chmod +x /etc/init.d/lvsr; chmod +x /etc/rc.d/init.d/functions; /etc/init.d/lvsr start # 加入开机启动 grep ^/etc/init.d/lvsr /etc/rc.local || echo '/etc/init.d/lvsr start' >> /etc/rc.local

三、查看状态、测试排错 1. 重启Keepalived，在LVS服务器上查看状态

/etc/init.d/keepalived restart; /etc/init.d/ipvsadm status;

2. 测试排查思路

# 查看配置文件，查看ip # 查看Iptables是否开启相应端口 # 查看网络，服务器间及与虚拟IP是否可以通信 # 查看Keepalived，ipvsadm是否正常启动 # 查看错误日志