The following example shows a configuration for non-sticky sessions. In this case there's no need for failoverNodes, as sessions are served by all tomcats round-robin and they're not bound to a single tomcat. For non-sticky sessions the configuration (for both/all tomcats) would look like this:
<Context> ... <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:host1.yourdomain.com:11211,n2:host2.yourdomain.com:11211" sticky="false" sessionBackupAsync="false" lockingMode="uriPattern:/path1|/path2" requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" /> </Context> 分别启动tomcat-1和tomcat-2,如下所示: [plain] view plain copy shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tomcat-1/bin/catalina.sh start Using CATALINA_BASE: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1 Using CATALINA_HOME: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1 Using CATALINA_TMPDIR: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/temp Using JRE_HOME: /home/hadoop/installation/jdk1.6.0_30 Using CLASSPATH: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/bin/bootstrap.jar:/home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/bin/tomcat-juli.jar shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tomcat-2/bin/catalina.sh start Using CATALINA_BASE: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2 Using CATALINA_HOME: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2 Using CATALINA_TMPDIR: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/temp Using JRE_HOME: /home/hadoop/installation/jdk1.6.0_30 Using CLASSPATH: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/bin/bootstrap.jar:/home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/bin/tomcat-juli.jar 可以查看两个tomcat的启动日志,对比一下: tomcat-1的日志 [plain] view plain copy shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-1/logs/catalina.out Jan 26, 2013 12:03:25 AM org.apache.catalina.core.AprLifecycleListener init INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init INFO: Initializing ProtocolHandler ["http-bio-8088"] Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init INFO: Initializing ProtocolHandler ["ajp-bio-8089"] Jan 26, 2013 12:03:26 AM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 1743 ms Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardService startInternal INFO: Starting service Catalina Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardEngine startInternal INFO: Starting Servlet Engine: Apache Tomcat/7.0.26 Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n1) 2013-01-26 00:03:27.441 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue 2013-01-26 00:03:27.442 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.RequestTrackingHostValve <init> INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$ 2013-01-26 00:03:27.459 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@5b0668 2013-01-26 00:03:27.460 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0 Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode INFO: Setting lockingMode to null Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n2] and failover node ids [n1] Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/host-manager Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/manager Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/docs Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/examples Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/ROOT Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start INFO: Starting ProtocolHandler ["http-bio-8088"] Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start INFO: Starting ProtocolHandler ["ajp-bio-8089"] Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.Catalina start INFO: Server startup in 1487 ms tomcat-2的日志 [plain] view plain copy shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-2/logs/catalina.out Jan 26, 2013 12:04:34 AM org.apache.catalina.core.AprLifecycleListener init INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init INFO: Initializing ProtocolHandler ["http-bio-9088"] Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init INFO: Initializing ProtocolHandler ["ajp-bio-9089"] Jan 26, 2013 12:04:34 AM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 742 ms Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardService startInternal INFO: Starting service Catalina Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardEngine startInternal INFO: Starting Servlet Engine: Apache Tomcat/7.0.26 Jan 26, 2013 12:05:07 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [33,094] milliseconds. Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n2) 2013-01-26 00:05:07.873 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue 2013-01-26 00:05:07.876 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.RequestTrackingHostValve <init> INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$ 2013-01-26 00:05:07.896 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@701a27 2013-01-26 00:05:07.897 INFO net.spy.memcached.MemcachedConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0 Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode INFO: Setting lockingMode to null Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n1] and failover node ids [n2] Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/host-manager Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/manager Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/docs Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/examples Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/ROOT Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start INFO: Starting ProtocolHandler ["http-bio-9088"] Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start INFO: Starting ProtocolHandler ["ajp-bio-9089"] Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.Catalina start INFO: Server startup in 34163 ms 已经成功启动了。 Nginx安装、配置、启动 在Ubuntu系统下,如果没有安装Nginx,可以执行如下命令进行安装: [plain] view plain copy sudo apt-get install nginx 默认情况下,Nginx使用默认的配置文件/etc/nginx/nginx.conf,该配置文件直接包含了虚拟目录配置文件/etc/nginx/sites-available/default,修改该文件的内容为,如下所示: [plain] view plain copy # You may add here your # server { # ... # } # statements for each of your virtual hosts to this file ## # You should look at the following URL's in order to grasp a solid understanding # of Nginx configuration files in order to fully unleash the power of Nginx. # http://wiki.nginx.org/Pitfalls # http://wiki.nginx.org/QuickStart # http://wiki.nginx.org/Configuration # # Generally, you will want to move this file somewhere, and start with a clean # file but keep this around for reference. Or just disable in sites-enabled. # # Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples. ## upstream dev.shirdrn.org { server 192.168.1.104:8088 weight=1; server 192.168.1.104:9088 weight=1; } server { #listen 80; ## listen for ipv4; this line is default and implied #listen [::]:80 default ipv6only=on; ## listen for ipv6 root /usr/share/nginx/www/session; # session是我们配置的虚拟目录,实际直接指向Tomcat下名为session的web应用 index index.html index.htm; # Make site accessible from http://localhost/ ####### server_name localhost; server_name dev.shirdrn.org; # Nginx服务所在主机 charset utf-8; location / { proxy_pass http://dev.shirdrn.org; #直接代理tomcat集群 proxy_set_header X-Real-IP $remote_addr; client_max_body_size 100m; # First attempt to serve request as file, then # as directory, then fall back to index.html ####### try_files $uri $uri/ /index.html; # Uncomment to enable naxsi on this location # include /etc/nginx/naxsi.rules } location ~ ^/(WEB-INF)/ { # 禁止访问Tomcat下web应用的WEB-INF目录下的资源 deny all; } location /doc/ { alias /usr/share/doc/; autoindex on; allow 127.0.0.1; deny all; } # Only for nginx-naxsi : process denied requests #location /RequestDenied { # For example, return an error code #return 418; #} #error_page 404 /404.html; # redirect server error pages to the static page /50x.html # #error_page 500 502 503 504 /50x.html; #location = /50x.html { # root /usr/share/nginx/www; #} # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 # #location ~ \.php$ { # fastcgi_split_path_info ^(.+\.php)(/.+)$; # # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini # # # With php5-cgi alone: # fastcgi_pass 127.0.0.1:9000; # # With php5-fpm: # fastcgi_pass unix:/var/run/php5-fpm.sock; # fastcgi_index index.php; # include fastcgi_params; #} # deny access to .htaccess files, if Apache's document root # concurs with nginx's one # #location ~ /\.ht { # deny all; #} } # another virtual host using mix of IP-, name-, and port-based configuration # #server { # listen 8000; # listen somename:8080; # server_name somename alias another.alias; # root html; # index index.html index.htm; # # location / { # try_files $uri $uri/ /index.html; # } #} # HTTPS server # #server { # listen 443; # server_name localhost; # # root html; # index index.html index.htm; # # ssl on; # ssl_certificate cert.pem; # ssl_certificate_key cert.key; # # ssl_session_timeout 5m; # # ssl_protocols SSLv3 TLSv1; # ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP; # ssl_prefer_server_ciphers on; # # location / { # try_files $uri $uri/ /index.html; # } #} 上面配置中根目录为/usr/share/nginx/www,因为我们使用到了一个虚拟目录来对应Tomcat下部署的web应用的path,所以需要在/usr/share/nginx/www下面创建这个对应的虚拟目录: [plain] view plain copy shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo mkdir /usr/share/nginx/www/session 最后,可以启动Nginx服务器,执行如下命令: [plain] view plain copy shirdrn@dev:~$ sudo /etc/init.d/nginx start 查询Nginx服务启动状态: [plain] view plain copy shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo ps -ef | grep nginx root 3199 1 0 Jan25 ? 00:00:00 nginx: master process /usr/sbin/nginx www-data 3200 3199 0 Jan25 ? 00:00:00 nginx: worker process www-data 3201 3199 0 Jan25 ? 00:00:00 nginx: worker process www-data 3202 3199 0 Jan25 ? 00:00:00 nginx: worker process www-data 3203 3199 0 Jan25 ? 00:00:00 nginx: worker process shirdrn 3327 3095 0 00:01 pts/5 00:00:00 grep --color=auto nginx 这是就可以通过Nginx访问(代理)Web应用首页了。 集群验证 上面配置完成,服务都已经启动成功,可以直接请求链接 http://dev.shirdrn.org/session,就会直接访问部署在Tomcat服务器上的Java Web应用,看到登录页面。 使用前面准备的Web应用session中账号数据进行模拟登录,就可以查看Memcached缓存的数据: telnet 192.168.1.104 11211 telnet到memcached服务端口,查看缓存统计数据。 [plain] view plain copy shirdrn@dev:~$ telnet 192.168.1.104 11211 Trying 192.168.1.104... Connected to 192.168.1.104. Escape character is '^]'. ERROR stats STAT pid 3300 STAT uptime 2148 STAT time 1359131741 STAT version 1.4.13 STAT libevent 2.0.16-stable STAT pointer_size 32 STAT rusage_user 0.052003 STAT rusage_system 0.084005 STAT curr_connections 7 STAT total_connections 9 STAT connection_structures 8 STAT reserved_fds 20 STAT cmd_get 23 STAT cmd_set 6 STAT cmd_flush 0 STAT cmd_touch 0 STAT get_hits 2 STAT get_misses 21 STAT delete_misses 0 STAT delete_hits 0 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT touch_hits 0 STAT touch_misses 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 2675 STAT bytes_written 2835 STAT limit_maxbytes 67108864 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT hash_power_level 16 STAT hash_bytes 262144 STAT hash_is_expanding 0 STAT expired_unfetched 0 STAT evicted_unfetched 0 STAT bytes 858 STAT curr_items 2 STAT total_items 6 STAT evictions 0 STAT reclaimed 0 END telnet 192.168.1.104 11311 [plain] view plain copy shirdrn@dev:~$ telnet 192.168.1.104 11311 Trying 192.168.1.104... Connected to 192.168.1.104. Escape character is '^]'. stats STAT pid 3308 STAT uptime 2398 STAT time 1359132000 STAT version 1.4.13 STAT libevent 2.0.16-stable STAT pointer_size 32 STAT rusage_user 0.004000 STAT rusage_system 0.128008 STAT curr_connections 7 STAT total_connections 9 STAT connection_structures 8 STAT reserved_fds 20 STAT cmd_get 1 STAT cmd_set 2 STAT cmd_flush 0 STAT cmd_touch 0 STAT get_hits 0 STAT get_misses 1 STAT delete_misses 1 STAT delete_hits 0 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT touch_hits 0 STAT touch_misses 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 644 STAT bytes_written 2109 STAT limit_maxbytes 67108864 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT hash_power_level 16 STAT hash_bytes 262144 STAT hash_is_expanding 0 STAT expired_unfetched 1 STAT evicted_unfetched 0 STAT bytes 0 STAT curr_items 0 STAT total_items 2 STAT evictions 0 STAT reclaimed 1 END 我们可以停掉某个Tomcat服务,然后观察Session会话数据的复制过程,能够保证当前集群中的会话数据不丢失,另一个结点仍然能提供基于该存在Session的服务。 参考链接 https://code.google.com/p/memcached-session-managerhttps://code.google.com/p/memcached-session-manager/wiki/SetupAndConfigurationhttp://code.google.com/p/kryo/http://wiki.ubuntu.org.cn/Nginxhttp://www.iteye.com/topic/676347