Many customers I work with have Lotus Quickr 8.2 Services for Domino clusters, fronted by IP Sprayer technology for load-balancing and failover.  This solution works well for availability and performance purposes, but takes quite a lot of configuration.  

As usual, the Infocenter gives most of the information needed, but doesn't provide much context as to how or why these steps are necessary, nor indeed what to do if things go wrong.

An example of this occurred this morning, when one cluster showed signs of poor performance, rendering of blank pages after login, and lots of errors in the server consoles, along the lines of:

HTTP JVM: java.net.ConnectException: Connection timed out: connect
HTTP JVM: at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
HTTP JVM: at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:59)
HTTP JVM: at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:41)
HTTP JVM: at java.lang.reflect.Constructor.newInstance(Constructor.java:528)
HTTP JVM: at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1305)
HTTP JVM: at java.security.AccessController.doPrivileged(AccessController.java:251)
HTTP JVM: at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1299)
HTTP JVM: at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:953)
HTTP JVM: at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:386)
HTTP JVM: at com.lotus.quickplace.util.QuickrSupportUtil.doPost(QuickrSupportUtil.java:270)
HTTP JVM: at com.lotus.quickplace.util.QuickrSupportUtil.doGet(QuickrSupportUtil.java:133)
HTTP JVM: at javax.servlet.http.HttpServlet.service(HttpServlet.java:499)
HTTP JVM: at javax.servlet.http.HttpServlet.service(HttpServlet.java:588)
HTTP JVM: at lotus.domino.servlet.DominoServletInvoker.executeServlet(DominoServletInvoker.java:296)
HTTP JVM: at lotus.domino.servlet.DominoServletInvoker.service(DominoServletInvoker.java:237)
HTTP JVM: at lotus.domino.servlet.ServletManager.service(ServletManager.java:258)
HTTP JVM: Caused by:
HTTP JVM: java.net.ConnectException: Connection timed out: connect
Firebug was also throwing up some very odd Dojo and Javascript errors.

By checking the output of NetStat commands, it appeared that the Quickr/Domino servers were trying to talk directly to the IP Sprayer to retrieve content (assumed to be CSS/javascript bundles), and the firewalls present in this environment were preventing this from occurring.

We tried opening the ports between the servers and the IP Sprayer, which improved the situation a little but performance was still poor and the errors occurred (though less frequently).

The next tactic was to add entries in the hosts files on the Quickr servers so that the FQDN for the IP Sprayer resolved to localhost, thus forcing lookups to be serviced by the local Quickr Domino server:
127.0.0.1  localhost    quickr.myorg.com

After restarting both servers, this worked a treat, with the errors no longer littering the server console, and performance being back to expected levels.  Hopefully, this solution will be of use to someone someday!



By: Stuart McIntyre | 4 Comments | On: 8 July 2010 13:10:26 | Tags:  quickr  domino  8.2 

Opus Neo Ad


Comments

1) my question
Ruby 7/13/10 6:01:34

Hi, Is there any reason why Quickr trying to get the JS scripts from Load Balancer instead of from itself? per the blog entry you gave, is it because of poor configuration on Quickr side? If yes, then what would be needed to perform on Quickr configuration to fix this issue?

2) Good question
Stuart McIntyre 7/13/10 8:53:34

I wish I knew why Quickr is trying to make this connection - it could of course be a configuration issue in this environment, but our checks have suggested that this is not the case.

This is a workaround that worked for us, if you can find a better resolution then we'd love to know the answer!

3) respond this post
GREENE24Alicia 8/15/10 1:04:00

Cars and houses are quite expensive and not everybody is able to buy it. Nevertheless, <a href="{ Link } loans</a> was created to aid people in such kind of situations.

4) respond this topic
buy term paper 8/22/10 11:38:50

I would never have opined the buy paper service could be able to assist me with my essays term papers creating. But, I changed my mind just after ordered the custom term papers. Hence I did this usually from that time.



Add a comment

Subject:
   
Name:
E-mail:
Web Site:
 
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?