Yellow Status on Lotus Traveler – What to do ?

Scenario:  Every time you connect to the traveler server and issue a “tell traveler status” command from the Domino console, you get a yellow status returned or more particularly

“The response times for opening databases on mail server X are above the acceptable threshold.”

Mail users on server “X”  are perfectly happy with how quickly messages get to their devices and you want to understand how this “threshold” works and  to confirm if it is doing what you want it to do.

The first thing to explain is that Traveler will go into yellow status if it take more than 2 (yes, that’s 2) seconds to open a mailfile on a remote server.  This is fine when all mail servers are on the same LAN and in the same location.  But what about actual remote servers that are in different locations? Maybe they are on a decent connection but  you may expect the remote mailfile to take longer than 2 seconds  to open as the norm.

The good news is that you can change this threshold to make your Traveler Server “Green” again !!! and here’s how:

The next thing to note is exactly how many seconds mailfiles on server X are taking to open.

You can do this by issuing a command “tell traveler stat show” from the domino console, this will return a list of results.
(as an aside you can make more sense out of these results from this link – http://www-10.lotus.com/ldd/dominowiki.nsf/dx/System_status_results_LNT853)

Look for DCA.DB_OPEN.Time.Histogram.CN=X/O=YourOrg (where Server X is X/YourOrg)

You’ll see a list of results that look like this
DCA.DB_OPEN.Time.HistogramCN=X/O=YourOrg.000-001 = 10
DCA.DB_OPEN.Time.Histogram.CN=CN=X/O=YourOrg.001-002 = 20
DCA.DB_OPEN.Time.Histogram.CN=CN=X/O=YourOrg.002-005 = 1

In the above example it’s saying is that in the last fifteen minutes  (the default interval for status checks) 10 times it took less than one second to open a remote mailfile on Server X, 20 times it took 1 second to 2 seconds, and once it took between 2 and 5 seconds.  (you may see more results if it took longer than 5 seconds).

You may want to do this a few times over the course of a day or week to see are the times consistent.

Let’s continue the scenario and say your remote server is in another country or even continent and you wouldn’t be shocked if it took up to a minute to open the mailfile.  You’re happy to accept that this is a network restriction and you shouldn’t be constantly alerted about it i.e. you want to change the threshold to something more reasonable.

For this threshold the following “interval buckets” values apply:

1 – “000-001” seconds to open a mailfile on a remote server.
2 – “001-002” seconds to open a mailfile on a remote server.
3 – “002-005” seconds to open a mailfile on a remote server.
4 – “005-010” seconds to open a mailfile on a remote server.
5 – “010-030” seconds to open a mailfile on a remote server.
6 – “030-060” seconds to open a mailfile on a remote server.
7 – “060-120” seconds to open a mailfile on a remote server.
8 – “120-Inf” seconds to open a mailfile on a remote server.

The default for a yellow status is a value 2 (so anything above 001-002 seconds) and the default for red is 8 (so anything from 120 seconds upwards).

We’ll give an example that from looking at the “tell traveler stat show” over the course of a week, you don’t want the traveler server to go into yellow status unless it takes over a minute to open a remote mailfile.  To do this your status value will need to equal 6 (“030-060″ seconds to open a mailfile on a remote server).

In order to apply this setting you need to modify the NTSConfig.xml file (located in <domino data directory>\traveler\cfg\ on the Traveler server) to include the following line:
<PROPERTY NAME=”STATUS_DB_OPEN_INTERVAL_YELLOW” VALUE=”6″/>
and on a new line below the following line
<COMPONENT COMPONENT_TYPE=”TrueSyncServer”>

Followed by issuing a “tell traveler quit“, followed by a “load traveler” from the Domino console.

You can see if the setting has been applied correctly by doing a “tell traveler systemdump“,  go to <domino data directory>\IBM_TECHNICAL_SUPPORT\traveler\logs\dumps\ and look for the latest system dump. Open the file and search for “—– adminForTSS —–“, and then for  STATUS_DB_OPEN_INTERVAL_YELLOW” below that, the value that is applied here is the one applied to the server (ie if it hasn’t changed then this value will equal the default of 2)

Obviously the status of the server will have changed as well (assuming there are no other yellow status’).

The next step from here from IBM’s point of view would be if you could set individual thresholds for individual servers (maybe even in the server document)!!

You can apply similar changes to other thresholds in the same way, more details on the status messages is here – http://www-10.lotus.com/ldd/dominowiki.nsf/dx/Status_command_details_LNT8521

Much kudos to Cormac McCarthy, one of our Senior Admins for taking the time for writing up his findings to share.