Working with Views outside of .nsfs

Ok, so it’s been a long time coming.  Been promised and “in the pipeline” for a long time.  Well with Domino 9.0.1. Feature Pack 8 we can now move views outside of .nsfs.

Why would you want to do this?  Well the two primary reasons:

1.  For potential performance improvements on large databases.  This is a simple statement that assumes much.  Generally it assumes you can re-point the views (to another dedicated physical disk, or dedicated LUN), though there would generally be performance improvements even if you didn’t.  There may be many other reasons why your database/application isn’t working as well as it should.  Contact us and we can set up a review for you.

2. To improve back up speeds on weekly full backups.

I say weekly backups explicitly here, as unless you’ve a really small footprint of disk, (or have some other compelling reason) you should be ONLY backing up archive based transaction logs, every day.  Please again contact us if you are not.  It is relatively straight forward to implement and it will improve the amount of time backups take (As well as allowing point in time recovery).

So how do you implement views outside of indexes?

There’s a couple of simple prerequistes. Install Domino 9.0.1 Feature Pack 8 (if it’s a Traveler server or Verse on premise server note the known issues), and make sure the ODS is up to date (ODS52).

Beyond that there’s only two real steps to implement:

1. Add the following notes.ini settings
NIFNSFEnable=1 is essential

NIFBasePath=path is optional (This is where the view files will be stored, by default it’s similar to full text indexes, i.e. the same location as the .nsf, but I changed it to NIFBasePath=d:\views\ for the sake of testing the setting).

(if you want to set the views to go outside ALL files by default then you can set Create_NIFNSF_Databases=1).

Then restart the server.

2. You can then apply it by issuing

load compact -c -nifnsf on path

So in practice

load compact -c -nifnsf on mail\cmccarth.nsf

Physical Size Before:

Physical Size After:

The .ndx file is created on the drive/view specified:

If you right click and manage views in the admin client, it looks exactly the same as before.  The performance on my mailfile was perfect before, and it’s perfect now, so it’s probably not the ultimate test.  (i.e. a huge database that currently has performance issues would be good to test with). But it works fine and is easy to implement.

To undo it’s just

load compact -c -nifnsf off mail\cmccarth.nsf

Posted in Backups, IBM Domino 9, Transaction logs, Verse on premise, Views | Leave a comment

Orphaned Devices on Traveler HA SQL

Hi all,

Had a recent issue with a customer using Traveler 9.0.1.4 on Domino 9.0.1.4 where one user had three devices listed all with the same device ID (so essentially the same device). All devices were inactive and we wanted to clear them up. So, I issued the usual console commands:

1. Tell Traveler Delete * <username>
2. Tell Traveler Security Delete * <username>

but Traveler kept saying that no devices were found for the user. Issuing Tell Traveler User <username> confirmed that there were indeed no devices associated with the user. However, in the back-end SQL database i.e. https://lotustraveler.nsf the devices were still showing !

So that means that devices still have records within the SQL database itself and so I approached an SQL DBA (after reading an IBM technote suggesting this) and there were hundreds of tables in the Traveler SQL database so that line of investigation ended right there.

IBM came to the rescue though with a couple of very handy commands which my good friend Google search could not find which is why I’m sharing this with you:

To manually remove the orphaned users and devices from the SQL db, you
may execute the following command:
.
tell traveler sql delete * from USERS where username=’fully qualified name’
Note: The name has to be within single quote.
.
tell traveler sql delete * from DEVICE where username=’fully qualified name’
Note: The name has to be within single quote.

So as an example:

tell traveler sql delete * from USERS where username=’CN=Jo Bloggs/OU=Sales/O=Acme’

tell traveler sql delete * from DEVICE where username=’CN=Jo Bloggs/OU=Sales/O=Acme’

And there you have it…

Posted in IBM Notes Traveler, IBM Notes Traveler 9, Lotus Traveler, Traveler 9 | 1 Comment

Working with SHA-2 in Domino

Any production web site on Domino will have been flagged for months (if not years) on lack of support for SHA-2.  Since the poodle scares last year, a lot of third party certifiers made the decision to no longer support SHA-1 certs, just in time for Domino to begin supporting SHA-2 (phew).

So by now you’ll probably be aware of the prerequistes

and the steps from IBM to a point are actually very clear. (though it does make sense for it to be packaged in the server certificate admin tool soon – please IBM)

There’s two steps that need to be different though.  The first is the same with any third party supplier of certs and that’s getting the correct root and intermediate certs for your SSL cert.  That’s a problem independent of Domino though and anyone who has installed a cert previously will have done similar to this.

The second issue is the step:

“6. Import the RSA keypair and self-signed certificate into the new keyring file

6a. Concatenate server.key and server.pem into a single file:

This step varies from the self-signed case. You will have more than one certificate in your “.pem” file, and will want to place them in order with your server’s SSL “leaf” certificate first and the root certificate last. Verify step 6b will check to ensure that the ordering is correct. If it returns any warnings or errors, edit the PEM file and verify it again.

Note the following:

  • Certificate Authorities will frequently return a signed certificate in a .crt file. If they also provide the root certificates when returning the CSR file, then you can concatenate all of the .crt files to the private key by using the “type” command from a DOS prompt.
  • The files should be concatenated with the server key first, the server’s cert next, the intermediate cert next, and the root cert last. Concatenation can be done from a DOS prompt using the TYPE command. The type command takes a list of files, and appends them together into an output file designated with a greater-than symbol. For example, type server.key server.crt intermediate.crt root.crt > server.txt In this example “server.txt” is the file provided to the kyrtool for import into a Domino keyring. You can display this output file in Notepad.
  • If the root and intermediate certs are not provided with the signed certificate, export the intermediate and root certificates by opening the server certificate with Windows Crypto Extensions. This will display the server in a three-tabbed user interface. On the third tab, select each of the signing certificates, select display, and then export that certificate using the “save to file” command on the second tab. Save each cert file using Base 64 format.

I’ve had continious issues with concatenating certs using type server.key server.crt intermediate.crt root.crt > server.txt

The last cert usually gets cut off (you can see when you open server.txt)., but it’s not immediately obvious and you get various errors saying the chain is invalid if you go any further.

The simple workaround is rather than concatenating all the certs in one step, simply do them one by one.

i.e

type server.key >key.txt

type server.crt >sslcert.txt

type intermediate.crt > intermediate.txt

type root.crt > root.txt

Then create a new text file in notepad called server.txt, and manually copy the contents of key.txt, sslcert.txt, intermediate.txt and root.txt in one by one.  The rest of the steps should now work for you.

Let us know if that helps or if you’ve any more gotchas!

Cormac McCarthy – Domino People Ltd

 

Posted in HTTP, HTTPS, IBM Domino 9, Poodle, SHA-1, SHA-2 | Leave a comment

Notes 9 and SmartCloud Activities – Heuston we have a…

Bit of a moan here.

I just upgraded from Notes R8.5.3 to R9.0.1 FP1. There are a lot a features that I really like in R9.x but something is REALLY bugging me.

In R8.5.3 I was an avid user of SmartCloud Activities in the sidebar – it worked really well.

In R9.0.1 “Social Edition”, you would have expected it to be truly social. But no. SmartCloud Activities just will not work. I understand that Connections Activities do but why the regression of functionality? Why Connections but no SmartCloud – it’s bizarre.

I’ve searched technotes and nearly every page on the whole Internet to look for a solution to this without luck.

If anyone can tell me how to get this working, my gratitude will be eternal as it’s like an old friend has left me….

Thanks,

Steven.

Posted in Uncategorized | Leave a comment

It’s Friday ! – Undocumented Notes Table Shortcut you will love

I’ve never seen this before and a colleague came across this brilliant shortcut by accident.

Here’s how it works (in Notes 8 and 9 anyway):

1. Create a table in Notes (or use an existing table)

2. Hold the shift key and hover the mouse pointer over a row line (not the top one)

3. The cursor will change to a thick black line with a little blue diamond above it

4. Double click while still holding down shift

5. A new row appears !!!

This also works for columns as well. I think it’s great but that might be just me.

Enjoy,

Steven.

Posted in IBM Notes 9, Lotus Notes Client | 7 Comments

Unread marks auditing plus Design Dumps…

Every Domino Admin worth their salt will have experienced  incidents with unread marks not working correctly and will have “fixed” these in variety of ways, mainly either marking all documents as read, or exchanging unread marks between replicas

There’s another type of unread marks issue, where someone wants to audit unread marks (i.e. find out who has read a document and when).  Well IBM can provide with a free auditing tool that does just that (amongst other useful Developer type functions).

From the about page of application:

“Welcome to the Toolbox Database

Introduction

The Toolbox database (toolbox.nsf) is designed to help administrators or advanced users troubleshoot issues with Unread Marks in a Lotus® Notes® database.  It can also be used for troubleshooting replication issues and design template issues.  Currently there are six major features in the Toolbox database. These are as follows:

  • Dump design documents: This feature enables users to dump information about all design documents in the database. Toolbox provides information on these attributes: NoteID, UNID, Sequence Time, Sequence, and UpdatedBy.  The output is sorted by Class Name and UNID.  This feature enable users to quickly identify who updated design documents, when they where updated, and how many times the documents were updated.
  • Dump design documents (long format) : This feature is the same as “Dump design documents” but provides more detailed information about each design document.  Additional information includes the document Title, Creation Time, and Last Modification Time.
  • Dump the NoteID and UNID :  This feature dumps the NoteID, UNID, Create time and Last Modified time for a database, database view, or single document to a log file. This log enables the user to take a quick inventory of the documents in the database or view. Then, when unread marks or replication issues are suspected, the user can dump the NoteID and UNID log again and compare the two logs to identify which document was changed.
  • Dump unread activity logs (local db only) :  This feature dumps the user’s unread activity log for a database, database view, or a single document to a log file. This feature works only on a local database that is not encrypted.  Also, the database must be enabled for unread replication.
  • Dump unread mark list : This feature dumps the unread mark list for a database, database view, or a single document into a log file. This feature works for any accessible local or remote database. This feature works only on a database that is not encrypted.  Also, the database must be enabled for unread replication.
  • Toolbox Add-in Menu : This feature adds the following Toolbox menu items to the Actions menu:  “Display info on selected Document”, “Dump design documents”, “Dump design documents (long format)”. These menus are enabled whenever the user is in a view to execute the action.”

There isn’t an Internet link per se for for the tool, but if you ask IBM Support nicely for it in a PMR they will provide it for you and it is intuitive to use.  IBM should probably have a technote on this as I think people would use it if they knew about it and it was freely downloadable (IBM take note !).

Please let us know if you end up using the application as a result of this post!

Cormac McCarthy – Domino People Ltd.

 

Posted in Domino Designer, Domino Server, IBM Domino 9, IBM_TECHNICAL_SUPPORT, Lotus Notes Diagnostic | Leave a comment

Traveler 9/Apple issue – “mails from other people appear to come from me”

An issue was reported by multiple users on a customer site recently.   Since a Traveler 9.0.1 upgrade :- mails coming into user’s inbox that are from external Internet senders are showing up in the “who” column as being from the recipient’s name rather than the sender. This is only for some mails though, not all.

The first steps taken were checking the SMTP headers to see had the message been modified by the mail gateway (McAfee).  Everything looked normal.  Checked the Domino logs, nothing unusual there.

Eventually we figured out, the common linkage was that the messages effected were last modified by the Traveler server.  The issue affected Traveler 9 users (who are all Apple iPhone/iPad users on this site) setting follow up flags from their devices.  Once a follow up flag is set on an external sender mail, then the “Who” column displays the person who received the mail rather than the sender in the Notes client (It remains fine on the iPhone/iPad however).

After opening a PMR with IBM, they  didn’t offer an immediate solution but started using the term “principal” field and have escalated to development (at the time of this writing we are still waiting to hear back) we managed to search online ourselves and find this -

http://www-01.ibm.com/support/docview.wss?uid=swg1LO77840

I have to say the title “SET AS FOLLOW UP MAY SET PRINCIPAL TO RECIPIENT” isn’t intatuively understood, but when we read :-

“If user flags a mail for follow up on a Mobile device and the message does not have the standard Principal field, Notes Traveler will mistakenly set the user as principal instead of the sender. This problem is only seen with the 9.0.1 Gold release.”

We understood the issue was exactly the same as ours, though it wasn’t exactly easy to find.  The issue is fixed in 9.0.1 IFI 1.  We’ve installed 9.0.1 IFI2 and the issue was resolved. Implementing the fix does not change back mails that were previously “changed” by the Traveler server.

Quite a simple fix, but even IBM’s own support couldn’t tell us the documented fix!

Another example of the good practice of ensuring that your Traveler server has the latest fix packs and interim fixes applied.

Hope this helps someone else out!  Let us know if it does.

Cormac McCarthy – Domino People Ltd

Posted in Fix Pack, IBM Domino 9, IBM Notes Traveler, IBM Notes Traveler 9, IBM_TECHNICAL_SUPPORT, iPhone, Lotus Traveler, Traveler 9, Traveler 9.0 | Leave a comment

Domino 9 – DBMT in program documents

The Database Maintenance Tool in R9, promises an awful lot; essentially, that it does everything compact, fixup and updall does, with more options including:

  • controls on number of threads for each task
  • conditional options such as running a fixup if compact doesn’t run a certain number of times
  • all from one single command/program document.

However, I’ve come across this on a couple of sites.  Admin is “trying out” new R9 features.  Admin reads help files for DBMT and gets really excited.  Admin tries a load DBMT command, it works, Admin tries to to turn on DBMT via program documents.  Admin gets “invalid syntax error”.  Admin thinks it doesn’t work and continues to compact, fixup updall individually via program documents as before.

The issue here is that DBMT will only work in a Program Document when it’s set to run “At Server Startup Only”.  i.e. the “range” switch is used to set which times it runs. It does say this in the help file, but it doesn’t highlight it particularly well, and isn’t particularly intuitive.

For all those who’ve been caught in this scenario here’s an example that works:

 

(runs with 4 compact threads, 4 updall threads from 3:00am to 7:00am, it will run a fix up on Sunday if there hasn’t been a valid compact for 5 days for more on the different switch options see here).

When the server starts you’ll see this task, it gives you the details of the current DBMT task:-

(what the DBMT program document was set to before server before it was restarted)

Note if you do not want the next DBMT to run, you can issue a “tell ndbmt quit” to the Domino Console.

Then when it’s in the time specified in range, you’ll see the threads under server tasks:

And when it’s run successfully, you’ll see the following statistics the console/log:

DBMT.Compact.Began
DBMT.Compact.Finished
DBMT.Compact.Successful
DBMT.Compact.Unsuccessful
DBMT.Compact.Unsuccessful.InUse
DBMT.Compact.Unsuccessful.TimeLimit
DBMT.Compact.BackLog
DBMT.Compact.Fixup
DBMT.Updall.Began
DBMT.Updall.Finished
DBMT.Updall.Processed

Let us know if you found this useful.

Cormac McCarthy – Domino People Ltd

 

 

Posted in DBMT, IBM Domino 9 | 4 Comments

FLASH: Some users stop receiving updates after upgrade to Notes Traveler 8.5.3 UP2 IF3 (2013.07.23)

IBM have sent out an update today, about 8.5.3 UP2 IF3 where users stop receiving mail.

In short the fix is to upgrade to 8.5.3 UP2 IF4.

The details are documented here

Cormac McCarthy – Domino People Ltd

 

Posted in Fix Pack, Lotus Traveler | Leave a comment

Domino Fix Packs on Virtual Windows 2008 R2 (Windows Instrumentation Service problems)

This is an addendum to an earlier blog post  but I found it so common on sites this year it’s worth it’s own blog post.

As per my earlier post, it is often “difficult” to install 8.5.3 fix packs on Windows 2008 R2,  (there’s a couple of work arounds already listed, namely “run as administrator”, uninstalling previous fix packs and disabling SNMP, coincidently another one I’d add is disabling UAC).

But the main and most consistent culprit I’ve found (on Virtual Servers anyway),  is Windows Instrumentation Service (WmiPrvSE.exe).

It is documented here in SPR GFAL95TK9X  which has been closed due to ”The problem was caused by a user error or user misunderstanding.”

If you are having this problem please open a PMR and add your instance to this SPR.

Let us know if you found this useful or if you’ve found any other “gotchas” with Windows 2008 R2.

Cormac McCarthy – Domino People Ltd

Posted in Domino Server, Fix Packs, SNMP, UAC, Virtual Server, Windows, Windows Instrumentation Service, Windows Server, Windows Server 2008 R2 | Leave a comment