Are your Lotus Notes databases bloated with view indexes? Here’s a trick to help you cut file sizes
This post recounts a database optimization trick and Lotus Notes secret that was shared by Eric Houvenaghel at this year’s Lotusphere (Hat tip: it was first suggested to us by our friend Alexandre Denis):
When NSF files start looking a little flabby
Have you ever seen a Lotus Notes application grow mysteriously large in size? And have you ever tried copying one of these bloated databases in Notes and found the copy to be significantly smaller?
These things are tell-tale signs of an application with too many views and, by extension, too many bloat-creating view indexes.
In theory this is an easy problem to deal with: just delete the views that end-users are no longer using. But the tricky bit is knowing which views are safe to remove, as no statistics are available to tell us which views are being used and which are not.
So who used that view? A simple test to perform
The following steps can be quite helpful in determining which views aren’t getting used. After finishing these steps, you can use this information trim down your databases intelligently:
1) Make sure the View Index Refresh option is set to “Auto, after first use” for all the views in question. You can do this in either Domino Designer or viewEZ (viewEZ offers the advantage of letting you modify this property on many views at the same time)
2) Now purge all view indexes. You can do this in viewEZ by clicking Actions>Purge Index for Selected View(s) or in your Domino Administrator client by right-clicking a database and choosing ‘Manage Views‘).
viewEZ lets you select numerous views and purge their indexes all at once
3) Next you’ll need to wait a reasonable amount of time (perhaps a few weeks), then go to the Administrator client and revisit the Manage Views dialog box (this dialog gives you the size of all your view indexes) and look to see which views still have no index built—these are the ones that haven’t been used. So now you can delete these views with a bit more confidence.
Some caveats and other considerations
This trick for finding unused views is only designed to help you narrow the search. It is neither a panacea nor is it entirely foolproof, so please read the notes below before you start gutting views from your applications.
- Rarely used views can still be extremely important (e.g. annual report views) so you’ve really got to use your common sense before you go and delete things.
- It goes without saying that you should have a backup of the design of any view that you delete.
- Furthermore, this method doesn’t always show you the views that haven’t been used; view indexes can sometimes be needlessly triggered by server tasks or Lotuscript calls. And remember that if anyone with sufficient access clicks Ctrl + F9 on a database that you’re testing, all the view indexes will be rebuilt regardless of whether or not the views have been used. Any of these things could really mess up your test by giving a false impression of a view’s value to your end-users.
- If the views you’re testing are big, the first time a user opens one during the test period he or she might be in for a seriously unpleasant delay.
- If the “Discard: If inactive for 45 days” property was already set, you could use whether or not a view has an index as an indicator of its popularity rather than going through the trouble of performing the test in this post.
Another caveat: if your database is largish your users will kill you for purging the views. 😉 The wait introduced by forcing them to wait for the first view update is not a very nice thing to do to users and the procedure can seriously disturb the servers for hours the first morning after purging.
Besides, this doesn’t show you how many users are using a given view. If you have 2000 users and just one of them uses a specific view, that view creates nearly as much load on the server as the other used views and should be marked for deletion unless that user has no other options or is buying donuts for the admin team. Or is the CEO.
Then again, having many users in few views in a large database is a recipe for disaster. This is bound to cause semaphore locks and flatline all clients and the server at 0% throughput. Spread out the activity across many views if at all possible. Many databases is much better if you can do it.
The best option for reducing overall database size is to ensure that “Discard if unused” is set properly. Turn the number of days before purging down if you can. This will let Domino decide what to do based on the real usage pattern. Check your use cases first though, someone using a view for monthly reports is not going to be happy if s/he has to wait for a complete rebuild every time.
Also, the 7 day period for keeping views fresh can be set globally on the server with the proper INI incantations so there’s plenty of room for improving server performance without bothering the users.
Even with all these options tweaked don’t expect your databases to shrink much. Very large databases will, even with proper care and feeding, spend up 80 or 90% of their space on view indexes. This is not necessarily a problem that must be fixed. Remember: a large but mostly quiet database is not a problem for Domino, only for the person in charge of buying disks. A large database constantly needing view updates is.