Better Living Through Reverse Proxies

  • May 30, 2013

A while ago, I set up Apache as a reverse proxy in front of Domino. Initially, that was in order to serve a WordPress site on the same server, but since then it has more than proven its worth in other ways, mostly in the areas of fault tolerance and SSL support.

IBM recognizes those benefits, which is why Domino 9 on Windows comes bundled with IHS; however, I am a responsible admin, so I don't run Windows on my main servers. Fortunately, stock Apache does the job nicely... or it does usually. Filled with enthusiasm for this kind of setup, I rolled out a small Linode VM dedicated entirely to running Apache as a proxy (sort of like their NodeBalancers, but manual). Unfortunately, I started running into a problem where sometimes sites fronted by it (such as this one) would not properly include their host information and would instead load the main I Know Some Guys site, which is the default on Domino. I wasn't able to find a fix that actually worked, so I decided to use that as an excuse to switch to a cute little number named Nginx.

So far, my experience with Nginx has been fantastic. The config files are like a cleaned-up version of Apache's and it matches it for each feature I've needed (namely, load balancing, easy config, and multiple SSL certificates). As a nice bonus, I didn't have to do any of the config massaging I had to for Apache in order to get XSP's funky resource-aggregation rules to work. If you have the means, I highly recommend it.


My latest foray into proxying also gave me an opportunity to look back into my main bugbears with the setup: Domino tracking the proxy server's IP instead of the original requester's and its lack of knowledge of SSL (which causes it to redirect from an SSL login page to a non-SSL one). Fortunately, it turns out that these problems have been sort-of-solved for years via a notes.ini setting added as part of Domino's terminal WebSphere infection: HTTPEnableConnectorHeaders.

By enabling that on my Domino server, I was able to start providing some of those headers. The remote-host headers ("$WSRA" and "$WSRH") work perfectly: setting that to the incoming host causes Domino to act just like that was the original requester, namely filling that in for the REMOTE_HOST field in classic and facesContext.getExternalContext().getRequest().getRemoteAddr() in XPages.

Unfortunately, I was stymied when I set "$WSIS" to True. Though it does indeed cause Domino to acknowledge that the incoming request is via SSL, it does it TOO well: Domino appears to revert to its behavior of only acknowledging a single SSL site, so it caused requests to essentially ignore the Host (and "$WSSN") headers. So that problem remains unsolved.


Still, I feel pretty good about my switch to Nginx and my abuse of the HTTP connector headers and look forward to tinkering some more. For reference, here is the config I use for the standard HTTP proxy ("arcturus" is the short name I gave for the main upstream target):

server {
    listen 80;
    location / {
        proxy_pass http://arcturus;
        proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
        proxy_redirect off;
        proxy_buffering off;
        proxy_set_header    Host            $host;
        proxy_set_header    X-Real-IP       $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    $WSRA           $remote_addr;
        proxy_set_header    $WSRH           $remote_addr;
        proxy_set_header    $WSSN           $host;

How I Got XPiNC Run-On-Server With SSO Working

  • May 17, 2013

Among the new features in Domino 9 is this little guy, found on the Launch pane of a database's properties when you set it to open an XPage:

If you've ever used an XPiNC application before, you'll know this is a godsend, promising the vast performance benefits of running an app on a server combined with the "the users are stuck using the Notes client" benefits of XPiNC.

I turned this on for a new app on one of my client's servers (let's say the server name is ClientName-2/ClientName) and took it out for a spin. However, I immediately noticed something was amiss: the status bar declared that it was unable to find the server "". With a bit of searching, I found I wasn't alone: Notes assumes that the Domino server is available as its common name in local DNS for some reason, and fails in various ways depending on your network's DNS failover behavior and whether or not is a real web site.

So, okay, as the last comment on that linked question indicates, you can "fix" this behavior by manually adding entries to the "XPages Performance" preferences pane (I have two because it's on two clustered servers, you see):

(Though it's blurred here, I note that the "Server" column stores only the common name of the server, not its full hierarchy. Let's hope all your common names are distinct!)

Alright, now that that is sorted out, Notes should use the URL you typed in rather than its own ludicrously-naïve guess. Open up the app again and bam:

Oh. Okay. Here's where things got fun. At this point, the Notes client is essentially using an embedded web browser to point to and carrying with it no authentication information. The documentation cheekily suggests that this means that your "configuration is not correctly set up" and basically leaves it at that.

I'll spare you the intervening days of frustration and cut to the chase: I needed to create an Account document in the client, set up SSO on the server (I had been using normal session auth), and disable loading Internet configurations from Server\Internet Sites documents (seriously).

Because I wasn't about to have my users use the Preferences -> Accounts pane if I could avoid it, I discovered that you can apply Accounts via policies. I went into the "Accounts" view of the "Configuration" tab of Administrator and created an account like so:

(That bit about "PreferredUsernameField" was because I stole the instructions from something about the social stuff in 9. I don't know if it's required, but I don't dare remove it now.)

Once I had that account document created, I went to the active Desktop policy document, then the "Accounts" tab, clicked "Update Links", and chose "All Supported" (because I only had the one anyway):

So at this point, I had it so that the Notes client was properly picking up the Account document, but it still wasn't signing in via SSO properly. That's when I had flashbacks to setting up a Sametime server, I felt a dread in the pit of my stomach, and I disabled Internet Sites for the server. Once I generated a new SSO configuration (with no "Organization" specified, so it was choosable in the server document) and restarted HTTP, it started to work. Hooray-ish!

Fortunately, in my case, I'm lucky: there's a spare server available that doesn't need to handle normal web requests (at least barring a catastrophe), so I can afford to disable Internet Sites configuration for it. However, long-term, I would be delighted to be wrong in my diagnosis - so if anybody knows a way to get this working while still supporting Internet Sites, please let me know. I would also love to know if there's a way to either avoid using those "run these apps on the server" entries in the client prefs or distribute those via policy as well.

Public Service Announcement - NotesIn9 is down.

  • May 7, 2013

David Leedy has run into some trouble with being down and he asked if I could help him get the word out about it, which I'm more than happy to do:



Hi - Just wanted to drop a note out there about my website.  Currently it's redirecting to someplace else for some unknown reason.  I assume it's been php hacked but I don't know.

My Wordpress site is generously hosted by Chris Miller and I've sent him a note.  Though since he's in the middle of the whole IamLug thing, it might be down for a little while before he can look at it.
My other non wordpress sites appear fine - so you can use (XPages.TV) or ( for the time being.
If you use the index site - try NOT to use Internet Explorer.  There's a bug with that browser where every video link you click will play the SAME video.  I'm not sure what's up with that yet.
I am getting more active on redoing my websites - but with all my travel recently it's just very slow going.
I'm actually heading out to Orlando for the day job for a couple days so I'll have limited ability to try and correct this problem.  But I will get this fixed as soon as I can.
Sorry for any inconveniece. 
David Leedy
NotesIn9 ScreenCast