Quick Tip: JDK Null Annotations for Eclipse

Dec 10, 2020, 3:17 PM

  1. The Cleansing Flame of Null Analysis
  2. Quick Tip: JDK Null Annotations for Eclipse

A few years back, I more-or-less found the religion of null analysis, and I've stuck with it with at least my larger projects.

One of the sticking points all along, though, has been Eclipse's lack of knowledge about what code not annotated with nullness annotations does, with the biggest blind spot being the JDK itself. For example, take this bit of code:

1
BigDecimal foo = BigDecimal.valueOf(10).add(1);

That will never throw a NullPointerException, but, since BigDecimal#valueOf isn't annotated at all, Eclipse doesn't know that for sure, and so it may flag it as a potential problem. To deal with this, Eclipse has the concept of external annotations, where you can associate a specially-formatted file with a set of classes and Eclipse will act as if those classes had nullness annotations already.

Core JDK Annotations

Unfortunately - and as opposed to things like IntelliJ - Eclipse for some reason doesn't ship with this knowledge out of the box. For a while, I just dealt with it, throwing in technically-unnecessary checks around things like Optional#get that are guaranteed to return non-null. The other day, though, I decided to look into it more and found lastNPE.org, which is a community-driven project to provide such external annotations.

Better still, they also provide an Eclipse plugin (404 expected on that link - Eclipse knows what to do with it) to apply rules from your project's Maven configuration to the IDE. This not only covers applying external annotations, but also synchronizing compiler configurations.

Sidebar: The Eclipse Compiler

By default, a Java project is compiled with javac, the stock Java compiler. Eclipse maintains its own compiler, varyingly called ECJ or (as shorthand) JDT. Eclipse's compiler is, unsurprisingly, well-geared towards IDE use, and part of that is that it can flag and process a great deal of semantic and stylistic issues that the stock compiler doesn't care about. This included null annotations.

Maven Configuration

With this information in hand, I went to configure my project's Maven build. The first step was to change it to use Eclipse's compiler, since I had recently switched the project away from being Tycho-based (which uses ECJ by default). This can be done by configuring maven-compiler-plugin:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.8.1</version>
      <configuration>
        <failOnWarning>false</failOnWarning>
        <compilerId>jdt</compilerId>
        <compilerArguments>
          <properties>${project.basedir}/../../config/org.eclipse.jdt.core.prefs</properties>
        </compilerArguments>
      </configuration>
      <dependencies>
        <dependency>
          <groupId>org.eclipse.tycho</groupId>
          <artifactId>tycho-compiler-jdt</artifactId>
          <version>1.7.0</version>
        </dependency>
      </dependencies>
    </plugin>
  <plugins>
<build>

I use an inline dependency on Tycho's tycho-compiler-jdt to provide the compiler. I stuck with version 1.7.0 for now because Tycho 2.0+ uses the newer core runtime that requires Java 11, which this project does not yet for platform lag reasons. I also find it useful to set <failOnWarning>false</failOnWarning> here because ECJ throws many more (legitimate) warnings than javac. Long-term, it's cleanest to keep this enabled.

I also configured Eclipse's compiler settings like I wanted for one of the project's modules, then copied the settings file to a common location. That's where the compilerArguments bit comes from.

Then, I went through the available libraries from lastNPE.org, found the ones that match the libraries we use, and added them as dependencies in my root project:

<properties>
  <lastnpe-version>2.2.1</lastnpe-version>
</properties>

<dependencies>
  <dependency>
    <groupId>org.lastnpe.eea</groupId>
    <artifactId>jdk-eea</artifactId>
    <version>${lastnpe-version}</version>
    <scope>provided</scope>
  </dependency>
  <!-- and so on -->
</dependencies>

Once I updated the project configurations, Eclipse churned for a while, and then I got to work cleaning up the giant pile of new errors and warnings it brought up. As usual with null checks, this was a mix of "oh, nice catch" and "okay, sure, technically, but come on". For example, it flags System.out.println as a potential NPE because System.out is assignable - this is true, but realistically my app's code is going to be the least of my concerns when System.out is set to null.

In any event, I was pleased as punch to find this. Now, I have a way to not only properly check nullness with core classes and common libraries, but it's a way that's shared among the whole project team automatically and enforced at compile time.

The Intricate Work of OSGi Dependencies on Domino

Dec 2, 2020, 3:03 PM

Tags: domino osgi
  1. Converting Tycho Projects to maven-bundle-plugin, Initial Phase
  2. Winter Project #2: Maven P2 Repository Resolver
  3. OpenNTF Fork of p2-maven-plugin
  4. The Intricate Work of OSGi Dependencies on Domino

One of the main goals of OSGi is proper runtime dependency management, not only allowing a bundle to declare what its dependencies are to ensure that they're there, but even to select one of multiple available versions. For example, you might have one bundle that expects Guava 15 but not higher and another that expects 18 or above, and both can be loaded successfully if you have multiple Guava versions installed. The goal is that you're supposed to be able to just throw a whole bunch of bundles into a pot and the system will figure it out.

Before I get into why this system is hobbled on Domino, I'll add a little more background.

Mechanisms

For our purposes here, bundles have two main ways to declare what they are (there are more than this, but they're not relevant right now):

  • The bundle's symbolic name combined with its bundle version. For example, the core bundle for ODA is named "org.openntf.domino" and has a version like "11.0.1.202006091416".
  • The bundle's exported packages, which are Java class packages and might also individually have versions. Using ODA again as an example, it exports a slew of packages, such as org.openntf.domino and org.openntf.domino.nsfdata, though it doesn't specify versions for any of these.

The versions of the bundle and the exported packages don't need to be the same, nor do all the exported packages need to have the same version. This shows up a lot in "spec bundles", where a vendor will wrap a standard spec API in their own bundle for various reasons. For example, Apache Geronimo has a bundle called "org.apache.geronimo.specs.geronimo-jaxrs_2.1_spec", which provides the JAX-RS 2.1 spec. The bundle itself has a version of "1.1.0", while the exported packages are all version "2.1".

To require a bundle by name, you use the Require-Bundle header, like Require-Bundle: org.openntf.domino;bundle-version="11.0.0", which requires specifically the ODA bundle, version 11 or above. To require a package, you use Import-Package, like Import-Package: javax.ws.rs;version="2.1.0". The two methods have some implications when it comes to how the ClassLoaders work, which I touched on a bit earlier this year.

Culture Differences

The package-based mechanism is generally preferred nowadays over the bundle-based one, largely for the kind of flexibility and division-of-responsibilities it provides. For example, if I want version 2.1.0 of javax.ws.rs, my code shouldn't care at all whether it comes from Geronimo's bundle, JBoss's, or anywhere else - it's all the same thing (in theory). This is extremely common for projects created with bnd and tools based on it, which can generated imported and exported packages automatically based on your code and some small configuration. For example, the bundles that make up Open Liberty use bnd config files that have some loose configuration, which then is processed out to full listings of packages with version information. This is also often paired with OSGi's "Capability" system, but that's one of the "not important for now" things.

Domino - and I believe this is inherited from Eclipse, which does the same thing - is largely based on bundle requirements. For example, the org.eclipse.jdt.ui bundle (part of the Java development tools in Eclipse and Designer) requires a bevy of bundles by name version and doesn't tag any of its exported packages with versions. This is similarly reinforced in the tools. Eclipse, unsurprisingly, uses Tycho to bring the "Eclipse PDE" style to the table, where the MANIFEST.MF file is more hand-crafted, and the tools don't do as much for you automatically. You can go full Import-Package and attach versions to your exported packages with Eclipse, but the tooling doesn't encourage it.

Conflicts in Practice

That brings us back to how this all contributes to making working with OSGi a little extra annoying on Domino. This is something that comes up constantly in my XPages Jakarta EE Support project: I want to bring in an implementation component for a JEE spec, but it will either already have or will have generated for it OSGi rules that lean towards the "package-style" of doing things, versions and all. Because there are common packages (like, say, javax.activation) that are supplied by bundles present in the Domino runtime, I want to use those. However, since the packages from Domino don't have versions specified, I need to re-wrap the bundles to import the packages without versions. Thus begins this ongoing nightmare.

Another approach I could take would be to bring my own, nicely-OSGi-ified versions of the afflicted packages, and options abound. However, that leads to a sneaky other trouble: because some of the Domino bundles are multi-spec monsters - with com.ibm.designer.lib.javamail being the absolute worst culprit - some other bundle on the system might casually import javax.activation and javax.mail. If I have this other spec implementation floating around, then it could get matched to my bundle for the former and IBM's for the latter... and crash right into "exposed to a package via two dependency chains" problem. That wouldn't necessarily be an issue if everything used versions on packages, but leaving them off means it's kind of up to the container to match bundles to each other, and it's entirely happy creating impossible conflicts.

Ways Around It

When working through this sort of trouble, I've found a few ways around the things I've run into. The first is what I mentioned above: using p2-maven-plugin to re-wrap bundles with instructions that make them more Domino-friendly. This involves a few tricks:

Aside from reworking existing bundles, I have a few times ended up creating fragment bundles that glom onto one bundle to tie it to another one after the fact, usually for the components that bridge different JEE specs.

The Wildcard: The System Bundle

In general, with OSGi, you can expect the packages you need to be provided in bundles, but it also allows for the core JDK to be implicitly available - that is, things like java.lang don't need to be imported, and are always present. That's fine, since it's generally well-enough-defined what makes up the JDK and what you'd need to instead bring in.

However, the Domino core classpath doesn't contain just the JDK, but also anything in jvm/lib/ext and (as a curveball) ndext from the Domino directory. Above, I specifically pointed out the javax.activation package, and that's because it suffers the most from both the javamail bundle as well as this. If you run tell http osgi packages javax.activation on Domino's console, you should see that bundles get this from two places: some use com.ibm.designer.lib.javamail, while others use the system bundle, org.eclipse.osgi. That "system bundle" concept is not only the thing in charge, but it also passively provides access to classes found in the lower-level classpath. In a fully OSGi environment, that system classpath will be pretty clean, but Domino's isn't that.

The good news here is that it isn't usually a big problem. If you import packages by a non-zero version, they'll never match from the system bundle this way. Still, it's always there, lurking, and the problems increase if you start adding more JAR files to jvm/lib/ext to avoid policy or amgr-memory issues. We've seen this periodically with ODA, where we used to support the use case of putting the core files there and using just a shim at the XSP layer, before it became too much of a hassle to manage. I ran into it again more recently when Guava showed up there: because I hadn't specified a version, it was able to match from the system bundle, and ended up with classes available at compile time but missing at runtime.

The Upshot

The upshot here is that there's no simple advice for dealing with this. Cultural and implementation factors make bringing third-party code into Domino unusually difficult, but it can generally be dealt with via various patching mechanisms. The XPages JEE project's dependency module ended up turning into a trove of such workarounds, so perhaps it can be useful if you ever run into this sort of thing yourself.

Java Discontinuities in Practice

Nov 23, 2020, 11:04 AM

Tags: java

Earlier this year, I wrote a post about the lay of the Java land, and in it I mentioned the oddities of post-8 Java releases as well as the then-oncoming namespace conversion in Jakarta EE. Those changes are a bit more "real" now, so I think it's worth taking the opportunity to expand on them and how they relate to Java with Domino.

Jakarta EE 9

With Jakarta EE 9 officially out now, I think it's all the more important to keep an eye on what these changes are. For Jakarta's part, there's a convenient post up on Eclipse's site detailing the specifics of what's going on, and most of what I say here is really just going to rehash that.

The "namespace conversion" in question is the switch from javax.* to jakarta.* for EE-related packages like Servlet, due to Oracle not granting rights to the "javax" term. This has involved a lot of fiddly work internally for the Jakarta project as a whole, and all of the included specs have received a major-version bump to reflect the break. In general, these new versions are functionally equivalent to the previous release, but use the different package names - so Servlet 5 has the same capabilities as 4, JSF 3 as 2.3, and on down the line.

A bit of a quirk in this is that not all classes in the javax.* namespace will be moving to jakarta.*, because not everything in there was part of Java EE. For example, Swing is in javax.swing, but it's not going anywhere. It gets fiddlier, too, especially when it comes to XML. The JDK traditionally (more on that in a bit) contained a couple distinct technologies wrapped up under the javax.xml package space, but some of those are actually part of Java EE and make the transition to Jakarta. For example, the javax.xml.transform package (covering XSLT) is part of what was originally termed "Java API for XML Processing", or "JAX-P", and is still part of the Java SE core. The javax.xml.bind package (covering mapping between XML and Java objects) was part of the "Java Architecture for XML Binding" API, or "JAX-B", and is not part of Java SE anymore. It's now "Jakarta XML Binding" and is receiving a package change to jakarta.xml.bind. I think it's cases like these that will be hairy for a lot of people not doing full Jakarta EE 9 work.

For the most part, this won't have an effect on Java development on Domino for a good while. Domino has never tracked changes in the EE world - XPages was a partial fork of Java EE 5 and that's been about it. I think that the ways it will affect Domino development (other than if you just outright do Jakarta EE development, which you should) is that code examples and third-party libraries are going to gradually transition over to the new namespaces, making them incompatible with code in the Domino stack. This will certainly affect things like my XPages Jakarta EE Support project, where future versions of the implementation components won't be usable directly if they use the Servlet spec, even if they don't require Servlet 3+ functionally.

So I think it's worth being aware of what's going on, even if there's not (yet) anything you need to do about it. The same applies to the changes in the core Java runtime itself.

Java 11 and Beyond

After 8, Java switched to a peculiar numbering system, where new major-version-numbered releases come out every six months, but only the ones that come out every three years are Long-Term-Service releases. As of right now, the current version of Java is 15, but 11 is the active LTS one, and so 11 is effectively the "real" current version for concerns like platform vendors. Java 8 is now in the same spot that Java 6 was for a while, where it's been the baseline expectation for a long time, and it's a slog of a process to move the full community past it.

Still, Java 11 is certainly hitting critical mass now. Eclipse-the-IDE started requiring it in the 2020-09 release, and the various app servers have either supported it for a while or are on the cusp of doing so.

There are a lot of nice things added to the language in the releases past 8, but they've also gotten more aggressive about removing things from the core Java SE runtime, and those changes are the things likely to be immediately noticeable Domino-wise. As I mentioned above, JAX-B was always technically an EE specification, but it was shipped with Java SE for a good long time. As of Java 11, though, it's gone, and instead must be either provided by the app server or brought in as an explicit dependency. The same goes for some less-important packages, such as org.omg - though that package sounds fun, it stands for "Object Management Group" and it just included some classes used for CORBA.

I imagine that few Domino developers use JAX-B or CORBA directly, but our old nemesis Notes.jar sure does! If you're doing any project builds outside of Domino that make use of the Notes.jar API, you likely already have or will soon run into this. For Tycho, I made a patch fragment that provides the required API to the com.ibm.notes.java.api bundle a good while back. For non-Tycho projects, your best bet is generally to include a dependency on the GlassFish-packaged variant and a pre-3.0 version of the Jakarta XML Bind API.

There will be some further removals down the line, like RMI Activation, but I don't think any currently on the horizon will be as pertinent as those.

Java With Domino Roundtable Recording

Nov 17, 2020, 4:37 PM

Tags: java

I hosted my "Java With Domino" roundtable earlier today, and I think it went pretty well! We ended up having just about the ideal number of participants, and it was not only great hearing how people feel on the topic, but also seeing and hearing from everyone.

I've put the video up on YouTube:

I'm thinking of doing more of these, and kind of making them a looser, more-casual companion to OpenNTF's webinar series. I don't know whether they'd all be on similar topics or what, but it seems like it'll be worth continuing.

OpenNTF Fork of p2-maven-plugin

Nov 14, 2020, 1:56 PM

Tags: maven tycho
  1. Converting Tycho Projects to maven-bundle-plugin, Initial Phase
  2. Winter Project #2: Maven P2 Repository Resolver
  3. OpenNTF Fork of p2-maven-plugin
  4. The Intricate Work of OSGi Dependencies on Domino

It's been one of my long-running goals to reduce my use of Tycho for my work. While Tycho does what it says on the tin, the way PDE works in Eclipse means it's an ongoing nightmare to deal with when I want to do simple things like add a new dependency. This isn't really Tycho's fault as such, and the project itself is making major steps to alleviate some issues, but it's the nature of the surrounding tooling. Even beyond that, the shaky support in IntelliJ and total lack of support in Visual Studio Code and similar editors makes it a real thorn in my side.

Still, though, it brings a lot to the table, particularly when dealing with Domino-targeted projects. Because Domino's OSGi layout is... fiddly, it's often safest to use the "Manifest-first" approach for dependencies, and it's definitely important to still be able to do feature projects and p2 repositories for importing into Designer and Domino.

But I've still been trying to whittle away at the constraints over time, and I got fed up enough yesterday to make some major strides.

The Original Project

One of the major tools in my toolbelt for years has been the p2-maven-plugin, which does a lot of heavy lifting when it comes to taking non-Tycho or non-OSGi-focused projects and making them palatable for an OSGi environment. Even when I don't use it as the backbone of a project, I tend to use it to gather third-party dependencies and process them to make them Domino-friendly.

The Fork

It has its limitations, though, that have kept me from using it to replace the final steps of a Tycho build, and those are the ones that I set out to improve. Yesterday, I forked the project and got to work. Most of my work centered around letting it pull more information out of existing p2 repositories. While it already has some knowledge of such repos, it was still geared heavily towards only using them to pick up a bundle here or there. The big annoyance for me there was that I wanted to bring in entire existing p2-housed features into the final update site.

For example, one of my big projects consumes and redistributes a bunch of upstream projects, such as ODA and the XPages Jakarta EE support. While the p2-maven-plugin made it possible to reference those projects as Maven artifacts or individual bundles, I couldn't do what I wanted and just say "bring X and Y features in, including all their bundles".

I also went in and added a few other niceties needed for Domino: generation of the antiquated "site.xml" file for the NSF Update Site, archiving of the final site for distribution, and so forth.

The Implications

With my changes, I was able to delete all of the feature projects in the tree, which lowers the mental complexity a bit. That also means that the only parts "controlled" by Tycho now are the actual bundle projects, and those have a clear path to de-Tycho-ization. Though doing that will make it a little more difficult to know when dependencies are Domino-suitable ahead of time, the conversion should save a ton of hassle overall.

So now, I have a toolchain that should be able to work together to replace Tycho while still working with the Equinox-heavy target:

  • maven-bundle-plugin to generate the OSGi metadata in META-INF/MANIFEST.MF. I could also use bnd-maven-plugin directly for this and bndtools in Eclipse, but I'm not sure that it'd gain me much in practice
  • generate-domino-update-site to create p2 repositories from post-9.0.1 Domino releases' XPages framework, which remains damnably non-Mavenized
  • p2-layout-provider to resolve p2-housed artifacts like those from above and OpenNTF projects and make them available as normal-enough Maven dependencies on the fly
  • The forked p2-maven-plugin to generate features and update sites, as well as to repackage existing bundles to be more Domino-friendly

What's missing now is an ability to run compile-time test suites in a true Equinox environment. I'm hemming and hawing on how important that really is, though. The tests I write only rarely expect the presence of OSGi - the main way it comes into play is for extensions, which are papered over by IBM Commons anyway. I've had a delightful time lately running tests of JAX-RS resources with Liberty's dev mode, and I'm pretty sure I saw some examples somewhere of building up and tearing down a scaffolding to run them during compilation, so maybe I'll switch to that anyway.

In any event, just having a tool to do this stuff is a huge weight off my back, and now the goal of a fully-normal-enough Maven project tree is tantalizingly in sight.

Upcoming Event: Java With Domino Roundtable

Nov 12, 2020, 3:31 PM

Tags: java

The other day, I floated the idea of running an unstructured roundtable discussion of working with Java either on or accessing Domino, and I think it'll be worth giving a shot.

Since Java with Domino is in a weird place, the goal would be to discuss the various ways that people are or want to use it. So that can include XPages, OSGi, REST services generally, Jakarta EE, Spring, Vert.x, and so forth. I'd also like it to be open generally. I imagine I'll have some preliminary remarks, but otherwise the goal is to be less like a webinar and more like a free-flowing discussion, in the vein of the "happy hour" and "coffee break" rooms from CollabSphere and Digital Week.

My current plan is to run it on short notice, next week:

Tuesday, November 17th
2:00 PM US Eastern (19:00 UTC)
https://zoom.us/j/99514285138
Password: Computers!

I'll share the password I come up with on Twitter on the day of the event, so look for it there.

CollabSphere 2020 Slides and Video

Oct 29, 2020, 4:09 PM

One of the nice bonuses of an all-online conference is that session recording comes built-in, so I was able to snag that and put it up on YouTube for posterity:

Additionally, I uploaded my slides to SlideShare, though that loses out on the extremely-fancy 5-second videos I used:

CollabSphere 2020: DEV101 - Add Continuous Delivery to Domino with the NSF ODP Tooling

Oct 26, 2020, 9:54 AM

CollabSphere 2020 is starting tomorrow, this year naturally taking the form of an online conference, which has the nice benefit of meaning that you can still sign up if you haven't done so, and you're only restricted by your time zone offset for attending.

For my part, I'll be giving a presentation on the NSF ODP Tooling, currently slated for tomorrow:

DEV101 - Add Continuous Delivery to Domino with the NSF ODP Tooling

Domino applications, stored in NSFs, have been historically difficult to add to Continuous Integration tools like Jenkins and to have participate in Continous Delivery workflows. This session will discuss the NSF ODP Tooling project on OpenNTF, which allows you to take Domino-based projects - whether targetting the Notes client or web, XPages or not - and integrate them with modern tooling and flows. It will demonstrate use with projects ranging from a single NSF to a suite of a dozen OSGi plguins and two dozen NSFs, showing how they can be built and packaged automatically and consistently.

I hope you'll be able to attend - there are definitely some very-interesting topics lined up.

A Notes-Client-Friendly Way To Access JWT-Protected Resources

Oct 2, 2020, 3:32 PM

Tags: lotusscript

A Notes-Client-Friendly Way To Access JWT-Protected Resources

I recently had call to access the Zoom REST API in a Notes client app that will be maintained by other Notes programmers, so I figured it'd be as good an opportunity as any to use the HTTP and JSON classes added in V10 and 11.

The basics there are fine enough - though those classes aren't featureful, they can get the job done. However, the Zoom API needs specialized authentication, beyond the username/password type that you can kind of work your way to in LotusScript alone. Since my needs will be administrative as opposed to multiple users acting as themselves, I decided to go the JWT route instead of OAuth.

JWT

JWT stands for "JSON Web Token", and it's one of the now-common ways to do secure authorization without passing passwords around. It's simple at its core - just some JSON objects to indicate the type of token and the payload of app-specific claims you're going to make, then a cryptographic signature.

It's that last part that moves it out of the realm of LotusScript (barring some way to wrangle the SEC* functions in the C API to do it), so I went to Java and LS2J to bridge the gap.

The Java Side

I lucked out in that the Zoom API uses a pretty simple path for generating the signature - my previous experience with JWT involved public/private key pairs, which is still doable but is more annoying. Additionally, the payload is pretty simple, just asserting that you're logging in, with nothing like the specialized user ID lookups I had to do with SharePoint. This meant I could get away with writing out the token "manually" rather than going through the onerous process of creating script libraries out of one of the available libraries and its dependency tree.

One gotcha is that the JDK doesn't actually ship with JSON support. Fortunately, in this case, the only values going in were JSON-friendly and didn't need escaping, but I'd suggest using even a basic library like the agent-friendly JSON-java for normal uses.

I ended up making a static method in a single-class Java script library:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
package us.iksg;

import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.concurrent.TimeUnit;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

public class JWTGenerator {
    public static final long TIMEOUT = TimeUnit.HOURS.toMillis(1);
    
    public static String generateJWT(String apiKey, String apiSecret) {
        try {
            long now = System.currentTimeMillis();
            long exp = now + TIMEOUT;
            
            // Note to the future: I apologize for writing JSON via string concatenation, but it
            //   _should_ be safe here.
            
            // Header: alg: HS256, typ: JWT
            String headerJson = "{\"alg\": \"HS256\", \"typ\": \"JWT\"}";
            String headerB64 = Base64.getUrlEncoder().encodeToString(headerJson.getBytes(StandardCharsets.UTF_8));
            
            // Payload: iss: API_KEY, exp: exp
            String payloadJson = "{" +
                    "\"iss\": \"" + apiKey + "\"," +
                    "\"exp\": \"" + exp + "\"" +
                "}";
            String payloadB64 = Base64.getUrlEncoder().encodeToString(payloadJson.getBytes(StandardCharsets.UTF_8));
            
            // Codec: HMAC SHA256 (HS256)
            Mac mac = Mac.getInstance("HmacSHA256");
            SecretKeySpec spec = new SecretKeySpec(apiSecret.getBytes(StandardCharsets.UTF_8), "HmacSHA256");
            mac.init(spec);
            byte[] signature = mac.doFinal((headerB64 + "." + payloadB64).getBytes(StandardCharsets.UTF_8));
            String signatureB64 = Base64.getUrlEncoder().encodeToString(signature);
            
            return headerB64 + '.' + payloadB64 + '.' + signatureB64;
        } catch(Throwable t) {
            throw new RuntimeException(t);
        }
    }
}

All of those classes come with the JDK, so it's nice and self-contained.

The LotusScript Side

Back on the LotusScript side, I brought out my trusty old friend LS2J:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Uselsx "*javacon"
Use "JWT Generator"

Sub Click(Source As Button)
    On Error Goto errorHandler
    
    Dim session As New NotesSession, ws As New NotesUIWorkspace, doc As NotesDocument
    Set doc = ws.CurrentDocument.Document
    
    Dim jsession As New JAVASESSION, jwtGenerator As JavaClass
    Set jwtGenerator = jsession.GetClass("us.iksg.JWTGenerator")
    
    Dim apiKey As String, apiSecret As String
    apiKey = doc.ZoomAPIKey(0)
    apiSecret = doc.ZoomAPISecret(0)
    
    Dim generate As JavaMethod
    Set generate = jwtGenerator.GetMethod("generateJWT", "(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;")
    
    Dim token As String
    token = generate.Invoke(Empty, apiKey, apiSecret)
    ' In this case, it's in a "developer playground" form I made for testing.
    ' Do not store JWT tokens long-term - they should be generated for each script.
    doc.ZoomJWTToken = token
    
    Exit Sub
errorHandler:
    Msgbox Erl & ": " & Error
    End
End Sub

The only unusual bit here is that, since I used a static method, I pass Empty as the first parameter to Invoke. I tend to use the reflection-based approach like this out of habit after consistently running into trouble with LS2J's mapping of methods to their Java counterparts, but it'd probably be a little cleaner if I made it an instance method and just called it directly.

Once I had the generated token, I was able to include it in my HTTP requests:

1
2
3
4
5
6
7
8
9
Dim req As NotesHTTPRequest
Set req = session.CreateHTTPRequest()
Call req.SetHeaderField("Authorization", "Bearer " & token)
' Since we just want to plunk this into the field, request a string back
req.PreferStrings = True

Dim result As String
result = req.Get("https://api.zoom.us/v2/users")
doc.Users = result

Not too shabby overall, for the Notes client. I may end up putting all these calls into run-on-server agents regardless just to avoid trouble should the client end up having their users use the Web Assembly or mobile Notes clients, but even then this still ends up very Notes-client-developer-friendly.

Writing Domino Server Addins With GraalVM Native Image

Sep 27, 2020, 3:35 PM

Tags: graalvm domino

I was thinking the other day about the task of writing a Domino server addin, the kind that you run by typing load foo on the server console. The way this is generally done is via C or the like: you write a program using your dusty old copy of the C API Toolkit and have an AddinMain function as the entrypoint. That's fine enough if you want to write in C, but, even beyond the language, it carries the tremendous overhead of a fiddly compilation chain that differs per-platform.

I got to thinking, then, about GraalVM, and specifically its Native Image capability. Before I get into what I did, I figure this warrants some background.

What is GraalVM?

GraalVM is a project from Oracle that is, roughly, an alternative core Java Virtual Machine. It's designed to serve a number of goals, but the main ways that I've seen it used is to improve the speed and efficiency of Java-based programs. It also has some neat-looking capabilities for running multiple languages in one app space, but I have yet to look into that.

The Native Image capability is a way to compile Java applications to native executables for a given platform. So, instead of having a JAR file that you then run with an installed JVM, you'd have an executable that you run directly, and which effectively acts as its own "VM". This means you end up with just "some executable" on your system, and the lack of bootstrapping needed to run it opens up some possibilities.

Domino Server Addins

Though Domino server addins have their own set of functions within the Notes C API, they're really just an executable that Domino launches as a sub-process. If you have a basic executable named foo in your Domino program directory, you can type load foo and it'll run it, whether or not the executable does anything with the Notes API at all. It won't necessarily be useful if it doesn't use the Notes API, but it'll run.

It's this "just an executable" bit, though, that was a contributing factor to making Java not a practical language for this. That's also where RunJava fit in: the runjava executable just initialized a JVM and loads the named class, which is afterward responsible for everything, but that was nonetheless obligatory work to get a Java app loaded this way.

The Combination

Once I realized these things, it wasn't a far reach to try implementing an addin this way. One of my initial concerns was the way addins use AddinMain as a C-type entrypoint - my knowledge of how that sort of thing works is limited enough that I wasn't sure if GraalVM's annotations would suffice. However, the C API documentation relieved my worry: using that function name is just a convenience that handles some of the bootstrapping for you. If you just use a normal main(...) entrypoint, the only difference is that you're on the hook for managing your status line more (the thing that shows up when you do show tasks).

Fortunately, the addin-related methods in the lotus.notes.addin.JavaServerAddin class in Notes.jar are extremely-thin wrappers around native calls and aren't actually specific to RunJava in any way. You can subclass it and use it in essentially the same way as in a RunJava addin:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
package frostillicus.graalvm;

import lotus.domino.NotesException;
import lotus.notes.addins.JavaServerAddin;

public class Main extends JavaServerAddin {
	static {
		System.setProperty("java.library.path", "/opt/hcl/domino/notes/11000100/linux"); //$NON-NLS-1$ //$NON-NLS-2$
		System.loadLibrary("notes"); //$NON-NLS-1$
		System.loadLibrary("lsxbe"); //$NON-NLS-1$
	}
	
	public static void main(String[] args) {
		new Main().start();
	}
	
	public Main() {
		setName("GraalVM Test");
	}
	
	@Override
	public void runNotes() throws NotesException {
		AddInLogMessageText("GraalVM Test initialized");
		int taskId = AddInCreateStatusLine(getName());
		try {

			// Do your work here

		} catch(Throwable t) {
			t.printStackTrace();
		} finally {
			AddInDeleteStatusLine(taskId);
		}
	}

}

GraalVM-specific configuration

The GraalVM project provides a Maven plugin to do native compilation for you, and I make use of that in the project's pom.xml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
<plugin>
	<groupId>org.graalvm.nativeimage</groupId>
	<artifactId>native-image-maven-plugin</artifactId>
	<version>20.2.0</version>
	<configuration>
		<imageName>${project.name}</imageName>
		<mainClass>frostillicus.graalvm.Main</mainClass>
		<!-- snip <buildArgs> -->
	</configuration>
	<executions>
		<execution>
			<goals>
				<goal>native-image</goal>
			</goals>
			<phase>package</phase>
		</execution>
	</executions>
</plugin>

Including that in your project will produce a native executable for your current platform in the target folder, alongside the normal JAR file.

The bit I snipped out, though, ends up being important. In a similar way to what happens during Android "Java" compilation, the GraalVM native compiler builds a map of all of the code used in your project to create its native representation. Additionally, it doesn't support reflection as casually as a normal JVM does, and doing a compilation like this shows just how common reflection is in Java.

Reflection and JNI Configuration

What reflection (and JNI) in Java generally needs is a mapping table of class/method/field names to their class representations, and GraalVM doesn't build this for everything by default. Instead, it does its best guess based on your actual code, but then it's up to you to explicitly specify the parts you'll be accessing dynamically.

For the normal case, Oracle wrote a tool that will monitor an actively-running app in Java for such calls. You build your app and run it non-native with this agent, and then it will spit out a configuration file based on the actually-called reflective methods.

However, as with everything else to do with Domino, it's not the normal case: since what I'm running only reasonably exists when launched explicitly from a server, I had to do it the "hard" way. Fortunately, the it's actually just mostly tedious: build the app, launch the Domino Docker container, wait to look for a NoClassDefFoundError or related problem, add that to the config file, and repeat until it stops yelling. Some cases are a little fiddlier, like how JNA's native component misrepresents the class name it was trying to find, but overall it's just time-consuming.

Practicality

So, this is possible, but is it worth doing? Depending on what you want to do, maybe. It's mildly less unsupported than RunJava, and has the huge advantage of not polluting the server's classpath with all of your application code. Additionally, it should be pretty zippy, as GraalVM boasts some impressive performance numbers. Additionally, at least for Java developers, it's much, much easier to use the native-image-maven-plugin than it is to set up cmake or manual makefiles for a C/etc. project.

However, it can also be a real PITA to get working, especially for a reflection-heavy project. Additionally, though you're technically using Addin* functions with a native executable, it's not like HCL would take your call if you run into trouble with a monstrosity like this (I assume). Most importantly, it's restricted to the sort of thing that would make sense as a server addin to begin with - for example, this wouldn't help with building web apps unless you were planning to use it to (again, just as an example) run a web server that's written in Java.

Future Tinkering

I think that this warrants some more investigation. I'd be curious if this process would work for writing other native components, such as DSAPI filters and ExtMgr addins. In those cases, it absolutely would be important to have the right entrypoints, so it wouldn't be quite so easy. Still, it'd be neat if that worked.

And GraalVM and the Native Image component are definitely worth some time even aside from anything Domino-related. I'm curious about what you can do with the "polyglot" features, for example.

Example Project

I've put an example project up on GitHub, which is a basic example that just accepts strings via tell graalvm-test foo and echoes them back. It also includes a Dockerfile for running via HCL's official Domino 11.0.1 image. I haven't actually tested it any other way, so that's the best way to give it a shot.