When you're designing a REST API or MVC app of more than basic complexity, you're likely to end up with a lot of different endpoints with the same path components. For example, in the new OpenNTF home app, projects have a lot of sub-pages with URL paths like:

• projects/{projectName}
• projects/{projectName}/releases
• projects/{projectName}/releases/{releaseId}
• projects/{projectName}/screenshots
• projects/{projectName}/screenshots/{screenshotId}

This will likely start with a class like this, using Jakarta Data to get your model objects:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

package controller.projects;

import jakarta.inject.Inject;
import jakarta.mvc.Controller;
import jakarta.mvc.Models;
import jakarta.mvc.View;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.NotFoundException;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import model.projects.Project;

@Path("projects")
@Controller
public class ProjectsController {
	@Inject
	private Project.Repository projectsRepository;

	@Inject
	private Models models;

	@Path("{projectName}")
	@GET
	@View("project/summary.jsp")
	public void showProject(@PathParam("projectName") String projectName) {
		String key = projectName.replace('+', ' ');
		Project project = projectsRepository.findByProjectName(key)
			.orElseThrow(() -> new NotFoundException("Could not find project for key " + key));
		models.put("project", project);
	}

	@Path("{projectName}/releases")
	@GET
	@View("project/releases.jsp")
	public void showProjectReleases(@PathParam("projectName") String projectName) {
		String key = projectName.replace('+', ' ');
		Project project = projectsRepository.findByProjectName(key)
			.orElseThrow(() -> new NotFoundException("Could not find project for key " + key));
		models.put("project", project);
	}

	// And so forth
}

That's fine and all, and you can scale it out as you add more "sub"-resources by having another controller with e.g. @Path("projects/{projectName}/releases"). But even in the basic case, there's a clear problem: I've repeated the same logic of getting the project based on the name parameter, and that will be repeated in almost every method in this and related resources.

This can be alleviated a bit by moving @PathParam("projectName") String projectName to the class level and having a utility method to do the lookup, but that's not quite ideal. What we really want is a way to encapsulate the meaning of this translation in a way that can be used transparently any time we need it.

Path Param Converters

By default, path params are naturally strings, but the REST framework knows how to convert them to basic values - "1" to integer 1, "true" to boolean true, and the like. This is an extensible mechanism, though, and so we can teach our app to convert project names to Project model objects.

To do this, we'll make a new class that implements jakarta.ws.rs.ext.ParamConverter:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

package rest.ext;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import jakarta.ws.rs.NotFoundException;
import jakarta.ws.rs.ext.ParamConverter;
import model.projects.Project;

@ApplicationScoped
public class ProjectParamConverter implements ParamConverter<Project> {
	
	@Inject
	private Project.Repository projectRepository;

	@Override
	public Project fromString(String value) {
		String key = value.replace('+', ' ');
		return projectRepository.findByProjectName(key)
			.orElseThrow(() -> new NotFoundException("Unable to find project for name: " + key));
	}

	@Override
	public String toString(Project value) {
		return value.getName();
	}

}

We make this a CDI bean so that it can participate in the whole CDI environment and get our data repository. Here's where we put our logic, allowing us to centralize what it means to go from a string to a model object. We also provide a way to convert back, for whatever the framework uses that for.

Just creating this as a bean isn't quite enough - we still need to tell the REST runtime about it. To do that, we create a class that implements jakarta.ws.rs.ext.ParamConverterProvider:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

package rest.ext;

import java.lang.annotation.Annotation;
import java.lang.reflect.Type;

import jakarta.inject.Inject;
import jakarta.ws.rs.ext.ParamConverter;
import jakarta.ws.rs.ext.ParamConverterProvider;
import jakarta.ws.rs.ext.Provider;
import model.projects.Project;

@Provider
public class ProjectParamConverterProvider implements ParamConverterProvider {
	
	@Inject
	private ProjectParamConverter projectParamConverter;

	@SuppressWarnings("unchecked")
	@Override
	public <T> ParamConverter<T> getConverter(Class<T> rawType, Type genericType, Annotation[] annotations) {
		if(Project.class.equals(rawType)) {
			return (ParamConverter<T>)projectParamConverter;
		}
		return null;
	}

}

With the @Provider annotation, REST will pick up on this and ask it for a converter whenever it encounters a type it doesn't inherently understand. For now, we cover just Project, but we can expand this as more types come into play (for example, project releases, which will also be in path parameters).

Putting It To Use

With this in place, we can go back to our ProjectsController class and clean it up:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

package controller.projects;

import jakarta.inject.Inject;
import jakarta.mvc.Controller;
import jakarta.mvc.Models;
import jakarta.mvc.View;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.NotFoundException;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import model.projects.Project;

@Path("projects")
@Controller
public class ProjectsController {
	@Inject
	private Models models;

	@PathParam("project")
	private Project project;

	@Path("{project}")
	@GET
	@View("project/summary.jsp")
	public void showProject() {
		models.put("project", project);
	}

	@Path("{project}/releases")
	@GET
	@View("project/releases.jsp")
	public void showProjectReleases() {
		models.put("project", project);
	}

	// And so forth
}

That's noticeably cleaner! It also means that our controller layer now has less to know about: if we change how projects are looked up (say, allowing UNIDs as well as names), this class doesn't need to change. The code is also much more explicit - we don't really care about the project name as such, but rather just that we have a URL that includes whatever the identifier for a project is. Someone entirely new to Jakarta REST may not know how that translation happens, but, once you know how it works, it's much clearer.

Figuring this out reinforced an important lesson for me: whenever I find that I'm building something out in a way that feels gangly, there's usually an idiomatic solution to the problem that improves code readability and maintainability. It's one of the advantages to using a mature and broad-scoped framework like this.

Inspired by Toby Samples's new blog series on JAX-RS in Domino, I'd like to share a description of how I made use of it to write the REST services in the frostillic.us Framework. This is not intended to be a from-scratch introduction - Toby is handling that well so far - but instead assumes a certain amount of knowledge with OSGi development and why you would want to do this in the first place.

The goal of my REST services is to provide an automatic REST/JSON API for any Framework model objects used in a database without having to include any servlet code in the database itself. It's a business-logic-friendly analogue to the Domino Access Services and "borrows" heavily from that code base. It doesn't use the DAS extension point, though, in large part because I didn't know that existed until recently. As far as I can tell, using that extension point saves you some bootstrapping work and makes it possible to enable/disable the service in the server config, but otherwise the work will likely be fairly similar.

Initial Setup

To get started, this will all have to take place in a plugin, unless it turns out there's a way to do it in-NSF. In this case, this made sense anyway, since I wanted the servlets to be available for everything. The first step was to make a stub class to act as the base of the servlet, even though it doesn't really do anything:

package frostillicus.xsp.model.servlet;

import javax.servlet.ServletException;
import com.ibm.domino.services.AbstractRestServlet;

public class ModelServlet extends AbstractRestServlet {
	private static final long serialVersionUID = 1L;

	public static ModelServlet instance;

	public ModelServlet() {
		instance = this;
	}

	@Override
	protected void doInit() throws ServletException {
		super.doInit();
	}
}

Once that class existed, I registered it in the plugin.xml as a servlet extension:

<extension id="frostillicus.xsp.model.Servlet" name="fmodelservlet" point="org.eclipse.equinox.http.registry.servlets">
	<servlet alias="/fmodel" class="frostillicus.xsp.model.servlet.ModelServlet">
		<init-param name="applicationConfigLocation" value="/WEB-INF/fmodelapplication"/>
		<init-param name="propertiesLocation" value="/WEB-INF/fmodelservlet.properties"/>
		<init-param name="DisableHttpMethodCheck" value="true"/>
	</servlet>
</extension>

In addition to that servlet class, it also references two text files to provide configuration for Wink, the JAX-RS implementation packaged with the Extension Library. The first is a list of resource classes to use in the servlet:

frostillicus.xsp.model.servlet.resources.ApiRootResource
frostillicus.xsp.model.servlet.resources.ManagersResource
frostillicus.xsp.model.servlet.resources.ManagerResource
frostillicus.xsp.model.servlet.resources.ModelResource

The second is a properties file with configuration options (I don't remember why this option is important):

wink.defaultUrisRelative=false

Implementing a Resource

The classes listed above are what receive a REST request (funneled through Wink/JAX-RS) and provide a response. As an example, here's the ManagerResource class:

package frostillicus.xsp.model.servlet.resources;

import java.io.IOException;
import java.net.URI;
import java.util.Map;
import java.util.HashMap;

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.ResponseBuilder;
import javax.ws.rs.core.UriInfo;

import org.openntf.domino.Database;

import com.ibm.commons.util.io.json.JsonException;
import com.ibm.commons.util.io.json.JsonGenerator;
import com.ibm.commons.util.io.json.JsonJavaFactory;
import com.ibm.domino.commons.util.UriHelper;
import com.ibm.domino.das.utils.ErrorHelper;

import frostillicus.xsp.model.ModelManager;
import frostillicus.xsp.model.ModelObject;
import frostillicus.xsp.model.ModelUtils;
import frostillicus.xsp.util.FrameworkUtils;

@SuppressWarnings("unused")
@Path("{managerName}")
public class ManagerResource {

	@GET
	@Produces(MediaType.APPLICATION_JSON)
	public Response getManager(@Context final UriInfo uriInfo, @PathParam("managerName") final String managerName) {
		try {
			Map<String, Object> result = new HashMap<String, Object>();
			Database database = FrameworkUtils.getDatabase();
			if(database == null) {
				result.put("status", "error");
				result.put("message", "Must be run in the context of a database.");
			} else {
				Class<? extends ModelManager<?>> managerClass = ModelUtils.findModelManager(database, managerName);
				if(managerClass == null) {
					result.put("status", "failure");
					result.put("message", "No manager found for name '" + managerName + "'");
				} else {
					result.put("status", "success");
					result.put("managerClass", managerClass.getName());
				}
			}

			return ResourceUtils.createJSONResponse(result, false);
		} catch (Throwable e) {
			return ResourceUtils.createErrorResponse(e);
		}
	}

	@POST
	@Consumes(MediaType.APPLICATION_JSON)
	public Response createModel(final String requestEntity, @Context final UriInfo uriInfo, @PathParam("managerName") final String managerName) {

		Database database = FrameworkUtils.getDatabase();
		Class<? extends ModelManager<?>> managerClass = ModelUtils.findModelManager(database, managerName);
		if(managerClass == null) {
			return ErrorHelper.createErrorResponse("Manager '" + managerName + "' not found.", Response.Status.NOT_FOUND);
		}

		URI location;
		try {
			ModelManager<? extends ModelObject> manager = managerClass.newInstance();
			ModelObject model = manager.create();
			ResourceUtils.updateModelObject(requestEntity, model, false);

			location = UriHelper.appendPathSegment(uriInfo.getAbsolutePath(), model.getId());
		} catch(Throwable t) {
			return ResourceUtils.createErrorResponse(t);
		}

		ResponseBuilder builder = Response.created(location);
		Response response = builder.build();
		return response;
	}
}

There are quite a few concepts at work here, as well as tons of logic wrapped up in the referenced ResourceUtils and ModelUtils classes. The term "manager" in this class has no special meaning for JAX-RS or servlets - it's the term the Framework uses for the objects that provide access to models, like the "Posts" manager that maps requests for "all" to a back-end view named "Posts\All" and returning "Post" objects.

This is where things get hairy and diverge from basic servlet creation and head into Domino/XPages-specific eccentricities.

A Secret Double Life

The job of the model servlet is a bit strange, in that it doesn't want to just read document data from an NSF, but also should read and process Java classes. Framework model classes are defined in the NSF, not in plugins, and so the servlet has no real knowledge of what's inside the NSF, other than that it should look for classes that implement the appropriate interfaces.

When code is executing in an OSGi servlet context like this, it sits in a strange grey area. It's a better spot than agents - which have no knowledge of OSGi plugins or the XSP runtime - but it's not quite an XPages context, either, and there's no FacesContext available. Instead, a class called ContextInfo provides access to the session (running as the currently-authenticated Domino user) and the current database, if applicable. That "if applicable" comes in because an OSGi servlet can be accessed either as "http://foo.com/servletname" or as "http://foo.com/bar.nsf/servletname". I modified my utility class to paper over the difference between these two environments. The manager resource calls ModelUtils.findModelManager with this database context to try to find the requested manager. For example, if the request comes in as "/fmodel/Posts", it will search the database for a class or managed bean named "Posts".

This is where the ability to treat an NSF as a "bag of classes" comes in handy. It may be possible to do this another way, but I'm using the DatabaseClassLoader provided by ODA to perform searches on the Java classes contained in an NSF. For this purpose, the actual names and structure of the classes are irrelevant, only that they implement ModelManager. If such a class is found, then the servlet knows enough about it, thanks to the interface, to fetch collections and individual model objects as necessary.

Bits and Bobs

In addition to the trickery required to pull manager and model classes out of an NSF, there are also a number of other techniques and components, mostly lifted from the ExtLib, used to make this work.

The code makes heavy use of JsonWriter to produce JSON in a lightweight manner, rather than building up and then spitting out large blobs of JSON, which is particularly important with large amounts of data.

The AbstractDominoModel class needed a lot of reworking to exist in the not-quite-XPages environment. In an XSP context, it makes use of an inner DominoDocument object in order to be able to deal with file attachments more easily, but that doesn't work without the full context. Accordingly, it uses a holder class to paper over the difference between DominoDocument and "manually" accessing the ODA Document. Of particular note is the handling of rich text for export into the REST display. It uses the HTML converter class included in DominoUtils, which I assume uses the HTMLConvertItem C function underneath the hood. It then uses the converter object to output an HTML version of the rich-text item as well as URLs for any attachments.

There is a certain amount of number fiddling and off-by-one-error-prone work done to determine the first and last entries in a model collection to show. As with DAS, it supports both the "Range" header and "start" and "count" GET parameters.

I nabbed wholesale IBM's shim implementation of the PATCH method, which isn't included in the version of Wink shipped with the ExtLib (at least not when I wrote this). Ideally, PATCH would mean that the JSON provided to the server would only be used to update fields in-place (leaving any fields in the model not included in the JSON untouched), while PUT would replace the model object entirely (removing any model fields not present in the JSON). In reality, they both act as PATCH.

As with XPages access to model objects, the REST APIs work with the JPA annotations used in model objects for validation. The model objects check their context - in an XPages context, failed validations result in FacesMessages, while otherwise it throws a ConstraintViolationException. When this exception occurs in the servlet, the error-response method picks up on that and generates specialized JSON to provide an explanation of the failed constraints to the user.

So Yeah

If you haven't tried out JAX-RS servlets yet, don't let this list of caveats and complicated code daunt you. This specific case of working with model objects in a very generic way naturally leads to complicated code, and the annotation-based coding system of JAX-RS/Wink reduces the amount of code dramatically. None of my code has to deal with fetching HTTP requests or parsing query parameters into useful objects - the API does that for me. There's no doubt a good deal more I could have it do for me as well. This is a pretty clean way to write servlets and is absolutely the best way to write them when the code makes sense to exist in a plugin.

I've been giving a lot of thought lately to how I feel about different structures for web apps. Specifically, in this case, the "structure" I'm thinking about is the "client-server" balance in the app itself and the associated method of data access.

The impetus has been my minor fiddling with Angular over the past half a year or so. If you're not familiar with it, Angular is a client-side JavaScript framework focused on building full-fledged apps that run in the browser. The way it (and frameworks like it) distinguish themselves from more server-heavy methods of development is that the primary logic of running the UI happens in client-side code - resources and data are fetched from the server (including HTML snippets), but little or no presentation logic takes place there. You should go read Marky Roden's posts on the topic and then attend his session at ConnectED.

For a while, I thought of this as an interesting idea, but mostly just another method of writing apps, like PHP vs. Ruby vs. XPages. However, the idea kept churning away in the back of my brain, offering itself up as the right answer as I started writing the REST APIs in my Framework (I swear I'll get to that final post in the series eventually) and thought more about mobile development. The thing that's really latched onto my psyche isn't that Angular or JavaScript are particularly compelling, but the idea of writing a decoupled app is. Writing an app in Angular isn't really like writing an app in XPages or PHP - instead, it's (at a very high level) like writing a native mobile or desktop application. You split your app into two main tiers: the model/data/connection tier on the server and the app tier on the client.

That is something that is starting to seem really right to me. Even if it did nothing else, having an architecture like that forces you to think in terms of "services" instead of just data access - once you've written a model layer that provides REST services with access and validation rules enforced, then you have a single interface that can be used by browser-based Angular apps, mobile and desktop apps, and remote applications and services you didn't write. There's also no reason you couldn't write a server-side app that consumes those services, continuing to build XPages apps that don't use the normal data sources. This is sort of what my framework has evolved towards, just with a first-among-equals twist where the XPage app gets direct Java access to the same objects REST serves up.

Once you have this sort of setup, the answers to what your data-access and client UI frameworks should be both become "whatever". Want to write your REST API in Node and consume it in XPages? Sure, go ahead. Decide later that you'd rather have the data served up by Rails? Can do - the XPages side wouldn't need a change if the API is the same. Similarly, if you want to swap out XPages for Angular served up as static files or, god forbid, a PHP app, the way is smoothed.

Even though it starts as a small thing - switching from accessing data "directly" on the server to always thinking about that REST-API abstraction - this really seems compelling to me. Not so compelling that I've actually written any non-test apps this way yet, but I've opened the door for myself with my framework.