Showing posts for tag "designer"

AbstractCompiledPage, Missing Plugins, and MANIFEST.MF in FP10 and V10

Oct 19, 2018 11:48 AM

Since 9.0.1 FP 10, and including V10 because it's largely identical for this purpose, I've encountered and seen others encountering a couple strange problems with compiling XPages projects. This is a perfect opporunity for me to spin a tale about the undergirding frameworks, but I'll start out with the immediate symptoms and their fixes.

The Symptoms

There are three broad categories of problems I've seen:

  • "AbstractCompiledPage cannot be resolved to a type"
  • Missing third-party XPages libraries, such as ODA, resulting in messages like "The import org.openntf cannot be resolved"
  • Complaints about MANIFEST.MF, like "MANIFEST.MF has no main section" and others

The first two are usually directly related and have the same fix, while the second can also be caused by some other sources, and the last one is entirely distinct.

Fix #1: The Target Platform

For the first two are based on problems in the active Target Platform, namely one or both of the standard platform components go missing. The upshot is that you want your Target Platform preferences to look something like this:

Working Target Platform

There should be a selected platform (the name doesn't matter, but "Running Platform" is the default name) with entries at least for ${eclipse_home} and for a directory inside your Notes data dir, here C:\Notes\Data\workspace\applications\eclipse. If they're missing, modify an existing platform or create a new one and add an "Installation"-type entry for ${eclipse_home} and a "Directory"-type one for the eclipse directory within your data dir.

Fix #2: Broken Plugins, Particularly ODA

Though V10 didn't change much when it comes to XPages, there are a few small differences. One in particular bit ODA: we had a dependency on the com.ibm.domino.commons plugin, which was in the standard Notes environment previously but is not as of V10 (though it's still present on the server). We fixed that one in the V10 release, and so you should update your ODA version if you hit this trouble. I don't think I've seen other plugins with this issue in the V10 transition, but it's a possibility if Fix #1 doesn't do it.

Fix #3: MANIFEST.MF

This one barely qualifies as a "fix", but it worked for me: if you see Designer complaining about MANIFEST.MF, you can usually beat it into submission by cleaning/rebuilding the project in question. The trouble is that Designer is, for some reason, skipping a step of the XPages compilation process, and cleaning usually kicks it into gear.

I've also seen others have success by deleting the error entry in the Errors view (which is actually a thing that you can do) and never seeing it again. I suspect that the real fix here is the same as above: during the next build, Designer creates the file properly and it goes away on its own.

The Causes

So what are the sources of these problems? The root reason is that Designer is a sprawling mountain of code, built on ancient frameworks and maintained by a diminished development team, but the immediate causes have to do with OSGi.

The first type of trouble - the target platform - most likely has to do with a change in the way Eclipse manages target platforms (look at the same prefs screen in 9.0.1 stock and you'll see it's quite different), and I suspect that there's a bug in the code that migrates between the two formats, possibly due to the dramatic age difference in the underlying Eclipse versions.

The second type of trouble - the MANIFEST.MF - is due to a behind-the-scenes switch in how Designer (and maybe the server) handles dependencies in XPages projects.

Target Platforms

The mechanism that OSGi projects - such as XPages applications - use for determining their dependencies at build time is the notion of a "Target Platform". The "target" refers to the notion that this is the platform that is expected to be available at runtime for what you're building - loosely equivalent to a basic Java classpath. An OSGi project is checked against this Target Platform to determine which classes are available based on their bundle names and versions.

This is distinct from the related concept of a "Running Platform". Designer, being based on Eclipse, is itself built on and runs using OSGi. Internally, it uses the same mechanisms that an XPages application does to determine what plugins it knows about and what services those plugins provide.

This distinction has historically been hidden from XPages developers due to the way the default Target Platform is set up, pointing at the same Running Platform it's using. So Designer itself has the core XPages plugins running, and it also exposes them to XPages applications as the Target. Similarly, the way we install XPages Libraries like ODA is to install them outright into the Designer Running Platform. This allows Designer to know about the library service provided, which it uses to populate the list of available plugins in the Xsp Properties editors.

However, as our trouble demonstrates, they're not inherently the same thing. In standalone OSGi development in Eclipse, it's often useful to have a Target Platform distinct from the Running Platform - such as the XPages environment for plugins - to ensure that you only depend on plugins that will be available at runtime. But when the two diverge in Designer, you end up with situations like this, where Designer-the-application knows about the XPages runtime and plugins and constructs an XPages project and translates XSP to Java using them, but then the compilation process with its empty Target Platform has no idea how to actually compile the generated code.

MANIFEST.MF

I've mentioned that an OSGi project "determines its dependencies" out of the Target Platform, but didn't mention the way it does that. The specific mechanism has changed over time (which is the source of our trouble), but the idea is that, in addition to the Java classes and resources, an OSGi bundle (or plugin) has a file that declares the names of the plugins it needs, including potentially a version range. So, for example, a plugin might say "I need org.apache.httpcomponents.httpclient at least version 4.5, but not 5.0 or higher". The compiler uses the Target Platform to find a matching plugin to compile the code, and the runtime environment (Domino in our case) does the same with its Running Platform when loading.

(Side note: you can also specify Java packages to include from any plugin instead of specific plugin names, but Designer does not do that, so it's not important for this purpose.)

(Other side note: this distinction comes, I believe, from Eclipse's switch from its own mechanism to OSGi in its 3.0 release, but I use "OSGi" to cover the general concept here.)

The old way to do this was in a file called "plugin.xml". If you look inside any XPages application in Package Explorer, you'll see this file and the contents will look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.0"?>
<plugin class="plugin.Activator"
  id="Galatea2dVCC_2fIKSG_dev5csyncagent_nsf" name="Domino Designer"
  provider="TODO" version="1.0.0">
  <requires>
    <!--AUTOGEN-START-BUILDER: Automatically generated by null. Do not modify.-->
    <import plugin="org.eclipse.ui"/>
    <import plugin="org.eclipse.core.runtime"/>
    <import optional="true" plugin="com.ibm.commons"/>
    <import optional="true" plugin="com.ibm.commons.xml"/>
    <import optional="true" plugin="com.ibm.commons.vfs"/>
    <import optional="true" plugin="com.ibm.jscript"/>
    <import optional="true" plugin="com.ibm.designer.runtime.directory"/>
    <import optional="true" plugin="com.ibm.designer.runtime"/>
    <import optional="true" plugin="com.ibm.xsp.core"/>
    <import optional="true" plugin="com.ibm.xsp.extsn"/>
    <import optional="true" plugin="com.ibm.xsp.designer"/>
    <import optional="true" plugin="com.ibm.xsp.domino"/>
    <import optional="true" plugin="com.ibm.notes.java.api"/>
    <import optional="true" plugin="com.ibm.xsp.rcp"/>
    <import optional="true" plugin="org.openntf.domino.xsp"/>
    <!--AUTOGEN-END-BUILDER: End of automatically generated section-->
  </requires>
</plugin>

You can see it here declaring a name for the pseudo-plugin that "is" the XPages application (oddly, "Domino Designer"), a couple other metadata bits, and, most importantly, the list of required plugins. This is the list that Designer historically (and maybe still; it's not clear) uses to populate the "Plug-in Dependencies" section in the Package Explorer view. It trawls through the Target Platform, finds a matching version of each of the named plugins (the latest version, since these have no specified ranges), adds it to the list, and recursively does the same for any re-exported dependencies of those plugins. "Re-exported" isn't exposed here as a concept, but it is a distinction in normal OSGi plugins.

Designer derives its starting points here from implicit required libraries in XPages applications (all those "org.eclipse" and "com.ibm" ones above) as well as through the special mechanism of XspLibrary extension contributions from plugins installed in the Running Platform. This is why a plugin like ODA has to be installed in Designer itself: in the runtime, it asks its plugins if they have any XspLibrary classes and uses those to determine the third-party plugin to load. Here, ODA declares that its library needs org.openntf.domino.xsp, so Designer adds that and its re-exported dependencies to the Plug-in Dependencies group.

With its switch to OSGi in the 3.x series circa 2005, most of the functionality of plugin.xml moved to a file called "META-INF/MANIFEST.MF". This starkly-named file is a standard part of Java, and OSGi extends it to include bundle/plugin metadata and dependency declarations. As of 9.0.1 FP10, Designer also generates one of these (or is supposed to) when assembling the XPages project. For the same project, it looks like this:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Domino Designer
Bundle-SymbolicName: Galatea2dVCC_2fIKSG_dev5csyncagent_nsf;singleton:=true
Bundle-Version: 1.0.0
Bundle-Vendor: TODO
Require-Bundle: org.eclipse.ui,
  org.eclipse.core.runtime,
  com.ibm.commons,
  com.ibm.commons.xml,
  com.ibm.commons.vfs,
  com.ibm.jscript,
  com.ibm.designer.runtime.directory,
  com.ibm.designer.runtime,
  com.ibm.xsp.core,
  com.ibm.xsp.extsn,
  com.ibm.xsp.designer,
  com.ibm.xsp.domino,
  com.ibm.notes.java.api,
  com.ibm.xsp.rcp,
  org.openntf.domino.xsp
Eclipse-LazyStart: false

You can see much of the same information (though oddly not the Activator class) here, switched to the new format. This matches what you'll work with in normal OSGi plugins. For Eclipse/Equinox-targeted plugins, like XPages libraries, plugin.xml still exists, but it's reduced to just declaring extension points and contributions, and no longer includes dependency or name information.

Eclipse had moved to full OSGi by the time of Designer's pre-FP10 basis (2008's 3.4 Ganymede), but XPages's history goes back further, so I guess that the old-style Eclipse plugin.xml route is a relic of that. For a good while, Eclipse worked with the older-style plugins without batting an eye. FP10 brought a move to 2016's Eclipse 4.6 Neon, though, and I'm guessing that Eclipse dropped the backwards compatibility somewhere in the intervening eight years, so the XPages build process had to be adapted to generate both the older plugin.xml files for backwards compatibility as well as the newer MANIFEST.MF files.

I can't tell what the cause is, but, sometimes, Designer fails to populate the contents of this file. It might have something to do with the order of the builders in the internal Eclipse project file or some inner exception that manifests as an incomplete build. Regardless, doing a project clean and usually jogs Designer into doing its job.

Conclusion

The mix of layering a virtual Eclipse project over an NSF, the intricacies of OSGi, and IBM's general desire to insulate XPages developers from the black magic behind the scenes leads to any number of opportunities for bugs like these to crop up. Honestly, it's impressive that the whole things holds together as well as it does. Even though it doesn't seem like it to look at the user-visible changes, the framework changes in FP10 were massive, and it's not at all surprising that things like this would crop up. It's just a little unfortunate that the fixes are in no way obvious unless you've been stewing in this stuff for years.

SNTT: Designer Target Platform

Sep 27, 2018 4:38 PM

When working on an XPages project, my development environment is generally set up like so:

  1. Eclipse running in the Mac environment editing Maven-structured plugins and ODPs
  2. Domino running in a local Windows VM, set to run plugins out of the Mac Eclipse workspace
  3. Designer running in the same Windows VM to compile the ODP and work with legacy elements
  4. Firefox DE running in the Mac environment

This has proven to be a fairly comfortable setup, particularly ever since Serdar added the ability to use a remote workspace to the XPages SDK. When I make a change in the plugin, I only need to restart HTTP on Domino and all is well.

But there's been one major annoyance: if I change a method or class that I use in in-NSF Java, I have to install the updated plugins and restart Designer, which is a big distruption to my flow. Even if I was using Eclipse on Windows with the old method of running Designer out of Eclipse (if that even still works), it'd still involve a restart.

Today, though, I realized that I've been doing it the dumb way all along. The reason is that I've been conflating the two aspects of an XPages Library plugin when it comes to Designer.

The first aspect is to tell Designer-the-IDE that an XPages Library named, say, org.example.XspLibrary is available for use in XPages applications, and that it's associated with the plugin org.example.plugin. This is provided by the plugin being installed into the running Designer environment, and absolutely needs a restart when it changes. Designer uses this information to compose the OSGi pseudo-project that makes up the NSF in the workspace (by adding it to the plugin.xml historically and the MANIFEST.MF in 9.0.1 FP10+).

The second aspect is that the plugin is then available in the Target Platform. The Target Platform is an OSGi-ism basically meaning the OSGi runtime's view of the world - in Eclipse, it means the plugins that Eclipse knows about and which can be referenced by projects, such as the NSF. This doesn't have to be related at all to the running platform - and, in fact, it's common in OSGi development to have an entirely-distinct target platform to properly represent a server or other runtime environment.

The reason that these aspects are comingled is that, by default, Designer is configured to have a default Target Platform that's based on its runtime environment and any installed plugins. We, as XPages developers, thus generally don't have to think too much about it. "Install the plugin in Designer" is the single step that handles registering the library and making its classes available.

However, the platform is just a setting in the preferences, and it usually looks something like this:

Default Designer Target Platform

Since it's just a setting, and one that is commonly modified in Eclipse, there's nothing stopping us from modifying it. That's where I realized I was doing it the inefficient way all along. So I modified the active platform to point to the target/site folder of my update site Maven project:

Modified Designer Target Platform

With that change, Designer will see both the installed version as well as the latest results of the Maven build. So now, I can do a Maven build and then, in this dialog, click "Reload..." to get Designer to notice the changes. Once I do, voilà - the new methods/classes show up and I don't have to restart and lose my workspace state.

Quick Tip: Re-Enabling Disabled Designer Plugins

May 19, 2015 12:58 PM

Tags: designer

Recently, I had a case where my installed Designer plugins stopped appearing, immediately made obvious by the libraries disappearing from XPages applications and Designer listing hundreds of class-not-found errors. At first, I figured that the local plugins had been deleted, but trying to install from update sites curtly informed me that they contained nothing new for me.

It turned out that my local plugins had been somehow marked disabled by Designer. The fix for this was to go to File → Application → Application Management (you may have to launch Designer to see this option) and to enable them there. Crucially, the disabled plugins didn't show up until I clicked the "Show Disabled Features" button (forgive the grossly-outdated Notes version on this client machine):

Once I did that, the second category of plugins (in the data folder) listed everything I expected, and I was able to re-enable them there. One hitch to this process is that it requires sticking to the dependency order, so some plugins may refuse to be enabled until you enable others (commonly, any that depend on the Extension Library).

I'm not sure what specifically caused all these plugins to take a nap, but I suspect it's related to a recent Interim Fix or the Java update, since it happened around when I installed those, and I've heard others report the same behavior.

"Build Automatically", "Refresh automatically", and "Team Development"

Jan 23, 2014 6:06 PM

TL;DR: "Build Automatically" and "Refresh automatically" are unrelated, but work together when both enabled when using SourceTree for source control.

I just had a conversation on Twitter where I went a little crazy talking about my hopefully-correct view of how the various sync processes in Designer work when you're dealing with source control.

The original impetus was a post by Mark Leusink about the fact that you have to build your NSF project in order to sync it with an on-disk-project (ODP) set up via Team Development. From there, that got into the topic of the "Refresh automatically" Eclipse setting that is helpful when using source control tools outside of Designer (e.g. SourceTree instead of eGit). The problem is that this stack of builders and settings is hard to get a hold of conceptually.

There are three "entities" to know about when it comes to how these things work together, along with a couple processes that can get your source code from one to the other. I tossed together this terrible image to visualize them:

I'll work from right to left, starting with the NSF. That part is nothing new, but the thing to know is how the NSF "knows" how to sync to an ODP. It's done via something called an Eclipse Builder. A Builder is a task you can attach to a project in Eclipse to do essentially arbitrary things when the build process happens for a project (or an individual part of a project). NSFs have a ton of them:

The one in question is the unnamed one at the top (don't ask me why it's nameless): that's the Builder that exports your changes from the NSF to the ODP. There's a companion Builder, also unnamed, in the ODP, that does the reverse when you make a change to the files there (normally you don't, but source control does). Because this sync is implemented with a Builder, it doesn't run automatically when you have "Build Automatically" disabled. Instead, it will run when you manually do a "Build All" or "Clean", or when you choose "Team Development" → "Sync with On-Disk Project" from the application list.

So the Builder handles all of the syncing between the ODP and the NSF. Where does "Refresh automatically" (which is very different from "Build Automatically") come in, and why has it only become a thing recently with the cultural shift away from eGit and towards SourceTree?

The culprit is the way Eclipse/DDE deals with the filesystem. Because OSes have traditionally been bad at notifying programs of when files change on disk, DDE doesn't actually know when a file is changed on disk. This is expected for something like an NSF (say, if someone else adds a view to an NSF you have open, it's no surprise that you have to refresh to see it in Designer (also, don't edit the same NSF from two places like that)), but counterintuitive with local files. The view of the ODP you see in Designer's Package Explorer is analogous to running an "ls" or "dir" command in the terminal and then adding a new file: the output from your command won't change until you run it again. What "Refresh automatically" does is attempt to keep its file listing up-to-date. Later versions of Eclipse modified this option to be more explicit and technical:

The upshot is that turning this on will (hopefully) keep the ODP in step with the filesystem. The chart I made is misleading here: the ODP and filesystem are technically the same storage area, but the ODP you see in DDE is really a listing of what Designer saw the last time it looked.

This is useful when using a tool like SourceTree to handle your source-control operations. Because DDE doesn't normally "know" about filesystem changes, you could update your local source repository from the server with tons of changes, but Designer wouldn't know until you kicked it. With "Refresh automatically", it should notice those changes on its own, update the ODP in Package Explorer, and, if "Build Automatically" is on, sync those changes into your NSF.

The reason this matters now and didn't/doesn't when using eGit is that, because eGit is an Eclipse plugin and does its filesystem access through Eclipse, Designer already knows when a source-control-related operation changes files on the filesystem: since it made the change, it doesn't need to refresh. So if you're using an Eclipse plugin for source control and not editing files on the filesystem manually, you can remain in blissful ignorance about "Refresh automatically."

Basic Eclipse Plugin Installation in Designer

Mar 15, 2012 2:50 PM

Tags: designer

Since Domino Designer is based on Eclipse, one of the nifty advantages is that you can use some of the same plugins as vanilla Eclipse. That "some of" is a big caveat, since Designer's base isn't the latest Eclipse, so some plugins won't install, won't work, or will even cause Designer to stop launching until you manually remove them. So... proceed with caution.

The first thing you have to do is to enable Eclipse plug-in installation, which is something you've likely done if you've installed the Extension Library. You used to have to go into your workspace directory and edit a config file to enable it, but 8.5.3 made it much easier. There's now a simple checkbox in the "Domino Designer" section of Designer's preferences (the fuzzy fonts come from my fiddling with GDI++ in my VM):

Once that's enabled, you can go to File -> Application -> Install. On the resultant dialog, pick "Search for new features to install" and click Next. What you do next depends on whether your plugin is an update site you downloaded (like the Extension Library) or one that's hosted (as it seems most Eclipse plugins are). Since the one in question is Eclipse Color Themes, you can click "Add Remote Location...", enter in a Name of your choosing, and enter "http://eclipse-color-theme.github.com/update" as the URL:

After that, you can click "Next", check the boxes for the plugin you want, and "Next"/"Finish"/"Yes" your way through however many prompts it gives you before it's done. Then, after a possibly-necessary restart of Designer, you'll be able to find the configuration in the preferences: