Tycho and Tribulations
Mar 14, 2015, 3:02 PM
For the last few weeks, a large part of my work has involved dealing with Maven, and much of that also involves Eclipse. It's been quite the learning experience - very frustrating much of the time, but valuable overall. In particular, any time OSGi comes into play, things get very complicated and arcane, in non-obvious ways. Fair warning: this blog post will likely start out as an even-keeled description of the task at hand and descending into just ranting about Maven.
To start out, it's important to know that this sort of development involves three warring factions, each overlapping and having distinct views of the world. In theory, there are plugins that sort out all the differences, but this doesn't play out very smoothly in reality. Our players are:
- Maven. This is the source of our trouble, but overall worth it. Maven is a build system for Java (and other) projects that brings with it great powers to do with dependency management, project organization, packaging, distribution, and any number of other things. I've been increasingly dealing with it, initially as an observer while the ODA team descended into madness to convert that project, and then Maven-izing my own framework. Its view of the world is of "artifacts" - conceptual units like junit, poi, or other dependencies, plus your own project components - organized in a tree of modules and available via repositories. Its build process is a multi-stage lifecycle with hooks for plugins at each step of the way.
- Eclipse. Eclipse-the-IDE has its own view of how a Java project should be organized and built, and it doesn't involve Maven. There is a plugin for Eclipse and Maven, m2eclipse, that is meant to patch over these differences, but it can only go so far - while it helps Eclipse know a bit about Maven dependencies and its plugins, it's very dodgy and often involves trying to sync the Eclipse build configuration to be an imperfect representation of the Maven config.
- OSGi. OSGi is a packaging, dependency, and runtime model, and is spoken natively by Eclipse (and Domino). However, it butts heads with Maven: they both cover the "packaging and dependencies" ground, and this creates a mess. Again, there are plugins to help bridge the gap, but these bring another layer of complexity and brittleness to the process.
Maven, Eclipse, and OSGi go together like oil, water, and a substance that dissolves in water but reacts explosively with oil.
The interaction with OSGi deserves a bit of further explanation. Unlike the Maven/Eclipse bridge, where there's basically one tool to work with, imperfect as it is, dealing with Maven+OSGi has two distinct plugins, which may or may not be required for your needs:
- Tycho. This is the big one, intended to give Maven a thorough understanding of OSGi's view of the world, parsing the MANIFEST.MF files and hunting down dependencies using both an OSGi environment and Maven's normal scheme (if you tell it to correctly). If you're writing a full-on Eclipse/OSGi plugin/feature set (like ODA or my Framework), Tycho will be involved.
- The Maven Bundle Plugin. This confusingly-named plugin is specifically referring to "bundle" in the OSGi sense, which is the elemental form of an OSGi plugin (the terminology begins to really overlap here). Its role is to take a non-OSGi project - say, a "normal" Maven project you or a third party wrote - and generate a MANIFEST.MF for you, allowing you to create an OSGi-friendly project usable as a dependency elsewhere.
These two projects, though often both required, are not related, and are crucially incompatible in one major way: Tycho's dependency resolution runs before the Bundle Plugin can do its job. So you can't, for example, have a Bundle project that generates an OSGi-friendly plugin and then depend on it in a "real" OSGi context inside the same Maven build. As far as I can tell, the "fix" is to separate these out into separate Maven projects. So, if you want to consume Maven projects and convert them into OSGi plugins without also manually managing plugin stuff and dependency copying, you have to make it a two-step process. The reason for this is that computers suck.
Eclipse and Maven
Throughout this sort of development, there's a constant gremlin on your back: the distinct worlds of Eclipse and Maven. Many changes to the pom.xml (Maven's project descriptor file) will prompt Eclipse to tell you that its project config is out of date and that you must click a menu item to sync it, which it refuses to do itself for some reason. Additionally, you will frequently run into a case where you'll paste in a block of Maven XML from somewhere and it will be legal for Maven, but Eclipse will complain about not having lifecycle support for it. If you're lucky, you can click the "quick fix" to download an adapter automatically, or failing that tell it to ignore that part. Other times, it'll give you some cryptic error about packaging or the like and offer no solution. The "fix" at that point is often to stop trying to do what you want to do.
Because of these and other conditions, it's fairly easy to get into a situation where the project will compile in Eclipse but not in Maven or vice-versa. Sometimes, this isn't too bad to fix, such as when you just need to add a dependency to a given project in Maven. Other times, things will get more arcane, requiring seeking out more blocks of Maven XML (this is a common task) to either let Maven or Eclipse know about the other, or to at least tell Eclipse to not bother trying to process part of the Maven project. This process is most similar to an adventure game, trying different combinations of plugins and pasted XML until it works or you quit and try a different career path.
Capping these problems off is the peculiar nature of documentation for all this. From my experience, it comes in a couple forms:
- Official documentation that is either a very basic getting-started tutorial or assumes you have a complete understanding of Maven's conceits and idioms to read what they're talking about.
- Individual plugin pages with varing levels of thoroughness, and usually no mention of interaction with other components.
- Blog posts and Stack Overflow questions from 3-5 years ago, half of which amount to "X doesn't work", and most of the rest of which contain blocks of XML to try pasting into your pom without much explanation.
After working with Maven long enough, you start developing a vague, disjointed understanding of how it works - how the "plugins" inside "build" differ from those inside "pluginManagement", for example - but it's slow going. It seems to be the sort of thing where you just have to pay your dues.
Conclusion For Now
Things are very gradually coming together, and the benefits of Maven are paying off as I start avoiding the pitfalls and implementing things like Jenkins. Once I properly sort out the projects I'm working on, I'll post more about what I learn to be the right ways to accomplish these goals, but for now my assessment remains "Maven is a huge PITA, but overall probably worth it".
René Winkelmeyer - Mar 15, 2015, 5:19 AM
I'm really looking forward going down this road next month.
Cameron Gregor - Mar 15, 2015, 8:45 AM
Jesse I think this is a really important post and I appreciate the honesty within it about how difficult it is to get started with maven + osgi + eclipse.
When I reached the point of automating the build for our XPages plugins, I had a look at maven and tycho, but from the initial difficulties I had (which were pretty much what you have described) I felt that the learning curve would just not be worth the end result. I thought, "I'll just wait for someone else to figure it out and blog about it". And when I heard that the extlib and openntf api were going mavenised I thought that would be great and it would only be a matter of time before a few blog posts came out describing how to set up a plugin project with maven, and listing all the benefits of doing so, but yours is the first post I've seen which mention any negatives about it and by the sounds of it I am still glad I haven't gone down the maven path yet. Whilst I have managed to clone the extlib and build it using the instructions on openntf, I still have no idea how to set up an eclipse / osgi / maven project or how it all fits together.
I went with eclipse's PDE Headless Build system because it was a simpler solution, any restructuring required to the projects was minimal and it uses the exact same code that eclipse uses to build the plugins when you do it using the eclipse IDE (e.g. building an update site project).
I run it using an ant task with Jenkins.
It doesn't manage all the dependencies like maven however I find that although this requires some manual maintenance, the occasional annoyance of this is not bad enough for me to bother with maven yet, but I am still very curious about how it all works so it is great to see posts like yours with some realities of the learning experience.