What Maven Should Have Been, but Probably Couldn’t Be

I’m not a fan of Maven, more precisely, I’m not a fan of using it as a developer tool.  As a build engineering tool, it’s fine; that’s for build engineers though, not developers.

Maven is a package management system, largely for Java though I’ve seen it used for other languages.  Essentially it allows you to organize your projects in a hierarchical manner where dependencies are inherited, and automatically resolve those dependencies at build time.

Metacello, in some ways, is the Smalltalk equivalent, the Smalltalk package manager. However rather than being a CLI tool like Maven, Metacello, like the majority of Smalltalk tools, is a GUI based tool (the GUI development environment for Smalltalk, or at least the core of it, is part of the language specification).

My other project right now involves the use of Smalltalk to handle XML data streams (using the Xtreams Smalltalk library) that are at too high a rate for Java on any deployment machine we can justify.  These streams are filtered in Smalltalk and the relevant data then passed to a Java EE Portal, as well as stored in Oracle and MongoDB. They’re also stored locally on the machine, which monitors various things, in the OSGi container (Java) VM that also resides there in an XML DB called eXist DB.

So much for the project – it’s nothing fancy.

As a developer, everything is always too slow, always takes too long. It gets to the point where whether you wait a few minutes or a few hours, it seems the same – the only speed anything goes is too slow.  Yesterday though I happenstantially ran a comparison between Metacello and Maven.

Since Pharo 5.0 was recently released, I’d been meaning to migrate the Smalltalk portion to the new version.  Since my laptop’s old hard drive bit the dust last week, I was doing a build, which uses Maven, for the Portal my software has to integrate with.  Since that build was taking a long time, too long, as I said, I decided it would be a good time to migrate the Smalltalk code.  So I fired up a fresh copy of Pharo 5 with only the default libs in the image.

One of my shortcuts in building a new environment is that I know that certain projects use a good set of dependencies, ones I also use, and by calling the Metacello configuration on the relevant project, it will configure not only that project but all its dependencies.  It will actually take Pharo from a bare Smalltalk IDE into a running web application server with various other necessities, such as an XML parser and writer, the aforementioned Xtreams, JQuery and various JQuery widgets for the UI, bootstrap.js and a set of good starting points for making the CSS look like the rest of the Portal, but doing it programmatically (I’m not big on configuration files, I trust my code far more).  And of course all the IDE tooling for all those components.

As it happens the parent application that uses all these dependencies is a Smalltalk CMS called Pier3.  While it is a different app, with different aims, to the Portal I was building, it uses very equivalent technologies (as far as Java and Smalltalk have equivalents) and is about the same complexity, if anything Portal is simpler, because it’s mainly a veiewer, whereas Pier3 is a fully functional CMS such as Liferay, WordPress or other web CMS’s written in various ways, so it has more in terms of editing facilities. So I loaded the Metacello ConfigurationOfPier3 specifying version 3.1.1 of Pier3, which would give me the dependency versions I wanted.  So far, this won’t sound unfamiliar to someone who uses Maven, although the ConfigurationOfPier3 is also programmatic, rather than configuration file based.

However, since Pharo is an interpreted, open source Smalltalk dialect, unlike Maven it doesn’t want built artifacts, for the good reason that Smalltalk doesn’t have them.  But to put it in Java terms, what this code does equates to the following:

  1. Start from a bare Eclipse Java install, without any plugins for Java EE, persistence, web, web services, REST, etc. etc., just the simple Eclipse for Java Developers distribution.
  2. Run some sort of script, realistically it would have to be its own Eclipse plugin or set of plugins, but assume its part of the base distro.
  3. Eclipse will then do a GIT clone of the repositories of the parent project, and all dependent projects, from wherever they’re hosted, and checkout the revision specified in the configuration.  It will also install all the necessary Eclipse plugins to handle everything Portal uses – Oracle connectivity, Java EE,  JavaScript, bootstrap.js, etc. etc.It will also download the server runtime (in Java that might be Virgo OSGi, Glassfish, Wildfly, etc.)
  4. Unlike Maven which fetches built artifacts (usually JAR files) it will pull everything as source, run a test compile of all the code as it loads it, and if there are any issues (say, a conflict with something in the new base Pharo, it will pop up the conflict resolution browser for you to fix it before it blows up your VM.
  5. Once loaded every object is live, even with the state it had when the last committer on the project committed his objects.  The server is also live and there’s no need to deploy it, it is already available to the server, just point your browser at localhost:8080/Pier3 and voila, a working CMS.

This process also took a while (particularly since I live in the middle of nowhere and have satellite internet, so not much bandwidth for all those downloads.  In fact it took just over 20 minutes.  I’d love to have the kind of hardware and bandwidth that would allow me to do the same in Java in quadruple that time.

Instead, during that time Maven was still building the same project, one of 9 projects in the Portal.

Since Metacello’s Configurations wouldn’t gain much by using tons of parallelism (nor would the Maven build – both mainly use one thread with a second occasionally when it’s an advantage, so on a 4 core CPU there’s no significant contention), and it guarantees order of loading which implies it has to be primarily serially implemented. The only significant potential contention involved was the rather scarce network resources, and at least as far as source goes, the repositories online are the same, since the Monticello versionner and Metacello are really just different clients to the GIT object repository.  Thus the libs could be in smalltalkhub, github, etc, whether Java or Smalltalk, so there’s no ‘magic’ helping out Metacello on the server side.  The download speed of  the Java artifacts, in any case, was no slower than it had been prior to the Metacello configuration running. Even with my limited bandwidth, if the server doesn’t send data any faster it makes no difference.

Maven couldn’t, of course, be designed this way fully.  Firstly, as a language which has changed over different releases, but must compile to bytecode that runs on a largely unchanged VM, different Java libs they may all run fine on the same VM,  but may need to be built with different Java versions and different IDE configurations.  More importantly, installing the Eclipse plugins to create a sufficient environment to build the source projects and account for scripted use of libs and other Maven plugins requires a restart of Eclipse, not all plugins are compatible with one another, and not all plugins used to build the artifacts in the Maven repositories would work in the current version of Eclipse.  For various reasons the complexity of creating a “MetaJavaCello”) explodes the moment you think about what it would actually take.

NPM could have been designed this way, since it does in fact, like Metacello, pull source.  Unfortunately as with everything in the NodeJS sphere, if you don’t use Vi(le) you’re not a real cockroach-coder.

For anyone interested in Metacello and open source Smalltalk with Pharo 5.0, and who can maybe go beyond me in figuring out how to write a Java equivalent, the latest description of Metacello from the book Deep into Pharo is linked here.


The Open Source Immersive Programming Environment in Smallt



Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s