Gradle Plugin for EBean, JPA, and JDO Enhancing along with Minification and Overlaying

During the development of the Tangram framework project a set of build related things went into a plugin for the Gradle tool.

The functionality of this Tangram Gradle Plugin is only in very small parts directly related to Tangram. It is more or less a general purpose plugin for applications needing

• Byte-code transformation of model classes for
  JDO, JPA, and Ebean ORM layers
• Minification of CSS and JavaScript codes to be placed in WAR artifacts
• Support underlying of WAR files into others (similar to overlays)

The good news for today is, that with the latest version 1.0.5 the plugin can be used from the central Gradle plugins repository with the - not very surprising - id tangram.gradle.plugin. So some of the usage notes have to be aligned with this situation.

Usage

Just a few lines have to be added to your Gradle build script to use the plugin:

plugins {
  id "tangram.gradle.plugin" version "1.0.5"
}

All of the following steps described here take place without any additional configuration.

Prepare EBean, JPA, and JDO Model Classes

When used with Java projects - and when some data model classes are discovered, - the plugin tries to prepare them for use the respective Object Relational Mapper (ORM). The ORM APIs supported are

• Java Data Objects (JDO)
• Java Persistence API (JPA)
• EBean

These APIs in turn are supported by a number of implementations. The supported implementations are

• DataNucleus Access Platform (JDO and JPA)
• EBean 
• EclipseLink 
• Hibernate 
• OpenJPA

These OR-Mapper API implementations require (DataNucleus and EBean) or recommend (the others) to apply byte-code transformations called "Enhancing" or "Weaving" to the class files. The compiled code is extended with some database access support to implement the active record pattern more or less seamless.

The API and implementation library in use is discovered from the names of the elements of the class path of the project. If one of the mentioned libraries is found, the corresponding byte-code transformation is applied to the appropriate step (post compile or pre jar creation).

martin@nelson:~/proj/tangram/sites/naturinspiriert$ gradle clean build
:clean
:compileJava
Performing DataNucleus JPA byte code transformation.
ENHANCED (Persistable) : org.naturinspiriert.RootGroup
ENHANCED (Persistable) : org.naturinspiriert.Topic
ENHANCED (Persistable) : org.naturinspiriert.AbstractGroup
ENHANCED (Persistable) : org.naturinspiriert.ImageData
ENHANCED (Persistable) : org.naturinspiriert.Article
ENHANCED (Persistable) : org.naturinspiriert.Linkable
ENHANCED (Persistable) : org.naturinspiriert.Group
DataNucleus Enhancer completed with success for 7 classes. Timings : input=60 ms, enhance=54 ms, total=114 ms. Consult the log for full details
7 classes enhanced.
:processResources
:classes
:jar
:war
:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build

BUILD SUCCESSFUL

Total time: 4.53 secs
(Highlighted message to indicate use of the DataNucleus Enhancer)

The byte-code transformations directly use the transformer of the respective library in use except for the OpenJPA case, where the ant task of the enhancer is integrated. Some of the transformers issue some logging.

Switching Off

Additionally it is possible to switch of the byte code transformation for JPA annotated classes by adding to your build file

// build.gradle: 
enhancer.enabled=false

in case this might be necessary e.g. to only use the other parts of the plugin. Also EclipseLink, Hibernate, and OpenJPA support the use of plain Java classes without the byte-code transformation.

JavaScript and CSS Minification

When used in conjunction with the war plugin, CSS and JavaScript resources are automatically minified.

The plugin checks for resources with a filename extension .css for Cascading Style Sheets and .js for JavaScript. Matching resources are minified using the YUI Compressor.

It is not possible to minify resource included from archive files but only file resources local to your project. So contents from archives - while being included in the resulting web archive - cannot be minified. (We would expect WAR files to contain minified resources like WAR files generated using this plugin do.)

Web Application Underlying

The plugin introduces a configuration named webapp for modules using the war plugin. Dependencies given for this configuration are extracted into the resulting war artifact.

// build.gradle:
dependencies {
  webapp "tangram:tangram-core:$tangram_version:war@war"
  webapp "tangram:tangram-editor:$tangram_version:war@war"

  compile "tangram:tangram-core:$tangram_version"
  // Persistence API JPA
  compile "tangram:tangram-jpa:$tangram_version:nucleus"
  compile "org.datanucleus:datanucleus-api-jpa:$versions.datanucleus"
  compile "org.datanucleus:datanucleus-core:$versions.datanucleus"
  compile "$versions.jdo_api"
  compile "$versions.persistence_api"
  runtime "org.datanucleus:datanucleus-mongodb:$versions.datanucleus"

  compile "tangram:tangram-editor:$tangram_version"
  runtime "tangram:tangram-dinistiq:$tangram_version"
  runtime "org.slf4j:slf4j-log4j12:$versions.slf4j"

  providedCompile "$versions.servlet_api"
  providedCompile "$versions.jsp_api"
}

This process is not really described well if called overlay so I call in underlying, since your web application's directory in fact is the overlay so the other archives referenced and included must be an underlying.

If your WAR relies on the contents of another pre-packaged or incomplete WAR, the contents of the latter will be copied into your resulting web application while you can override any file in this archive from your local web application contents directory.

Version List

The plugin introduces a version object which collects some version strings for a number of libraries. This ensures that any project using the plugin can use these libraries with recent versions and version changes are applied in sync. The use of this part is optional and you have to explicitly use the versions in your build file since this cannot be applied transparently.

Some random examples:

dependencies {
  compile "org.pac4j:pac4j-openid:$versions.pac4j"

  compile ("org.apache.openjpa:openjpa:$versions.openjpa") {
    exclude group: 'asm'
  }

  compile "org.eclipse.persistence:org.eclipse.persistence.jpa:$versions.eclipselink"
  compile "$versions.persistence_api"

  compile "org.hibernate:hibernate-core:$versions.hibernate"
   
  compile "org.datanucleus:datanucleus-api-jpa:$versions.datanucleus"
  compile "org.dataucleus:datanucleus-core:$versions.datanucleus"
  compile "$versions.jdo_api"
  runtime "org.slf4j:slf4j-log4j12:$versions.slf4j"

  testCompile "org.testng:testng:$versions.testng"

  // your container will have this for you
  providedCompile "$versions.servlet_api"
  providedCompile "$versions.jsp_api"
}

Manual Mode

Of course it is still possible to call the methods performing the different tasks directly like described in the 0.9 plugin blog post. This should only be necessary if you e.g. want to enhance files in the unit test section of your code, which is considered a very rare case.

Gradle Plugin for JDO, JPA, Ebean etc in Tangram grown up

As part of the Tangram framework project a set of build related things went into a plugin for the Gradle build tool.
Only in part this plugin is directly related to Tangram but offers some generic tools suitable for many projects.
The plugin does

• Support underlying of WAR files into others (similar to overlays)
• Minify CSS and JavaScript codes to be placed in WAR artifacts
• Byte-code transform model classes for JDO, JPA, and Ebean ORM layers

Up to version 0.9 the plugin was more or less a collection of utility methods. It is now redesigned to do the same job transparently when appropriate and at the appropriate stage of the build process.

Usage

Just a few lines have to be added to your Gradle build script to use these add on functionality:

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath 'tangram:gradle-plugin:1.0.0'
  }
}

apply plugin: 'tangram'

All of the following steps described here take place without any additional configuration.

Web application underlying

The plugin introduces a configuration named webapp for modules using the war plugin. Dependencies given for this configuration are extracted into the resulting war artifact.
This process is not really described well if called overlay so I call in underlying, since your web application's directory in fact is the overlay so the other archives referenced and included must be an underlying.
If your WAR relies on the contents of another pre-packaged or incomplete WAR, the contents of the latter will be copied into your resulting web application while you can override any file in this archive from your local web application contents directory.

Minification

When - again - used in a project also using the war plugin, all files included from the project directory will be checked for CSS and JavaScript file extensions. Matching files are minified using the YUI Compressor.
It is not possible to minify resource included from archive files. So contents from archives while being included in the resulting WAR cannot be minified. (We would expect WAR files to contain minified resources like WAR files generated using this plugin do.)

Handle Model Classes

When used with Java projects and when some data model classes are discovered, the plugin tries to prepare them for use with one of the ORM frameworks Java Data Objects (JDO), Java Persistence API (JPA), or Ebean.
The OR-Mappers require or recommend to apply byte-code transformations called "Enhancing" or "Weaving" to the class files for the data model related classes.
The plugin supports DataNucleus Access Platform (JDO and JPA), EBean, EclipseLink, Hibernate, and OpenJPA. All the calling options from the 0.9 version of the plugin have be removed and the APIs are auto-discovered so the whole process is now - except for logging output - transparent during build.

Version List

The plugin introduces a version object which collects some version strings for a number of libraries. This ensures that any project using the plugin can use these libraries with recent versions and version changes are applied in sync. The use of this part is optional and you have to explicitly use the versions in your build file since this cannot be applied transparently.
Some random examples:
dependencies {
  compile "org.pac4j:pac4j-openid:$versions.pac4j"

  compile ("org.apache.openjpa:openjpa:$versions.openjpa") {
    exclude group: 'asm'
  }

  compile "org.eclipse.persistence:org.eclipse.persistence.jpa:$versions.eclipselink"
  compile "$versions.persistence_api"

  compile "org.hibernate:hibernate-core:$versions.hibernate"
   
  compile "org.datanucleus:datanucleus-api-jpa:$versions.datanucleus"
  compile "org.dataucleus:datanucleus-core:$versions.datanucleus"
  compile "$versions.jdo_api"
  runtime "org.slf4j:slf4j-log4j12:$versions.slf4j"

  testCompile "junit:junit:$versions.junit"

  // your container will have this for you
  providedCompile "$versions.servlet_api"
  providedCompile "$versions.jsp_api"
}

Manual Mode

Of course it is still possible to call the methods performing the different tasks directly like described in the 0.9 plugin blog post.
Also it is possible to switch of the byte code transformation for JPA annotated classes by adding to your build file

// build.gradle: 
enhancer.enabled=false

in case this might be necessary e.g. to only use the other parts of the plugin.

Gradle Plugin for JPA, JDO, and EBean Bytecode Transformers

The plugin referenced here is meant for use with the tangram framework but there are a few calls which are helpfull for any user of Gradle and

• EclipseLink
• OpenJPA
• DataNucleus (JDO or JPA style)
• Ebean

It simply has easy to use wrappers for the enhancer/wever components for the persistence frameworks above since the solutions presented by those projects each for its own reason needed some cosmetic enhancement to fit into my gradle builds.

Prerequisites

Of course you will have some classes with persistence annotations. I'm assuming that you have a (sub)module containing the model classes for the API to be used (JDO, JPA with OpenJPA, DataNucleus, or EclipseLink, or Ebean with JPA annotations) which get compiled by some task of your project.

Preparation

The plugin can be obtained from the tangram snapshots repository.

// build.gradle
buildscript {
  repositories {
    mavenCentral()
    maven { url "http://repository-tangram.forge.cloudbees.com/snapshot" }
  }
  dependencies {
    classpath "tangram:gradle-plugin:0.9-SNAPSHOT"
  }
}

And of course it must be applied

apply plugin: 'tangram'

When using EclipseLink the bytecode transformer called weaver will already be contained in the compile dependencies of your module. The same applies for OpenJPA except that here the bytecode transformer is called enhancer.
For DataNucleus und Ebean the bytecode transformer is called enhancer again and is a dependency of the plugin itself. There is no generic need to have it in the compile dependencies of your module.
So in short, you will not have to modify your dependencies and the resulting package will not contain any additional classes or jars.

Solution

The plugin does not introduce any new tasks but just some methods that can be placed anywhere in the build process. In a standard gradle task-wiring of the java plugin the methods the following locations make sense:

// JDO with DataNucleus
compileJava.doLast {
  nucleusJdoEnhance()
}

// JPA with DataNucleus
compileJava.doLast {
  nucleusJpaEnhance()
}

// JPA with OpenJPA
jar.doFirst {
  openjpaEnhance()
}

// Ebean
compileJava.doLast { 
  ebeanEnhance()
}

// JPA with EclipseLink
compileJava.doLast {
  eclipselinkWeave()
}

But you may decide to use it at other points within your build process.

Background

For OpenJPA the wrapper presented here is a simple wrapper for the ant tasks provided by this project wired up for the given gradle build setup.
For EclipseLink the weaver the jar is included as a dependency for the plugin itself and the weaver is called called via its Java API. So woven codes can be generated independent of the using projects build setup.
For DataNucleus and Ebean the solution is somewhat more complicated since the enhancers are in separate jars which you most likely don't want to include in your resulting packages.
So those jars are also included as a dependency of the plugin, and again the plugin itself calls the Java APIs of the enhancers directly.