Sunday 14 May 2017

Building OpenJDK 9 with OpenJFX 9 on Ubuntu (VBox and WSL)

Wanting to test Windows Subsystem for Linux, I've refreshed my steps for building on Ubuntu.
(Dependencies come from the helpful configure messages for the JDK and from the OpenJFX wiki)

Quick steps for installing WSL:
  1. Activate the developer's mode in settings
  2. In an administrator PowerShell console run:
    Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
  3. Open Bash and install Ubuntu
 (the following also applies to a VirtualBox VM)

 To build OpenJFX and OpenJDK 9:
  1. Install the dependencies:
    sudo apt install mercurial openjdk-8-jdk make unzip zip g++ libx11-dev libxext-dev libxrender-dev libxtst-dev libcups2-dev libfreetype6-dev libasound2-dev ksh bison flex gperf libasound2-dev libgl1-mesa-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libjpeg-dev libpng-dev libxml2-dev libxslt1-dev libxt-dev libxxf86vm-dev pkg-config x11proto-core-dev x11proto-xf86vidmode-dev libavcodec-dev libgtk2.0-dev libgtk-3-dev libxtst-dev libudev-dev libavformat-dev
  2. Create the dev directory:
    mkdir ~/dev
    cd ~/dev
  3. Retrieve specific dependencies for OpenJFX:
    wget https://services.gradle.org/distributions/gradle-3.1-bin.zip
    unzip gradle-3.1-bin.zip
    export PATH=$PATH:~/dev/gradle-3.1/bin

    wget http://archive.apache.org/dist/ant/binaries/apache-ant-1.8.2-bin.zip
    unzip apache-ant-1.8.2-bin.zip
    export PATH=$PATH:~/dev/apache-ant-1.8.2/bin

    wget http://download.java.net/java/jdk9/archive/168/binaries/jdk-9-ea+168_linux-x64_bin.tar.gz
    gunzip jdk-9-ea+168_linux-x64_bin.tar.gz
    tar x -f jdk-9-ea+168_linux-x64_bin.tar
  4. Get the sources:
    hg clone http://hg.openjdk.java.net/openjfx/9-dev/rt jfx

    hg clone http://hg.openjdk.java.net/jdk9/jdk9/
    cd ~/dev/jdk9
    bash ./get_source.sh
  5. Set the env variables for OpenJFX:
    export _JAVA_OPTIONS="-Dsun.reflect.debugModuleAccessChecks=true --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED -Dorg.gradle.daemon=false"

    export JAVA_HOME=~/dev/jdk-9/
    export JDK_HOME=~/dev/jdk-9/
  6. Build OpenJFX:
    cd ~/dev/jfx
    gradle
  7. Unset the env variables for OpenJFX (to avoid interfering with the configuration of OpenJDK):
    export -n _JAVA_OPTIONS
    export -n JAVA_HOME
  8. Configure and Build OpenJDK:
    cd ~/dev/jdk9
    bash configure --with-import-modules=`echo ~/dev/jfx/build/modular-sdk/`
    make images
  9. Enjoy

Sunday 2 April 2017

WorldClock and Jigsaw (the Java 9 edition)

Although I have other applications running as Jigsaw modules, I hadn't migrated WorldClock to it. Now it is.


WorldClock is currently composed of a set of 'modules':
  • config
    that will read/write a configuration from/to disk
  • geonames4lhwc
    that will connect to the Geonames service and retrieve a list of cities for a search string
  • application
    the application
  • editor
    an editor for the configuration
  • panel
    the WorldClock panel that shows the day and night on Earth
  • schema
    the XML schema for the configuration file
where
  • config depends on schema
  • editor depends on schema, config, geonames4lhwc and org.jdesktop:appframework
  • application depends on panel
  • schema and geonames4lhwc depend on XML schemas

With a new version of Java, I usually
  1. get my program to run with the new version
  2. get my program to compile with the new version

With Jigsaw the following steps can be added
  1. create modules
  2. run in 'modular mode'
  3. create a custom image

Let see how it goes.

1. Run WorldClock with JDK 9


The first step to move WorldClock to Jigsaw is to try to run the code compiled with Java 8 on JDK 9:
  • This works without issue for application.
  • But fails for editor with:
    Exception in thread "AWT-EventQueue-0" java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
        at lh.worldclock.editor.EditorView.initComponents(EditorView.java:113)
    This because it belongs to a Java EE API and in JDK 9 Java EE APIs are not resolved by default for code on the class path (this to make it easier for the applications servers... see http://openjdk.java.net/jeps/261#EE-modules)
    To fix it '--add-modules java.xml.bind' needs to be added to the command line.

2. Compile WorldClock with JDK 9


The second step is to compile with JDK 9:

Building requires using Maven compiler 3.6.1 plugin in the main pom:
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <version>3.6.1</version>
</plugin>
 As well setting the release to 9, via the property:
<maven.compiler.release>9</maven.compiler.release>
 (for NetBeans, <maven.compiler.source>9</maven.compiler.source> and <maven.compiler.target>9</maven.compiler.target> are also added)

Then for the subprojects some ajustments are required to add the Java EE API modules:
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <compilerArgs>
      <arg>--add-modules</arg>
      <arg>java.xml.bind</arg>
    </compilerArgs>       
  </configuration>
</plugin>
for config, editor, geonames4lhwc

and
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <compilerArgs>
      <arg>--add-modules</arg>
      <arg>java.xml.ws.annotation,java.xml.bind</arg>
    </compilerArgs>       
  </configuration>
</plugin>
for schema.

geonames4lhwc and schema create some classes from XSDs, this used to be done with maven-jaxb-plugin, however this plugin currently does not work with JDK 9, the jaxb2-maven-plugin does not work either, but in its 43rd issue https://github.com/mojohaus/jaxb2-maven-plugin/issues/43 Gunnar Morling has a workaround consisting in calling maven-antrun-plugin and exec-maven-plugin to create the classes in a less integrated manner.

This is the workarround for geonames4lhwc:
<!-- Create target dir -->
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-antrun-plugin</artifactId>
  <version>1.4</version>
  <executions>
    <execution>
      <phase>generate-sources</phase>
      <configuration>
        <tasks>
          <echo message="Creating target/generated-sources/jaxb"/>
          <mkdir dir="./target/generated-sources/jaxb"/>
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

<!-- Invoke xjc -->
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>generate schema types</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>exec</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <executable>xjc</executable>
    <arguments>
      <argument>-enableIntrospection</argument>
      <argument>-encoding</argument>
      <argument>UTF-8</argument>
      <argument>-p</argument>
      <argument>lh.worldclock.geonames.schema</argument>
      <argument>-extension</argument>
      <argument>-target</argument>
      <argument>2.1</argument>
      <argument>-d</argument>
      <argument>target/generated-sources/jaxb</argument>
      <argument>src/main/schema/geonames.xsd</argument>
    </arguments>
  </configuration>
</plugin>

<!-- Add target dir to compilation -->
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>add-source</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>add-source</goal>
      </goals>
      <configuration>
        <sources>
          <source>target/generated-sources/jaxb</source>
        </sources>
      </configuration>
    </execution>
  </executions>
</plugin>
And this is the workarround for schema:
<!-- Create target dir -->
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-antrun-plugin</artifactId>
  <version>1.4</version>
  <executions>
    <execution>
      <phase>generate-sources</phase>
      <configuration>
        <tasks>
          <echo message="Creating target/generated-sources/jaxb"/>
          <mkdir dir="./target/generated-sources/jaxb"/>
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

<!-- Invoke xjc -->
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>generate schema types</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>exec</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <executable>xjc</executable>
    <arguments>
      <argument>-enableIntrospection</argument>
      <argument>-encoding</argument>
      <argument>UTF-8</argument>
      <argument>-p</argument>
      <argument>lh.worldclock.config.schema</argument>
      <argument>-extension</argument>
      <argument>-target</argument>
      <argument>2.1</argument>
      <argument>-d</argument>
      <argument>target/generated-sources/jaxb</argument>
      <argument>src/main/resources/worldclock.xsd</argument>
    </arguments>
  </configuration>
</plugin>

<!-- Add target dir to compilation -->
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>add-source</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>add-source</goal>
      </goals>
      <configuration>
        <sources>
          <source>target/generated-sources/jaxb</source>
        </sources>
      </configuration>
    </execution>
  </executions>
</plugin>

With those the projects now compile.

3.  Create modules


The third step is to create 'Jigswaw' modules for the 'Maven' modules:
This by adding a module-info.java in src/main/java for each module.

The simplest module-info is for panel:
module lh.worldclock.panel
{
  exports lh.worldclock.core;
  requires java.desktop;
}
It exports the package lh.worldclock.core and requires the module java.desktop (which contains Swing).


Next is application:
module lh.worldclock.application
{
  requires lh.worldclock.panel;
  requires java.xml;
  requires java.desktop;
}
It requires the modules lh.worldclock.panel (to access the classes in the exported lh.worldclock.core package),  java.xml (for loading the XML configuration) and java.desktop (for Swing and AWT).



Next is schema:
module lh.worldclock.schema
{
  exports lh.worldclock.config.schema;
  opens lh.worldclock.config.schema to java.xml.bind;
  requires java.xml.ws.annotation;
  requires java.xml.bind;
}
It exports the generated package lh.worldclock.config.schema and
requires java.xml.bind and java.xml.ws.annotation,
it also opens lh.worldclock.config.schema to java.xml.bind that is it allows java.xml.bind (aka JAXB)
to use reflection on lh.worldclock.config.schema in order to instanciate its classes.


Next is config:
module lh.worldclock.config
{
  exports lh.worldclock.config;
  requires lh.worldclock.schema;
  requires java.xml.bind;
}
It exports lh.worldclock.config and requires java.xml.bind (JAXB) and lh.worldclock.schema (for lh.worldclock.config.schema).


Next is geonames4lhwc:
module lh.worldclock.geonames4lhwc
{
  exports lh.worldclock.geonames4lhwc;
  exports lh.worldclock.geonames.schema;
  opens lh.worldclock.geonames.schema;
  requires java.xml.bind;
  requires java.logging;
}
It exports the package lh.worldclock.geonames4lhwc, the generated package lh.worldclock.geonames.schema,
opens that same package,
requires java.xml.bind and java.logging (the later so that the exceptions in
lh.worldclock.geonames4lhwc.GeonamesWSWrapper can be logged)


Finally is the editor:
open module lh.worldclock.editor
{
  exports lh.worldclock.editor to appframework;
  requires lh.worldclock.schema;
  requires lh.worldclock.config;
  requires lh.worldclock.geonames4lhwc;
  requires appframework;
  requires swing.worker;
  requires java.desktop;
  requires java.logging;
  requires java.xml.bind;
}
It requires java.xml.bind (JAXB), java.logging, java.desktop (Swing), the schema, config and geonames4lhwc worldclock modules,
the swing.worker and appframework are modules providing the application framework on which
the editor is based.
The package lh.worldclock.editor is only exported to the appframework module so that it can't
be used by another module.
And the module is open so that its resources (icons and properties) can be accessed
(opening the resources packages does not work since they are not known in the compile phase of Maven and opened packages are validated by the compiler).

4. Run in 'modular mode'


The fourth step is to run the application and editor in 'modular mode'.

For that first lets create the module path, the class path for modules, by copying all the jars created by the project as well as appframework and swing-worker to a single directory. Although appframework and swing-worker don't have module-info by being put on the module path they be treated as (automatic) modules.
(on Windows:
rmdir /S /Q target\mods
mkdir target\mods
copy %USERPROFILE%\.m2\repository\org\jdesktop\appframework\1.0.3\appframework-1.0.3.jar target\mods
copy %USERPROFILE%\.m2\repository\org\jdesktop\swing-worker\1.1\swing-worker-1.1.jar target\mods
copy application\target\application-0.8-SNAPSHOT.jar target\mods
copy config\target\config-1.1-SNAPSHOT.jar target\mods
copy editor\target\editor-1.1-SNAPSHOT.jar target\mods
copy geonames4lhwc\target\geonames4lhwc-1.1-SNAPSHOT.jar target\mods
copy panel\target\panel-0.8-SNAPSHOT.jar target\mods
copy schema\target\schema-1.1-SNAPSHOT.jar target\mods
)

Then to run application:
java --module-path target\mods --module lh.worldclock.application/lh.worldclock.WorldClock

Likewise for editor:
java --module-path target\mods --module lh.worldclock.editor/lh.worldclock.editor.EditorApp

5. Create a custom image


The last step is to make use of a new feature in JDK 9 that allow to create a custom JDK/JRE image with just the modules needed by the application. It is provided by running the command jlink.

For application it is simple:
jlink --output app --module-path target\mods;"%JDK%\jmods" --add-modules lh.worldclock.application --launcher worldclock=lh.worldclock.application/lh.worldclock.WorldClock
where
  • app is the directory that will contain the custom image
  • %JDK% is the path to the JDK, its jmods directory contains the JDK modules (packaged as .jmod to also include extra data, like binaries or configurations)
  • --add-modules will list the root modules, with other modules added from the module paht when they are dependencies of the root modules
  • --launcher creates a launcher script/.bat that will launch java in the custom image with the given module/main class
    (on Windows, the java command can be replaced with javaw to avoid the console window by editing the .bat)
For the editor it is more involved... as it depends on appframework and swing-worker which automatic module (thus don't have explicit dependencies as they don't have a module-info and they may have some access to the class path which could not quite be included in the image) and so can't be added to an image.

So appframework and swing-worker need to be turned into proper modules.
This means adding a module-info for each then updating their jars to include it.

Module-info for appframework (in modextra\appframework\module-info.java):
open module appframework
{
  exports org.jdesktop.application;
  requires swing.worker;
  requires java.desktop;
  requires java.logging;
}
Module-info for swing-worker (in modextra\swingworker\module-info.java):
module swing.worker
{
  exports org.jdesktop.swingworker;
  requires java.desktop;
}
Since appframework has a dependency on swing-worker, lets first modularise swing-worker.
First extract its content in a directory:
cd target
mkdir swing-worker-1.1
cd swing-worker-1.1
jar --extract --file ..\..\target\mods\swing-worker-1.1.jar
cd ..
Then compile the module-info (the compilation requires the exploded jar):
javac -d swing-worker-1.1 ..\modextra\swingworker\module-info.java
Then update the jar with the compiled module-info:
jar --update --file mods\swing-worker-1.1.jar -C swing-worker-1.1 module-info.class

Similarly for appframework:

Extract its content in a directory:
mkdir appframework-1.0.3
cd appframework-1.0.3
jar --extract --file ..\..\target\mods\appframework-1.0.3.jar
cd ..
Compile (note the -p mods to include the swing-worker module):
javac -p mods -d appframework-1.0.3 ..\modextra\appframework\module-info.java
Jar update:
jar --update --file mods\appframework-1.0.3.jar -C appframework-1.0.3 module-info.class

Now the editor's image can be created with:
jlink --output editor --module-path mods;"%JDK%\jmods" --add-modules lh.worldclock.editor --launcher editor=lh.worldclock.editor/lh.worldclock.editor.EditorApp

And that's it.
WorldClock is modularised and even have 'stand alone' application and editor.

The full source code as well as the batch files can be found on GitHub on the Jigsaw branch of WorldClock.