In this post, we will create a modular Java project that uses Maven as its build tool.
Motivation. A modular project consists of several modules (e.g. an app + libraries) that are developed together with the app. On one hand, it’s convenient to create a Git repository and a Maven artifact per module. This gives us flexibility.
On the other hand, the changes should often be done across several modules, and it’s important to change the sources and test the changes in several modules at once. So the following cycle is pretty painful:
- change module A;
- build and upload an artifact to Maven;
- download the change from Maven to module B project;
- test if module B works fine with the updated module A;
- if not, rinse repeat.
And this is just with 2 modules. With 3 or more modules it becomes real trouble!
In this article, we will create a project that unites the best of both worlds: the modules will still be split into separate Git repositories and Maven artifacts while it will be possible to build and test them altogether.
Assumptions. We will use Maven 3.6.0 to build the project, more recent Maven versions will also be ok.
We will also use Atlassian Bitbucket Server/Data Center for hosting Git repositories. Any other Git hosting software would also be ok, though the steps could be different.
Finally, we will use Git X-Modules. For Bitbucket Server/Data Center there’s a dedicated app. For other Git servers, there’re other implementations of Git X-Modules.
Project structure. We will create a command-line tool that calculates SHA-1/MD5 checksums of the file specified. So there will be 2 modules:
- a core engine calculating the checksums (‘core’);
- a command-line tool as the user interface (‘cli’).
We will also create a ‘parent’ Git repository that would unite both ‘core’ and ‘cli’ repositories under a single project.
Step 1. Create ‘core’ repository.
Create ‘core’ repository using Atlassian Bitbucket Server/Data Center UI.
Clone this newly created empty repository:
git clone http://example.org/scm/maven-example/core.git core/
cd core/
Run “archetype:generate” goal to generate the project:
mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DoutputDirectory=../
and answer interactive questions. As the “artifactId” coincides with the repository name, pom.xml will be generated in the current directory.
...
Define value for property 'groupId': org.example
Define value for property 'artifactId': core
Define value for property 'version' 1.0-SNAPSHOT: :
Define value for property 'package' org.example: :
Confirm properties configuration:
groupId: org.example
artifactId: core
version: 1.0-SNAPSHOT
package: org.example
Y: :
Maven generates the default project for us with App.java and AppTest.java. Delete these classes and replace them with CheckSumCalculator.java:
package org.example;
import java.io.IOException;
import java.io.InputStream;
import java.math.BigInteger;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.DigestInputStream;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
public class ChecksumCalculator {
private final MessageDigest messageDigest;
public ChecksumCalculator(String checksumAlgorithm) throws NoSuchAlgorithmException{
this.messageDigest = MessageDigest.getInstance(checksumAlgorithm);
}
public String calculate(Path file) throws IOException {
try (final InputStream is = Files.newInputStream(file);
final DigestInputStream dis = new DigestInputStream(is, messageDigest)) {
final byte[] buffer = new byte[8192];
while (true) {
final int bytesRead = dis.read(buffer);
if (bytesRead < 0) {
break;
}
}
}
return toHex(messageDigest.digest());
}
private static String toHex(byte[] digest) {
return String.format("%032x", new BigInteger(1, digest));
}
}
and ChecksumCalculatorTest.java:
package org.example;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.NoSuchAlgorithmException;
import org.junit.Assert;
import org.junit.Test;
public class ChecksumCalculatorTest {
@Test
public void shouldAnswerWithTrue() throws Exception {
final Path file = Paths.get("target/test-data/emptyFile");
if (Files.exists(file)) {
Files.delete(file);
}
Files.createDirectories(file.getParent());
Files.createFile(file);
Assert.assertEquals("d41d8cd98f00b204e9800998ecf8427e", checksum("MD5", file));
Assert.assertEquals("da39a3ee5e6b4b0d3255bfef95601890afd80709", checksum("SHA-1", file));
}
private String checksum(String algorithm, Path file) throws IOException, NoSuchAlgorithmException {
return new ChecksumCalculator(algorithm).calculate(file);
}
}
Make sure Maven can build the project:
mvn clean package
You can specify Maven URL to upload the artifact to in pom.xml:
<distributionManagement>
<repository>
<id>target</id>
<url>${yourCompanyMavenReposUrl}</url>
</repository>
</distributionManagement>
Commit and push the change:
git add pom.xml
git add src
git commit -m "Initial."
git push origin master
You can run “mvn clean install” to install the artifact to the local Maven repository or “mvn clean deploy” to upload the artifact to your company’s Maven repository.
Step 2. Create the ‘cli’ repository.
Similarly create ‘cli’ repository with Bitbucket UI.
Clone the empty Git repository:
git clone http://example.org/scm/maven-example/cli.git cli/
cd cli/
And similarly, initialize the Maven project there:
mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DoutputDirectory=../
Define value for property 'groupId': org.example
Define value for property 'artifactId': cli
Define value for property 'version' 1.0-SNAPSHOT: :
Define value for property 'package' org.example: :
Confirm properties configuration:
groupId: org.example
artifactId: cli
version: 1.0-SNAPSHOT
package: org.example
Y: :
Edit pom.xml to depend on ‘core’ module (org.example:cli:1.0-SNAPSHOT). Also, add a dependency on Picocli library that allows creating a command-line interface easily.
<dependencies>
...
<dependency>
<groupId>org.example</groupId>
<artifactId>core</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>info.picocli</groupId>
<artifactId>picocli</artifactId>
<version>4.5.2</version>
</dependency>
</dependencies>
Also add “appassembler-maven-plugin” to pom.xml:
<properties>
...
<program.name>jchecksum</program.name>
<program.main.class>org.example.JChecksumCommand</program.main.class>
...
</properties>
<build>
...
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>2.1.0</version>
<executions>
<execution>
<id>appassemble</id>
<goals>
<goal>assemble</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
<configuration>
<repositoryName>lib</repositoryName>
<repositoryLayout>flat</repositoryLayout>
<assembleDirectory>${project.build.directory}/${program.name}-${project.version}</assembleDirectory>
<programs>
<program>
<mainClass>${program.main.class}</mainClass>
<id>${program.name}</id>
</program>
</programs>
</configuration>
</plugin>
</plugins>
</build>
This plugin creates a launching shell script for our command line tool.
<program.main.class>org.example.JChecksumCommand</program.main.class>
defines the main class to be run by the script. So from the src/ remove all the classes and create org.example.JChecksumCommand:
package org.example;
import java.nio.file.Path;
import java.util.concurrent.Callable;
import picocli.CommandLine;
@CommandLine.Command(name = "jchecksum",
mixinStandardHelpOptions = true,
version = "jchecksum 1.0",
description = "Compute checksum of the file specified.")
public class JChecksumCommand implements Callable<Integer> {
@CommandLine.Parameters(index = "0", description = "The file to calculate checksum.")
private Path file;
@CommandLine.Option(names = {"--algo"},
description = "SHA-1, SHA-256, MD5, ... (default: ${DEFAULT-VALUE})",
defaultValue = "MD5")
private String algorithm;
@Override
public Integer call() throws Exception {
final String checksum = new ChecksumCalculator(algorithm).calculate(file);
System.out.println(checksum);
return 0;
}
public static void main(String[] args) {
final int exitCode = new CommandLine(new JChecksumCommand()).execute(args);
System.exit(exitCode);
}
}
If you have ‘core’ artifact installed into your local Maven repository, you can build the project with:
mvn clean package
The resulting utility will be in target/jchecksum-1.0-SNAPSHOT directory. Now let’s test it:
echo line > /tmp/file
target/jchecksum-1.0-SNAPSHOT/bin/jchecksum --algo MD5 /tmp/file
5b4bd9815cdb17b8ceae19eb1810c34c
md5sum /tmp/file
5b4bd9815cdb17b8ceae19eb1810c34c /tmp/file
target/jchecksum-1.0-SNAPSHOT/bin/jchecksum --algo SHA-1 /tmp/file
6bfa09d82ce3e898ad4641ae13dd4fdb9cf0d76b
sha1sum /tmp/file
6bfa09d82ce3e898ad4641ae13dd4fdb9cf0d76b /tmp/file
Commit and push the module:
git add pom.xml
git add src/
git commit -m "Initial."
git push origin master
Step 3. Create the ‘parent’ repository.
As I wrote before, it’s painfully unconvenient to develop ‘core’ and ‘cli’ modules in parallel: change ‘core’, build and install it with “mvn clean install”, then open ‘cli’ project, test the updates…
So let’s create a project that gets both modules together, so both modules can be modified and tested on the fly. We will name this project ‘parent’.
So create ‘parent’ repository with Bitbucket UI.
Clone this empty repository:
git clone http://example.org/scm/maven-example/parent.git parent/
cd parent/
Create pom.xml file there:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>Parent Project for 'core' and 'cli'</name>
<description>Checksum Aggregate Project</description>
<modules>
<module>core</module>
<module>cli</module>
</modules>
</project>
This simple Maven configuration expects 2 ‘core’ module in core/ subdirectory, ‘cli’ module in cli/ subdirectory. So it’s enough to insert corresponding Git repositories into corresponding directories. We will do that at the next step. So far, commit and push the changes:
git add pom.xml
git commit -m "Initial."
git push origin master
Step 4. Inserting ‘core’ and ‘cli’ modules.
To insert one Git repository into another we will be using X-Modules. For Atlassian Bitbucket Server/Data Center there’s a dedicated app. For other Git servers, visit [link].
Now make sure you have the X-Modules app installed into Atlassian Bitbucket Server/Data Center. If not, visit Administration | Find new apps | Search the Marketplace and type “X-Modules” from Bitbucket Server/Data Center UI.
Now go to the ‘parent’ Git repository page on Bitbucket. As the X-Modules app is now installed, there should be an X-Modules button, click it.
Click ‘Add Module’ to insert the first module.
Choose ‘core’ repository and ‘master’ branch.
Make sure “This Repository Path” is ‘core’. This is ‘core’ repository insertion path, for Maven it should coincide with the module name.
Click ‘Add Module’.
Without applying changes click ‘Add Module’ again to add the second module.
Choose ‘cli’ repository and ‘master’ branch.
Also make sure “This Repository Path” is ‘cli’
Click ‘Add Module’.
Apply the changes.
Now the ‘parent’ repository has 2 subdirectories ‘core’ and ‘cli’ with corresponding Git repositories inserted into them.
Get the updates from the Git repository:
cd parent/
git pull --rebase
The directory content is now:
├── cli
│ ├── pom.xml
│ └── src
│ └── main
│ └── java
│ └── org
│ └── example
│ └── JChecksumCommand.java
├── core
│ ├── pom.xml
│ └── src
│ ├── main
│ │ └── java
│ │ └── org
│ │ └── example
│ │ └── ChecksumCalculator.java
│ └── test
│ └── java
│ └── org
│ └── example
│ └── ChecksumCalculatorTest.java
└── pom.xml
You can now build it with
mvn clean package
and the result will be in cli/target/jchecksum-1.0-SNAPSHOT/ subdirectory.
You can open the ‘parent’ repository in IDE, modify and push changes to it. Changes corresponding to ‘core’ and ‘cli’ subdirectories will be translated to the corresponding Git repositories.
Synchronization in the other direction works similarly. When one pushes changes to ‘core’ or ‘cli’ Git repositories, the corresponding subdirectories of the ‘parent’ repository will be automatically updated by X-Modules.
This configuration is optimal for any Maven-based project. It has the advantages of modular structure and of monorepo allowing to modify several projects at once without “mvn install” or “mvn deploy” round-trip.