[FIXED] How to make a plugin-like architecture runnable?

Issue

We have a couple of legacy Java projects, which we converted to Maven projects / modules. Previously, all projects were NetBeans projects and had no real dependency management. External dependencies existed on the companies network drive and were directly included as JARs in the NetBeans projects of each module. For the internal dependencies, simple project references were used. It was a pain to build everything because the programmer had to build everything in the right order.

Now, we are in the position that we can open all the Maven modules in IntelliJ IDEA and NetBeans. However, I am having trouble figuring out the best way to combine the different modules and external dependencies in a specific way, which conforms to in-house plugin-like structure. Especially with NetBeans (developing with both IDEs must be possible).

Here is how the git repositories / project structure roughly looks like. The folder structure of the modules is the default Maven structure for each module. The list feature of this site was too clumsy, so I included it as screenshot...

We have an internal maven repository for the stuff and building with maven etc. is working. For Intellij IDEA i can run and debug the end product for customer1 via a custom run configuration, which copies the needed files in the needed structure:

With IntelliJ IDEA, I can debug the software, but I think that the approach (custom IntelliJ run config I created, pointing to all needed JARs and files directly) is rather ugly, and for NetBeans I could not find a similar "run configuration" mechanism.

So I tried to achieve this build process by creating a new "Customer1Runnable" Maven project as a sort of build description, which points to all needed Maven modules. Based on this, I believed I could achieve and automatism to create the needed software structure. Ergo copy all modules into a plugin folder and all dependencies of the modules into a lib folder inside the Customer1Runnable project, using the maven-assembly-plugin.

First off, is my assumption correct that this is a possible use case for the maven-assembly-plugin?

The project itself does not have any source files, it is only a pom.xml and the assembly-config.xml descriptor. I attached the assembly-plugin to the package phase. When running the mvn package command all connected modules are built, but for the execution of the assembly-plugin I get the following output:

For starters, I only tried to include one module in the assembly descriptor. This is the XML (opicom-assembly.xml) for it:

<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 http://maven.apache.org/xsd/assembly-2.1.0.xsd">
    <id>opicom-assembly</id>
    <formats>
        <format>dir</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <moduleSets>
        <moduleSet>
            <useAllReactorProjects>true</useAllReactorProjects>
                <includes>
                    <include>my.company.reporting:module1</include>
                </includes>
        </moduleSet>
    </moduleSets>
</assembly>

pom.xml of Customer1Runnable project

 <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <version>1.6</version>
    <groupId>my.company.customer1</groupId>
    <artifactId>OpicomRunnable</artifactId>
    <packaging>pom</packaging>
    <name>OpicomRunnable</name>
    
    <repositories>
        <repository>
            <id>Company-Maven-Repo</id>
            <url>file:\\\\MyCompany\TFSDrop\MavenRepo</url>
        </repository>
    </repositories>
    
    <modules>
        <module>../my.company.customer1.module1</module>
        <module>../my.company.customer1.module2</module>
        .
        .
        .
        <module>../../MyCompany_Common/Report/my.company.reporting.module1</module>
    </modules>  
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>3.3.0</version>
                <inherited>true</inherited>
                <executions>
                    <execution>
                        <id>copy-dependencies</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <descriptors>
                        <descriptor>opicom-assembly.xml</descriptor>
                    </descriptors>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

The pom of a module looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>my.company</groupId>
        <artifactId>reporting</artifactId>
        <version>1.3</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

    <artifactId>module1</artifactId>
    <version>1.3</version>
    <packaging>jar</packaging>



    <dependencies>
        <!-- external dependencies -->
        <dependency>
            <groupId>commons-pool</groupId>
            <artifactId>commons-pool</artifactId>
            <version>1.6</version>
        </dependency>
        <dependency>
            <groupId>com.oracle.database.jdbc</groupId>
            <artifactId>ojdbc8</artifactId>
            <version>21.1.0.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>3.3.0</version>
                <configuration>
                    <finalName>my-company-${project.artifactId}</finalName>
                    <appendAssemblyId>false</appendAssemblyId>
                    <outputDirectory>../build</outputDirectory>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Thanks for any input on what I am doing wrong here / how to achieve this with Maven.

EDIT: As requested, here an example project as ZIP-File. https://drive.google.com/drive/folders/1ilJeTrOPgYgUTdOP0J4BQcBnPT5fls0k?usp=sharing The parent directories ModuleGroupCustomer and ModuleGroupCommon do represent git repositories in the real scenario. The relative module path is caused, because the maven project which should be my "run config" points to maven projects in both repositories.

Maybe I am misunderstanding Maven in general? I thought of it in terms of use cases for dependency management similar to .Net nuget packages, but also as "project configuration" like ordinary NetBeans/Intellij projects.

Is it better to simply stick to the existing NetBeans projects for day to day development?

Solution

After a long and tedious process of trial and error, I have found a solution which is working for me. So I decided to share the solution online, in case someone else runs into a similar problem. Here is a link to the final zip archive containing working example projects => File CustomerRunnable_RunningAssemblyPluginStackoverflowExample.zip https://drive.google.com/drive/u/0/folders/1ilJeTrOPgYgUTdOP0J4BQcBnPT5fls0k

My error was that I misunderstood how the assembly-plugin works. The approach that I executed the plugin inside my aggregator pom (CustommerRunnable) is wrong, as this maven project only exists as parent pom.

The CustommerRunnable pom.xml references all customer plugins as modules. Those modules have not the CustommerRunnable as parent, but a different pom. Then I created a separate maven project "distribution". The pom.xml of the distribution defines all the plugins (needed customer maven modules) as dependencies. It also has the CustommerRunnable pom.xml as parent. Hence when I run the project in NetBeans, all connected modules are also build(if necessary).

It also configures the assembly plugin. The assembly plugin is attached to the maven package-phase and thus executed with it. It also uses a custom assembly descriptor, which copies all the previously defined plugins into the right folders. This is done by using dependencySets with include and exclude patterns. See https://maven.apache.org/plugins/maven-assembly-plugin/advanced-descriptor-topics.html for details on this. So one dependencySet copies all jar files of all plugins to a /plugin folder by using an include pattern. Then this approach is inversed to copy the jar files of all external dependencies to a /lib folder.

The descriptor also defines some extra files to copy to a particular location. exec-maven-plugin, so I can comfortably start the customer software out of NetBeans. I didn't yet manage to configure the execute plugin correctly regarding the needed classpath arguments.

Endresult looks like this:

It is also worth noting that the configurations of the "Build project", "Run project" and "Debug project" inside NetBeans need a tiny bit of modification. (Right Click Module "distribution" -> "Properties" -> point "Actions"

Answered By - tilt32
Answer Checked By - Timothy Miller (JavaFixing Admin)