[FIXED] Create a large zip assembly with maven-assembly-plugin

Issue

I'm having a problem with creating a larger zip assembly (uncompressed takes over 3GB) using maven-assembly-plugin. The problem occurs when building the output zip file (compressed less than 1GB). Running maven with option -e gives me a more detailed info:

[INFO] Building zip: xxx/HG19-UCSC-dist.zip
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:40.199s
[INFO] Finished at: Wed Oct 02 11:08:44 BST 2013
[INFO] Final Memory: 13M/723M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single (make-assembly) on project HG19: Execution make-assembly of goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single failed: invalid entry size -> [Help 1]
...
...
...
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution make-assembly of goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single failed: invalid entry size
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:115)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
    ... 19 more
Caused by: java.lang.IllegalArgumentException: invalid entry size
    at java.util.zip.ZipEntry.setSize(ZipEntry.java:135)
    at org.codehaus.plexus.archiver.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:352)
    at org.codehaus.plexus.archiver.zip.ZipOutputStream.finish(ZipOutputStream.java:316)

Looking around I found that the issue comes from missing support for Zip64 in my JDK/JRE (https://blogs.oracle.com/xuemingshen/entry/zip64_support_for_4g_zipfile), which was added to OpenJDK in 2011.

However, I've updated my JDK to the newest one provided by Oracle:

$ mvn -version
Apache Maven 3.1.0 (893ca28a1da9d5f51ac03827af98bb730128f9f2; 2013-06-28 03:15:32+0100)
Maven home: /usr/local/apache-maven-3.1.0
Java version: 1.7.0_40, vendor: Oracle Corporation
Java home: /usr/local/java/jdk1.7.0_40/jre
Default locale: en_GB, platform encoding: ISO-8859-1
OS name: "linux", version: "2.6.32-279.2.1.el6.x86_64", arch: "amd64", family: "unix"

and it reports the same issue, again. Does the newest Oracle JDK have no support for Zip64 or it's something else?

I guess, I could try building OpenJDK b147 from sources but would like to avoid that unless inevitable.

Solution

I found the culprit, so reporting the issue just to save others time.

It seems that the problem lays in org.codehaus.plexus.archiver.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:352) as reported in my exception stack trace. Looking at the sources of org.codehaus...ZipOutputStream.closeEntry I can see:

entry.setSize( def.getTotalIn() );

and that def is java.util.zip.Deflater. However, Deflater.getTotalIn returns int not long. Instead the code should use Deflater.getBytesRead() which I'm gonna report to the developers.

[EDIT] It's been a while but I see people reading this, so FYI the problem has been fixed since version 2.5 of the maven assembly plugin.

Answered By - Jacek
Answer Checked By - Mary Flores (JavaFixing Volunteer)