Issue
I am using the layered jar file approach [1] for optimizing our Docker image build times. I have noticed that the jar files that are extracted from the following command do not preserve the timestamp of the individual jar files that get extracted.
java -Djarmode=layertools -jar my-uber-jar.jar extract
If I run the above command, check the timestamps on the extracted files, and then re-run the command and recheck the timestamps they are always updated to the current time.
example:
$ java -Djarmode=layertools -jar my-uber-jar.jar extract
$ ls -l dependencies/BOOT-INF/lib/commons-*
-rw-r--r-- 1 legacy staff 327K Oct 1 09:23 dependencies/BOOT-INF/lib/commons-codec-1.11.jar
-rw-r--r-- 1 legacy staff 204K Oct 1 09:23 dependencies/BOOT-INF/lib/commons-io-2.5.jar
-rw-r--r-- 1 legacy staff 490K Oct 1 09:23 dependencies/BOOT-INF/lib/commons-lang3-3.8.1.jar
-rw-r--r-- 1 legacy staff 60K Oct 1 09:23 dependencies/BOOT-INF/lib/commons-logging-1.2.jar
-rw-r--r-- 1 legacy staff 2.1M Oct 1 09:23 dependencies/BOOT-INF/lib/commons-math3-3.6.1.jar
... wait ~5 minutes ...
$ java -Djarmode=layertools -jar my-uber-jar.jar extract
$ ls -l dependencies/BOOT-INF/lib/commons-*
-rw-r--r-- 1 legacy staff 327K Oct 1 09:29 dependencies/BOOT-INF/lib/commons-codec-1.11.jar
-rw-r--r-- 1 legacy staff 204K Oct 1 09:29 dependencies/BOOT-INF/lib/commons-io-2.5.jar
-rw-r--r-- 1 legacy staff 490K Oct 1 09:29 dependencies/BOOT-INF/lib/commons-lang3-3.8.1.jar
-rw-r--r-- 1 legacy staff 60K Oct 1 09:29 dependencies/BOOT-INF/lib/commons-logging-1.2.jar
-rw-r--r-- 1 legacy staff 2.1M Oct 1 09:29 dependencies/BOOT-INF/lib/commons-math3-3.6.1.jar
The reason this is an issue for me is that I am attempting to use Docker buildkit [2] optimizations to reduce the amount of build context sent to a remote Docker daemon. The idea behind using the layered jar file approach and Docker buildkit was to avoid sending any dependency jar files to the remote docker daemon unless they changed in some way. Docker buildkit seems to notice the changed timestamps and then sends all of the files over to the remote server even though a checksum on those files is identical to the previous build. I have confirmed that if I just do the build without re-extracting the files that very little data is sent to the remote Docker daemon.
Using the layered jar file approached worked well for reducing the amount of data sent on a docker push
but our workflow requires a docker build on a remote server (via IntelliJ's remote docker build feature)
That leads me to my questions:
a) is it possible to extract the layers while preserving the timestamps?
b) is it possible to make docker buildkit look at checksums instead of timestamps?
[1] https://spring.io/blog/2020/08/14/creating-efficient-docker-images-with-spring-boot-2-3 [2] https://docs.docker.com/develop/develop-images/build_enhancements/
Solution
is it possible to extract the layers while preserving the timestamps?
Spring Boot's layer tools doesn't support this at the moment. It sounds like a useful feature to add, though, and I think it should probably be the default behaviour. We already take care to preserve the timestamps when creating the jar so I think it makes sense to preserve them during extraction as well.
Please open a Spring Boot issue and we'll take a look.
Answered By - Andy Wilkinson