[FIXED] log4j vulnerability check: how to quickly detect if log4j is used, in mavevn/in an image? "mvn dependency:tree" does not give a full picture

Issue

OK here's my question:

• how to output the full dependency tree of your service/application, and find out if somewhere in the corner, log4j is used? Considering that mvn dependency:tree does not allow you to traverse to the bottom of bottom. Check this: when I do mvn dependency:tree on my pom.xml, it gives something like this:

[INFO] |  |  +- io.quarkus:quarkus-core:jar:2.4.1.Final:compile
[INFO] |  |  |  +- jakarta.enterprise:jakarta.enterprise.cdi-api:jar:2.0.2:compile
[INFO] |  |  |  |  \- jakarta.el:jakarta.el-api:jar:3.0.3:compile
[INFO] |  |  |  +- jakarta.inject:jakarta.inject-api:jar:1.0:compile
[INFO] |  |  |  +- io.quarkus:quarkus-ide-launcher:jar:2.4.1.Final:compile
[INFO] |  |  |  +- io.quarkus:quarkus-development-mode-spi:jar:2.4.1.Final:compile
[INFO] |  |  |  +- org.jboss.logging:jboss-logging:jar:3.4.2.Final:compile
[INFO] |  |  |  +- org.jboss.logmanager:jboss-logmanager-embedded:jar:1.0.9:compile
[INFO] |  |  |  +- org.jboss.logging:jboss-logging-annotations:jar:2.2.1.Final:compile
[INFO] |  |  |  +- org.jboss.threads:jboss-threads:jar:3.4.2.Final:compile
[INFO] |  |  |  +- org.jboss.slf4j:slf4j-jboss-logmanager:jar:1.1.0.Final:compile
[INFO] |  |  |  +- org.graalvm.sdk:graal-sdk:jar:21.2.0:compile

Now I get it, that +- and \- are used to replace ├ and └ somehow(but till 5 min ago I found it confusing), so they are not "not expanded". But, if the pom of some dependencies uses log4j, am I safe without doing anything? For example, when I check org.jboss.logging:jboss-logging:jar:3.4.2.Final:compile in https://search.maven.org/artifact/org.jboss.logging/jboss-logging/3.4.2.Final/jar, I see org.apache.logging.log4j:log4j-core:2.11 is used, but then it's excluded later. So I understand that Quarkus is safe. But, do I have to check the pom of each and every jar I find in the dependency tree to find out what is used and what is not?

• if I am using a base image from somewhere, how do I do this process in step 1? How can I know that my base image is vulnerable because it uses, in some corner of the dependency tree, log4j2 < 2.15?

Some background here: https://nvd.nist.gov/vuln/detail/CVE-2021-44228

EDIT: Now I think maybe setting the env var is better, so it will solve it once and for all.

Solution

Well the case we are facing, is that we are building an app, foo, based on Red Hat JBoss AMQ 6. With jib maven plugin, our code would be built into an uber jar and put under /opt/xxx to execute, and the base image would put other jars into the image under /opt/amq/lib.

Step 1

For our code, we do:

mvn dependency:tree | grep log4j

And we found some dep from other teams which brings transitive log4j 1.17 or so. The other team would handle it; before that, as a workaround, we would remove

• org/apache/log4j/net/SocketServer.class
• org/apache/log4j/net/SimpleSocketServer.class (just in case)
• org/apache/log4j/net/SocketAppender.class
• org/apache/log4j/net/SMTPAppender$1.class
• org/apache/log4j/net/SMTPAppender.class
• org/apache/log4j/net/JMSAppender.class
• org/apache/log4j/net/JMSSink.class
• org/apache/log4j/net/JDBCAppender.class
• org/apache/log4j/chainsaw/*.class

from the jar manually in our internal artifactory, so that every jar downloaded from there would lack the class file.

If you cannot fix your internal Nexus repo/artifactory, you can find the jar of log4j in local Maven registry (under ~/.m2) and remove the class; then you build your app again; but remember don't use -U to redownload the jar from remote registry.

Step 2

To find other libs in the base image containing log4j is more complicated. After all, tampering the layers removing the classes files cannot go undetected by Docker daemon. The sha256 value changes, you have to replace the sha256 value in the json file in the main dir with new sha256sum layer.tar; but even with that, Docker daemon will give error when you load the tar: Cannot open /var/lib/docker/tmp-xxxx/...: file not found or so. So at last, I created a script to remove the classes at runtime, right before running the app, and define a new entrypoint in jib to run it before running the app.

The script:

#!/bin/bash
# Script to fix log4j 1.x CVEs. Initially it is only for CVE-2021-4104, but
# since there are multiple CVEs regarding log4j 1.x, they are all fixed here:

# Class File                                        CVE
# org/apache/log4j/net/SocketAppender.class         CVE-2019-17571
# org/apache/log4j/net/SocketServer.class           CVE-2019-17571
# org/apache/log4j/net/SMTPAppender$1.class         CVE-2020-9488
# org/apache/log4j/net/SMTPAppender.class           CVE-2020-9488
# org/apache/log4j/net/JMSAppender.class            CVE-2021-4104
# org/apache/log4j/net/JMSSink.class                CVE-2022-23302
# org/apache/log4j/net/JDBCAppender.class           CVE-2022-23305
# org/apache/log4j/chainsaw/*.class                 CVE-2022-23307

cves=(
'CVE-2019-17571'
'CVE-2019-17571'
'CVE-2020-9488'
'CVE-2020-9488'
'CVE-2021-4104'
'CVE-2022-23302'
'CVE-2022-23305'
'CVE-2022-23307'
)

size() {
    stat -c %s "$1"
}

extract_remove_repackage() {
    before=$1
    # jar xf -C some_dir only extract to current dir, we have to cd first
    jar_dir=$(dirname "$2")
    jar_file=$(basename "$2")
    temp_dir=$jar_dir/temp
    mkdir "$temp_dir"
    cp list.txt "$temp_dir"/ && cp "$2" "$temp_dir"/
    cd "$temp_dir"
    jar xf "$jar_file"
    # provide file and dir names to rm with list.txt
    xargs rm -rvf < list.txt && rm list.txt "$jar_file"
    jar cf "$jar_file" .
    mv "$jar_file" ../
    # go back and clean up
    cd "$before" && rm -rf "$temp_dir"
}

find_vulnerable_jars() {
    jar -tvf "$1" | grep -E "$pattern" | awk '{ print $8 }' > list.txt
    if [ "$(size list.txt)" -gt 0 ]; then
        echo ">>>>> Removing class file from $(realpath $1)":
        extract_remove_repackage "$(pwd)" "$1"
    else
        return
    fi
}
remove_classes_from_jars() {
    echo Starting to fix all CVEs regarding Log4j 1.x...
    cd $root_dir
    # exclude jolokia.jar(link)
    find . -name "*.jar" -not -type l -exec bash -c 'cd "$root_dir"; find_vulnerable_jars "$@"' bash {} \;
    echo All vunerable classes removed. CVE addressed:
    printf '%s\n' "${cves[@]}"
}

# to be able to use in find -exec child shell, we need to export all vars and functions
export root_dir=/opt/amq
export pattern=".*(JMS|JDBC|SMTP|Socket)Appender.*.class|.*SocketServer.class|.*JMSSink.class|org/apache/log4j/chainsaw/.*"
export -f size
export -f extract_remove_repackage
export -f find_vulnerable_jars
remove_classes_from_jars

New jib entrypoint:

<!-- was "INHERITED" -->
<entrypoint>/opt/amq/bin/fix_and_launch.sh</entrypoint>

The new endpoint script:

#!/bin/sh
/opt/amq/bin/fix_log4j_1.x_cves.sh
/opt/amq/bin/launch.sh # the original, inherited entrypoint in jib

More discussions about security vulnerabilities scan report handling, see here

Answered By - WesternGun
Answer Checked By - David Marino (JavaFixing Volunteer)