Monday, July 18, 2022

[FIXED] Adding logstash-gelf log4j2 plugin to Karaf application

July 18, 2022 apache-karaf, gelf, log4j2, maven, pax

Issue

I have a Karaf (v4.2.15) application using the default PAX logging with associated config file org.ops4j.pax.logging.cfg. I am trying to update my graylog appender from log4j v1, in an older version of the application, to the current log4j2 implementation.

I have replaced the old gelf library (org.graylog2/gelfj) with the recommended biz.paluch.logging/logstash-gelf. I have had to wrapped it as an OSGI bundle making sure the biz.paluch.logging.gelf.log4j2 package is exported and that the Log4j2Plugins.dat file is included. Here is the pom.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>

    <groupId>osgi.biz.paluch.logging</groupId>
    <artifactId>logstash-gelf</artifactId>
    <version>1.15.0</version>
    <packaging>bundle</packaging>
    <name>logstash-gelf OSGi Bundle</name>
    <description>This OSGi bundle simply wraps logstash-gelf-1.15.0jar artifact.</description>

    <dependencies>
      <dependency>
        <groupId>biz.paluch.logging</groupId>
        <artifactId>logstash-gelf</artifactId>
        <version>1.15.0</version>
      </dependency>
    </dependencies>

    <build>
      <defaultGoal>install</defaultGoal>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-shade-plugin</artifactId>
          <version>3.3.0</version>
          <executions>
            <execution>
              <phase>package</phase>
              <goals>
                <goal>shade</goal>
              </goals>
              <configuration>
                <artifactSet>
                  <includes>
                    <include>biz.paluch.logging:logstash-gelf</include>
                  </includes>
                </artifactSet>
                <filters>
                  <filter>
                    <includes>
                      <include>META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat</include>
                    </includes>
                  </filter>
                </filters>
              </configuration>
            </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.felix</groupId>
        <artifactId>maven-bundle-plugin</artifactId>
        <version>5.1.2</version>
        <extensions>true</extensions>
        <configuration>
          <instructions>
            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
            <Export-Package>biz.paluch.logging.gelf.log4j2</Export-Package>
            <_versionpolicy>[$(version;==;$(@)),$(version;+;$(@)))</_versionpolicy>
            <_removeheaders>Ignore-Package,Include-Resource,Private-Package,Embed-Dependency</_removeheaders>
          </instructions>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

I've then added it to the karaf startup.properties as I had done with gelfj so that it can log the application startup.

# Bundles to be started on startup, with startlevel
mvn\:org.apache.karaf.features/org.apache.karaf.features.extension/4.2.15 = 1
mvn\:org.ops4j.pax.url/pax-url-aether/2.6.7 = 5
mvn\:org.ops4j.pax.logging/pax-logging-log4j2/1.11.13 = 8
mvn\:org.ops4j.pax.logging/pax-logging-api/1.11.13 = 8
mvn\:org.fusesource.jansi/jansi/1.18 = 8
mvn\:org.osgi/org.osgi.util.promise/1.1.1 = 9
mvn\:org.apache.felix/org.apache.felix.coordinator/1.0.2 = 9
mvn\:org.osgi/org.osgi.util.function/1.1.0 = 9
mvn\:org.apache.felix/org.apache.felix.configadmin/1.9.22 = 10
mvn\:org.apache.felix/org.apache.felix.fileinstall/3.7.4 = 11
mvn\:org.apache.karaf.features/org.apache.karaf.features.core/4.2.15 = 15
# My bundles
mvn\:osgi.biz.paluch.logging/logstash-gelf/1.15.0 = 7`

I've confirmed it is installed:

START LEVEL 100 , List Threshold: 0
 ID │ State    │ Lvl │ Version                    │ Name
────┼──────────┼─────┼────────────────────────────┼───────
  0 │ Active   │   0 │ 5.6.12                     │ System Bundle, Fragments: 1
  1 │ Resolved │   1 │ 4.2.15                     │ Apache Karaf :: Features :: Extension, Hosts: 0
  2 │ Active   │   5 │ 2.6.7                      │ OPS4J Pax Url - mvn:
  3 │ Active   │   8 │ 1.11.13                    │ OPS4J Pax Logging - Log4Jv2 implementation
  4 │ Active   │   8 │ 1.11.13                    │ OPS4J Pax Logging - API
  5 │ Active   │   8 │ 1.18.0                     │ jansi
  6 │ Active   │   9 │ 1.1.1.201810101357         │ org.osgi:org.osgi.util.promise
  7 │ Active   │   9 │ 1.0.2                      │ Apache Felix Coordinator Service
  8 │ Active   │   9 │ 1.1.0.201802012106         │ org.osgi:org.osgi.util.function
  9 │ Active   │  10 │ 1.9.22                     │ Apache Felix Configuration Admin Service
 10 │ Active   │  11 │ 3.7.4                      │ Apache Felix File Install
 11 │ Active   │  15 │ 4.2.15                     │ Apache Karaf :: Features :: Core
 12 │ Active   │   7 │ 1.15.0                     │ logstash-gelf OSGi Bundle

My understanding is that log4j2 use a bundleListner to look for plugins and so should automatically find the "Gelf" plugin. I've configured the appender as:

log4j2.appender.graylog.type = Gelf
log4j2.appender.graylog.name = Graylog
log4j2.appender.graylog.host = log.server.com
log4j2.appender.graylog.port = 12201
log4j2.appender.graylog.originHost = myapp.server.com

Unfortunately I just get an error that the Gelf plugin can't be found, this occurs whether the config is present at startup or loaded dynamically after pax logging has initialized

org.ops4j.pax.logging.pax-logging-log4j2 [log4j2] ERROR : Unable to locate plugin type for Gelf Ignored FQCN: org.apache.logging.log4j.spi.AbstractLogger

In case the BundleListener is not working I've tried declaring the package both in the config log4j2.packages = biz.paluch.logging.gelf.log4j2 and also on the commandline -Dlogj4.plugin.packages=biz.paluch.logging.gelf.log4j2

Any solutions or suggestions how to debug why the log4j2/pax can't find the Gelf plugin gratefully received.

Solution

After following the rabbit hole of GELF appender plugins I went back a looked through the the log4j2 document. I found that there is inbuilt support for the GELF layout and that you can use the Socket appender to direct logs to your Graylog server. This is a far cleaner way to achieve logging from startup and it works!

As far as I can tell it provides parity with the functionality I had before and appears to render logstash-gelf's log4j2 support unnecessary. Please comment if you know otherwise. Here is a example with additional fields:

# Socket GELF/Graylog Appender
log4j2.rootLogger.appenderRef.Graylog.ref = Graylog

log4j2.appender.graylog.type = Socket
log4j2.appender.graylog.name = Graylog
log4j2.appender.graylog.host = graylog.server.com
log4j2.appender.graylog.port = 12201
log4j2.appender.graylog.protocol = UDP
log4j2.appender.graylog.immediateFail = true
log4j2.appender.graylog.layout.type = GelfLayout
log4j2.appender.graylog.layout.host = my.server.com
log4j2.appender.graylog.layout.includeStacktrace = true
log4j2.appender.graylog.layout.additionalField1.type = KeyValuePair
log4j2.appender.graylog.layout.additionalField1.key = environment
log4j2.appender.graylog.layout.additionalField1.value = test

Answered By - JamesP
Answer Checked By - Marie Seifert (JavaFixing Admin)

This Answer collected from stackoverflow and tested by JavaFixing community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Monday, July 18, 2022

[FIXED] Adding logstash-gelf log4j2 plugin to Karaf application

Issue

Solution

Popular Posts

Labels