Log4j2 - Support the use of async appenders when using the standard Pattern Layout

[jtduffy]

Original GTSE: https://new-relic.atlassian.net/jira/software/c/projects/NR/boards/289?selectedIssue=NR-323639

When using a Log4j async appender and the Pattern layout, trace id and span id are not properly added to the log event metadata because of a thread hop, which results in the active transaction no longer being available.

Module: https://github.com/newrelic/newrelic-java-agent/tree/main/instrumentation/apache-log4j-2.11

It might be possible to instrument a super class of the layout classes and instrument the toSerializable() method similarly to how the current AbstractJacksonLayout_Instrumentation is instrumented.

To enable async appenders in a project do the following:

• Create a file named log4j2.component.properties in the same folder where the log4j2.xml file exists
• Add the following line to the file:

Log4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector

An example log4j2.xml that configures a simple pattern layout with a console appender:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
  <Appenders>
    <Console name="console_log" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX},%-5p,%m%n" />
    </Console>
  </Appenders>
  <Loggers>
    <Root level="info">
      <AppenderRef ref="console_log" />
    </Root>
  </Loggers>
</Configuration>

https://new-relic.atlassian.net/browse/NR-323639

[workato-integration]

https://new-relic.atlassian.net/browse/NR-329669

[jtduffy]

We're going to place this back on the backlog.

Implementing a fix for this will require a significant refactoring of the log4j instrumentation modules, as well as changes to the log sender service which is used across all logger instrumentation modules.

Two options were explored:

• After the log4j appenders are called, add the linking metadata to the message (like we’re doing local decorating) and submit the LogEvent to the “non transaction” reservoir. Force priority value to .99
  Cons: Sampling could remove LogEvents that should be attached to a transaction.

• Add methods to the “Logs” interface (exposed through AgentBirdge.getAgent().getLogSender()) to allow the cache to live in the agent component
  Cons: Pollution of the Logs API interface