dd-trace-ot should not bundle unshaded com.kenai.jffi

[hvtuananh]

com.kenai.jffi used to be a dependency in dd-trace-ot 0.x, then bundled as a shaded library in 1.x until 1.8.x, then unshaded from 1.9.x onward by this commit: #4779

I understood the reason for not relocating it, but why not declare com.kenai.jffi as an external dependency instead?

[motlin]

Any chance of getting this changed? We need to run our build with the maven-enforcer-plugin rule banDuplicateClasses to prevent a different problem, but it's running into the duplicate classes in com.kenai.jffi.

[motlin]

Here's the error that we see:

[INFO] -------< datadog-dependency-example:datadog-dependency-example >--------
[INFO] Building datadog-dependency-example 1.0.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- enforcer:3.5.0:enforce (enforce) @ datadog-dependency-example ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.490 s (Wall Clock)
[INFO] Finished at: 2024-12-24T10:00:07-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.5.0:enforce (enforce) on project datadog-dependency-example: 
[ERROR] Rule 0: org.codehaus.mojo.extraenforcer.dependencies.BanDuplicateClasses failed with message:
[ERROR] Duplicate classes found:
[ERROR] 
[ERROR]   Found in:
[ERROR]     com.datadoghq:dd-trace-ot:jar:1.44.1:compile
[ERROR]     com.github.jnr:jffi:jar:1.2.23:compile
[ERROR]   Duplicate classes:
[ERROR]     com/kenai/jffi/ObjectParameterInfo.class
[ERROR]     com/kenai/jffi/Union.class
[ERROR]     com/kenai/jffi/ClosureManager.class
... many more ...

Here's a minimal maven build that shows the error. Note the config value of ignoreWhenIdentical=true which means that the duplicate classes aren't byte-for-byte identical. I think this means that if I declare a dependency on com.github.jnr:jffi at the top of my pom.xml, I'll wind up with different classes chosen at runtime. This is the reason we turned on this enforcer rule - though for problems we faced with a different library.

<project>
    <modelVersion>4.0.0</modelVersion>

    <groupId>datadog-dependency-example</groupId>
    <artifactId>datadog-dependency-example</artifactId>
    <version>1.0.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>com.datadoghq</groupId>
            <artifactId>dd-trace-ot</artifactId>
            <version>1.44.1</version>
        </dependency>

        <dependency>
            <groupId>com.datadoghq</groupId>
            <artifactId>java-dogstatsd-client</artifactId>
            <version>4.4.3</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>

            <plugin>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>3.5.0</version>
                <dependencies>
                    <dependency>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>extra-enforcer-rules</artifactId>
                        <version>1.9.0</version>
                    </dependency>
                </dependencies>
                <configuration>
                    <rules>
                        <banDuplicateClasses>
                            <ignoreWhenIdentical>true</ignoreWhenIdentical>
                            <findAllDuplicates>true</findAllDuplicates>
                        </banDuplicateClasses>
                    </rules>
                </configuration>
                <executions>
                    <execution>
                        <id>enforce</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

        </plugins>
    </build>

</project>

Here's some output from mvn dependency:tree which shows how the two jars get on the classpath.

[INFO] --- dependency:3.8.1:tree (default-cli) @ datadog-dependency-example ---
[INFO] datadog-dependency-example:datadog-dependency-example:jar:1.0.0-SNAPSHOT
[INFO] +- com.datadoghq:dd-trace-ot:jar:1.44.1:compile
[INFO] |  +- com.datadoghq:dd-trace-api:jar:1.44.1:compile
[INFO] |  +- io.opentracing:opentracing-api:jar:0.32.0:compile
[INFO] |  +- io.opentracing:opentracing-noop:jar:0.32.0:compile
[INFO] |  +- io.opentracing:opentracing-util:jar:0.32.0:compile
[INFO] |  +- io.opentracing.contrib:opentracing-tracerresolver:jar:0.1.6:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.30:compile
[INFO] \- com.datadoghq:java-dogstatsd-client:jar:4.4.3:compile
[INFO]    \- com.github.jnr:jnr-unixsocket:jar:0.36:compile
[INFO]       +- com.github.jnr:jnr-ffi:jar:2.1.16:compile
[INFO]       |  +- com.github.jnr:jffi:jar:1.2.23:compile
[INFO]       |  +- com.github.jnr:jffi:jar:native:1.2.23:runtime
[INFO]       |  +- org.ow2.asm:asm:jar:7.1:compile
[INFO]       |  +- org.ow2.asm:asm-commons:jar:7.1:compile
[INFO]       |  +- org.ow2.asm:asm-analysis:jar:7.1:compile
[INFO]       |  +- org.ow2.asm:asm-tree:jar:7.1:compile
[INFO]       |  +- org.ow2.asm:asm-util:jar:7.1:compile
[INFO]       |  +- com.github.jnr:jnr-a64asm:jar:1.0.0:compile
[INFO]       |  \- com.github.jnr:jnr-x86asm:jar:1.0.2:compile
[INFO]       +- com.github.jnr:jnr-constants:jar:0.9.17:compile
[INFO]       +- com.github.jnr:jnr-enxio:jar:0.30:compile
[INFO]       \- com.github.jnr:jnr-posix:jar:3.0.61:compile
[INFO] ------------------------------------------------------------------------

In addition to the linked ticket, this may also be related to #7789.

Tagging @mcculls since you participated in both issues and may have more context.

[motlin]

It occurred to me that this might be a simple diamond dependency problem, if the embedded version of jffi is newer (1.3.13), and the version that maven pulls in transitively is older (1.2.23) through this path starting with an older version of jnr-unixsocket.

[INFO] \- com.datadoghq:java-dogstatsd-client:jar:4.4.3:compile
[INFO]    \- com.github.jnr:jnr-unixsocket:jar:0.36:compile
[INFO]       +- com.github.jnr:jnr-ffi:jar:2.1.16:compile
[INFO]       |  +- com.github.jnr:jffi:jar:1.2.23:compile

Unfortunately that didn't help. Adding this to my build:

        <dependency>
            <groupId>com.github.jnr</groupId>
            <artifactId>jffi</artifactId>
            <version>1.3.13</version>
        </dependency>

Just changed the error message to show the newer version in the duplicate classes:

[ERROR] Duplicate classes found:
[ERROR] 
[ERROR]   Found in:
[ERROR]     com.datadoghq:dd-trace-ot:jar:1.44.1:compile
[ERROR]     com.github.jnr:jffi:jar:1.3.13:compile

[mcculls]

This will be fixed in 1.46.0 by making jnr-unixsocket an explicit dependency of dd-trace-ot

You can try out a snapshot containing this fix by patching the above pom.xml as follows:

@@ -9,7 +9,7 @@
         <dependency>
             <groupId>com.datadoghq</groupId>
             <artifactId>dd-trace-ot</artifactId>
-            <version>1.44.1</version>
+            <version>1.46.0-SNAPSHOT</version>
         </dependency>

         <dependency>
@@ -54,4 +54,18 @@
         </plugins>
     </build>

+    <repositories>
+        <repository>
+            <id>oss-snapshots</id>
+            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
+            <releases>
+                <enabled>false</enabled>
+            </releases>
+            <snapshots>
+                <enabled>true</enabled>
+            </snapshots>
+        </repository>
+    </repositories>
+
 </project>

[github-actions]

🤖 This issue has been addressed in the latest release. Check full details in the Release Notes.