CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot)

I am having an issue wen running my application with QueryDSL on OpenJDK.

I have 2 classes that I generate QueryDsl classes for. One(Project) is a MongoEntity class that I annotated with @QueryEntity and the other(Service) is a POJO also with @QueryEntity.

I use com.querydsl.apt.QuerydslAnnotationProcessor for generating the classes.

When I compile and run it locally everything works fine. This is using the following JDK:


java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

But when built for production using this JDK:

java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

...and then run in a container with this JDK:

openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

...then I get this exception for the Service class, but not the Project class:

com.mysema.codegen.CodegenException: Compilation of public class Q_2124358794_1275614662_1275614662_1195259493 {

    public static Iterable<dk.codezoo.meerkat.model.Service> eval(Iterable<dk.codezoo.meerkat.model.Service> service_, String a1) {
java.util.List<dk.codezoo.meerkat.model.Service> rv = new java.util.ArrayList<dk.codezoo.meerkat.model.Service>();
for (dk.codezoo.meerkat.model.Service service : service_) {
    try {
        if (service.getName().toLowerCase().contains(a1)) {
            rv.add(service);
        }
    } catch (NullPointerException npe) { }
}
return rv;    }

}

 failed.
warning: Supported source version 'RELEASE_6' from annotation processor 'org.jvnet.hudson.annotation_indexer.AnnotationProcessorImpl' less than -source '1.8'
/Q_2124358794_1275614662_1275614662_1195259493.java:3: error: package dk.codezoo.meerkat.model does not exist
    public static Iterable<dk.codezoo.meerkat.model.Service> eval(Iterable<dk.codezoo.meerkat.model.Service> service_, String a1) {
                                                                                                   ^
/Q_2124358794_1275614662_1275614662_1195259493.java:3: error: package dk.codezoo.meerkat.model does not exist
    public static Iterable<dk.codezoo.meerkat.model.Service> eval(Iterable<dk.codezoo.meerkat.model.Service> service_, String a1) {
                                                   ^
2 errors
1 warning

	at com.mysema.codegen.JDKEvaluatorFactory.compile(JDKEvaluatorFactory.java:74)
	at com.mysema.codegen.AbstractEvaluatorFactory.createEvaluator(AbstractEvaluatorFactory.java:128)
	at com.querydsl.collections.DefaultEvaluatorFactory.createEvaluator(DefaultEvaluatorFactory.java:151)
	at com.querydsl.collections.DefaultQueryEngine.evaluateSingleSource(DefaultQueryEngine.java:166)
	at com.querydsl.collections.DefaultQueryEngine.list(DefaultQueryEngine.java:80)
	at com.querydsl.collections.AbstractCollQuery.fetch(AbstractCollQuery.java:182)
	at dk.codezoo.meerkat.orchestration.DockerOrchestrationClient$DockerServiceClient.findAll(DockerOrchestrationClient.java:108)
	at dk.codezoo.meerkat.service.ServiceService.findAll(ServiceService.java:64)
	at dk.codezoo.meerkat.controller.ServiceController.list(ServiceController.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.oauth2.provider.authentication.OAuth2AuthenticationProcessingFilter.doFilter(OAuth2AuthenticationProcessingFilter.java:176)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

package dk.codezoo.meerkat.model does not exist

Can you check this? Is it only happening with those JDKs?

That package does exist. I have tried to unpack the jar just to check it, and it is there.
The two classes are both in the package. One works the other does not.

I just tried to force our build process to update the docker image it uses(openjdk:8-jdk), and that changed the JDK to this:

openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

But to no avail. It fails with the same exception.

~~But I am only seeing it when I run it on OpenJDK.~~

Okay. I just extracted the Jar built for production and ran that locally... And it fails...
So its not only OpenJDK.

Weird. If I built it with maven locally on 1.8.0_121(mac) and run it locally, there is not issue. If built it with 1.8.0_161(linux) and I run that locally, then it fails.

I upgraded my mac to JDK 1.8.0_161 and still I am unable to reproduce it when I build i locally.

I then tried to login to our build server(JDK 1.8.0_161 linux), checked out the code, did a fresh build(mvn clean install) and copied the jar-file to my local machine and it fails.

So weird... I'll try to setup a virtual linux locally and reproduce it there.

Yes please do, I can't fathom where it would go wrong but can go right in some cases.
So I'd like to hear any progress!

I think I found the source of the issue: classpath.

This is the start of the resolved classpath when I run the project via spring-boot:run from intellij:

/Users/michael/Development/meerkat/meerkat/target/classes:/Users/michael/.m2/repository/org/springframework/boot/spring-boot-starter-web/1.5.3.RELEASE/spring-boot-starter-web-1.5.3.RELEASE.jar:/Users/michael/.m2/repository/org/springframework/boot/spring-boot-starter/1.5.3.RELEASE/spring-boot-starter-1.5.3.RELEASE.jar:/...

And this is the start of the classpath when I run the final artifact locally from the commandline (java -jar meerkat.jar):

/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!:/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-starter-web-1.5.3.RELEASE.jar!:/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-starter-1.5.3.RELEASE.jar!:/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-1.5.3.RELEASE.jar!:...

Weird classpath

/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes! looks a bit weird, doesn't it?

It comes down to

codegen/src/main/java/com/mysema/codegen/SimpleCompiler.java

Lines 71 to 72 in 60b65f9

    
           String decodedPath = URLDecoder.decode(url.getPath(), "UTF-8"); 
        
           paths.add(new File(decodedPath).getAbsolutePath());

The first URL retrieved in my case is jar:file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!/

At line 71 it is being decoded to file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!/

At line 72 the absolute path becomes /Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!

I am unable to make it work.
SimpleCompiler.getClassPath(parent); at https://github.com/querydsl/codegen/blob/master/src/main/java/com/mysema/codegen/JDKEvaluatorFactory.java#L57 returns a classpath that does not work when the application is run as an executable jar built with Spring Boot (https://docs.spring.io/spring-boot/docs/current/reference/html/build-tool-plugins-maven-plugin.html).

The classloader is a TomcatEmbeddedWebappClassLoader which resolves jars in the executable jar(Fx. /Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-starter-tomcat-1.5.3.RELEASE.jar!). But the classpath returned from SimpleCompiler is then broken and classes can then not be resolved.

My knowledge about JavaCompiler and the codegen project is to sparse. I have been unable to create a fix.

Yes the issue is known, I'll continue with my fix

Thanks for looking into it though, much appreciated!

@Shredder121

@Shredder121 I just created a PR to add the support for Codegen to generate code when running in a fat JAR. It should fix this bug and others related to this.
I'm using this fix and it works for me with Spring Boot, Spring Data Key Value, spring-data-hazelcast and QueryDsl Collections.
The PR is: #43
If you can, please review it and let me know if there are possible improvements to do.

This issue is indeed fixed by #43.

#43 was never accepted even after this many years.

Is there any solution in sight or is building a custom jar still the only viable solution in order to use this with spring boot?

Any updates here?

@michaelkrog

@michaelkrog do you how to build custom jar to make it work on ubuntu/macos?

Navigation Menu

Repository navigation

CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

Weird classpath

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	String decodedPath = URLDecoder.decode(url.getPath(), "UTF-8");
	paths.add(new File(decodedPath).getAbsolutePath());

CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

Description

Activity

Shredder121 commented on Mar 1, 2018

michaelkrog commented on Mar 1, 2018

michaelkrog commented on Mar 1, 2018

michaelkrog commented on Mar 2, 2018

Shredder121 commented on Mar 2, 2018

michaelkrog commented on Mar 2, 2018

Weird classpath

michaelkrog commented on Mar 2, 2018

michaelkrog commented on Mar 6, 2018

Shredder121 commented on Mar 6, 2018

Shredder121 commented on Mar 6, 2018

matteo-gallo-bb commented on Mar 27, 2018

michaelkrog commented on Jun 15, 2018

michaelkrog commented on Jan 21, 2021

viktorzub commented on Jul 8, 2021

viktorzub commented on Jul 12, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions