-
Notifications
You must be signed in to change notification settings - Fork 273
[Improvement] Shade Jetty in the Solr module to avoid classpath conflicts with Storm #1859
Copy link
Copy link
Open
Description
What would you like to be improved?
Context
As discovered during the Solr 10 migration (#1833), the Solr module's Jetty dependencies conflict with the Jetty version shipped by Apache Storm at runtime.
Solr 10 (via solrj) depends on Jetty 12.0.27, while Storm 2.8.5 ships Jetty 12.1.6. Because both end up on the same classpath, the version mismatch causes NoSuchMethodError at runtime:
18:50:11.431 [Thread-37-status-executor[5, 5]] ERROR o.a.s.u.Utils - Async loop died!
java.lang.NoSuchMethodError: 'boolean org.eclipse.jetty.http2.client.HTTP2Client.installBean(java.lang.Object)'
at org.eclipse.jetty.http2.client.HTTP2Client.<init>(HTTP2Client.java:133)
at org.apache.solr.client.solrj.jetty.HttpJettySolrClient.createHttpClient(HttpJettySolrClient.java:292)
at org.apache.solr.client.solrj.jetty.HttpJettySolrClient.<init>(HttpJettySolrClient.java:152)
at org.apache.solr.client.solrj.jetty.HttpJettySolrClient$Builder.build(HttpJettySolrClient.java:987)
at org.apache.stormcrawler.solr.SolrConnection.getConnection(SolrConnection.java:364)
at org.apache.stormcrawler.solr.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:89)
at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
at org.apache.storm.utils.Utils$1.run(Utils.java:393)
at java.base/java.lang.Thread.run(Thread.java:840)
How should we improve?
Proposed Solution
Use the Maven Shade plugin to relocate the Solr client's Jetty dependencies under a StormCrawler-specific package (e.g. org.apache.stormcrawler.solr.shaded.jetty).
This way Storm's Jetty and Solr's Jetty coexist on the classpath without interfering with each other, regardless of what either project ships.
A similar approach is used in Apache Tomee: tomee/deps/commons-dbcp2-shade/pom.xml.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels