[jira] [Comment Edited] (SOLR-11087) Get rid of jar duplicates in release

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[jira] [Comment Edited] (SOLR-11087) Get rid of jar duplicates in release

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16089721#comment-16089721 ]

Uwe Schindler edited comment on SOLR-11087 at 7/17/17 12:11 PM:
----------------------------------------------------------------

Hi,

I like the cleanup. Personally, I'd go the other route: I would remove the whole webapp folder and instead of using start.jar, I'd assemble the servlet context in a simple Java main() method, where the JAR files are picked from the server or dist folder. I have done this several times. You can build a webapp without a web.xml in code with about 30 lines of code to startup jetty and link servlet filters and code. The good thing with that is on top, that you don't even need a webapp folder anywhere. And static resources can also be delivered directly from a JAR file! Here is a simple example I did for a micro-service:

{code:java}
      final Server server = new Server();
      server.setStopAtShutdown(true);
      server.setStopTimeout(1000L);
     
      // setup connectors...
      final ServerConnector ipv6 = new ServerConnector(server);
      ipv6.setPort(PORT);
      ipv6.setHost("::1");
      ipv6.setIdleTimeout(IDLE_TIMEOUT);
      server.addConnector(ipv6);

      // add servlet context:
      final ServletContextHandler context = new ServletContextHandler(ServletContextHandler.NO_SECURITY | ServletContextHandler.NO_SESSIONS);
      context.insertHandler(new ResourceHandler());
      context.setBaseResource(Resource.newClassPathResource("/webroot/"));
      context.addServlet(SelectServlet.class, "/select");
      context.addServlet(RecordServlet.class, "/record/*");
      server.setHandler(context);

      // start webserver
      server.start();
      server.join();
{code}

This is just an example binding 2 servlets, i just removed other non-servlet and logging stuff, so you can add your own access/jetty logging, too. The good thing with that is also that we can get rid of the stupid internal "/c" redirects as you have full flexibility where you bind what. And finally the WAR file is gone and the server does not unpack it on startup. In addition, we can do all the startup logic before spawning jetty (e.g. starting embedded zookeeper, checking log4j config,...)

Static files and stuff are loaded using {{context.insertHandler(new ResourceHandler()); context.setBaseResource(Resource.newClassPathResource("/webroot/"));}} directly from one of the JAR files (where a {{webroot/}} folder is part of the JAR's contents/resources). This can be a separate JAR file or simply in main solr.jar. This makes it very small and unpacking Solr would take only a second then (currently unzipping all those small static files is a mess).

So I'd go that route and as a first step refactor the web.xml file into a simple startup main() method as seen before. I can help with that. I have some time this week, so I may make a quick and dirty mockup - if you agree.


was (Author: thetaphi):
Hi,

I like the cleanup. Personally, I'd go the other route: I would remove the whole webapp folder and instead of using start.jar, I'd assemble the servlet context in a simple Java main() method, where the JAR files are picked from the server or dist folder. I have done this several times. You can build a webapp without a web.xml in code with about 30 lines of code to startup jetty and link servlet filters and code. The good thing with that is on top, that you don't even need a webapp folder anywhere. And static resources can also be delivered directly from a JAR file! Here is a simple example I did for a micro-service:

{code:java}
      final Server server = new Server();
      server.setStopAtShutdown(true);
      server.setStopTimeout(1000L);
     
      // setup connectors...
      final ServerConnector ipv6 = new ServerConnector(server);
      ipv6.setPort(PORT);
      ipv6.setHost("::1");
      ipv6.setIdleTimeout(IDLE_TIMEOUT);
      server.addConnector(ipv6);

      // add servlet context:
      final ServletContextHandler context = new ServletContextHandler(ServletContextHandler.NO_SECURITY | ServletContextHandler.NO_SESSIONS);
      context.insertHandler(new ResourceHandler());
      context.setBaseResource(Resource.newClassPathResource("/webroot/"));
      context.addServlet(SelectServlet.class, "/select");
      context.addServlet(RecordServlet.class, "/record/*");
      server.setHandler(context);

      // start webserver
      server.start();
      server.join();
{code}

This is just an example binding 2 servlets, i just removed other non-servlet and logging stuff, so you can add your own access/jetty logging, too. The good thing with that is also that we can get rid of the stupid internal "/c" redirects as you have full flexibility where you bind what. And finally the WAR file is gone and the server does not unpack it on startup. In addition, we can do all the startup logic before spawning jetty (e.g. starting embedded zookeeper, checking log4j config,...)

So I'd go that route and as a first step refactor the web.xml file into a simple startup main() method as seen before. I can help with that. I have some time this week, so I may make a quick and dirty mockup - if you agree.

> Get rid of jar duplicates in release
> ------------------------------------
>
>                 Key: SOLR-11087
>                 URL: https://issues.apache.org/jira/browse/SOLR-11087
>             Project: Solr
>          Issue Type: Sub-task
>          Components: Build
>            Reporter: Jan H√łydahl
>             Fix For: master (8.0), 7.1
>
>         Attachments: SOLR-11087.patch
>
>
> Sub task of SOLR-6806
> The {{dist/}} folder contains many duplicate jar files, totalling 10,5M:
> {noformat}
> 4,6M   ./dist/solr-core-6.6.0.jar (WEB-INF/lib)
> 1,2M   ./dist/solr-solrj-6.6.0.jar (WEB-INF/lib)
> 4,7M   ./dist/solrj-lib/* (WEB-INF/lib and server/lib/ext)
> {noformat}
> The rest of the files in dist/ are contrib jars and test-framework.
> To weed out the duplicates and save 10,5M, we can simply add a {{dist/README.md}} file listing what jar files are located where. The file could also contain a bash one-liner to copy them to the dist folder. Another possibility is to ship the binary release tarball with symlinks in the dist folder, and advise people to use {{cp -RL dist mydist}} which will make a copy with the real files. Downside is that this won't work for ZIP archives that do not preserve symlinks, and neither on Windows.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...