Re: svn commit: r378655 - in /lucene/nutch/trunk/src/plugin: ./ analysis-de/ analysis-fr/ clustering-carrot2/ creativecommons/ index-basic/ index-more/ languageidentifier/ lib-commons-httpclient/ lib-http/ lib-jakarta-poi/ lib-log4j/ lib-lucene-analy

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: svn commit: r378655 - in /lucene/nutch/trunk/src/plugin: ./ analysis-de/ analysis-fr/ clustering-carrot2/ creativecommons/ index-basic/ index-more/ languageidentifier/ lib-commons-httpclient/ lib-http/ lib-jakarta-poi/ lib-log4j/ lib-lucene-analy

Jérôme Charron
> Calling "compile-core" for every plugin makes builds really slow.

I was surprised that nobody complain about this...   ;-)


>   I
> think it's safe to assume that the core has already been compiled before
> plugins are compiled.  Don't you?

It just ensure that the last modified core version is automatically compiled
while compiling a single plugin.
From my point of view the time for a whole build is not a problem.
If I just work on core, then I can use the "fast" compile-core target.
And if I just work on a plugin, I only compile the plugin.
Finally I use the global compilation very rarely.
But perhaps that's not your case, and so it makes sense to reduce time of
the whole build.

Jérôme
Reply | Threaded
Open this post in threaded view
|

Re: svn commit: r378655 - in /lucene/nutch/trunk/src/plugin: ./ analysis-de/ analysis-fr/ clustering-carrot2/ creativecommons/ index-basic/ index-more/ languageidentifier/ lib-commons-httpclient/ lib-http/ lib-jakarta-poi/ lib-log4j/ lib-lucene-analy

Doug Cutting
Jérôme Charron wrote:
> It just ensure that the last modified core version is automatically compiled
> while compiling a single plugin.
> From my point of view the time for a whole build is not a problem.
> If I just work on core, then I can use the "fast" compile-core target.
> And if I just work on a plugin, I only compile the plugin.
> Finally I use the global compilation very rarely.
> But perhaps that's not your case, and so it makes sense to reduce time of
> the whole build.

In a distributed configuration one needs to rebuild the job jar each
time anything changes, and hence must check all plugins, etc.  So I
would appreciate it if this didn't take quite so long.

Doug
Reply | Threaded
Open this post in threaded view
|

Re: svn commit: r378655 - in /lucene/nutch/trunk/src/plugin: ./ analysis-de/ analysis-fr/ clustering-carrot2/ creativecommons/ index-basic/ index-more/ languageidentifier/ lib-commons-httpclient/ lib-http/ lib-jakarta-poi/ lib-log4j/ lib-lucene-analy

Jérôme Charron
> In a distributed configuration one needs to rebuild the job jar each
> time anything changes, and hence must check all plugins, etc.  So I
> would appreciate it if this didn't take quite so long.

Make sense!
Here is my proposal. For each plugin:
* Define a target containing core (will be used when building single plugin)
* Define a target not containing core (will be used when building whole
code)
I commit this as soon as possible.

Jérôme

--
http://motrech.free.fr/
http://www.frutch.org/
Reply | Threaded
Open this post in threaded view
|

Re: svn commit: r378655 - in /lucene/nutch/trunk/src/plugin: ./ analysis-de/ analysis-fr/ clustering-carrot2/ creativecommons/ index-basic/ index-more/ languageidentifier/ lib-commons-httpclient/ lib-http/ lib-jakarta-poi/ lib-log4j/ lib-lucene-analy

Doug Cutting
Jérôme Charron wrote:
> Here is my proposal. For each plugin:
> * Define a target containing core (will be used when building single plugin)
> * Define a target not containing core (will be used when building whole
> code)
> I commit this as soon as possible.

That sounds perfect.  Thanks!

Doug
Reply | Threaded
Open this post in threaded view
|

Re: svn commit: r378655 - in /lucene/nutch/trunk/src/plugin: ./ analysis-de/ analysis-fr/ clustering-carrot2/ creativecommons/ index-basic/ index-more/ languageidentifier/ lib-commons-httpclient/ lib-http/ lib-jakarta-poi/ lib-log4j/ lib-lucene-analy

Jérôme Charron
On 3/3/06, Doug Cutting <[hidden email]> wrote:

>
> Jérôme Charron wrote:
> > Here is my proposal. For each plugin:
> > * Define a target containing core (will be used when building single
> plugin)
> > * Define a target not containing core (will be used when building whole
> > code)
> > I commit this as soon as possible.
>
> That sounds perfect.  Thanks!

Committed.
Quick benchs:
* Before : around 70s
* After : around 50s

Better, but not so perfect...  :-(

Jérôme