[lucy-user] ClusterSearcher

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[lucy-user] ClusterSearcher

Thal Asure
Hello,

I'm keen on LucyX::Remote::ClusterSearcher for obvious reasons.  I may be
wrong from my casual clicking around, but docs for
LucyX::Remote::ClusterSearcher
appears to be "hidden", like an embarrassing cousin who's parentage is iffy.

I see it mentioned here:
http://mail-archives.apache.org/mod_mbox/lucy-user/201301.mbox/browser
and here:
http://lucy.apache.org/docs/test/LucyX/Remote/ClusterSearcher.html
and here: http://search.cpan.org/~creamyg/Lucy-0.4.1/

...but I can't find a reference to it directly from here:
http://lucy.apache.org/docs/perl/

Is anyone using it successfully on large(ish) indexes (say, 1TB+ sharded
across N nodes)?
How stable/mature is it?  Any gotchas,gaps?

Cheers!
Tha
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] ClusterSearcher

Marvin Humphrey
On Tue, Dec 23, 2014 at 11:32 PM, Thal Asure <[hidden email]> wrote:

> I'm keen on LucyX::Remote::ClusterSearcher for obvious reasons.  I may be
> wrong from my casual clicking around, but docs for
> LucyX::Remote::ClusterSearcher
> appears to be "hidden", like an embarrassing cousin who's parentage is iffy.

All children are precious, including ClusterSearcher. :)

Though ClusterSearcher's documentation had gone missing on lucy.apache.org,
it has always been available on search.cpan.org, metacpan.org, etc.

    http://search.cpan.org/perldoc?LucyX::Remote::ClusterSearcher

> I see it mentioned here:
> http://mail-archives.apache.org/mod_mbox/lucy-user/201301.mbox/browser
> and here:
> http://lucy.apache.org/docs/test/LucyX/Remote/ClusterSearcher.html
> and here: http://search.cpan.org/~creamyg/Lucy-0.4.1/
>
> ...but I can't find a reference to it directly from here:
> http://lucy.apache.org/docs/perl/

Thank you for the report.  Its absence was due to a flaw in Lucy's release
runbook -- regenerating our website docs requires the Report Manager to take
manual steps which have been inadequately specified.  I've now performed the
regeneration.

> Is anyone using it successfully on large(ish) indexes (say, 1TB+ sharded
> across N nodes)?
> How stable/mature is it?  Any gotchas,gaps?

The thing about ClusterSearcher is that it is not accompanied by a
complementary tool to perform turnkey sharded indexing.  Implementing such a
tool requires that you make some decisions about index structure -- almost
certainly you'll want a primary key, which Lucy doesn't require by default.

Rather than build on ClusterSearcher, my colleagues at Eventful rolled their
own solution, which is not general enough to consider open-sourcing.  I
suspect we're not the only ones who have taken that path.

Marvin Humphrey