[lucy-dev] ElasticSearch

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[lucy-dev] ElasticSearch

Peter Karman
Ran across this today:

http://www.elasticsearch.com/

I've just looked at it briefly and wanted others to be aware of the project.

--
Peter Karman  .  http://peknet.com/  .  [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-dev] ElasticSearch

Marvin Humphrey
On Mon, Dec 06, 2010 at 01:02:34PM -0600, Peter Karman wrote:
> Ran across this today:
>
> http://www.elasticsearch.com/
>
> I've just looked at it briefly and wanted others to be aware of the project.

For what it's worth, ElasticSearch is built on top of Lucene.  It exists at
the same level as Solr, one level up from Lucene and Lucy.

Marvin Humphrey

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-dev] ElasticSearch

Peter Karman
Marvin Humphrey wrote on 12/6/10 1:16 PM:

> On Mon, Dec 06, 2010 at 01:02:34PM -0600, Peter Karman wrote:
>> Ran across this today:
>>
>> http://www.elasticsearch.com/
>>
>> I've just looked at it briefly and wanted others to be aware of the project.
>
> For what it's worth, ElasticSearch is built on top of Lucene.  It exists at
> the same level as Solr, one level up from Lucene and Lucy.
>

ah ok. Yes, I saw it was Java-based but hadn't dug deep enough. thanks.

--
Peter Karman  .  http://peknet.com/  .  [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-dev] ElasticSearch

Dobrica Pavlinusic
On Mon, Dec 06, 2010 at 01:18:20PM -0600, Peter Karman wrote:

> Marvin Humphrey wrote on 12/6/10 1:16 PM:
> > On Mon, Dec 06, 2010 at 01:02:34PM -0600, Peter Karman wrote:
> >> Ran across this today:
> >>
> >> http://www.elasticsearch.com/
> >>
> >> I've just looked at it briefly and wanted others to be aware of the project.
> >
> > For what it's worth, ElasticSearch is built on top of Lucene.  It exists at
> > the same level as Solr, one level up from Lucene and Lucy.
> >
>
> ah ok. Yes, I saw it was Java-based but hadn't dug deep enough. thanks.

In similar vain, I found Riak search very isteresting:

http://www.basho.com/riaksearch.html

In fast, I love everything about it (including ability to throw xml at
it and get it indexed), but I would love to have something similar in
perl.

Which leads me to practical question: what's current state of schema in
KinoSearch and/or lucy?

I seems to remember API for adding fields on the fly while indexing in
older version of KinoSearch, but I can't find anything similar in
documentation for 0.31.

I would love to index CouchDB _changes feed with (much like
ElasticSearch CouchDB river) but I don't really know schema in advance.

--
Dobrica Pavlinusic               2share!2flame            [hidden email]
Unix addict. Internet consultant.             http://www.rot13.org/~dpavlin
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-dev] ElasticSearch

Marvin Humphrey
On Mon, Dec 06, 2010 at 11:54:32PM +0100, Dobrica Pavlinusic wrote:
> Which leads me to practical question: what's current state of schema in
> KinoSearch and/or lucy?
>
> I seems to remember API for adding fields on the fly while indexing in
> older version of KinoSearch, but I can't find anything similar in
> documentation for 0.31.
>
> I would love to index CouchDB _changes feed with (much like
> ElasticSearch CouchDB river) but I don't really know schema in advance.

Dynamic schemas are supported, same as in KS 0.2x.  Supplying a document with
an unknown field still triggers an exception, so you just have to iterate over
all fields in the unknown document and call spec_field() for anything which
hasn't been seen before.

    while (my $doc = get_next_doc_from_somewhere()) {
        for my $field (keys %$doc) {
            next if $schema->fetch_type($field);
            $schema->spec_field(name => $field, type => $type);
        }
        $indexer->add_doc($doc);
    }
    $indexer->commit;

Marvin Humphrey

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-dev] ElasticSearch

Peter Karman
In reply to this post by Dobrica Pavlinusic
Dobrica Pavlinusic wrote on 12/06/2010 04:54 PM:

> In similar vain, I found Riak search very isteresting:
>
> http://www.basho.com/riaksearch.html
>
> In fast, I love everything about it (including ability to throw xml at
> it and get it indexed), but I would love to have something similar in
> perl.
>

Swish3 handles the XML parsing, using KinoSearch on the backend. It'll
use Lucy as soon as we can get a Lucy release out the door.

Dynamic, on-the-fly schemas based on parsing XML could be supported as
well in Swish3. That's already on the todo list.

Help us get a Lucy release done so we can build on it. :)

--
Peter Karman  .  http://peknet.com/  .  [hidden email]