ElasticSearch

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

ElasticSearch

Robust Links
Hi

A client is considering moving from Lucene to ElasticSearch. What is the community's opinion on ES?

thank you

Peyman
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Bill Mitchell
Under the covers, ElasticSearch contains mutliple lucene indexes -- so the
full expressiveness of lucene queries are translatable to ElasticSearch --
but the benefit of using ES as an abstraction layer to give sharded
searches is something attractive enough that we're looking at it too.  ;)

We typically see fairly large lucene index over time, and having
ElasticSearch to 'purge' sets of data, but still get federated searches is
very very attractive.

Perhaps a discussion of moving from SOLR to ElasticSearch would be a more
fair comparison.

regards,
Bill


On Wed, Nov 16, 2011 at 9:12 AM, Peyman Faratin <[hidden email]>wrote:

> Hi
>
> A client is considering moving from Lucene to ElasticSearch. What is the
> community's opinion on ES?
>
> thank you
>
> Peyman
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Federico Fissore
In reply to this post by Robust Links
Peyman Faratin, il 16/11/2011 15:12, ha scritto:
> Hi
>
> A client is considering moving from Lucene to ElasticSearch. What is the community's opinion on ES?
>
> thank you


we have recently compared ES to Solr to estimate the effort of evolving
our search infrastructure (it was game like: two teams tried to "sell"
each software to the other one)

the ES team in the end has found it good as a storage but difficult to
extend for a lucene expert. the Solr one provided some extension
examples and argued that raw lucene API access was easier. Solr was "sold"

Hope it's clear how relative this analysis was WRT to the quality of the
teams and the time they had for the evaluation, but we are extending
solr ATM, so we hope not to have missed the point completely

regards

federico

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Shashi Kant-2
I had posted this earlier on this list, hope this provides some answers

http://engineering.socialcast.com/2011/05/realtime-search-solr-vs-elasticsearch/



On Wed, Nov 16, 2011 at 9:53 AM, Federico Fissore <[hidden email]>wrote:

> Peyman Faratin, il 16/11/2011 15:12, ha scritto:
>
>  Hi
>>
>> A client is considering moving from Lucene to ElasticSearch. What is the
>> community's opinion on ES?
>>
>> thank you
>>
>
>
> we have recently compared ES to Solr to estimate the effort of evolving
> our search infrastructure (it was game like: two teams tried to "sell" each
> software to the other one)
>
> the ES team in the end has found it good as a storage but difficult to
> extend for a lucene expert. the Solr one provided some extension examples
> and argued that raw lucene API access was easier. Solr was "sold"
>
> Hope it's clear how relative this analysis was WRT to the quality of the
> teams and the time they had for the evaluation, but we are extending solr
> ATM, so we hope not to have missed the point completely
>
> regards
>
> federico
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.**apache.org<[hidden email]>
> For additional commands, e-mail: [hidden email].**org<[hidden email]>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Yonik Seeley-2-2
On Wed, Nov 16, 2011 at 10:36 AM, Shashi Kant <[hidden email]> wrote:
> I had posted this earlier on this list, hope this provides some answers
>
> http://engineering.socialcast.com/2011/05/realtime-search-solr-vs-elasticsearch/

Except it's an out of date comparison.
We have NRT (near real time search) in Solr now.

http://wiki.apache.org/solr/NearRealtimeSearch

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Peter
 Hi,

its not really fair to compare NRT of Solr to ElasticSearch.
ElasticSearch provides NRT for distributed indices as well... also when
doing heavy indexing Solr lacks real NRT.

The only main disadvantages of ElasticSearch are:
 * only one (main) committer
 * no autowarming


> the ES team in the end has found it good as a storage but difficult to
extend for a lucene expert.

The nice thing with ES is that you can e.g. create lucene queries with
even high complexity as ES supports lucene-like query nesting via JSON.
Also when implementing server side stuff you can take advantage of full
lucene power.

Ah, before I forgot it: it is very important to test the software
yourself. Do not trust me or anybody else :), also the software should
fit to your environment, requirements + team!

Regards,
Peter.


PS: here is my different comparison:
http://karussell.wordpress.com/2011/05/12/elasticsearch-vs-solr-lucene/


> On Wed, Nov 16, 2011 at 10:36 AM, Shashi Kant <[hidden email]> wrote:
>> I had posted this earlier on this list, hope this provides some answers
>>
>> http://engineering.socialcast.com/2011/05/realtime-search-solr-vs-elasticsearch/
> Except it's an out of date comparison.
> We have NRT (near real time search) in Solr now.
>
> http://wiki.apache.org/solr/NearRealtimeSearch
>
> -Yonik
> http://www.lucidimagination.com


--
http://jetsli.de news reader for geeks


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Jason Rutherglen
> even high complexity as ES supports lucene-like query nesting via JSON

That sounds interesting.  Where is it described in the ES docs?  Thanks.

On Wed, Nov 16, 2011 at 1:36 PM, Peter Karich <[hidden email]> wrote:

>  Hi,
>
> its not really fair to compare NRT of Solr to ElasticSearch.
> ElasticSearch provides NRT for distributed indices as well... also when
> doing heavy indexing Solr lacks real NRT.
>
> The only main disadvantages of ElasticSearch are:
>  * only one (main) committer
>  * no autowarming
>
>
>> the ES team in the end has found it good as a storage but difficult to
> extend for a lucene expert.
>
> The nice thing with ES is that you can e.g. create lucene queries with
> even high complexity as ES supports lucene-like query nesting via JSON.
> Also when implementing server side stuff you can take advantage of full
> lucene power.
>
> Ah, before I forgot it: it is very important to test the software
> yourself. Do not trust me or anybody else :), also the software should
> fit to your environment, requirements + team!
>
> Regards,
> Peter.
>
>
> PS: here is my different comparison:
> http://karussell.wordpress.com/2011/05/12/elasticsearch-vs-solr-lucene/
>
>
>> On Wed, Nov 16, 2011 at 10:36 AM, Shashi Kant <[hidden email]> wrote:
>>> I had posted this earlier on this list, hope this provides some answers
>>>
>>> http://engineering.socialcast.com/2011/05/realtime-search-solr-vs-elasticsearch/
>> Except it's an out of date comparison.
>> We have NRT (near real time search) in Solr now.
>>
>> http://wiki.apache.org/solr/NearRealtimeSearch
>>
>> -Yonik
>> http://www.lucidimagination.com
>
>
> --
> http://jetsli.de news reader for geeks
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Peter

>> even high complexity as ES supports lucene-like query nesting via JSON
> That sounds interesting.  Where is it described in the ES docs?  Thanks.

"Think of the Query DSL as an AST of queries"
http://www.elasticsearch.org/guide/reference/query-dsl/

For further info ask on ES mailing list.

Regards,
Peter.

--
http://jetsli.de news reader for geeks


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Jason Rutherglen
The docs are slim on examples.

On Wed, Nov 16, 2011 at 3:35 PM, Peter Karich <[hidden email]> wrote:

>
>>> even high complexity as ES supports lucene-like query nesting via JSON
>> That sounds interesting.  Where is it described in the ES docs?  Thanks.
>
> "Think of the Query DSL as an AST of queries"
> http://www.elasticsearch.org/guide/reference/query-dsl/
>
> For further info ask on ES mailing list.
>
> Regards,
> Peter.
>
> --
> http://jetsli.de news reader for geeks
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Chris Hostetter-3
In reply to this post by Peter

: "Think of the Query DSL as an AST of queries"
: http://www.elasticsearch.org/guide/reference/query-dsl/

I'm not familiar with ES, but FWIW: based on that one page the "Query DSL"
doesn't really sound much more powerful then what you can do with nested
queries, local params, and param refs using the various Solr QParsers.

Here's a (somewhat contrived) example from my Apachecon talk last week...

        ?q={!boost b=div(popularity,price) v=$qq}
        &qq={!dismax qf=desc^2,review}cheap
        &bq={!lucene df=keywords}lucene solr java
        &fq={!geofilt sfield=location pt=10.312,-20.556 d=3.5}
        &fq={!term f=$ff v=$vv}&ff=keywords&vv=solr
        &sort=query(keywords:lame) asc, score desc

https://people.apache.org/~hossman/apachecon2011/

-Hoss

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Simon Willnauer
On Wed, Nov 16, 2011 at 10:18 PM, Chris Hostetter
<[hidden email]> wrote:

>
> : "Think of the Query DSL as an AST of queries"
> : http://www.elasticsearch.org/guide/reference/query-dsl/
>
> I'm not familiar with ES, but FWIW: based on that one page the "Query DSL"
> doesn't really sound much more powerful then what you can do with nested
> queries, local params, and param refs using the various Solr QParsers.
>
> Here's a (somewhat contrived) example from my Apachecon talk last week...
>
>        ?q={!boost b=div(popularity,price) v=$qq}
>        &qq={!dismax qf=desc^2,review}cheap
>        &bq={!lucene df=keywords}lucene solr java
>        &fq={!geofilt sfield=location pt=10.312,-20.556 d=3.5}
>        &fq={!term f=$ff v=$vv}&ff=keywords&vv=solr
>        &sort=query(keywords:lame) asc, score desc
>

I agree, one big difference here is that I might spend 2h reading &
finding documentation to figure out what that does :D

simon
> https://people.apache.org/~hossman/apachecon2011/
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Peter

>> : "Think of the Query DSL as an AST of queries"
>> : http://www.elasticsearch.org/guide/reference/query-dsl/
>>
>> I'm not familiar with ES, but FWIW: based on that one page the "Query DSL"
>> doesn't really sound much more powerful then what you can do with nested
>> queries, local params, and param refs using the various Solr QParsers.
>>
>> Here's a (somewhat contrived) example from my Apachecon talk last week...
>>
>>        ?q={!boost b=div(popularity,price) v=$qq}
>>        &qq={!dismax qf=desc^2,review}cheap
>>        &bq={!lucene df=keywords}lucene solr java
>>        &fq={!geofilt sfield=location pt=10.312,-20.556 d=3.5}
>>        &fq={!term f=$ff v=$vv}&ff=keywords&vv=solr
>>        &sort=query(keywords:lame) asc, score desc
>>
> I agree, one big difference here is that I might spend 2h reading &
> finding documentation to figure out what that does :D
>
> simon

Did you mean the ElasticSearch or the Solr part ;) !! ;)

Peter.

--
http://jetsli.de news reader for geeks


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Simon Willnauer
On Thu, Nov 17, 2011 at 9:25 AM, Peter Karich <[hidden email]> wrote:

>
>>> : "Think of the Query DSL as an AST of queries"
>>> : http://www.elasticsearch.org/guide/reference/query-dsl/
>>>
>>> I'm not familiar with ES, but FWIW: based on that one page the "Query DSL"
>>> doesn't really sound much more powerful then what you can do with nested
>>> queries, local params, and param refs using the various Solr QParsers.
>>>
>>> Here's a (somewhat contrived) example from my Apachecon talk last week...
>>>
>>>        ?q={!boost b=div(popularity,price) v=$qq}
>>>        &qq={!dismax qf=desc^2,review}cheap
>>>        &bq={!lucene df=keywords}lucene solr java
>>>        &fq={!geofilt sfield=location pt=10.312,-20.556 d=3.5}
>>>        &fq={!term f=$ff v=$vv}&ff=keywords&vv=solr
>>>        &sort=query(keywords:lame) asc, score desc
>>>
>> I agree, one big difference here is that I might spend 2h reading &
>> finding documentation to figure out what that does :D
>>
>> simon
>
> Did you mean the ElasticSearch or the Solr part ;) !! ;)

dude, look at this query... its insane isn't it :)

simon

>
> Peter.
>
> --
> http://jetsli.de news reader for geeks
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Yonik Seeley-2-2
On Thu, Nov 17, 2011 at 2:53 PM, Simon Willnauer
<[hidden email]> wrote:
> dude, look at this query... its insane isn't it :)

Sorry... what's the equivalent you'd like instead?
Or if you're just unjustifiably bitching about Solr again, maybe I
should take a stroll through Lucene land and bitch about
incomprehensible code, APIs that are increasingly hard to use, APIs
that keep changing on a whim w/o regard to existing users, etc.  Your
attitude is getting tiring.

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Petite Abeille

On Nov 17, 2011, at 9:03 PM, Yonik Seeley wrote:

>> dude, look at this query... its insane isn't it :)
>
> Sorry... what's the equivalent you'd like instead?
> Or if you're just unjustifiably bitching about Solr again, maybe I
> should take a stroll through Lucene land and bitch about
> incomprehensible code, APIs that are increasingly hard to use, APIs
> that keep changing on a whim w/o regard to existing users, etc.  Your
> attitude is getting tiring.

Ohhh... touchy, touchy :)

One the other hand, since this project has joined Apache, it really has gone downhill... ah!... take that Lucene! :P


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Simon Willnauer
In reply to this post by Yonik Seeley-2-2
On Thu, Nov 17, 2011 at 8:03 PM, Yonik Seeley
<[hidden email]> wrote:
> On Thu, Nov 17, 2011 at 2:53 PM, Simon Willnauer
> <[hidden email]> wrote:
>> dude, look at this query... its insane isn't it :)
>
> Sorry... what's the equivalent you'd like instead?

oh, I think there are many ways to design a DSL for querying..
something I can read and understand without reading documentation
maybe? :)

> Or if you're just unjustifiably bitching about Solr again, maybe I
> should take a stroll through Lucene land and bitch about
> incomprehensible code, APIs that are increasingly hard to use, APIs
> that keep changing on a whim w/o regard to existing users, etc.

I think you should do it. I mean this can only take us further! Ask
for 400% and get 100% ....more than nothing. Sitting there thinking
the APIs suck doesn't move us anywhere. Its open source. please help
getting the APIs straight!

> Your attitude is getting tiring.

sorry about that... I think its not my attitude, I am not against solr
or anything but if you look at the differences we should try to
improve where the biggest gaps are visible instead of keeping quiet.

simon
>
> -Yonik
> http://www.lucidimagination.com
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: ElasticSearch

Uwe Schindler
In reply to this post by Yonik Seeley-2-2
> Or if you're just unjustifiably bitching about Solr again

Sorry, this query is really ununderstandable. Those complex queries should
have a meaningful language, e.g. a JSON object structure (like
XMLQueryParser, but instead JSON). Those queries are never entered by users
only by machines, why not make them easy interpretable and produceable by
using structured objects and not ugly string-concatenations - leads to
security bugs like SQL injection.

> maybe I should take a
> stroll through Lucene land and bitch about incomprehensible code, APIs
that
> are increasingly hard to use, APIs that keep changing on a whim w/o regard
to
> existing users, etc.  Your attitude is getting tiring.

This only affects bulk postings. In my opinion, the trunk APIs are much
easier to understand than the broken 3.x API.

Uwe


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Yonik Seeley-2-2
On Thu, Nov 17, 2011 at 3:18 PM, Uwe Schindler <[hidden email]> wrote:
> Sorry, this query is really ununderstandable. Those complex queries should
> have a meaningful language, e.g. a JSON object structure

There are upsides and downsides to that.  A big JSON object graph
would be easier to *read* but certainly not easier to write (much more
nesting).
These main Solr APIs are based around HTTP parameters... the upside
being you can add another parameter w/o worrying about nesting it
correctly.

Like simply adding another filter for example:
fq=instock:true

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

mark harwood
I don't think of queries as inherently flat in the way HTTP request parameters are with their name=value pairings.

JSON or XML can reflect more closely the hierarchy in the underlying Lucene query objects.
For me using a "flat" query interface feels a bit like when you start off trying to manage your application config in ".properties" files and quickly hit the limitations of their expressiveness. It's fine for simple things but complexity soon overwhelms.

As well as hierarchical syntax it's worth considering the role a formal query schema has to play. The XMLQueryParser has a set of DTDs that currently serve to generate HTML documentation but also could conceivably be used by tooling to drive query construction. "Runnable documentation" always feels like a useful combo.

Cheers
Mark




On 17 Nov 2011, at 20:21, Yonik Seeley wrote:

> On Thu, Nov 17, 2011 at 3:18 PM, Uwe Schindler <[hidden email]> wrote:
>> Sorry, this query is really ununderstandable. Those complex queries should
>> have a meaningful language, e.g. a JSON object structure
>
> There are upsides and downsides to that.  A big JSON object graph
> would be easier to *read* but certainly not easier to write (much more
> nesting).
> These main Solr APIs are based around HTTP parameters... the upside
> being you can add another parameter w/o worrying about nesting it
> correctly.
>
> Like simply adding another filter for example:
> fq=instock:true
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch

Michael McCandless-2
In reply to this post by Yonik Seeley-2-2
Maybe someone can post the equivalent query in ElasticSearch?  Then at
least we have a fair comparison of the two syntaxes, for this one
(complex) query at least...

Mike McCandless

http://blog.mikemccandless.com

On Thu, Nov 17, 2011 at 3:21 PM, Yonik Seeley
<[hidden email]> wrote:

> On Thu, Nov 17, 2011 at 3:18 PM, Uwe Schindler <[hidden email]> wrote:
>> Sorry, this query is really ununderstandable. Those complex queries should
>> have a meaningful language, e.g. a JSON object structure
>
> There are upsides and downsides to that.  A big JSON object graph
> would be easier to *read* but certainly not easier to write (much more
> nesting).
> These main Solr APIs are based around HTTP parameters... the upside
> being you can add another parameter w/o worrying about nesting it
> correctly.
>
> Like simply adding another filter for example:
> fq=instock:true
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12