re :Range queries

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

re :Range queries

Kinnar Kumar Sen, Noida

Hi

    When I am trying RANGE QUERY on my index it works fine for a  small
index but when the index is large such as 0 - 1000000 it gives an
exception

  Boolean Clause Exception  I have set the 1024 value in boolean to
integer.max but now  is giving a out of memory exception . Can some body
suggest me a way out or a technique to effectively use range queries  I
need to use this feature extensively for my application for ranges like
0 - 100000 ,





Regards and Thanks

Kinnar Kumar Sen







TO SUCEED BE DIFFERENT BE DARING AND BE THERE FIRST









DISCLAIMER:
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.
-----------------------------------------------------------------------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|

RE: Range queries

Ramana Jelda
Hi,

You can use  BooleanQuery.setMaxClauseCount() to set to your required
maximum clause count.
But ofcourse too big clause count is not advisable.

May be you need to find some strategy to reduce this range. Ex:Reduce date
ranges to 20060505 from including timestamp like 20060505120530 :)

All the best.

Jelda

> -----Original Message-----
> From: Kinnar Kumar Sen, Noida [mailto:[hidden email]]
> Sent: Tuesday, May 09, 2006 11:57 AM
> To: [hidden email]
> Cc: [hidden email]
> Subject: re :Range queries
>
>
> Hi
>
>     When I am trying RANGE QUERY on my index it works fine
> for a  small index but when the index is large such as 0 -
> 1000000 it gives an exception
>
>   Boolean Clause Exception  I have set the 1024 value in
> boolean to integer.max but now  is giving a out of memory
> exception . Can some body suggest me a way out or a technique
> to effectively use range queries  I need to use this feature
> extensively for my application for ranges like 0 - 100000 ,
>  
>  
> Regards and Thanks
>
> Kinnar Kumar Sen
>
>  
>  
>  
> TO SUCEED BE DIFFERENT BE DARING AND BE THERE FIRST
>
>  
>  
>  
>
>
> DISCLAIMER:
> --------------------------------------------------------------
> ---------------------------------------------------------
> The contents of this e-mail and any attachment(s) are
> confidential and intended for the named recipient(s) only. It
> shall not attach any liability on the originator or HCL or
> its affiliates. Any views or opinions presented in this email
> are solely those of the author and may not necessarily
> reflect the opinions of HCL or its affiliates. Any form of
> reproduction, dissemination, copying, disclosure,
> modification, distribution and / or publication of this
> message without the prior written consent of the author of
> this e-mail is strictly prohibited. If you have received this
> email in error please delete it and notify the sender
> immediately. Before opening any mail and attachments please
> check them for viruses and defect.
> --------------------------------------------------------------
> ---------------------------------------------------------


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Range queries

mark harwood
Typically the 3 most important things to remember when
using numerical range queries are:

1) Use a filter instead.
2) Use a filter instead.
3) Use a filter instead.

Seriously, number rangeQueries are normally a bad idea
because:

a) they can produce "too many term" errors (your
current problem)
b) they influence results ranking in ways which are
undesirable (do we typically care how rare one number
in the range is versus another? - Lucene does when
scoring RangeQueries)
c) they are slower than filters

See here for examples of filter-based alternatives -
http://wiki.apache.org/jakarta-lucene/FilteringOptions


Cheers
Mark

Send instant messages to your online friends http://uk.messenger.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

re :Range queries

Nadav Har'El-2
In reply to this post by Kinnar Kumar Sen, Noida
"Kinnar Kumar Sen, Noida" <[hidden email]> wrote on 09/05/2006 12:57:16
PM:
>     When I am trying RANGE QUERY on my index it works fine for a  small
> index but when the index is large such as 0 - 1000000 it gives an
> exception
>
>   Boolean Clause Exception  I have set the 1024 value in boolean to
> integer.max but now  is giving a out of memory exception . Can some body
> suggest me a way out or a technique to effectively use range queries  I
> need to use this feature extensively for my application for ranges like
> 0 - 100000 ,

Use "ConstantScoreRangeQuery" instead of "RangeQuery". It is still very
inefficient, and you still need to remember to pad all your numbers so
they sort properly *lexicographically* (e.g., 000000-1000000), but at
least you should not have exceptions any more.

--
Nadav Har'El


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]