Out of memory on Solr sorting

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Out of memory on Solr sorting

sundar shankar

Hi,
SOrry again fellos. I am not sure whats happening. The day with solr is bad for me I guess. EZMLM didnt let me send any mails this morning. Asked me to confirm subscription and when I did, it said I was already a member. Now my mails are all coming out bad. Sorry for troubling y'all this bad. I hope this mail comes out right.



    We are developing a product in a agile manner and the current implementation has a data of size just about a 800 megs in dev.
    The memory allocated to solr on dev (Dual core Linux box) is 128-512.

My config
=========

   



   

   

    true


My Field
=======

       
                   
           
               
                   
                         
       
       
           
               
               
               
       
   


Problem
======

I execute a query that returns 24 rows of result. I pick 10 out of it. I have no problem when I execute this.
But When I do sort it by a String field that is fetched from this result. I get an OOM. I am able to execute several
other queries with no problem. Just having a sort asc clause added to the query throws an OOM. Why is that.
What should I have ideally done. My config on QA is pretty similar to the dev box and probably has more data than on dev.
It didnt throw any OOM during the integration test. The Autocomplete is a new field we added recently.

Another point is that the indexing is done with a field of type string
 

and the autocomplete field is a copy field.

The sorting is done based on string field.

Please do lemme know what mistake am I doing?

Regards
Sundar

P.S: The stack trace of the exception is


Caused by: org.apache.solr.client.solrj.SolrServerException: Error executing query
 at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:86)
 at org.apache.solr.client.solrj.impl.BaseSolrServer.query(BaseSolrServer.java:101)
 at com.apollo.sisaw.solr.service.AbstractSolrSearchService.makeSolrQuery(AbstractSolrSearchService.java:193)
 ... 105 more
Caused by: org.apache.solr.common.SolrException: Java heap space  java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:403)  
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)  
at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:352)  
at org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSortedHitQueue.java:416)  
at org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:207)  
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)  
at org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)  
at org.apache.lucene.search.FieldSortedHitQueue.(FieldSortedHitQueue.java:56)  
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)  
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)  
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)  
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)  
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:156)  
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:128)  
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1025)  
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)  
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)  
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)  
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)  
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)  
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)  
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)  
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)  
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)  
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)  
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)  
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
_________________________________________________________________
Searching for the best deals on travel? Visit MSN Travel.
http://msn.coxandkings.co.in/cnk/cnk.do