[jira] Created: (SOLR-2266) java.lang.ArrayIndexOutOfBoundsException in field cache when using a tdate field in a boost function with rord()

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-2266) java.lang.ArrayIndexOutOfBoundsException in field cache when using a tdate field in a boost function with rord()

Sebastian Nagel (Jira)
java.lang.ArrayIndexOutOfBoundsException in field cache when using a tdate field in a boost function with rord()
----------------------------------------------------------------------------------------------------------------

                 Key: SOLR-2266
                 URL: https://issues.apache.org/jira/browse/SOLR-2266
             Project: Solr
          Issue Type: Bug
    Affects Versions: 1.4.1
         Environment: Mac OS 10.6
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261)
Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode)

            Reporter: Peter Wolanin



I have been testing a switch to long and tdate instead of int and date fields in the schema.xml for our Drupal integration.  This indexes fine, but search fails with a 500 error.

{code}
INFO: [d7] webapp=/solr path=/select params={spellcheck=true&facet=true&facet.mincount=1&indent=1&spellcheck.q=term&json.nl=map&wt=json&rows=10&version=1.2&fl=id,entity_id,entity,bundle,bundle_name,nid,title,comment_count,type,created,changed,score,path,url,uid,name&start=0&facet.sort=true&q=term&bf=recip(rord(created),4,19,19)^200.0} status=500 QTime=4
Dec 5, 2010 11:52:28 AM org.apache.solr.common.SolrException log
SEVERE: java.lang.ArrayIndexOutOfBoundsException: 39
        at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:721)
        at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:224)
        at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:692)
        at org.apache.solr.search.function.ReverseOrdFieldSource.getValues(ReverseOrdFieldSource.java:61)
        at org.apache.solr.search.function.TopValueSource.getValues(TopValueSource.java:57)
        at org.apache.solr.search.function.ReciprocalFloatFunction.getValues(ReciprocalFloatFunction.java:61)
        at org.apache.solr.search.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:123)
        at org.apache.solr.search.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:93)
        at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:297)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:250)
        at org.apache.lucene.search.Searcher.search(Searcher.java:171)
        at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
        at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
        at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
        at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
        at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
        at com.acquia.search.HmacFilter.doFilter(HmacFilter.java:62)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:285)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
{code}

The exception goes away if I remove the boost function param bf=recip(rord(created),4,19,19)^200.0

Omitting the recip() doesn't help, so just bf=rord(created)^200.0 still causes the exception.

In this schema,

{code}
 <field name="created" type="tdate" indexed="true" stored="true"/>
{code}

In the prior schema:

{code}
 <field name="created" type="date" indexed="true" stored="true"/>
{code}

This seems somewhat related to:  SOLR-324 which is marked as fixed.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-2266) java.lang.ArrayIndexOutOfBoundsException in field cache when using a tdate field in a boost function with rord()

Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12967022#action_12967022 ]

Yonik Seeley commented on SOLR-2266:
------------------------------------

OK, here's my guess: it's probably due to multiple indexed values per field value.  ord/rord uses the StringIndex to get the ord values, which can't handle multiple indexed tokens per field value.

The "tdate" type has a precisionStep > 0, meaning it will index multiple values per field value to speed up range queries.
If you don't need faster range queries on this type, then use "date" instead of "tdate".

But the ideal fix here is to eliminate the use of ord/rord since they also use up more memory... sorting by "created" will instantiate a per-segment long[] FieldCache entry.
It would be nice if that could be reused for the function queries too.  This is the case if you use ms().
http://wiki.apache.org/solr/FunctionQuery#ms

> java.lang.ArrayIndexOutOfBoundsException in field cache when using a tdate field in a boost function with rord()
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-2266
>                 URL: https://issues.apache.org/jira/browse/SOLR-2266
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4.1
>         Environment: Mac OS 10.6
> java version "1.6.0_22"
> Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261)
> Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode)
>            Reporter: Peter Wolanin
>
> I have been testing a switch to long and tdate instead of int and date fields in the schema.xml for our Drupal integration.  This indexes fine, but search fails with a 500 error.
> {code}
> INFO: [d7] webapp=/solr path=/select params={spellcheck=true&facet=true&facet.mincount=1&indent=1&spellcheck.q=term&json.nl=map&wt=json&rows=10&version=1.2&fl=id,entity_id,entity,bundle,bundle_name,nid,title,comment_count,type,created,changed,score,path,url,uid,name&start=0&facet.sort=true&q=term&bf=recip(rord(created),4,19,19)^200.0} status=500 QTime=4
> Dec 5, 2010 11:52:28 AM org.apache.solr.common.SolrException log
> SEVERE: java.lang.ArrayIndexOutOfBoundsException: 39
>         at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:721)
>         at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:224)
>         at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:692)
>         at org.apache.solr.search.function.ReverseOrdFieldSource.getValues(ReverseOrdFieldSource.java:61)
>         at org.apache.solr.search.function.TopValueSource.getValues(TopValueSource.java:57)
>         at org.apache.solr.search.function.ReciprocalFloatFunction.getValues(ReciprocalFloatFunction.java:61)
>         at org.apache.solr.search.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:123)
>         at org.apache.solr.search.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:93)
>         at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:297)
>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:250)
>         at org.apache.lucene.search.Searcher.search(Searcher.java:171)
>         at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
>         at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
>         at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
>         at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
>         at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>         at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>         at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>         at com.acquia.search.HmacFilter.doFilter(HmacFilter.java:62)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
>         at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>         at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>         at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>         at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>         at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
>         at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>         at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>         at org.mortbay.jetty.Server.handle(Server.java:285)
>         at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
>         at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
>         at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
>         at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
>         at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
>         at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
>         at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> {code}
> The exception goes away if I remove the boost function param bf=recip(rord(created),4,19,19)^200.0
> Omitting the recip() doesn't help, so just bf=rord(created)^200.0 still causes the exception.
> In this schema,
> {code}
>  <field name="created" type="tdate" indexed="true" stored="true"/>
> {code}
> In the prior schema:
> {code}
>  <field name="created" type="date" indexed="true" stored="true"/>
> {code}
> This seems somewhat related to:  SOLR-324 which is marked as fixed.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]