solr java.lang.NullPointerException on select queries

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

solr java.lang.NullPointerException on select queries

avenka
This post was updated on .
I have recently started getting the error pasted below with solr-3.6 on /select queries. I don't know of anything that changed in the config to start causing this error. I am also running a second independent solr server on the same machine, which continues to run fine and has the same configuration as the first one except for the port number. The first one seems to be doing dataimport operations fine and updating index files as usual, but fails on select queries. An example of a failing query (that used to run fine) is: http://<machine_name>/solr/select/?q=title%3Afoo&version=2.2&start=0&rows=10&indent=on

I also found that I get this error only when the result set is non-empty. I am stupefied. Any idea?

HTTP ERROR 500

Problem accessing /solr/select/. Reason:

null
java.lang.NullPointerException at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:186) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Can you tell us more about how you ran the second server? Is this an
independent installation or is it the same installation as your first
server just starting with a different port?

Because this is very strange. It half feels like there are conflicting
jars somewhere in your path, but that's a guess. This really shouldn't
be happening unless something _did_ change that doesn't seem
important but is....

And does this only happen when both servers are running? If you turn
the second one off and this still occurs, it _really_ sounds like clashing
jar files....

Best
Erick

On Sat, Jun 16, 2012 at 3:39 PM, avenka <[hidden email]> wrote:

> I have recently started getting the error pasted below with solr-3.6 on
> /select queries. I don't know of anything that changed in the config to
> start causing this error. I am also running a second independent solr server
> on the same machine, which continues to run fine and has the same
> configuration as the first one except for the port number. The first one
> seems to be doing dataimport operations fine and updating index files as
> usual, but fails on select queries. An example of a failing query (that used
> to run fine) is:
> http://<machine_name>/solr/select/?q=title%3Afoo&version=2.2&start=0&rows=10&indent=on
>
> I am stupefied. Any idea?
>
> HTTP ERROR 500
>
> Problem accessing /solr/select/. Reason:
>
> null
> java.lang.NullPointerException at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:186)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326) at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
For the first install, I copied over all files in the directory "example" into, let's call it, "install1". I did the same for "install2". The two installs run on different ports, use different jar files, are not really related to each other in any way as far as I can see. In particular, they are not "multicore". They have the same access control setup via jetty. I did a diff on config files and confirmed that only port numbers are different.

Both had been running fine in parallel importing from a common database for several weeks. The documents indexed by install1, the problematic one currently, is a vastly bigger (~2.5B) superset of those indexed by install2 (~250M).

At this point, select queries on install1 incurs the NullPointerException irrespective of whether install2 is running or not. The log file looks like it is indexing normally as always though. The index is also growing at the usual rate each day. Just select queries fail. :(
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Internal Lucene document IDs are signed 32 bit numbers, so having
2.5B docs seems to be just _asking_ for trouble. Which could
explain the fact that this just came out of thin air. If you kept adding
docs to the problem instance, you wouldn't have changed configs
etc, just added more docs....

I really think it's time to shard.

Best
Erick

On Wed, Jun 20, 2012 at 2:15 AM, avenka <[hidden email]> wrote:

> For the first install, I copied over all files in the directory "example"
> into, let's call it, "install1". I did the same for "install2". The two
> installs run on different ports, use different jar files, are not really
> related to each other in any way as far as I can see. In particular, they
> are not "multicore". They have the same access control setup via jetty. I
> did a diff on config files and confirmed that only port numbers are
> different.
>
> Both had been running fine in parallel importing from a common database for
> several weeks. The documents indexed by install1, the problematic one
> currently, is a vastly bigger (~2.5B) superset of those indexed by install2
> (~250M).
>
> At this point, select queries on install1 incurs the NullPointerException
> irrespective of whether install2 is running or not. The log file looks like
> it is indexing normally as always though. The index is also growing at the
> usual rate each day. Just select queries fail. :(
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3990476.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
Erick, thanks for pointing that out. I was going to say in my original post that it is almost like some limit on max documents got violated all of a sudden, but the rest of the symptoms didn't seem to quite match. But now that I think about it, the problem probably happened at 2B (corresponding exactly to the size of the signed int space) as my ID space in the database has roughly 85% holes and the problem probably happened when the ID hit around 2.4B.

It is still odd that indexing appears to proceed normally and the select queries "know" which IDs are used because the error happens only for queries with non-empty results, e.g., searching for an ID that doesn't exist gives a valid "0 numResponses" response. Is this because solr uses 'long' or more for indexing (given that the schema supports long) but not in the querying modules?

I hadn't used solr sharding because I really needed "rolling" partitions, where I keep a small index of recent documents and throw the rest into a slow "archive" index. So maintaining the smaller instance2 (usually < 50M) and replicating it if needed was my homebrewed sharding approach. But I guess it is time to shard the archive after all.

AV
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Let's make sure we're talking about the same thing. Solr happily
indexes and stores long (64) bit values, no problem. What it doesn't
do is assign _internal_ documents IDs as longs, those are ints.

on admin/statistics, look at maxDocs and numDocs. maxDocs +1 will be the
next _internal_ lucene doc id assigned, so if that's wonky or > 2B, this
is where the rub happens. BTW, the difference between numDocs and
maxDocs is the number of documents deleted from your index. If your number
of current documents is much smaller than 2B, you can get maxDocs
to equal numDocs if you optimize, and get yourself some more headroom.
whether your index will be OK I'm not prepared to guarantee though...

But if I'm reading your notes correctly, the "85% holes" applies to a value in
your document, and has nothing to do with the internal lucene ID issue.....

But internally, the int limit isn't robustly enforced, so I'm not
surprised that it
pops out (if, indeed, this is your problem) in odd places.

Best
Erick

On Wed, Jun 20, 2012 at 10:02 AM, avenka <[hidden email]> wrote:

> Erick, thanks for pointing that out. I was going to say in my original post
> that it is almost like some limit on max documents got violated all of a
> sudden, but the rest of the symptoms didn't seem to quite match. But now
> that I think about it, the problem probably happened at 2B (corresponding
> exactly to the size of the signed int space) as my ID space in the database
> has roughly 85% holes and the problem probably happened when the ID hit
> around 2.4B.
>
> It is still odd that indexing appears to proceed normally and the select
> queries "know" which IDs are used because the error happens only for queries
> with non-empty results, e.g., searching for an ID that doesn't exist gives a
> valid "0 numResponses" response. Is this because solr uses 'long' or more
> for indexing (given that the schema supports long) but not in the querying
> modules?
>
> I hadn't used solr sharding because I really needed "rolling" partitions,
> where I keep a small index of recent documents and throw the rest into a
> slow "archive" index. So maintaining the smaller instance2 (usually < 50M)
> and replicating it if needed was my homebrewed sharding approach. But I
> guess it is time to shard the archive after all.
>
> AV
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3990534.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
Yes, wonky indeed.
  numDocs : -2006905329
  maxDoc : -1993357870

And yes, I meant that the holes are in the database auto-increment ID space, nothing to do with lucene IDs.

I will set up sharding. But is there any way to retrieve most of the current index? Currently, all select queries even in ranges in the hundreds of millions return the NullPointerException. It would suck to lose all of this. :(
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
That indeed sucks. But I don't personally know of a good way to
try to split apart an existing index into shards. I'm afraid you're
going to be stuck with re-indexing....

Wish I had a better solution
Erick

On Wed, Jun 20, 2012 at 10:45 AM, avenka <[hidden email]> wrote:

> Yes, wonky indeed.
>  numDocs : -2006905329
>  maxDoc : -1993357870
>
> And yes, I meant that the holes are in the database auto-increment ID space,
> nothing to do with lucene IDs.
>
> I will set up sharding. But is there any way to retrieve most of the current
> index? Currently, all select queries even in ranges in the hundreds of
> millions return the NullPointerException. It would suck to lose all of this.
> :(
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3990542.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
Thanks. Do you know if the tons of index files with names like '_zxt.tis' in the index/data/ directory have the lucene IDs embedded in the binaries? The files look good to me and are partly readable even if in binary. I am wondering if I could just set up a new solr instance and move these index files there and hope to use them (or most of them) as is without shards? If so, I will just set up a separate sharded index for the documents indexed henceforth, but won't bother splitting the huge existing index.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Don't even try to do that. First of all, you have to have a reliable way to
index the same docs to the same shards. The docs are all mixed up
in the segment files and would lead to chaos. Solr/Lucene report
the same doc multiple times if it's indifferent shards, so if you
ever updated a document, you wouldn't know what shard to
send it to.

Second, the segments are all parts of a single index, and Solr expects
(well, actually Lucene) expects them to be consistent. Putting some on
one shard and some on another would probably not allow Solr to start
(but I confess I've never tried that).

So I really wouldn't even try to go there.....

Best
Erick

On Wed, Jun 20, 2012 at 12:35 PM, avenka <[hidden email]> wrote:

> Thanks. Do you know if the tons of index files with names like '_zxt.tis' in
> the index/data/ directory have the lucene IDs embedded in the binaries? The
> files look good to me and are partly readable even if in binary. I am
> wondering if I could just set up a new solr instance and move these index
> files there and hope to use them (or most of them) as is without shards? If
> so, I will just set up a separate sharded index for the documents indexed
> henceforth, but won't bother splitting the huge existing index.
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3990560.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
Erick, thanks for the advice, but let me make sure you haven't misunderstood what I was asking.

I am not trying to split the huge existing index in install1 into shards. I am also not trying to make the huge install1 index as one shard of a sharded solr setup. I plan to use a sharded setup only for future docs.

I do want to avoid trying to re-index the docs in install1 and think of them as a slow "tape archive" index server if I ever need to go and query the past documents. So I was wondering if I could somehow use the existing segment files to run an isolated (unsharded) solr server that lets me query roughly the first 2B docs before the wraparound problem happened. If the "negative" internal doc IDs have pervasively corrupted the segment files, this would not be possible, but I am not able to imagine an underlying lucene design that would cause such a problem. Is my only option to re-index the past 2B docs if I want to be able to query them at this point or is there any way to use the existing segment files?
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Ah, OK, I misunderstood. OK, here's a couple of off-the-top-of-my-head
ideas.

make a backup of your index before anything else <G>...

Split up your current index into two parts by segments. That is, copy the whole
directory to another place, and remove some of the segments from each. I.e.
when you're done, you'll still have all the segments you used to have,
but some of
them will be in one directory and some in another. Of course all of
the segments files
with a common prefix should be in place (e.g. all the _0.* files in
the same dir, not
split between the two dirs).

Now run CheckIndex on them. That'll take a long time, but it _should_
spoof Solr/Lucene
into thinking that there are two complete indexes out there. Now your
idea of having an archival
search should work, but with two places to look, not one. NOTE:
Whether this plays nice with
the over 2B docs or deleted documents I can't guarantee.... I believe
that the deleted docs
are per-segment, if so this should be fine. This won't work if you've recently
optimized..... when you're done you should have two cores out there
(hmmm, these could
also be treated as shards?) that you point your solr at.

You might want to optimize in this case when you're done.
I suspect you could, with a magnetized needle and a steady hand, edit
some of the auxiliary files (segments*) but
I would feel more secure letting CheckIndex to the heavy lifting.

Here's another possibility
> Try a delete-by-query from a bit before the date you think things went over 2B to now (really hope you have a date!)
    > perhaps you can walk the underlying index in Lucene somehow and
make this work if you don't have a date. Since the
           underlying Lucene IDs are segment_base +
local_segment_count this should be safely under 2B
           but I'm reaching here into areas I don't know much about.
> optimize (and wait. probably a really long time).
> re-index everything after the date (or whatever) you used above into a new shard
> now treat the big index just as you were talking about.

Please understand that the over 2B docs might cause some grief here,
but since the underlying index
is segment based (i.e. the internal Lucene doc IDs are a base+offset
for each segment), this has a
decent chance of working (but anyone who really understands, please
chime in. I'm reaching).

Oh, and if it works, please let us know...

Best
Erick

On Wed, Jun 20, 2012 at 6:37 PM, avenka <[hidden email]> wrote:

> Erick, thanks for the advice, but let me make sure you haven't misunderstood
> what I was asking.
>
> I am not trying to split the huge existing index in install1 into shards. I
> am also not trying to make the huge install1 index as one shard of a sharded
> solr setup. I plan to use a sharded setup only for future docs.
>
> I do want to avoid trying to re-index the docs in install1 and think of them
> as a slow "tape archive" index server if I ever need to go and query the
> past documents. So I was wondering if I could somehow use the existing
> segment files to run an isolated (unsharded) solr server that lets me query
> roughly the first 2B docs before the wraparound problem happened. If the
> "negative" internal doc IDs have pervasively corrupted the segment files,
> this would not be possible, but I am not able to imagine an underlying
> lucene design that would cause such a problem. Is my only option to re-index
> the past 2B docs if I want to be able to query them at this point or is
> there any way to use the existing segment files?
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3990615.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
Erick, much thanks for detailing these options. I am currently trying the second one as that seems a little easier and quicker to me.

I successfully deleted documents with IDs after the problem time that I do know to an accuracy of a couple hours. Now, the stats are:
  numDocs : 2132454075
  maxDoc : -2130733352
The former is nicely below 2^31. But I can't seem to get the latter to "decrease" and become positive by deleting further.

Should I just run an optimize at this point? I have never manually run an optimize and plan to just hit
  http://<machine_name>/solr/update?optimize=true
Can you confirm this?
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Right, if you optimize, at the end maxDocs should == numDocs.
Usually the document reclamation stuff is done when segments
merge, but that won't happen in this case since this index is
becoming static, so a manual optimize is probably indicated.

Something like this should also work, either way:
http://localhost:8983/solr/update?stream.body=<optimize/>

But be prepared to wait for a very long time.....

I'd copy it somewhere else first just for safety's sake....

Best
Erick

On Thu, Jun 21, 2012 at 12:52 PM, avenka <[hidden email]> wrote:

> Erick, much thanks for detailing these options. I am currently trying the
> second one as that seems a little easier and quicker to me.
>
> I successfully deleted documents with IDs after the problem time that I do
> know to an accuracy of a couple hours. Now, the stats are:
>  numDocs : 2132454075
>  maxDoc : -2130733352
> The former is nicely below 2^31. But I can't seem to get the latter to
> "decrease" and become positive by deleting further.
>
> Should I just run an optimize at this point? I have never manually run an
> optimize and plan to just hit
>  http://<machine_name>/solr/update?optimize=true
> Can you confirm this?
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3990798.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

avenka
So, I tried 'optimize', but it failed because of lack of space on the first machine. I then moved the whole thing to a different machine where the index was pretty much the only thing and was using about 37% of disk, but it still failed because of a "No space left on device" IOException. Also, the size of the index has since doubled to roughly 74% of the disk on this second machine now and the number of files has increased from 3289 to 3329. Actually even the 3289 files on the first machine were after I tried optimize on it once, so the "original" size must have been even smaller.

I don't think I can afford any more space and am close to giving up and reclaiming space on the two machines. A couple more questions before that:

1) I am tempted to try editing binary--the "magnetic needle" option. Could you elaborate on this? Would there be a way to go back to an index that is the original size from its super-sized current form(s)?

2) Will CheckIndex also need more than twice the space? Would there be a way to bring down the size to the original size without running 'optimize' if I try that route? Also how exactly do I run CheckIndex, e.g., the exact URL I need to hit?
Reply | Threaded
Open this post in threaded view
|

Re: solr java.lang.NullPointerException on select queries

Erick Erickson
Well, you'd have to understand the whole way the index structure is laid
out to do binary editing, and I don't know it well enough to even offer
a rough idea. There are detailed docs hanging around _somewhere_ that
will give you the formats, or you could go at the code. But that's probably
pretty hairy.

I'm surprised you're getting that many files, something sounds really screwy.
I'm not at all surprised that the index tries to double _temporarily_
when optimizing,
that's expected behavior. But it should go back down after the
optimization/forcemerge
is complete. Problem is that now you've got some garbage left around
that prevents your
original index from being copied while optimizing, and getting rid of
the unnecessary files
seems fraught.

I'm also surprised that you wound up with a space problem when you
tried to optimize
unless you have compound file format turned on, which I doubt. Could
you wipe the index
from that disk and re-copy the original and try again?

So I'm going to chicken out and leave it to my betters to offer any
advice, I don't play
in the lower-level file structures.

Sorry I can't be more help
Erick

On Tue, Jun 26, 2012 at 11:46 AM, avenka <[hidden email]> wrote:

> So, I tried 'optimize', but it failed because of lack of space on the first
> machine. I then moved the whole thing to a different machine where the index
> was pretty much the only thing and was using about 37% of disk, but it still
> failed because of a "No space left on device" IOException. Also, the size of
> the index has since doubled to roughly 74% of the disk on this second
> machine now and the number of files has increased from 3289 to 3329.
> Actually even the 3289 files on the first machine were after I tried
> optimize on it once, so the "original" size must have been even smaller.
>
> I don't think I can afford any more space and am close to giving up and
> reclaiming space on the two machines. A couple more questions before that:
>
> 1) I am tempted to try editing binary--the "magnetic needle" option. Could
> you elaborate on this? Would there be a way to go back to an index that is
> the original size from its super-sized current form(s)?
>
> 2) Will CheckIndex also need more than twice the space? Would there be a way
> to bring down the size to the original size without running 'optimize' if I
> try that route? Also how exactly do I run CheckIndex, e.g., the exact URL I
> need to hit?
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3991400.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

RE: solr java.lang.NullPointerException on select queries

Bryan Loofbourrow
In reply to this post by avenka
Regarding the large number of files, even after optimize, we found that
when rebuilding a large, experimental 1.7TB index on Solr 3.5, instead of
Solr 1.4.1, there were a ton of index files, thousands, in 3.5, when there
used to be just 10 (or 11?) segments worth (as expected with mergeFactor
set to 10) in 1.4.1.

The apparent cause was a Solr switch to use TieredMergePolicy by default
somewhere in the 3.x version series. TieredMergePolicy has a default
segment size limit of 5GB, so if your index goes over 50GB, a mergeFactor
of 10 effectively gets ignored. We remedied this by explicitly configuring
TieredMergePolicy's segment size (and some other things that may or may
not be making a difference) in solrconfig.xml:

    <mainIndex>
       ....
      <mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
        <int name="maxMergeAtOnce">10</int>
        <int name="segmentsPerTier">10</int>
        <double name="maxMergedSegmentMB">30000</double>
      </mergePolicy>
    </mainIndex>

-- Bryan

> -----Original Message-----
> From: avenka [mailto:[hidden email]]
> Sent: Tuesday, June 26, 2012 8:46 AM
> To: [hidden email]
> Subject: Re: solr java.lang.NullPointerException on select queries
>
> So, I tried 'optimize', but it failed because of lack of space on the
> first
> machine. I then moved the whole thing to a different machine where the
> index
> was pretty much the only thing and was using about 37% of disk, but it
> still
> failed because of a "No space left on device" IOException. Also, the
size
> of
> the index has since doubled to roughly 74% of the disk on this second
> machine now and the number of files has increased from 3289 to 3329.
> Actually even the 3289 files on the first machine were after I tried
> optimize on it once, so the "original" size must have been even smaller.
>
> I don't think I can afford any more space and am close to giving up and
> reclaiming space on the two machines. A couple more questions before
that:
>
> 1) I am tempted to try editing binary--the "magnetic needle" option.
Could
> you elaborate on this? Would there be a way to go back to an index that
is
> the original size from its super-sized current form(s)?
>
> 2) Will CheckIndex also need more than twice the space? Would there be a
> way
> to bring down the size to the original size without running 'optimize'
if
> I
> try that route? Also how exactly do I run CheckIndex, e.g., the exact
URL
> I
> need to hit?
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-
> java-lang-NullPointerException-on-select-queries-tp3989974p3991400.html
> Sent from the Solr - User mailing list archive at Nabble.com.