Lucene › Solr › Solr - User

Memory use during merges (OOM)

Classic

List

Threaded

15 messages Options

Tom Burton-West-2 Memory use during merges (OOM) – Hello all, Are there any general guidelines for determining the main factors in memory use during merges? We recently changed our indexing c...

Upayavira How long does it take to reach this OOM situation? Is it possible for you to try a merge with each setting in turn, and evaluate what impact the...

Michael McCandless-2 RAM usage for merging is tricky. First off, merging must hold open a SegmentReader for each segment being merged. However, it's not necessari...

Petersen, Robert RE: Memory use during merges (OOM) – Hello we occasionally bump into the OOM issue during merging after propagation too, and from the discussion below I guess we are doing thousands o...

Michael McCandless-2 Re: Memory use during merges (OOM) – It's not that it's "bad", it's just that Lucene must do extra work to check if these deletes are real or not, and that extra work requi...

Tom Burton-West-2 RE: Memory use during merges (OOM) – Thanks Mike, >>But, if you are doing deletions (or updateDocument, which is just a >>delete + add under-the-hood), then this will ...

Petersen, Robert RE: Memory use during merges (OOM) – Thanks Mike! When you say 'term index of the segment readers', are you referring to the term vectors? In our case our index of 8 million docs ...

Michael McCandless-2 Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 2:09 PM, Burton-West, Tom <tburtonw@...> wrote: > Thanks Mike, > >>>But, if you are doing deletion...

Michael McCandless-2 Re: Memory use during merges (OOM) – Actually terms index is something different. If you don't use CFS, go and look at the size of *.tii in your index directory -- those are the t...

Robert Muir Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 2:09 PM, Burton-West, Tom <tburtonw@...> wrote: > > I always get confused about the two different divisors a...

Tom Burton-West-2 RE: Memory use during merges (OOM) – >>Your setting isn't being applied to the reader IW uses during >>merging... its only for readers Solr opens from directories >&g...

Michael McCandless-2 Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 4:03 PM, Burton-West, Tom <tburtonw@...> wrote: >>>Your setting isn't being applied to the reader IW uses ...

Robert Muir Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 4:03 PM, Burton-West, Tom <tburtonw@...> wrote: >>>Your setting isn't being applied to the reader IW uses ...

Yonik Seeley-2-2 On Thu, Dec 16, 2010 at 5:51 AM, Michael McCandless <lucene@...> wrote: > If you are doing false deletions (calling .updateDocument whe...

Tom Burton-West-2 RE: Memory use during merges (OOM) – Thanks Robert, We will try the termsIndexInterval as a workaround. I have also opened a JIRA issue: https://issues.apache.org/jira/browse/SO...