Quantcast

Memory use during merges (OOM)

15 messages Options
Memory use during merges (OOM) – Hello all, Are there any general guidelines for determining the main factors in memory use during merges? We recently changed our indexing c...
How long does it take to reach this OOM situation? Is it possible for you to try a merge with each setting in turn, and evaluate what impact the...
RAM usage for merging is tricky. First off, merging must hold open a SegmentReader for each segment being merged. However, it's not necessari...
RE: Memory use during merges (OOM) – Hello we occasionally bump into the OOM issue during merging after propagation too, and from the discussion below I guess we are doing thousands o...
Re: Memory use during merges (OOM) – It's not that it's "bad", it's just that Lucene must do extra work to check if these deletes are real or not, and that extra work requi...
RE: Memory use during merges (OOM) – Thanks Mike, >>But, if you are doing deletions (or updateDocument, which is just a >>delete + add under-the-hood), then this will ...
RE: Memory use during merges (OOM) – Thanks Mike! When you say 'term index of the segment readers', are you referring to the term vectors? In our case our index of 8 million docs ...
Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 2:09 PM, Burton-West, Tom <tburtonw@...> wrote: > Thanks Mike, > >>>But, if you are doing deletion...
Re: Memory use during merges (OOM) – Actually terms index is something different. If you don't use CFS, go and look at the size of *.tii in your index directory -- those are the t...
Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 2:09 PM, Burton-West, Tom <tburtonw@...> wrote: > > I always get confused about the two different divisors a...
RE: Memory use during merges (OOM) – >>Your setting isn't being applied to the reader IW uses during >>merging... its only for readers Solr opens from directories >&g...
Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 4:03 PM, Burton-West, Tom <tburtonw@...> wrote: >>>Your setting isn't being applied to the reader IW uses ...
Re: Memory use during merges (OOM) – On Thu, Dec 16, 2010 at 4:03 PM, Burton-West, Tom <tburtonw@...> wrote: >>>Your setting isn't being applied to the reader IW uses ...
On Thu, Dec 16, 2010 at 5:51 AM, Michael McCandless <lucene@...> wrote: > If you are doing false deletions (calling .updateDocument whe...
RE: Memory use during merges (OOM) – Thanks Robert, We will try the termsIndexInterval as a workaround. I have also opened a JIRA issue: https://issues.apache.org/jira/browse/SO...
Loading...