[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890905#action_12890905 ]

Jason Rutherglen commented on LUCENE-2324:
------------------------------------------

{quote}Implement logic to discard deletes from the deletes
buffer{quote}

Michael, where in the code is this supposed to occur?

{quote}Implement flush-by-ram logic{quote}

I'll make a go of this.

{quote}Maybe change delete logic: currently deletes are applied
when a segment is flushed. Maybe we can keep it this way in the
realtime-branch though, because that's most likely what we want
to do once the RAM buffer is searchable and deletes are cheaper
as they can then be done in-memory before flush{quote}

I think we'll keep things this way for this issue (ie, per
thread document writers), however for LUCENE-2312 I think we'll
want to implement foreground deletes (eg, updating the deleted
docs sequences int[]).

> Per thread DocumentsWriters that write their own private segments
> -----------------------------------------------------------------
>
>                 Key: LUCENE-2324
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2324
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: lucene-2324.patch, lucene-2324.patch, LUCENE-2324.patch
>
>
> See LUCENE-2293 for motivation and more details.
> I'm copying here Mike's summary he posted on 2293:
> Change the approach for how we buffer in RAM to a more isolated
> approach, whereby IW has N fully independent RAM segments
> in-process and when a doc needs to be indexed it's added to one of
> them. Each segment would also write its own doc stores and
> "normal" segment merging (not the inefficient merge we now do on
> flush) would merge them. This should be a good simplification in
> the chain (eg maybe we can remove the *PerThread classes). The
> segments can flush independently, letting us make much better
> concurrent use of IO & CPU.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]