[jira] [Commented] (LUCENE-4658) Per-segment tracking of external/side-car data

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4658) Per-segment tracking of external/side-car data

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604626#comment-13604626 ]

Michael McCandless commented on LUCENE-4658:
--------------------------------------------

bq. As I understand it, Lucene's faceting module uses a side-car index. If so, then if the feature proposed here is a good API then the faceting module will use it. No?

It does use a side-car (taxonomy) index, so that facet labels use global ords, which makes counting/NRT reopen fast.

But, that index is global, vs this patch which adds a per-segment side-car, so it wouldn't quite fit, until/unless we change taxonomy writer/reader to work per-segment.
               

> Per-segment tracking of external/side-car data
> ----------------------------------------------
>
>                 Key: LUCENE-4658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4658
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>         Attachments: LUCENE-4658.patch, LUCENE-4658.patch
>
>
> Spinoff from David's idea on LUCENE-4258
> (https://issues.apache.org/jira/browse/LUCENE-4258?focusedCommentId=13534352&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13534352 )
> I made a prototype patch that allows custom per-segment "side-car
> data".  It adds an abstract ExternalSegmentData class.  The idea is
> the app implements this, and IndexWriter will pass each Document
> through to it, and call on it to do flushing/merging.  I added a
> setter to IndexWriterConfig to enable it, but I think this would
> really belong in Codec ...
> I haven't tackled the read-side yet, though this is already usable
> without that (ie, the app can just open its own files, read them,
> etc.).
> The random test case passes.
> I think for example this might make it easier for Solr/ElasticSearch
> to implement things like ExternalFileField.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]