Can I add new field values to a existing lucene index ?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Can I add new field values to a existing lucene index ?

Anupam Bhattacharya
Does Lucene API allows to add new field values to a existing doc.

For example,
Initially

  Doc 1    FieldName Value  id 1  schoolname xyz  zipcode pqr


After update can I keep the existing field values and add 2 more fields.

  Doc 1    FieldName Value  id 1  schoolname xyz  zipcode pqr  *telephone* *
112233*  *grade* *10*
Reply | Threaded
Open this post in threaded view
|

Re: Can I add new field values to a existing lucene index ?

Michael McCandless-2
Alas, no, not yet.  This is an oft-requested feature, but challenging to build.

That said, there is a possible start towards making something possible in 4.0:

  https://issues.apache.org/jira/browse/LUCENE-3837

Mike McCandless

http://blog.mikemccandless.com

On Wed, Mar 28, 2012 at 8:16 AM, Anupam Bhattacharya
<[hidden email]> wrote:

> Does Lucene API allows to add new field values to a existing doc.
>
> For example,
> Initially
>
>  Doc 1    FieldName Value  id 1  schoolname xyz  zipcode pqr
>
>
> After update can I keep the existing field values and add 2 more fields.
>
>  Doc 1    FieldName Value  id 1  schoolname xyz  zipcode pqr  *telephone* *
> 112233*  *grade* *10*

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Can I add new field values to a existing lucene index ?

Tim Eck
Excuse my ignorance of lucene internals, but is the problem any easier if
the requirement is just to allow the addition/removal of stored only fields
(as opposed to indexed)?

I suspect this wasn't intent of the original question, but a document can
certainly be deleted and re-added to the index with a new set of fields.

-----Original Message-----
From: Michael McCandless [mailto:[hidden email]]
Sent: Wednesday, March 28, 2012 5:31 AM
To: [hidden email]
Subject: Re: Can I add new field values to a existing lucene index ?

Alas, no, not yet.  This is an oft-requested feature, but challenging to
build.

That said, there is a possible start towards making something possible in
4.0:

  https://issues.apache.org/jira/browse/LUCENE-3837

Mike McCandless

http://blog.mikemccandless.com

On Wed, Mar 28, 2012 at 8:16 AM, Anupam Bhattacharya
<[hidden email]> wrote:

> Does Lucene API allows to add new field values to a existing doc.
>
> For example,
> Initially
>
>  Doc 1    FieldName Value  id 1  schoolname xyz  zipcode pqr
>
>
> After update can I keep the existing field values and add 2 more fields.
>
>  Doc 1    FieldName Value  id 1  schoolname xyz  zipcode pqr  *telephone*
*
> 112233*  *grade* *10*

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can I add new field values to a existing lucene index ?

Michael McCandless-2
On Wed, Mar 28, 2012 at 2:30 PM, Tim Eck <[hidden email]> wrote:
> Excuse my ignorance of lucene internals, but is the problem any easier if
> the requirement is just to allow the addition/removal of stored only fields
> (as opposed to indexed)?

It would substantially simplify the problem... but even this
simplification is quite a bit of work.

Lucene's segments are write-once, so when some number of documents in
a given segment are updated, we'd need to write new files against that
segment (we call this a "stacked segment"), sort of like a patch over
the original segment.

But Lucene's stored fields writer/reader impls are set up to handle
this sort of "patching", so some work must be done to correlate docIDs
in the patched segment against docIDs in the base segment...

> I suspect this wasn't intent of the original question, but a document can
> certainly be deleted and re-added to the index with a new set of fields.

Ahh, right, I forgot to mention that: fully removing the original
document, and then fully adding the new document (with all original
fields, plus the new fields) is always possible... just not "ideal" if
your update is tiny and your original fields are huge.

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]