Conditional copyField

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Conditional copyField

David Stuart
Hi,
I am pushing data to solr from two different sources nutch and a cms.  
I have a data clash in that in nutch a copyField is required to push  
the url field to the id field as it is used as  the primary lookup in  
the nutch solr intergration update. The other cms also uses the url  
field but also populates the id field with a different value. Now I  
can't really change either source definition so is there a way in  
solrconfig or schema to check if id is empty and only copy if true or  
is there a better way via the updateprocessor?

Thanks for your help in advance
Regards

David
Reply | Threaded
Open this post in threaded view
|

Re: Conditional copyField

iorixxx
> Hi,
> I am pushing data to solr from two different sources nutch
> and a cms. I have a data clash in that in nutch a copyField
> is required to push the url field to the id field as it is
> used asĀ  the primary lookup in the nutch solr
> intergration update. The other cms also uses the url field
> but also populates the id field with a different value. Now
> I can't really change either source definition so is there a
> way in solrconfig or schema to check if id is empty and only
> copy if true or is there a better way via the
> updateprocessor?

copyField declaration has three attributes: source, dest and maxChars.
Therefore it can be concluded that there is no way to do it in schema.xml

Luckily, Wiki [1] has a quick example that implements a conditional copyField.

[1] http://wiki.apache.org/solr/UpdateRequestProcessor



Reply | Threaded
Open this post in threaded view
|

Re: Conditional copyField

Grant Ingersoll-2
Nice find, Amhet,  I'd love to see this formalized in the Solr schema  
syntax, as it is something I've often wanted to.  Max Chars is OK,  
too, but would like to see max tokens as well.

On Oct 12, 2009, at 6:31 PM, AHMET ARSLAN wrote:

>> Hi,
>> I am pushing data to solr from two different sources nutch
>> and a cms. I have a data clash in that in nutch a copyField
>> is required to push the url field to the id field as it is
>> used as  the primary lookup in the nutch solr
>> intergration update. The other cms also uses the url field
>> but also populates the id field with a different value. Now
>> I can't really change either source definition so is there a
>> way in solrconfig or schema to check if id is empty and only
>> copy if true or is there a better way via the
>> updateprocessor?
>
> copyField declaration has three attributes: source, dest and maxChars.
> Therefore it can be concluded that there is no way to do it in  
> schema.xml
>
> Luckily, Wiki [1] has a quick example that implements a conditional  
> copyField.
>
> [1] http://wiki.apache.org/solr/UpdateRequestProcessor
>
>
>

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search