Replication and querying

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Replication and querying

Jhille
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Replication and querying

Jan Høydahl / Cominvent
Hi,

Index replication in Solr makes an exact copy of the original index.
Is it not possible to add the 6 extra fields to both instances?
An alternative to replication is to feed two independent Solr instances -> full control :)
Please elaborate on your specific use case if this is not useful answer to you.

--
Jan Høydahl  - search architect
Cominvent AS - www.cominvent.com

On 9. feb. 2010, at 13.21, Julian Hille wrote:

> Hi,
>
> id like to know if its possible to have a solr Server with a schema and lets say 10 fields indexed.
> I know want to replicate this whole index to another solr server which has a slightly different schema.
> There are additional 6 fields these fields change the sort order for a product which base is our solr database.
>
> Is this kind of replication possible?
>
> Is there another way to interact with data in solr? We'd like to calculate some fields when they will be added.
> I cant seem to find a good documentation about the possible calls in the query itself nor documentaion about queries/calculation  which should be done on update.
>
>
> so far,
> Julian Hille
>
>
> ---------------------------------------------------------------
> NetImpact KG
> Altonaer Straße 8
> 20357 Hamburg
>
> Tel: 040 / 6738363 2
> Mail: [hidden email]
>
> Geschäftsführer: Tarek Müller
>

Reply | Threaded
Open this post in threaded view
|

Re: Replication and querying

Jhille
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Replication and querying

Jan Høydahl / Cominvent
Hi again,

I would still keep all fields in the original schema of the global Solr, just for the sake of simplicity.

For custom sort order, you can look at ExternalFileField which is a text file that you can add to your local Solr index independently of the pre-built index. However, this only supports float and cannot be returned in result.  http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html

The Solr replication does a binary copy of the index - i.e. no  way to change docs in input.
But if you instead replicate source XML feed from master to slaves, you can hook into that stream to modify/add fields (see http://wiki.apache.org/solr/SolrConfigXml#UpdateRequestProcessorChain_section). But then you need to index locally of course. 1.5 mill docs isn't that much, so why not?

--
Jan Høydahl  - search architect
Cominvent AS - www.cominvent.com

On 10. feb. 2010, at 10.22, Julian Hille wrote:

> Hi,
>
> its would be possible to add that to the main solr but the problem is:
> Lets face it (example):
> We have kind of 1.5 million documents in the solr master. These Documents are books.
> These books have fields like title, ids, numbers and authors and more.
> This solr is global.
>
> Now: The slave solr is for a local library which has all these books, but want to sort in another way,
> and wants to add their own fields. For sorting and output (these fields doesnt need to be indexed or searched through).
>
> So we try to replicate the whole database but have a slightly differen schema.xml in the slaves.
>
>
> Secondly we need for another Project to know if its possible to change data "oninsert", "onupdate".
> So that the replicationed data gets edited before its really inserted. Is there some kind of hook?
> As an exmaple lets take the book example from top:
> On replication the slave gets a updated document set. But before updated on the the slaves db
> we like to add fields which come from another database or we like to replace strings in some fields and such things.
>
> Is that possible?
>
> Thanks for any answers.
>
>
>
> Am 09.02.2010 um 16:53 schrieb Jan Høydahl / Cominvent:
>
>> Hi,
>>
>> Index replication in Solr makes an exact copy of the original index.
>> Is it not possible to add the 6 extra fields to both instances?
>> An alternative to replication is to feed two independent Solr instances -> full control :)
>> Please elaborate on your specific use case if this is not useful answer to you.
>>
>> --
>> Jan Høydahl  - search architect
>> Cominvent AS - www.cominvent.com
>>
>> On 9. feb. 2010, at 13.21, Julian Hille wrote:
>>
>>> Hi,
>>>
>>> id like to know if its possible to have a solr Server with a schema and lets say 10 fields indexed.
>>> I know want to replicate this whole index to another solr server which has a slightly different schema.
>>> There are additional 6 fields these fields change the sort order for a product which base is our solr database.
>>>
>>> Is this kind of replication possible?
>>>
>>> Is there another way to interact with data in solr? We'd like to calculate some fields when they will be added.
>>> I cant seem to find a good documentation about the possible calls in the query itself nor documentaion about queries/calculation  which should be done on update.
>>>
>>>
>>> so far,
>>> Julian Hille
>>>
>>>
>>> ---------------------------------------------------------------
>>> NetImpact KG
>>> Altonaer Straße 8
>>> 20357 Hamburg
>>>
>>> Tel: 040 / 6738363 2
>>> Mail: [hidden email]
>>>
>>> Geschäftsführer: Tarek Müller
>>>
>
> Mit freundlichen Grüßen,
> Julian Hille
>
>
> ---------------------------------------------------------------
> NetImpact KG
> Altonaer Straße 8
> 20357 Hamburg
>
> Tel: 040 / 6738363 2
> Mail: [hidden email]
>
> Geschäftsführer: Tarek Müller
>