Which fieldType to use for JSON Array in Solr 6.5.0?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Which fieldType to use for JSON Array in Solr 6.5.0?

Abhijit Pawar-3
Hello Guys,

I am trying to index a JSON array in one of my collections in mongoDB in
Solr 6.5.0 however it is not getting indexed.

I am using a DataImportHandler for this.

*Here's how the data looks in mongoDB:*
{
 "idStr" : "5ca38e407b154dac08913a96",
    "sampleAttr" : "sampleAttrVal",
*    "additionalInfo" : [ *
*        {*
*            "name" : "Manufacturer",*
*            "value" : "Videocon"*
*        }*
*    ]*
}

*data-source-config.xml:*
<dataConfig>
<dataSource name="mongod" type="JdbcDataSource"
driver="com.mongodb.jdbc.MongoDriver"
url="mongodb://<<IPAddress>>:<<Port>>/<<DBCollection>>"/>
<document name="products">
<entity name="products"
dataSource="mongod"
transformer="ProdsCatsFieldTransformer,TemplateTransformer"
onError="continue"
pk="uuid"
query="SELECT idStr,sampleAttr,additionalInfo.name,additionalInfo.value
FROM products"
deltaImportQuery="SELECT
idStr,sampleAttr,additionalInfo.name,additionalInfo.value FROM products
WHERE orgidStr = '${dataimporter.request.orgid}' AND idStr =
'${dataimporter.delta.idStr}'"
deltaQuery="SELECT idStr FROM products WHERE orgidStr =
'${dataimporter.request.orgid}' AND updatedAt &gt;
'${dataimporter.last_index_time}'"
>
<field column="idStr" name="prod_idStr"/>
<field column="sampleAttr" name="sampleAttr" />
*<field column="additionalInfo.name" name="additionalInfo.name" />*
* <field column="additionalInfo.value" name="additionalInfo.value" />*
</entity>
</document>
</dataConfig>

*managed-schema.xml:*
<field name="prod_idStr" type="string" indexed="true" stored="true"
required="false"/>
 <field name="sampleAttr" type="string" indexed="true" stored="true" />
 <field name="additionalInfo.name" type="strings" indexed="true"
stored="true" />
 <field name="additionalInfo.value" type="strings" indexed="true"
stored="true" />

What fieldType should I use for JSON Array.?I tried above - "strings"
however it doesn't seem to work.
Can someone help me on this?
Appreciate your response.Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Which fieldType to use for JSON Array in Solr 6.5.0?

Shawn Heisey-2
On 4/9/2019 2:04 PM, Abhijit Pawar wrote:

> Hello Guys,
>
> I am trying to index a JSON array in one of my collections in mongoDB in
> Solr 6.5.0 however it is not getting indexed.
>
> I am using a DataImportHandler for this.
>
> *Here's how the data looks in mongoDB:*
> {
>   "idStr" : "5ca38e407b154dac08913a96",
>      "sampleAttr" : "sampleAttrVal",
> *    "additionalInfo" : [ *
> *        {*
> *            "name" : "Manufacturer",*
> *            "value" : "Videocon"*
> *        }*
> *    ]*
> }

That is not a structure that Solr knows how to handle.  Essentially what
you have there is one document nested inside another.  Each of Solr's
documents has a completely flat structure -- there is no possibility of
a hierarchy within a single document.

Solr does have support for parent/child documents, but it wouldn't be
indexed like that.  I know almost nothing about how the parent/child
document support works.  You would have to get help from someone else or
consult the documentation.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Which fieldType to use for JSON Array in Solr 6.5.0?

David Hastings
Exactly, Solr is a search index, not a data store.  you need to flatten
your relationships.  Right tool for the job etc.

On Tue, Apr 9, 2019 at 4:28 PM Shawn Heisey <[hidden email]> wrote:

> On 4/9/2019 2:04 PM, Abhijit Pawar wrote:
> > Hello Guys,
> >
> > I am trying to index a JSON array in one of my collections in mongoDB in
> > Solr 6.5.0 however it is not getting indexed.
> >
> > I am using a DataImportHandler for this.
> >
> > *Here's how the data looks in mongoDB:*
> > {
> >   "idStr" : "5ca38e407b154dac08913a96",
> >      "sampleAttr" : "sampleAttrVal",
> > *    "additionalInfo" : [ *
> > *        {*
> > *            "name" : "Manufacturer",*
> > *            "value" : "Videocon"*
> > *        }*
> > *    ]*
> > }
>
> That is not a structure that Solr knows how to handle.  Essentially what
> you have there is one document nested inside another.  Each of Solr's
> documents has a completely flat structure -- there is no possibility of
> a hierarchy within a single document.
>
> Solr does have support for parent/child documents, but it wouldn't be
> indexed like that.  I know almost nothing about how the parent/child
> document support works.  You would have to get help from someone else or
> consult the documentation.
>
> Thanks,
> Shawn
>
Reply | Threaded
Open this post in threaded view
|

Re: Which fieldType to use for JSON Array in Solr 6.5.0?

Abhijit Pawar-3
In reply to this post by Shawn Heisey-2
Can parent-child relationship be used in this scenario?
Anyone?

I see it needs an update handler:
https://lucene.apache.org/solr/guide/6_6/transforming-and-indexing-custom-json.html#transforming-and-indexing-custom-json


curl 'http://localhost:8983/solr/my_collection/update/json/docs?split=/|/orgs'\
    -H 'Content-type:application/json' -d '{  "name": "Joe Smith",
"phone": 876876687,  "orgs": [    {      "name": "Microsoft",
"city": "Seattle",      "zip": 98052    },    {      "name": "Apple",
    "city": "Cupertino",      "zip": 95014    }  ]}'


On Tue, Apr 9, 2019 at 3:28 PM Shawn Heisey <[hidden email]> wrote:

> On 4/9/2019 2:04 PM, Abhijit Pawar wrote:
> > Hello Guys,
> >
> > I am trying to index a JSON array in one of my collections in mongoDB in
> > Solr 6.5.0 however it is not getting indexed.
> >
> > I am using a DataImportHandler for this.
> >
> > *Here's how the data looks in mongoDB:*
> > {
> >   "idStr" : "5ca38e407b154dac08913a96",
> >      "sampleAttr" : "sampleAttrVal",
> > *    "additionalInfo" : [ *
> > *        {*
> > *            "name" : "Manufacturer",*
> > *            "value" : "Videocon"*
> > *        }*
> > *    ]*
> > }
>
> That is not a structure that Solr knows how to handle.  Essentially what
> you have there is one document nested inside another.  Each of Solr's
> documents has a completely flat structure -- there is no possibility of
> a hierarchy within a single document.
>
> Solr does have support for parent/child documents, but it wouldn't be
> indexed like that.  I know almost nothing about how the parent/child
> document support works.  You would have to get help from someone else or
> consult the documentation.
>
> Thanks,
> Shawn
>