Customizing JSON response of a query

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Customizing JSON response of a query

Sarvothaman Madhavan
Hello all,
After a week of research I've come to the conclusion that there is no
mechanism within solr where I can create a nested json response like this:
https://pastebin.com/XavvUP94 . I am able to get something like this
https://pastebin.com/FeXRqG59.
1. Am I right in assuming that within solr this is not possbile?
2. Assuming it is, I imagine I would need to write custom response writer
in Java to customize the response. I am having a hard time locating the
right resource to get me started on writing this.

Any ideas?

Thanks,
Sarvo
Reply | Threaded
Open this post in threaded view
|

RE: Customizing JSON response of a query

Davis, Daniel (NIH/NLM) [C]
It should be possible with a custom response handler.

-----Original Message-----
From: Sarvothaman Madhavan [mailto:[hidden email]]
Sent: Wednesday, September 06, 2017 10:17 AM
To: [hidden email]
Subject: Customizing JSON response of a query

Hello all,
After a week of research I've come to the conclusion that there is no mechanism within solr where I can create a nested json response like this:
https://pastebin.com/XavvUP94 . I am able to get something like this https://pastebin.com/FeXRqG59.
1. Am I right in assuming that within solr this is not possbile?
2. Assuming it is, I imagine I would need to write custom response writer in Java to customize the response. I am having a hard time locating the right resource to get me started on writing this.

Any ideas?

Thanks,
Sarvo
Reply | Threaded
Open this post in threaded view
|

RE: Customizing JSON response of a query

Rick Leir-2
Sarvo,
What are you trying to achieve? Describe the use case.
Cheers -- Rick

On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" <[hidden email]> wrote:

>It should be possible with a custom response handler.
>
>-----Original Message-----
>From: Sarvothaman Madhavan [mailto:[hidden email]]
>Sent: Wednesday, September 06, 2017 10:17 AM
>To: [hidden email]
>Subject: Customizing JSON response of a query
>
>Hello all,
>After a week of research I've come to the conclusion that there is no
>mechanism within solr where I can create a nested json response like
>this:
>https://pastebin.com/XavvUP94 . I am able to get something like this
>https://pastebin.com/FeXRqG59.
>1. Am I right in assuming that within solr this is not possbile?
>2. Assuming it is, I imagine I would need to write custom response
>writer in Java to customize the response. I am having a hard time
>locating the right resource to get me started on writing this.
>
>Any ideas?
>
>Thanks,
>Sarvo

--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|

Re: Customizing JSON response of a query

Sarvothaman Madhavan
Rick,

My use case is this :

I have a set of documents each of which have "sub documents" associated
with it. I have this in the json format and I am able to load this into a
solr collection. When I search within this set of documents using solr, I
want the response in "grouped" json format

i.e

{

  "key": "value",

  "sub_doc": [

    {

      "inner_key": "inner_value"

    }

  ]

}



instead of solrs default flat json format:

i.e.

{

               "key":"value",

               "subdoc.inner_key"= ["inner_value"]

}



I think the "grouped" json format will be much more intuitive to my end
users who are going to use the search



P.S: Just to be clear I am not having any trouble querying children/parent
document since I have all of this stored using fully qualified names in
each document in the collection.





Regards,

Sarvo



On Wed, Sep 6, 2017 at 3:52 PM, Rick Leir <[hidden email]> wrote:

> Sarvo,
> What are you trying to achieve? Describe the use case.
> Cheers -- Rick
>
> On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" <
> [hidden email]> wrote:
> >It should be possible with a custom response handler.
> >
> >-----Original Message-----
> >From: Sarvothaman Madhavan [mailto:[hidden email]]
> >Sent: Wednesday, September 06, 2017 10:17 AM
> >To: [hidden email]
> >Subject: Customizing JSON response of a query
> >
> >Hello all,
> >After a week of research I've come to the conclusion that there is no
> >mechanism within solr where I can create a nested json response like
> >this:
> >https://pastebin.com/XavvUP94 . I am able to get something like this
> >https://pastebin.com/FeXRqG59.
> >1. Am I right in assuming that within solr this is not possbile?
> >2. Assuming it is, I imagine I would need to write custom response
> >writer in Java to customize the response. I am having a hard time
> >locating the right resource to get me started on writing this.
> >
> >Any ideas?
> >
> >Thanks,
> >Sarvo
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|

Re: Customizing JSON response of a query

Rick Leir-2
Sarvo,
I hope the users do not read JSON. I would have thought you'd have a web app in front of Solr and some Javascript in the browser. Either would be able to transform Solr's output into a display format. But I suspect there is more to the problem, and I do not understand it all.
Cheers -- Rick

On September 6, 2017 4:42:03 PM EDT, Sarvothaman Madhavan <[hidden email]> wrote:

>Rick,
>
>My use case is this :
>
>I have a set of documents each of which have "sub documents" associated
>with it. I have this in the json format and I am able to load this into
>a
>solr collection. When I search within this set of documents using solr,
>I
>want the response in "grouped" json format
>
>i.e
>
>{
>
>  "key": "value",
>
>  "sub_doc": [
>
>    {
>
>      "inner_key": "inner_value"
>
>    }
>
>  ]
>
>}
>
>
>
>instead of solrs default flat json format:
>
>i.e.
>
>{
>
>               "key":"value",
>
>               "subdoc.inner_key"= ["inner_value"]
>
>}
>
>
>
>I think the "grouped" json format will be much more intuitive to my end
>users who are going to use the search
>
>
>
>P.S: Just to be clear I am not having any trouble querying
>children/parent
>document since I have all of this stored using fully qualified names in
>each document in the collection.
>
>
>
>
>
>Regards,
>
>Sarvo
>
>
>
>On Wed, Sep 6, 2017 at 3:52 PM, Rick Leir <[hidden email]> wrote:
>
>> Sarvo,
>> What are you trying to achieve? Describe the use case.
>> Cheers -- Rick
>>
>> On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" <
>> [hidden email]> wrote:
>> >It should be possible with a custom response handler.
>> >
>> >-----Original Message-----
>> >From: Sarvothaman Madhavan [mailto:[hidden email]]
>> >Sent: Wednesday, September 06, 2017 10:17 AM
>> >To: [hidden email]
>> >Subject: Customizing JSON response of a query
>> >
>> >Hello all,
>> >After a week of research I've come to the conclusion that there is
>no
>> >mechanism within solr where I can create a nested json response like
>> >this:
>> >https://pastebin.com/XavvUP94 . I am able to get something like this
>> >https://pastebin.com/FeXRqG59.
>> >1. Am I right in assuming that within solr this is not possbile?
>> >2. Assuming it is, I imagine I would need to write custom response
>> >writer in Java to customize the response. I am having a hard time
>> >locating the right resource to get me started on writing this.
>> >
>> >Any ideas?
>> >
>> >Thanks,
>> >Sarvo
>>
>> --
>> Sorry for being brief. Alternate email is rickleir at yahoo dot com

--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|

RE: Customizing JSON response of a query

Davis, Daniel (NIH/NLM) [C]
Sarvo,

I agree with Rick.   It is better to put something in front of Solr (or any search engine), because the search engine sort of fits into a 3-tier hierarchy along with the database service:

Load Balancer/Httpd front-end ----> App -----> RDBMS

Becomes:

Load Balancer/Httpd front-end -----> App --+-> RDBMS
                                                                              |
                                                                              +-> Search Engine


This page, https://lucene.apache.org/solr/guide/6_6/response-writers.html, covers the response writers available out of the box.   There is a an XLSX Response Writer, and so the way to make your request into a feature would be to have some sort of JSON transformation response writer.  

As a temporary fix, you could probably use the VelocityResponseWriter to return JSON structured the way you want, but this is a bit of a hack.   Note here how I have avoided the issue of avoiding trailing commas, which any real implementation would need to deal with:

   [
     #foreach($doc in $response.results)
       {
            ...
            "subdoc": {
                 "innerkey": "$doc.subdoc.innerkey",
                 "multivalue": [
                     #foreach($val in $doc.subdoc.multivale)
                         "$val",                      
                     #end
                  ]
             }          
       },
     #end
   ]

If you must do this transformation on the Solr Server, a better approach would be to deploy a servlet listener that transformed the JSON response.  In the bad-old days, implementing a custom response writer would be the recommended way to go, but I think Solr has grown up to the point where it would be better to handle this transformation outside of Solr.

One point I want to make is that JSON is very malleable in an application, whether front-end or not.   Check out http://jmespath.org/ - it functions as a sort of XPath for JSON documents, even if it isn't supported by a standards organization like W3C.   In front-end, there are also very useful transformational packages such as https://lodash.com/.   I prefer jmespath, because I am often jumping from Java to Python to JavaScript.


-----Original Message-----
From: Rick Leir [mailto:[hidden email]]
Sent: Wednesday, September 06, 2017 8:21 PM
To: [hidden email]
Subject: Re: Customizing JSON response of a query

Sarvo,
I hope the users do not read JSON. I would have thought you'd have a web app in front of Solr and some Javascript in the browser. Either would be able to transform Solr's output into a display format. But I suspect there is more to the problem, and I do not understand it all.
Cheers -- Rick

On September 6, 2017 4:42:03 PM EDT, Sarvothaman Madhavan <[hidden email]> wrote:

>Rick,
>
>My use case is this :
>
>I have a set of documents each of which have "sub documents" associated
>with it. I have this in the json format and I am able to load this into
>a solr collection. When I search within this set of documents using
>solr, I want the response in "grouped" json format
>
>i.e
>
>{
>
>  "key": "value",
>
>  "sub_doc": [
>
>    {
>
>      "inner_key": "inner_value"
>
>    }
>
>  ]
>
>}
>
>
>
>instead of solrs default flat json format:
>
>i.e.
>
>{
>
>               "key":"value",
>
>               "subdoc.inner_key"= ["inner_value"]
>
>}
>
>
>
>I think the "grouped" json format will be much more intuitive to my end
>users who are going to use the search
>
>
>
>P.S: Just to be clear I am not having any trouble querying
>children/parent document since I have all of this stored using fully
>qualified names in each document in the collection.
>
>
>
>
>
>Regards,
>
>Sarvo
>
>
>
>On Wed, Sep 6, 2017 at 3:52 PM, Rick Leir <[hidden email]> wrote:
>
>> Sarvo,
>> What are you trying to achieve? Describe the use case.
>> Cheers -- Rick
>>
>> On September 6, 2017 12:36:08 PM EDT, "Davis, Daniel (NIH/NLM) [C]" <
>> [hidden email]> wrote:
>> >It should be possible with a custom response handler.
>> >
>> >-----Original Message-----
>> >From: Sarvothaman Madhavan [mailto:[hidden email]]
>> >Sent: Wednesday, September 06, 2017 10:17 AM
>> >To: [hidden email]
>> >Subject: Customizing JSON response of a query
>> >
>> >Hello all,
>> >After a week of research I've come to the conclusion that there is
>no
>> >mechanism within solr where I can create a nested json response like
>> >this:
>> >https://pastebin.com/XavvUP94 . I am able to get something like this
>> >https://pastebin.com/FeXRqG59.
>> >1. Am I right in assuming that within solr this is not possbile?
>> >2. Assuming it is, I imagine I would need to write custom response
>> >writer in Java to customize the response. I am having a hard time
>> >locating the right resource to get me started on writing this.
>> >
>> >Any ideas?
>> >
>> >Thanks,
>> >Sarvo
>>
>> --
>> Sorry for being brief. Alternate email is rickleir at yahoo dot com

--
Sorry for being brief. Alternate email is rickleir at yahoo dot com