Real time get - URL size limitation

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Real time get - URL size limitation

Zarski, Jacek
Let's say there is a collection with a schema that specifies 5000 elements.  I would like to make a "real time get" call to retrieve a large subset of those elements. Currently I am limited by the size restriction of the HTTP GET method if I want to make that request be "real time" since I would have to specify each field in the "fl" parameter. Is there a way to circumvent the limitation of url size? Maybe a POST call with a body or something of the sort?
Reply | Threaded
Open this post in threaded view
|

Re: Real time get - URL size limitation

Alexandre Rafalovitch
Two quick thoughts without computer access:
1) have you tried post? Usually they do work for all calls.
2) if the list does not change often, you can add it to the request handler
definition. Or even as a separate paramset to pass by reference. Either way
you would not need to have it in URL every time.

Regards,
   Alex

On Thu, Apr 11, 2019, 3:58 PM Zarski, Jacek, <[hidden email]>
wrote:

> Let's say there is a collection with a schema that specifies 5000
> elements.  I would like to make a "real time get" call to retrieve a large
> subset of those elements. Currently I am limited by the size restriction of
> the HTTP GET method if I want to make that request be "real time" since I
> would have to specify each field in the "fl" parameter. Is there a way to
> circumvent the limitation of url size? Maybe a POST call with a body or
> something of the sort?
>
Reply | Threaded
Open this post in threaded view
|

RE: Real time get - URL size limitation

Zarski, Jacek
1) I have not tried post as we are using SolrJ version 6.4 - There is no option to change the method of the request as far as I know from looking at the documentation.

2) The list changes very often but I will explore this option further to see if there is some subset of elements that is always requested.

-----Original Message-----
From: Alexandre Rafalovitch <[hidden email]>
Sent: Thursday, April 11, 2019 4:07 PM
To: solr-user <[hidden email]>
Subject: Re: Real time get - URL size limitation

CAUTION: This email originated from outside of D&B. Please do not click links or open attachments unless you recognize the sender and know the content is safe.


Two quick thoughts without computer access:
1) have you tried post? Usually they do work for all calls.
2) if the list does not change often, you can add it to the request handler definition. Or even as a separate paramset to pass by reference. Either way you would not need to have it in URL every time.

Regards,
   Alex

On Thu, Apr 11, 2019, 3:58 PM Zarski, Jacek, <[hidden email]>
wrote:

> Let's say there is a collection with a schema that specifies 5000
> elements.  I would like to make a "real time get" call to retrieve a
> large subset of those elements. Currently I am limited by the size
> restriction of the HTTP GET method if I want to make that request be
> "real time" since I would have to specify each field in the "fl"
> parameter. Is there a way to circumvent the limitation of url size?
> Maybe a POST call with a body or something of the sort?
>
Reply | Threaded
Open this post in threaded view
|

Re: Real time get - URL size limitation

Vincenzo D'Amore
Hi ZarskiJ

> 1) I have not tried post as we are using SolrJ version 6.4 - There is no option to change the method of the request as far as I know from looking at the documentation.

https://lucene.apache.org/solr/6_4_2/solr-solrj/org/apache/solr/client/solrj/SolrClient.html#query-org.apache.solr.common.params.SolrParams-org.apache.solr.client.solrj.SolrRequest.METHOD-

Have a look at this. Specifically the SolrRequest.METHOD

>
> 2) The list changes very often but I will explore this option further to see if there is some subset of elements that is always requested.
>
> -----Original Message-----
> From: Alexandre Rafalovitch <[hidden email]>
> Sent: Thursday, April 11, 2019 4:07 PM
> To: solr-user <[hidden email]>
> Subject: Re: Real time get - URL size limitation
>
> CAUTION: This email originated from outside of D&B. Please do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> Two quick thoughts without computer access:
> 1) have you tried post? Usually they do work for all calls.
> 2) if the list does not change often, you can add it to the request handler definition. Or even as a separate paramset to pass by reference. Either way you would not need to have it in URL every time.
>
> Regards,
>   Alex
>
> On Thu, Apr 11, 2019, 3:58 PM Zarski, Jacek, <[hidden email]>
> wrote:
>
>> Let's say there is a collection with a schema that specifies 5000
>> elements.  I would like to make a "real time get" call to retrieve a
>> large subset of those elements. Currently I am limited by the size
>> restriction of the HTTP GET method if I want to make that request be
>> "real time" since I would have to specify each field in the "fl"
>> parameter. Is there a way to circumvent the limitation of url size?
>> Maybe a POST call with a body or something of the sort?
>>