Pagination with streaming expressions

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Pagination with streaming expressions

Pratik Patel
Hello Everyone,

Is there a way to paginate the results of Streaming Expression?

Let's say I have a simple gatherNodes function which has count operation at
the end of it. I can sort by the count fine but now I would like to be able
to select specific sub set of result based on pagination parameters. Is
there any way to do that?

Thanks!
Pratik
Reply | Threaded
Open this post in threaded view
|

Re: Pagination with streaming expressions

Erick Erickson
This sounds like an XY problem. You’re asking now to paginate, but not explaining the problem you want to solve with paginating.

I don’t immediately see what purpose paginating serves here. What significance does a page have to do with the gatherNodes? How use would the _user_ have with these results? Especially for two unrelated queries. IOW if for query1 you count something for page 13, and for query2 you also count something for page 13 what information is the user getting in those two cases? Especially if the total result set for query1 is 1,000 docs but for query2 is 10,000,000 does?

But in general no, streaming is orthogonal to most use-cases for pagination and isn’t really supported except if you read through the returns and throw away the first N pages, probably pretty inefficient.

Erick

> On May 1, 2019, at 1:28 PM, Pratik Patel <[hidden email]> wrote:
>
> Hello Everyone,
>
> Is there a way to paginate the results of Streaming Expression?
>
> Let's say I have a simple gatherNodes function which has count operation at
> the end of it. I can sort by the count fine but now I would like to be able
> to select specific sub set of result based on pagination parameters. Is
> there any way to do that?
>
> Thanks!
> Pratik

Reply | Threaded
Open this post in threaded view
|

Re: Pagination with streaming expressions

Joel Bernstein
There is an open ticket which deals with this:

https://issues.apache.org/jira/browse/SOLR-12209

I've been very focused though on anything that enhances the Solr Math
Expressions or has been needed for the Fusion SQL engine, which is what I
work on at LucidWorks. SOLR-12209 doesn't fall into that category.
Eventually though I will clear that ticket if someone else doesn't resolve
it first.




Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, May 1, 2019 at 7:56 PM Erick Erickson <[hidden email]>
wrote:

> This sounds like an XY problem. You’re asking now to paginate, but not
> explaining the problem you want to solve with paginating.
>
> I don’t immediately see what purpose paginating serves here. What
> significance does a page have to do with the gatherNodes? How use would the
> _user_ have with these results? Especially for two unrelated queries. IOW
> if for query1 you count something for page 13, and for query2 you also
> count something for page 13 what information is the user getting in those
> two cases? Especially if the total result set for query1 is 1,000 docs but
> for query2 is 10,000,000 does?
>
> But in general no, streaming is orthogonal to most use-cases for
> pagination and isn’t really supported except if you read through the
> returns and throw away the first N pages, probably pretty inefficient.
>
> Erick
>
> > On May 1, 2019, at 1:28 PM, Pratik Patel <[hidden email]> wrote:
> >
> > Hello Everyone,
> >
> > Is there a way to paginate the results of Streaming Expression?
> >
> > Let's say I have a simple gatherNodes function which has count operation
> at
> > the end of it. I can sort by the count fine but now I would like to be
> able
> > to select specific sub set of result based on pagination parameters. Is
> > there any way to do that?
> >
> > Thanks!
> > Pratik
>
>