Streaming In Solr

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Streaming In Solr

Lucky Sharma
Hi I have some doubt regarding how the streaming expressions and
parallel SQL queries are evaluated in SOLR, I tried to dig deep in the
code but wasn't able to find much, A little help will be much
appreciated

--
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918
Reply | Threaded
Open this post in threaded view
|

RE: Streaming In Solr

Prakhar Nigam
HI Lucky Prakhar here

we have met at a training at Mahindra Comviva.  I have found this article it may be a little helpful

https://medium.com/@sarkaramrit2/getting-started-with-streaming-expressions-in-apache-solr-b49111a417e3





Regards,

Prakhar



From: Lucky Sharma<mailto:[hidden email]>
Sent: Wednesday, November 14, 2018 9:25 AM
To: [hidden email]<mailto:[hidden email]>
Subject: Streaming In Solr



Hi I have some doubt regarding how the streaming expressions and
parallel SQL queries are evaluated in SOLR, I tried to dig deep in the
code but wasn't able to find much, A little help will be much
appreciated

--
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918
This e-mail and all material transmitted with it are for the use of the intended recipient(s) ONLY and contains confidential information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken pursuant to the contents of the present e-mail is strictly prohibited and is unlawful. The recipient acknowledges that Comviva Technologies Limited or its management or directors, are unable to exercise control or ensure the integrity over /of the contents of the information contained in e-mail. Any views expressed herein are those of the individual sender only and no binding nature of the contents shall be implied or assumed unless the sender does so expressly with due authority of Comviva Technologies Limited. E-mail and any contents transmitted with it are prone to viruses and related defects despite all efforts to avoid such by Comviva Technologies Limited.
Reply | Threaded
Open this post in threaded view
|

Re: Streaming In Solr

Lucky Sharma
Hi Prakhar,
Thanks for the reply, But What I am actually curious to know how is it
implemented Internally?
On Wed, Nov 14, 2018 at 12:47 PM Prakhar Nigam
<[hidden email]> wrote:

>
> HI Lucky Prakhar here
>
> we have met at a training at Mahindra Comviva.  I have found this article it may be a little helpful
>
> https://medium.com/@sarkaramrit2/getting-started-with-streaming-expressions-in-apache-solr-b49111a417e3
>
>
>
>
>
> Regards,
>
> Prakhar
>
>
>
> From: Lucky Sharma<mailto:[hidden email]>
> Sent: Wednesday, November 14, 2018 9:25 AM
> To: [hidden email]<mailto:[hidden email]>
> Subject: Streaming In Solr
>
>
>
> Hi I have some doubt regarding how the streaming expressions and
> parallel SQL queries are evaluated in SOLR, I tried to dig deep in the
> code but wasn't able to find much, A little help will be much
> appreciated
>
> --
> Warm Regards,
>
> Lucky Sharma
> Contact No :+91 9821559918
> This e-mail and all material transmitted with it are for the use of the intended recipient(s) ONLY and contains confidential information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken pursuant to the contents of the present e-mail is strictly prohibited and is unlawful. The recipient acknowledges that Comviva Technologies Limited or its management or directors, are unable to exercise control or ensure the integrity over /of the contents of the information contained in e-mail. Any views expressed herein are those of the individual sender only and no binding nature of the contents shall be implied or assumed unless the sender does so expressly with due authority of Comviva Technologies Limited. E-mail and any contents transmitted with it are prone to viruses and related defects despite all efforts to avoid such by Comviva Technologies Limited.



--
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918
Reply | Threaded
Open this post in threaded view
|

Re: Streaming In Solr

Joel Bernstein
The implementation is as follows:

1) There are "stream sources" that generate results from Solr Cloud
collections. Some of these include: search, facet, knnSearch, random,
timeseries, nodes, sql etc...
2) There are "stream decorators" that wrap stream sources and operated over
the result set tuples. Some of these decorators operate over sorted result
sets and don't need to keep much data in memory. For example the innerJoin
stream decorator merge joins two sorted streams of tuples. Other stream
decorators read data into memory and perform operations entirely in memory.
An example of this is the sort stream decorator.
3) The are "stream evaluators" that evaluate expressions over the data.
This includes math expressions. These expressions can operate in both a
streaming context using the "select" expression or an in memory context
using the "let" expression to set variables and operate of vectors and
matrices in memory.

But basically you can think of it as decorators operating over streams of
data.





Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, Nov 14, 2018 at 3:26 AM Lucky Sharma <[hidden email]> wrote:

> Hi Prakhar,
> Thanks for the reply, But What I am actually curious to know how is it
> implemented Internally?
> On Wed, Nov 14, 2018 at 12:47 PM Prakhar Nigam
> <[hidden email]> wrote:
> >
> > HI Lucky Prakhar here
> >
> > we have met at a training at Mahindra Comviva.  I have found this
> article it may be a little helpful
> >
> >
> https://medium.com/@sarkaramrit2/getting-started-with-streaming-expressions-in-apache-solr-b49111a417e3
> >
> >
> >
> >
> >
> > Regards,
> >
> > Prakhar
> >
> >
> >
> > From: Lucky Sharma<mailto:[hidden email]>
> > Sent: Wednesday, November 14, 2018 9:25 AM
> > To: [hidden email]<mailto:[hidden email]>
> > Subject: Streaming In Solr
> >
> >
> >
> > Hi I have some doubt regarding how the streaming expressions and
> > parallel SQL queries are evaluated in SOLR, I tried to dig deep in the
> > code but wasn't able to find much, A little help will be much
> > appreciated
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No :+91 9821559918
> > This e-mail and all material transmitted with it are for the use of the
> intended recipient(s) ONLY and contains confidential information. If you
> are not the intended recipient, please contact the sender by reply e-mail
> and destroy all copies and the original message. Any unauthorized review,
> use, disclosure, dissemination, forwarding, printing or copying of this
> email or any action taken pursuant to the contents of the present e-mail is
> strictly prohibited and is unlawful. The recipient acknowledges that
> Comviva Technologies Limited or its management or directors, are unable to
> exercise control or ensure the integrity over /of the contents of the
> information contained in e-mail. Any views expressed herein are those of
> the individual sender only and no binding nature of the contents shall be
> implied or assumed unless the sender does so expressly with due authority
> of Comviva Technologies Limited. E-mail and any contents transmitted with
> it are prone to viruses and related defects despite all efforts to avoid
> such by Comviva Technologies Limited.
>
>
>
> --
> Warm Regards,
>
> Lucky Sharma
> Contact No :+91 9821559918
>
Reply | Threaded
Open this post in threaded view
|

Re: Streaming In Solr

Lucky Sharma
Hi Joel,
Thanks for the info :)
On Wed, Nov 14, 2018 at 8:13 PM Joel Bernstein <[hidden email]> wrote:

>
> The implementation is as follows:
>
> 1) There are "stream sources" that generate results from Solr Cloud
> collections. Some of these include: search, facet, knnSearch, random,
> timeseries, nodes, sql etc...
> 2) There are "stream decorators" that wrap stream sources and operated over
> the result set tuples. Some of these decorators operate over sorted result
> sets and don't need to keep much data in memory. For example the innerJoin
> stream decorator merge joins two sorted streams of tuples. Other stream
> decorators read data into memory and perform operations entirely in memory.
> An example of this is the sort stream decorator.
> 3) The are "stream evaluators" that evaluate expressions over the data.
> This includes math expressions. These expressions can operate in both a
> streaming context using the "select" expression or an in memory context
> using the "let" expression to set variables and operate of vectors and
> matrices in memory.
>
> But basically you can think of it as decorators operating over streams of
> data.
>
>
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Wed, Nov 14, 2018 at 3:26 AM Lucky Sharma <[hidden email]> wrote:
>
> > Hi Prakhar,
> > Thanks for the reply, But What I am actually curious to know how is it
> > implemented Internally?
> > On Wed, Nov 14, 2018 at 12:47 PM Prakhar Nigam
> > <[hidden email]> wrote:
> > >
> > > HI Lucky Prakhar here
> > >
> > > we have met at a training at Mahindra Comviva.  I have found this
> > article it may be a little helpful
> > >
> > >
> > https://medium.com/@sarkaramrit2/getting-started-with-streaming-expressions-in-apache-solr-b49111a417e3
> > >
> > >
> > >
> > >
> > >
> > > Regards,
> > >
> > > Prakhar
> > >
> > >
> > >
> > > From: Lucky Sharma<mailto:[hidden email]>
> > > Sent: Wednesday, November 14, 2018 9:25 AM
> > > To: [hidden email]<mailto:[hidden email]>
> > > Subject: Streaming In Solr
> > >
> > >
> > >
> > > Hi I have some doubt regarding how the streaming expressions and
> > > parallel SQL queries are evaluated in SOLR, I tried to dig deep in the
> > > code but wasn't able to find much, A little help will be much
> > > appreciated
> > >
> > > --
> > > Warm Regards,
> > >
> > > Lucky Sharma
> > > Contact No :+91 9821559918
> > > This e-mail and all material transmitted with it are for the use of the
> > intended recipient(s) ONLY and contains confidential information. If you
> > are not the intended recipient, please contact the sender by reply e-mail
> > and destroy all copies and the original message. Any unauthorized review,
> > use, disclosure, dissemination, forwarding, printing or copying of this
> > email or any action taken pursuant to the contents of the present e-mail is
> > strictly prohibited and is unlawful. The recipient acknowledges that
> > Comviva Technologies Limited or its management or directors, are unable to
> > exercise control or ensure the integrity over /of the contents of the
> > information contained in e-mail. Any views expressed herein are those of
> > the individual sender only and no binding nature of the contents shall be
> > implied or assumed unless the sender does so expressly with due authority
> > of Comviva Technologies Limited. E-mail and any contents transmitted with
> > it are prone to viruses and related defects despite all efforts to avoid
> > such by Comviva Technologies Limited.
> >
> >
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No :+91 9821559918
> >



--
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918