How do I add my own Streaming Expressions?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How do I add my own Streaming Expressions?

Eric Pugh-4
What is the process for adding new Streaming Expressions?  

It appears that the org.apache.solr.client.solrj.io.Lang method statically loads all the streaming expressions?

Eric

_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>  
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Reply | Threaded
Open this post in threaded view
|

Re: How do I add my own Streaming Expressions?

Christian Spitzlay
HI,

I assume you are trying to add a custom extension,
not preparing a general purpose contribution for solr itself ...

We succeeded in loading a custom reduce operation by adding the jar
with our code:

<lib dir="dir-with-the-custom-code" regex=".*\.jar" />

And then defining an "expressible":
<expressible name="thenewname" class="the.custom.class" />

Both changes were made in solrconfig_extra.xml, a file included from our solrconfig.xml.


HTH,
Christian


> Am 16.11.2019 um 03:43 schrieb Eric Pugh <[hidden email]>:
>
> What is the process for adding new Streaming Expressions?  
>
> It appears that the org.apache.solr.client.solrj.io.Lang method statically loads all the streaming expressions?
>
> Eric
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>  
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.


--  

Christian Spitzlay
Senior Software-Entwickler

bio.logis Genetic Information Management GmbH
Zentrale:
Olof-Palme-Str. 15
D-60439 Frankfurt am Main

T: +49 69 348 739 116
[hidden email]
biologis.com

Geschäftsführung: Prof. Dr. med. Daniela Steinberger
Firmensitz:  Altenhöferallee 3, 60438 Frankfurt am Main
Registergericht Frankfurt am Main, HRB 97945

Reply | Threaded
Open this post in threaded view
|

Re: How do I add my own Streaming Expressions?

Eric Pugh-4
In reply to this post by Eric Pugh-4
The documentation in the StreamHandler suggests adding into Solrconfig some streamFunctions:

     * <lst name="streamFunctions">
     *  <str name="group">org.apache.solr.client.solrj.io.stream.ReducerStream</str>
     *  <str name="count">org.apache.solr.client.solrj.io.stream.RecordCountStream</str>
     * </lst>


https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/StreamHandler.java#L114

What is happening in StreamHandler doesn’t seem to be working, however in the similar GraphHandler, there is a call to “streamFunctions”:

https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/GraphHandler.java#L90

I’m still debugging this…

Eric



> On Nov 15, 2019, at 9:43 PM, Eric Pugh <[hidden email]> wrote:
>
> What is the process for adding new Streaming Expressions?  
>
> It appears that the org.apache.solr.client.solrj.io.Lang method statically loads all the streaming expressions?
>
> Eric
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>  
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>

_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>  
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Reply | Threaded
Open this post in threaded view
|

Re: How do I add my own Streaming Expressions?

Joel Bernstein
Yeah this not documented. Here are two links that will be helpful:

https://issues.apache.org/jira/browse/SOLR-9103

Slide 40 Shows the solrconfig.xml approach to registering new streams:
https://www.slideshare.net/lucidworks/creating-new-streams-presented-by-dennis-gove-bloomberg-lp



Joel Bernstein
http://joelsolr.blogspot.com/


On Tue, Nov 19, 2019 at 3:04 PM Eric Pugh <[hidden email]>
wrote:

> The documentation in the StreamHandler suggests adding into Solrconfig
> some streamFunctions:
>
>      * <lst name="streamFunctions">
>      *  <str
> name="group">org.apache.solr.client.solrj.io.stream.ReducerStream</str>
>      *  <str
> name="count">org.apache.solr.client.solrj.io.stream.RecordCountStream</str>
>      * </lst>
>
>
>
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/StreamHandler.java#L114
>
> What is happening in StreamHandler doesn’t seem to be working, however in
> the similar GraphHandler, there is a call to “streamFunctions”:
>
>
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/GraphHandler.java#L90
>
> I’m still debugging this…
>
> Eric
>
>
>
> > On Nov 15, 2019, at 9:43 PM, Eric Pugh <[hidden email]>
> wrote:
> >
> > What is the process for adding new Streaming Expressions?
> >
> > It appears that the org.apache.solr.client.solrj.io.Lang method
> statically loads all the streaming expressions?
> >
> > Eric
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>