Streaming expressions malfunctioning

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Streaming expressions malfunctioning

jpereira
This post was updated on .
Hi there,

I just recently upgraded a SOLR instance to 6.0.1 version and while been trying the new streaming expressions feature I discovered what I think might be a bug in the HTTP interface.

I tried to create two simple streaming expressions, as described below:

innerJoin(
        search(collection1,zkhost="localhost:9983",qt="/export",fl="id",sort="id asc",q="*:*"),
        search(collection2,zkhost="localhost:9983",qt="/export",fl="id",sort="id asc",q="*:*"),
        on("id")
)

facet(
        collection3,
        q="*:*",
        buckets="field1",
        bucketSorts="sum(field2) asc",
        sum(field2),
        count(*)
)

What I noticed is while I can obtain the expressions results using JAVA, the feature does not seem to function when I try to get the same data via cURL. You can see the code snippets I used below. Can you tell me if I am doing anything wrong or if this is indeed a bug in this version?

Inner Join expression

HTTP/PHP (not working)

Request:
curl --data-urlencode 'expr=innerJoin(search(collection1,zkhost="localhost:9983",qt="/export",fl="id",sort="id asc",q="*:*"),search(collection2,zkhost="localhost:9983",qt="/export",fl="id",sort="id asc",q="*:*"),on("id"))' http://localhost:8983/solr/collection1/stream
Response:
{"result-set":{"docs":[{"EXCEPTION":"Unable to construct instance of org.apache.solr.client.solrj.io.stream.InnerJoinStream","EOF":true}]}}

JAVA (working)

Map props = new HashMap();
	props1.put("q", "*:*");
	props1.put("qt", "/export");
	props1.put("sort", "id asc");
	props1.put("fl", "id");

CloudSolrStream stream1 = new CloudSolrStream(
		"localhost:9983",
		"collection1",
		props
);
CloudSolrStream stream2 = new CloudSolrStream(
		"localhost:9983",
		"collection2",
		props
);
StreamEqualitor eq = new FieldEqualitor("id");

InnerJoinStream joinStream = new InnerJoinStream(stream1, stream2, eq);
joinStream.open()

while(true){
	Tuple tuple = joinStream.read();
	if(tuple.EOF) {
		break;
	}
	//Some code to consume tuples here
}

joinStream.close();

Facet Expression

HTTP/PHP

Request:
curl --data-urlencode 'expr=facet(collection3,q="*:*",buckets="field1",bucketSorts="sum(field2) asc",sum(field2),count(*))' http://localhost:8983/solr/collection3/stream
Response:
{"result-set":{"docs":[{"EXCEPTION":"Unable to construct instance of org.apache.solr.client.solrj.io.stream.FacetStream","EOF":true}]}}

JAVA

Map props = new HashMap();
props.put("q", "*:*");

Bucket[] buckets = new Bucket[1];
buckets[0] = new Bucket("field1");

Metric[] metrics = new Metric[2];
metrics[0] = new SumMetric("field2");
metrics[1] = new CountMetric();

FieldComparator[] bucketSorts = new FieldComparator[1];
bucketSorts[0] = new FieldComparator("field1", ComparatorOrder.ASCENDING);

FacetStream facetStream = new FacetStream(
	"localhost:9983",
	"collection3",
	props,
	buckets,
	metrics,
	bucketSorts,
	100
);

facetStream.open();

while(true) {
	Tuple tuple = facetStream.read();
	if(tuple.EOF) {
	   break;
	}
	//Some code to consume tuples here
}

facetStream.close();

Thanks for the help!

Regards,

João Pereira
Reply | Threaded
Open this post in threaded view
|

Re: Streaming expressions malfunctioning

jpereira
EDIT: I'll keep testing with other stream sources/decorators. So far only the search endpoint works both in the JAVA and cURL implementation

Cheers
Reply | Threaded
Open this post in threaded view
|

Re: Streaming expressions malfunctioning

Joel Bernstein
I'm not a big user of curl when I test the http interface. Most of the time
I use a browser and just send the request and view the results. This won't
be feasible if you're streaming a large number of documents but it will
work to quickly prototype expressions. I believe chrome will also handle
the URLencoding of the expression, but I usually URLencode the expression
before sending it down.


In Solr 6.1 the admin screen has a console for sending expressions which is
really nice. Also I believe Solr 6.1 does a better job of bubbling up the
error message all the way to the client.

In my testing all the expressions work from the http interface. But as they
get more complex it's easier to introduce errors into the syntax. If the
error that is returned to the client isn't clear enough you can check the
logs for the full stack trace.



Joel Bernstein
http://joelsolr.blogspot.com/

On Tue, Jun 7, 2016 at 10:38 AM, jpereira <[hidden email]> wrote:

> EDIT: I'll keep testing with other stream sources/decorators. So far only
> the
> search endpoint works both in the JAVA and cURL implementation
>
> Cheers
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Streaming-expressions-malfunctioning-tp4281016p4281019.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Streaming expressions malfunctioning

jpereira
Hi,

Actually there were errors in the expression synthax , examining the logs allowed me to see what the error was.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Streaming expressions malfunctioning

vrindavda
This post was updated on .
Hello,

I am looking for similar use case - Using LeftOuterJoin on collections(self join) with facets. Will it be possible for you to share the corrected syntax ?

Or let me know if there is any other way to self join on Solr Collections and get facets too.