Problem using curl in PHP to get Solr results

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem using curl in PHP to get Solr results

gearond
I finally figured out how to use curl to GET results, i.e. just turn all spaces
into '%20' in my type of queries. I'm using solar spatial, and then searching in
both the default text field and a couple of columns. Works fine on in the
browser.

But if I query for it using curl in PHP, there's an error somewhere in the JSON.
I don't know if it's in the PHP food chain or something else.


Just putting my solution to GETing from curl in PHP and my problem up here, for
others to find.

 Of course, if anyone knows the answer, all the better.

 Dennis Gearon


Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'


EARTH has a Right To Life,
otherwise we all die.

Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

pankaj bhatt
HI ,


On Wed, Dec 15, 2010 at 2:52 PM, Dennis Gearon <[hidden email]>wrote:

> I finally figured out how to use curl to GET results, i.e. just turn all
> spaces
> into '%20' in my type of queries. I'm using solar spatial, and then
> searching in
> both the default text field and a couple of columns. Works fine on in the
> browser.
>
> But if I query for it using curl in PHP, there's an error somewhere in the
> JSON.
> I don't know if it's in the PHP food chain or something else.
>
>
> Just putting my solution to GETing from curl in PHP and my problem up here,
> for
> others to find.
>
>  Of course, if anyone knows the answer, all the better.
>
>  Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

Stephen Weiss-2
In reply to this post by gearond
Forgive me if this seems like a dumb question but have you tried the Apache_Solr_Service class?

http://www.ibm.com/developerworks/library/os-php-apachesolr/index.html

It's really quite good at handling the nuts and bolts of making the HTTP requests and decoding the responses for PHP.  I almost always use it when working from PHP.  It's all over Google so I don't know how someone would miss it but I don't know why else someone would bother curling a GET to SOLR otherwise.

--
Steve

On Dec 15, 2010, at 4:22 AM, Dennis Gearon wrote:

> I finally figured out how to use curl to GET results, i.e. just turn all spaces
> into '%20' in my type of queries. I'm using solar spatial, and then searching in
> both the default text field and a couple of columns. Works fine on in the
> browser.
>
> But if I query for it using curl in PHP, there's an error somewhere in the JSON.
> I don't know if it's in the PHP food chain or something else.
>
>
> Just putting my solution to GETing from curl in PHP and my problem up here, for
> others to find.
>
> Of course, if anyone knows the answer, all the better.
>
> Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a better
> idea to learn from others’ mistakes, so you do not have to make them yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>

Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

gearond
I want to just pass the JSON through after qualifying the user's access to the
site.


Didn't want to spend the horse power to receive it as PHP array syntax, run the
risk of someone putting bad stuff in the contents and running 'exec()' on it,
and then spending the extra horsepower to putput it as json.

I had that page up in the browwser to look at it later. If it deons't do the
above, I will be glad to have the Solr access abstracted, thanks :-)


 Dennis Gearon


Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'


EARTH has a Right To Life,
otherwise we all die.



----- Original Message ----
From: Stephen Weiss <[hidden email]>
To: [hidden email]
Sent: Wed, December 15, 2010 1:36:11 AM
Subject: Re: Problem using curl in PHP to get Solr results

Forgive me if this seems like a dumb question but have you tried the
Apache_Solr_Service class?

http://www.ibm.com/developerworks/library/os-php-apachesolr/index.html

It's really quite good at handling the nuts and bolts of making the HTTP
requests and decoding the responses for PHP.  I almost always use it when
working from PHP.  It's all over Google so I don't know how someone would miss
it but I don't know why else someone would bother curling a GET to SOLR
otherwise.

--
Steve

On Dec 15, 2010, at 4:22 AM, Dennis Gearon wrote:

> I finally figured out how to use curl to GET results, i.e. just turn all spaces
>
> into '%20' in my type of queries. I'm using solar spatial, and then searching
>in
>
> both the default text field and a couple of columns. Works fine on in the
> browser.
>
> But if I query for it using curl in PHP, there's an error somewhere in the
>JSON.
>
> I don't know if it's in the PHP food chain or something else.
>
>
> Just putting my solution to GETing from curl in PHP and my problem up here, for
>
> others to find.
>
> Of course, if anyone knows the answer, all the better.
>
> Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
>better
>
> idea to learn from others’ mistakes, so you do not have to make them yourself.

> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

Andrew McCombe
Hi

You could use Solr's php serialized object output (wt=phps) and then convert
it to json in your php:
<?php
echo json_encode(unserialize($results_from_solr));
?>

Regards
Andrew McCombe

On 15 December 2010 17:49, Dennis Gearon <[hidden email]> wrote:

> I want to just pass the JSON through after qualifying the user's access to
> the
> site.
>
>
> Didn't want to spend the horse power to receive it as PHP array syntax, run
> the
> risk of someone putting bad stuff in the contents and running 'exec()' on
> it,
> and then spending the extra horsepower to putput it as json.
>
> I had that page up in the browwser to look at it later. If it deons't do
> the
> above, I will be glad to have the Solr access abstracted, thanks :-)
>
>
>  Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
>
> ----- Original Message ----
> From: Stephen Weiss <[hidden email]>
> To: [hidden email]
> Sent: Wed, December 15, 2010 1:36:11 AM
> Subject: Re: Problem using curl in PHP to get Solr results
>
> Forgive me if this seems like a dumb question but have you tried the
> Apache_Solr_Service class?
>
> http://www.ibm.com/developerworks/library/os-php-apachesolr/index.html
>
> It's really quite good at handling the nuts and bolts of making the HTTP
> requests and decoding the responses for PHP.  I almost always use it when
> working from PHP.  It's all over Google so I don't know how someone would
> miss
> it but I don't know why else someone would bother curling a GET to SOLR
> otherwise.
>
> --
> Steve
>
> On Dec 15, 2010, at 4:22 AM, Dennis Gearon wrote:
>
> > I finally figured out how to use curl to GET results, i.e. just turn all
> spaces
> >
> > into '%20' in my type of queries. I'm using solar spatial, and then
> searching
> >in
> >
> > both the default text field and a couple of columns. Works fine on in the
> > browser.
> >
> > But if I query for it using curl in PHP, there's an error somewhere in
> the
> >JSON.
> >
> > I don't know if it's in the PHP food chain or something else.
> >
> >
> > Just putting my solution to GETing from curl in PHP and my problem up
> here, for
> >
> > others to find.
> >
> > Of course, if anyone knows the answer, all the better.
> >
> > Dennis Gearon
> >
> >
> > Signature Warning
> > ----------------
> > It is always a good idea to learn from your own mistakes. It is usually a
> >better
> >
> > idea to learn from others’ mistakes, so you do not have to make them
> yourself.
>
> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
> >
> >
> > EARTH has a Right To Life,
> > otherwise we all die.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

Markus Jelsma-2
In reply to this post by gearond
The GeoDistanceComponent triggers the problem. It may be an issue in the
component but it could very well be a Solr issue. It seems you missed a very
recent thread on this one.

https://issues.apache.org/jira/browse/SOLR-2278

> I finally figured out how to use curl to GET results, i.e. just turn all
> spaces into '%20' in my type of queries. I'm using solar spatial, and then
> searching in both the default text field and a couple of columns. Works
> fine on in the browser.
>
> But if I query for it using curl in PHP, there's an error somewhere in the
> JSON. I don't know if it's in the PHP food chain or something else.
>
>
> Just putting my solution to GETing from curl in PHP and my problem up here,
> for others to find.
>
>  Of course, if anyone knows the answer, all the better.
>
>  Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better idea to learn from others’ mistakes, so you do not have to make
> them yourself. from
> 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

gearond
In reply to this post by Andrew McCombe
I will look into the security and processor power implications of that. Good
idea, thx.

 Dennis Gearon


Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'


EARTH has a Right To Life,
otherwise we all die.



----- Original Message ----
From: Andrew McCombe <[hidden email]>
To: [hidden email]
Sent: Wed, December 15, 2010 11:07:54 AM
Subject: Re: Problem using curl in PHP to get Solr results

Hi

You could use Solr's php serialized object output (wt=phps) and then convert
it to json in your php:
<?php
echo json_encode(unserialize($results_from_solr));
?>

Regards
Andrew McCombe

On 15 December 2010 17:49, Dennis Gearon <[hidden email]> wrote:

> I want to just pass the JSON through after qualifying the user's access to
> the
> site.
>
>
> Didn't want to spend the horse power to receive it as PHP array syntax, run
> the
> risk of someone putting bad stuff in the contents and running 'exec()' on
> it,
> and then spending the extra horsepower to putput it as json.
>
> I had that page up in the browwser to look at it later. If it deons't do
> the
> above, I will be glad to have the Solr access abstracted, thanks :-)
>
>
>  Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
>
> ----- Original Message ----
> From: Stephen Weiss <[hidden email]>
> To: [hidden email]
> Sent: Wed, December 15, 2010 1:36:11 AM
> Subject: Re: Problem using curl in PHP to get Solr results
>
> Forgive me if this seems like a dumb question but have you tried the
> Apache_Solr_Service class?
>
> http://www.ibm.com/developerworks/library/os-php-apachesolr/index.html
>
> It's really quite good at handling the nuts and bolts of making the HTTP
> requests and decoding the responses for PHP.  I almost always use it when
> working from PHP.  It's all over Google so I don't know how someone would
> miss
> it but I don't know why else someone would bother curling a GET to SOLR
> otherwise.
>
> --
> Steve
>
> On Dec 15, 2010, at 4:22 AM, Dennis Gearon wrote:
>
> > I finally figured out how to use curl to GET results, i.e. just turn all
> spaces
> >
> > into '%20' in my type of queries. I'm using solar spatial, and then
> searching
> >in
> >
> > both the default text field and a couple of columns. Works fine on in the
> > browser.
> >
> > But if I query for it using curl in PHP, there's an error somewhere in
> the
> >JSON.
> >
> > I don't know if it's in the PHP food chain or something else.
> >
> >
> > Just putting my solution to GETing from curl in PHP and my problem up
> here, for
> >
> > others to find.
> >
> > Of course, if anyone knows the answer, all the better.
> >
> > Dennis Gearon
> >
> >
> > Signature Warning
> > ----------------
> > It is always a good idea to learn from your own mistakes. It is usually a
> >better
> >
> > idea to learn from others’ mistakes, so you do not have to make them
> yourself.
>
> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
> >
> >
> > EARTH has a Right To Life,
> > otherwise we all die.
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: Problem using curl in PHP to get Solr results

gearond
well, it was three problems:

1/ I was saving the file as a 'complete web page', uknowingly, from firefox.
2/ I had a small message for troubleshooting being spit out after the json.
3/ My partner had output all the spatial solr 'tiers' information, and there's a
binary value in there that stops all the JSON viewer plugins for firefox from
going past it. At this point, PHP *AND* the firefox plugins consider the json to
be valid, but the plugin won't display past a character with value 0x007F. Or at
least, that's what the browser is displaying in a box at thatpoint.

 Dennis Gearon


Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'


EARTH has a Right To Life,
otherwise we all die.



----- Original Message ----
From: Dennis Gearon <[hidden email]>
To: [hidden email]
Sent: Wed, December 15, 2010 3:28:40 PM
Subject: Re: Problem using curl in PHP to get Solr results

I will look into the security and processor power implications of that. Good
idea, thx.

Dennis Gearon


Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better

idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'


EARTH has a Right To Life,
otherwise we all die.



----- Original Message ----
From: Andrew McCombe <[hidden email]>
To: [hidden email]
Sent: Wed, December 15, 2010 11:07:54 AM
Subject: Re: Problem using curl in PHP to get Solr results

Hi

You could use Solr's php serialized object output (wt=phps) and then convert
it to json in your php:
<?php
echo json_encode(unserialize($results_from_solr));
?>

Regards
Andrew McCombe

On 15 December 2010 17:49, Dennis Gearon <[hidden email]> wrote:

> I want to just pass the JSON through after qualifying the user's access to
> the
> site.
>
>
> Didn't want to spend the horse power to receive it as PHP array syntax, run
> the
> risk of someone putting bad stuff in the contents and running 'exec()' on
> it,
> and then spending the extra horsepower to putput it as json.
>
> I had that page up in the browwser to look at it later. If it deons't do
> the
> above, I will be glad to have the Solr access abstracted, thanks :-)
>
>
>  Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
>
> ----- Original Message ----
> From: Stephen Weiss <[hidden email]>
> To: [hidden email]
> Sent: Wed, December 15, 2010 1:36:11 AM
> Subject: Re: Problem using curl in PHP to get Solr results
>
> Forgive me if this seems like a dumb question but have you tried the
> Apache_Solr_Service class?
>
> http://www.ibm.com/developerworks/library/os-php-apachesolr/index.html
>
> It's really quite good at handling the nuts and bolts of making the HTTP
> requests and decoding the responses for PHP.  I almost always use it when
> working from PHP.  It's all over Google so I don't know how someone would
> miss
> it but I don't know why else someone would bother curling a GET to SOLR
> otherwise.
>
> --
> Steve
>
> On Dec 15, 2010, at 4:22 AM, Dennis Gearon wrote:
>
> > I finally figured out how to use curl to GET results, i.e. just turn all
> spaces
> >
> > into '%20' in my type of queries. I'm using solar spatial, and then
> searching
> >in
> >
> > both the default text field and a couple of columns. Works fine on in the
> > browser.
> >
> > But if I query for it using curl in PHP, there's an error somewhere in
> the
> >JSON.
> >
> > I don't know if it's in the PHP food chain or something else.
> >
> >
> > Just putting my solution to GETing from curl in PHP and my problem up
> here, for
> >
> > others to find.
> >
> > Of course, if anyone knows the answer, all the better.
> >
> > Dennis Gearon
> >
> >
> > Signature Warning
> > ----------------
> > It is always a good idea to learn from your own mistakes. It is usually a
> >better
> >
> > idea to learn from others’ mistakes, so you do not have to make them
> yourself.
>
> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
> >
> >
> > EARTH has a Right To Life,
> > otherwise we all die.
> >
>