solr + cocoon problem

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

solr + cocoon problem

mirko-9
Hi,

I am trying to implement a cocoon based application using solr for searching.
In particular, I would like to forward the request from my response page to
solr.  I have tried several alternatives, but none of them worked for me.

One which would seem a logical way to me is to have response page, which is
forwarded to solr with cocoon's file generator.  It works fine if I perform
queries which contain only alphanumeric characters, but it gives the following
error if I try to query for a string containing nonalphanum characters:

http://hostname/cocoon/mywebapp/response?q=a+b

java.io.IOException: Server returned HTTP response code: 505 for URL:
http://hostname/solr/select/?q=a b


The interesting thing is that if I access http://hostname/solr/select/?q=a b
directly it works.


The relevant part of my sitemap.xmap:

<map:match pattern="response">
  <map:generate
    src="http://hostname/solr/select/?q={request-param:q}"
    type="file" >
  </map:generate>
  <map:serialize type="xml"/>
</map:match>

Any ideas on how to implement a cocoon layer above solr?

thanks,
mirko

ps. I realize this question might be more of a cocoon question, but I am
posting it here because I have gotten the idea from
http://wiki.apache.org/solr/XsltResponseWriter to use cocoon on top of solr)
So, I assume some of you have already had run into similar issues and/or knows
the solution...
Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

Walter Lewis-2
[hidden email] wrote:
> Any ideas on how to implement a cocoon layer above solr?
You're far from the only one approaching solr via cocoon ... :)

The approach we took, passes the search parameters to a "solrsearch"
stylesheet, the heart of which is a <cinclude> block that embeds the
solr results.  A further transformation prepares the results of the solr
query for display.

The latest rewrite is getting more complicated as we work in flowscript
to manipulate the values more before presenting them to solr, but the
heart of the solution is below.

Walter


==== From the sitemap.xmap =====
    <map:match pattern="results">
        <map:generate type="request">
            <map:parameter name="generate-attributes" value="true"/>
        </map:generate>
        <map:transform type="xslt" src="style/solrsearch.xsl">
            <map:parameter name="use-request-parameters" value="true"/>
        </map:transform>
        <map:transform type="cinclude" />
        <map:transform type="xslt" src="style/search_result.xsl" />
        <map:serialize type="html"/>
    </map:match>

=== From solrsearch.xsl ====
[assuming parameters of q, start and rows]

            <cinclude:includexml>
               
<cinclude:src>http://localhost:8080/solr/select?q=<xsl:value-of
select='$q' />&amp;start=<xsl:value-of select='$start'
/>&amp;rows=<xsl:value-of select='$rows' />
                </cinclude:src>
            </cinclude:includexml>


Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

thorsten
On Tue, 2007-01-16 at 16:19 -0500, Walter Lewis wrote:
> [hidden email] wrote:
> > Any ideas on how to implement a cocoon layer above solr?

I just finished a forrest plugin (in the whiteboard, our testing ground
in forrest) that is doing what you asked for and some pagination.
Forrest is cocoon based so you just have to build the plugin jar and add
it to your cocoon project. Please ask on the forrest list if you have
problems.

http://forrest.apache.org/pluginDocs/plugins_0_80/org.apache.forrest.plugin.output.solr/

> You're far from the only one approaching solr via cocoon ... :)
>
> The approach we took, passes the search parameters to a "solrsearch"
> stylesheet, the heart of which is a <cinclude> block that embeds the
> solr results.  A further transformation prepares the results of the solr
> query for display.

That was my first version for above plugin as well, but since forrest
makes use of the cocoon crawler I needed something with a default search
string for offline generation.

You should have a closer look on
http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.output.solr/output.xmap?view=markup
and
http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.output.solr/input.xmap?view=markup

For the original use case of this thread I added a generator:

<map:generator name="solr-search"
        logger="sitemap.generator.searchgenerator"
        src="org.apache.forrest.solr.client.SolrSearchGenerator" />

and as well a paginator transformer that calculates the next pages based on start, rows and numFound:

 <map:transformer logger="sitemap.transformer.solr" name="solr" pool-max="16"
        src="org.apache.forrest.solr.client.SolrQueryTransformer" />

We use it as follows:

<!-- Will dispatch a query against the solr server -->
      <map:match pattern="solr-search.xml">
        <map:generate type="solr-search">
<!-- You would replace this with your own. In forrest we have a property system for that
<property name="solr.select.url" value="http://localhost:8983/solr/select"/>
-->
          <map:parameter name="destinationUrl" value="{properties:solr.select.url}"/>
        </map:generate>
<!-- here we add the pagination to the sax from the generator will result in something like:
...
<result name="response" numFound="17" start="0">
 <paginator found="17" start="0" rows="10" pages="2">
  <page id="1" queryString="q=forrest&amp;hl=true&amp;rows=10&amp;version=2.2&amp;indent=on&amp;hl.fl=content&amp;fl=id,title&amp;Search=Search&amp;start=0" current="true"/>
  <page id="2" queryString="q=forrest&amp;hl=true&amp;rows=10&amp;version=2.2&amp;indent=on&amp;hl.fl=content&amp;fl=id,title&amp;Search=Search&amp;start=10"/>
 </paginator>
<doc>
...-->
        <map:transform type="solr"/>
<!-- here we transform the solr response into xdocs (forrest internal format)
the pagination:
<xsl:template match="paginator">
<section>
      <title>Result pages</title>
      <p>
        <xsl:for-each select="page">
          <xsl:variable name="current" select="@current"/>
          <xsl:choose>
            <xsl:when test="$current='true'">
              <xsl:text> </xsl:text><xsl:value-of select="@id"/><xsl:text> </xsl:text>
            </xsl:when>
            <xsl:otherwise>
              <a href="{concat($searchForm,'?',@queryString)}">
                <xsl:value-of select="@id"/>
              </a>
            </xsl:otherwise>
          </xsl:choose>
        </xsl:for-each>
      </p>
    </section>
  </xsl:template>
-->
        <map:transform src="resources/stylesheets/solrQueryResult-to-xdocs.xsl">
          <map:parameter name="searchForm" value="{request:servletPath}"/>
        </map:transform>
        <map:serialize type="xml"/>
      </map:match>

You may be interested in the update generator as well.

Please give feedback to [hidden email].

It really needs more testing besides myself, you could be the first to provide feedback.

<map:generator name="solrUpdate"
        src="org.apache.forrest.solr.client.SolrUpdateGenerator"/>

<!-- Update generator will send the command to the solr server and
        generate the server response -->
      <map:match pattern="**.do">
        <map:generate type="solrUpdate" src="cocoon:/{1}">
          <map:parameter name="destinationUrl" value="{properties:solr.update.url}"/>
        </map:generate>
        <map:serialize type="xml"/>
      </map:match>

HTH

salu2
--
thorsten

"Together we stand, divided we fall!"
Hey you (Pink Floyd)


Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

thorsten
In reply to this post by mirko-9
On Tue, 2007-01-16 at 16:02 -0500, [hidden email] wrote:
> Hi,
>
> I am trying to implement a cocoon based application using solr for searching.
> In particular, I would like to forward the request from my response page to
> solr.  I have tried several alternatives, but none of them worked for me.
>

Please see http://wiki.apache.org/solr/SolrForrest.

salu2
--
thorsten

"Together we stand, divided we fall!"
Hey you (Pink Floyd)


Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

Chris Hostetter-3
In reply to this post by mirko-9

: java.io.IOException: Server returned HTTP response code: 505 for URL:
: http://hostname/solr/select/?q=a b
:
:
: The interesting thing is that if I access http://hostname/solr/select/?q=a b
: directly it works.

i don't know anything about cocoon, but that is not a legal URL, URLs
can't have spaces in them ... if you type a space into your browser, it's
probably being nice and URL escaping it for you (that's what most browsers
seem to do now a days)

i'm guessing Cocoon automaticaly un-escapes the input to your app, and you
need to re-URL escape it before sending it to Solr.




-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

mirko-9
Hi,

I agree, this is not a legal URL.  But the thing is that cocoon itself is
sending the unescaped URL.  That is why I thought I am not using the right
tools from cocoon.

mirko


Quoting Chris Hostetter <[hidden email]>:

>
> : java.io.IOException: Server returned HTTP response code: 505 for URL:
> : http://hostname/solr/select/?q=a b
> :
> :
> : The interesting thing is that if I access http://hostname/solr/select/?q=a
> b
> : directly it works.
>
> i don't know anything about cocoon, but that is not a legal URL, URLs
> can't have spaces in them ... if you type a space into your browser, it's
> probably being nice and URL escaping it for you (that's what most browsers
> seem to do now a days)
>
> i'm guessing Cocoon automaticaly un-escapes the input to your app, and you
> need to re-URL escape it before sending it to Solr.
>
>
>
>
> -Hoss
>


Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

Thorsten Scherler-3
On Wed, 2007-01-17 at 10:25 -0500, [hidden email] wrote:
> Hi,
>
> I agree, this is not a legal URL.  But the thing is that cocoon itself is
> sending the unescaped URL.

...because you told it so.

You use
<map:generate
    src="http://hostname/solr/select/?q={request-param:q}"
    type="file" >

The request param module will not escape the param by default.

salu2

Reply | Threaded
Open this post in threaded view
|

Re: solr + cocoon problem

mirko-9
Thanks Thorsten,

that really was helpful.  Cocoon's url-encode module does solve my problem.

mirko


Quoting Thorsten Scherler <[hidden email]>:

> On Wed, 2007-01-17 at 10:25 -0500, [hidden email] wrote:
> > Hi,
> >
> > I agree, this is not a legal URL.  But the thing is that cocoon itself is
> > sending the unescaped URL.
>
> ...because you told it so.
>
> You use
> <map:generate
>     src="http://hostname/solr/select/?q={request-param:q}"
>     type="file" >
>
> The request param module will not escape the param by default.
>
> salu2
>