Solr Drivers

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr Drivers

ben.demott
Hello - I am completely new to the Solr development list, I have been
using Solr for about a year now - So, first - Thank you everyone for
the great set of libraries!

I did previously search the list but didn't find anything discussing
this question in detail.

I'll keep it brief and to the point.


Is there any interest in a Solr binary server-daemon in addition to
it's HTTP interface/daemon?
A generalized specification would have to be written that would listen
on an alternative port from the HTTP interface.
This binary interface would accept binary streams instead of 'strings'.
A language-specific client driver would have to be written to support
this specification. (my interest is in writing a PHP Pecl extension to
implement the server interface/specification)

If there is any interest or development going on in this area - I am
interested in connecting with the other developers and working on a
specification/implementation.

Now that SOLR and Lucene development is combining I'm not sure how
this would change implementation details.

I realize  the HTTP interface is 'at the heart' of SOLR, but I think
the benefits of language-specific drivers would be worthwhile.

Reason1:
The time taken to establish an HTTP connection, the overhead of the
protocol, and the time taken to parse string results is often greater
than the time taken to 'perform' the query itself.

Reason 2:
When returning a 'single node' serialized structure it is not possible
to 'scan' the result without at some point in time storing the entire
string representation of the result, and the entire data-structure in
memory, thus duplicating the minimum required resources.

Reason 3:
If the client generates a set of binary-tokens no string-lexing has to
occur on the server-side.

Reason 4: If client tokens are generated from an OOP/Procedural
interface, Injection (like SQL injection) is all but impossible.

Reason 5: With a binary driver native code interactions with
result-sets like "foreach", "for var in list", etc are possible,
without any need to parse or unmarshal/unserialize the result.


10gen has implemented all binary Object-Oriented client drivers for
Python, Php, Ruby, Perl, and Java for their Document database
'MongoDB' - and these benefits are very clear in the use of their
product and the speed of the driver.


Thanks for your time!
Reply | Threaded
Open this post in threaded view
|

Re: Solr Drivers

Chris Hostetter-3

: Is there any interest in a Solr binary server-daemon in addition to
: it's HTTP interface/daemon?
: A generalized specification would have to be written that would listen
: on an alternative port from the HTTP interface.
: This binary interface would accept binary streams instead of 'strings'.
: A language-specific client driver would have to be written to support
: this specification. (my interest is in writing a PHP Pecl extension to
: implement the server interface/specification)

I guess my first quesion would be: why does this need to be a seperate
port? ... we already have a binary format for updates/searches that works
over HTTP, and it's pretty easy to deal with alternate formats (string
based or binary) over HTTP using the RequestHandler+ContentStream or
QueryResponseWriter APIs, so what exactly is it that you want to do using
some binary format that isn't possible now?

: I realize  the HTTP interface is 'at the heart' of SOLR, but I think
: the benefits of language-specific drivers would be worthwhile.

Again, we already have langauge specific *parsers* and *formatters* --
it's not clear to me what types of langauge specific *transports* might be
needed.

: The time taken to establish an HTTP connection, the overhead of the

this is what keep-alive is for.


My suggestion if you want to pursue this would be to implement it using
the RequestHandler abstraction -- your custom request handler could have a
trivial NOOP implementation of handleRequest, but during initialization it
could spin up it's own port, speaking whatever protocol it wants.


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]