[jira] Created: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

classic Classic list List threaded Threaded
98 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
A simple Java client with Java APIs for add(), delete(), commit() and optimize().
---------------------------------------------------------------------------------

         Key: SOLR-20
         URL: http://issues.apache.org/jira/browse/SOLR-20
     Project: Solr
        Type: New Feature

  Components: update  
 Environment: all
    Reporter: Darren Erik Vengroff
    Priority: Minor


I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Darren Erik Vengroff updated SOLR-20:
-------------------------------------

    Attachment: DocumentManagerClient.java

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Darren Erik Vengroff updated SOLR-20:
-------------------------------------

    Attachment: SolrClientException.java

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Darren Erik Vengroff updated SOLR-20:
-------------------------------------

    Attachment: DocumentManagerClient.java

Previous version didn't properly escape the query in the deleteByQuery() case.

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/SOLR-20?page=comments#action_12414382 ]

Erik Hatcher commented on SOLR-20:
----------------------------------

This looks quite good and well documented!   Thanks for this contribution.  The only issue my current project would have with this is the Map which prevents multiple fields of the same name from being added.  I use a lot of multi-valued fields.


> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/SOLR-20?page=comments#action_12414394 ]

Erik Hatcher commented on SOLR-20:
----------------------------------

One idea for dealing with multivalued fields is to check the type of the object in the Map and if is an array or Collection then iterate over it rather than just doing .toString() on it.  Would that logic work for your use as well?

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/SOLR-20?page=comments#action_12414414 ]

Darren Erik Vengroff commented on SOLR-20:
------------------------------------------

Iterating over an array or Collection of values is a great suggestion.  I'll change it and resubmit when I have some time later today.

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Darren Erik Vengroff updated SOLR-20:
-------------------------------------

    Attachment: DocumentManagerClient.java

Here is the latest, incorporating Erik's suggestion about supporting multi-valued fields.

BTW, is there any way to delete the older attached versions of this file from JIRA?  There's no real need for them to be there any more.

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Erik Hatcher updated SOLR-20:
-----------------------------

    Attachment:     (was: DocumentManagerClient.java)

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Erik Hatcher updated SOLR-20:
-----------------------------

    Attachment:     (was: DocumentManagerClient.java)

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, SolrClientException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Erik Hatcher
In reply to this post by JIRA jira@apache.org
Darren (Erik!) - Thanks for this very handy utility!   I'm happy to  
commit it soon (let me use it first) unless someone beats me to it.

I'll have to see what makes the most sense in terms of where this  
will live in the directory structure and how we'll distribute it so  
it can be used as a stand-alone API for a client.  I noticed it  
leverages Solr's XML class which makes this a little more tricky for  
someone to easily use in a standalone Java project, but not a big  
deal for us to arrange things so all someone would need is a solr-
client.jar or something like that.

I've deleted the older versions of the file on JIRA.

        Erik


On Jun 2, 2006, at 12:58 PM, Darren Erik Vengroff (JIRA) wrote:
> Here is the latest, incorporating Erik's suggestion about  
> supporting multi-valued fields.
>
> BTW, is there any way to delete the older attached versions of this  
> file from JIRA?  There's no real need for them to be there any more.
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Yonik Seeley
On 6/2/06, Erik Hatcher <[hidden email]> wrote:
> Darren (Erik!) - Thanks for this very handy utility!   I'm happy to
> commit it soon (let me use it first) unless someone beats me to it.
>
> I'll have to see what makes the most sense in terms of where this
> will live in the directory structure and how we'll distribute it so
> it can be used as a stand-alone API for a client.

Yeah, the fewer dependencies the better.
As for where it lives.... contrib/clients/java?
alongside contrib/clients/[ruby,python,cpp,etc]?

>  I noticed it
> leverages Solr's XML class
> which makes this a little more tricky for
> someone to easily use in a standalone Java project, but not a big
> deal for us to arrange things so all someone would need is a solr-
> client.jar or something like that.

Hmmm, yeah.  For the Query side of clients, we might want some other
classes like NamedList too?


-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Erik Hatcher

On Jun 2, 2006, at 1:25 PM, Yonik Seeley wrote:
> On 6/2/06, Erik Hatcher <[hidden email]> wrote:
>> I'll have to see what makes the most sense in terms of where this
>> will live in the directory structure and how we'll distribute it so
>> it can be used as a stand-alone API for a client.
>
> Yeah, the fewer dependencies the better.
> As for where it lives.... contrib/clients/java?
> alongside contrib/clients/[ruby,python,cpp,etc]?

I think this kind of library is more "core" than contrib.  This would  
be THE way Java clients would interact with Solr, I suspect.  But, of  
course it doesn't really matter much to me as it'll just be a JAR  
file I'll copy into my project once :) [I index with Java, query from  
Rails]

        Erik


>
>>  I noticed it
>> leverages Solr's XML class
>> which makes this a little more tricky for
>> someone to easily use in a standalone Java project, but not a big
>> deal for us to arrange things so all someone would need is a solr-
>> client.jar or something like that.
>
> Hmmm, yeah.  For the Query side of clients, we might want some other
> classes like NamedList too?
>
>
> -Yonik
> http://incubator.apache.org/solr Solr, the open-source Lucene  
> search server

Reply | Threaded
Open this post in threaded view
|

RE: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Darren Vengroff-2
In reply to this post by Erik Hatcher
Thanks Erik.

I wasn't thrilled about the XML dependency either, but it I figured using
the class already present in solr was better than adding new code (or
ripping off code I've written elsewhere) that does more or less the same
thing.

In situations like this I tend to go for a three jar approach, with a common
jar with stuff like XML in it, and then independent client and server jars
built on top of that.  It might be overkill in this case, given there is
only one small client class, but it is an option.

Maven2, by the way, is really good at managing that kind of interdependent
set of jars.  But I realize that I got ahead of myself when I proposed
http://issues.apache.org/jira/browse/SOLR-19?page=all.  I've since made
improvements that make it much easier for me to use solr in my
all-Maven2-all-the-time world, but it's somewhat unreasonable to push it
back on a community that is happy with ant.

-D

-----Original Message-----
From: Erik Hatcher [mailto:[hidden email]]
Sent: Friday, June 02, 2006 10:07 AM
To: [hidden email]
Subject: Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs
for add(), delete(), commit() and optimize().

Darren (Erik!) - Thanks for this very handy utility!   I'm happy to  
commit it soon (let me use it first) unless someone beats me to it.

I'll have to see what makes the most sense in terms of where this  
will live in the directory structure and how we'll distribute it so  
it can be used as a stand-alone API for a client.  I noticed it  
leverages Solr's XML class which makes this a little more tricky for  
someone to easily use in a standalone Java project, but not a big  
deal for us to arrange things so all someone would need is a solr-
client.jar or something like that.

I've deleted the older versions of the file on JIRA.

        Erik


On Jun 2, 2006, at 12:58 PM, Darren Erik Vengroff (JIRA) wrote:
> Here is the latest, incorporating Erik's suggestion about  
> supporting multi-valued fields.
>
> BTW, is there any way to delete the older attached versions of this  
> file from JIRA?  There's no real need for them to be there any more.

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Yonik Seeley
In reply to this post by JIRA jira@apache.org
Thanks for contributing this Darren!

Some thoughts about future directions:
How would someone go about maximizing throughput by using multiple
connections and multiple requests to a Solr server?

There are two cases from the client perspective:
  1) client has a single thread calling add()
  2) client has multiple threads generating documents and calling add()

Interface:
  1) We could have a pool of DocumentManagerClient objects, and create
a *new* interface that accepts updates and dispatches to an available
DocumentManagerClient.
  2) We could make DocumentManagerClient an interface, and have both a
single threaded version and a multi-threaded, multi-connection version
(which could simply use multiple single threaded versions).
  - if we opted for (2), then we should make the single-threaded
version MT safe so the client wouldn't have to know that one
implementation is and one isn't.  Wouldn't be too hard - might need to
get rid of getLastServerException().


Implementation:
  Java5 concurrency classes!

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Erik Hatcher
In reply to this post by Darren Vengroff-2

On Jun 2, 2006, at 1:35 PM, Darren Vengroff wrote:

> I wasn't thrilled about the XML dependency either, but it I figured  
> using
> the class already present in solr was better than adding new code (or
> ripping off code I've written elsewhere) that does more or less the  
> same
> thing.
>
> In situations like this I tend to go for a three jar approach, with  
> a common
> jar with stuff like XML in it, and then independent client and  
> server jars
> built on top of that.  It might be overkill in this case, given  
> there is
> only one small client class, but it is an option.

Sounds like we need a client with the code that is not shared within  
the Solr server, a Solr JAR that is shared across both the client and  
server, and then whatever other dependencies we need like the pull  
parser.

With Yonik's further directions with this sort of thing, I think the  
best thing for now is to go with his recommendation of putting it in  
a contrib directory and letting this stuff evolve.  I do see it  
eventually fitting into some core kinda of library without the  
"contrib" label.

> Maven2, by the way, is really good at managing that kind of  
> interdependent
> set of jars.  But I realize that I got ahead of myself when I proposed
> http://issues.apache.org/jira/browse/SOLR-19?page=all.  I've since  
> made
> improvements that make it much easier for me to use solr in my
> all-Maven2-all-the-time world, but it's somewhat unreasonable to  
> push it
> back on a community that is happy with ant.

I must admit a strong bias against Maven due to my deep ties with  
Ant, but the last time I tried it I was using the first version of  
Maven.  Maven2 has gotten a lot of good PR and is well worth  
considering for Solr and my own Java projects.  At the moment, I  
stick with raw Lucene, Solr, and Ruby on Rails so I don't deal with a  
whole lot of dependencies thankfully.

        Erik

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Chris Hostetter-3

: Sounds like we need a client with the code that is not shared within
: the Solr server, a Solr JAR that is shared across both the client and
: server, and then whatever other dependencies we need like the pull
: parser.

I think the org.apache.solr.util package may be the right boundray ... the
build process for "Solr core" could generate a solr-util.jar which
contains all of that code, and a solr.war (which also contains
that solr-util.jar for use within the Solr) external clients could then
use solr-util.jar as needed.

: With Yonik's further directions with this sort of thing, I think the
: best thing for now is to go with his recommendation of putting it in
: a contrib directory and letting this stuff evolve.  I do see it
: eventually fitting into some core kinda of library without the
: "contrib" label.

I personally don't have an opinion about a "contrib" label .. but as we
talk about making a place for client library code, I wonder wether we want
to distiguish client code intended for executing queries vs. client code
intended for executing updates? (vs. client code that encapsulates both)
... should we encourage a seperation?



-Hoss

Reply | Threaded
Open this post in threaded view
|

RE: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Darren Vengroff-2
In reply to this post by Yonik Seeley
The multi-threaded cases are interesting.  I tried to keep the initial
implementation super-simple, though arguably it would have been better to
skip getLastServerException() and just throw an exception on the client side
when a non-zero server side status came back.  That would improve thread
safety.

One additional issue that comes to mind in option 2 below is that different
kinds of clients may have different requirements or expectations when it
comes to dealing with errors coming back from the server.  In the standard
async kind of model with one or more producers (those calling add())
dropping work into a queue, and one or more consumers pulling work from the
queue and actually contacting the server, there is little or nothing a
consumer can do to alert the original producer of an add that a server-side
error occurred.  Indeed the consumer is unlikely to be able to do much other
than log the error, hope that is enough, and move on.  In many cases this is
perfectly good enough.  But in cases where the original producer actually
cares there was an error, it's problematic.  I suspect we would just punt
and go for the logging solution, but that then drags in log4j or commons
logging or whatever else you want to use, all of which have the feature that
they are different than what someone who wants to call this very low level
client code is using in the rest of their application. Or you could have a
callback mechanism, and let people write logging callbacks if they want.

I'm not saying we shouldn't do all of this, but merely that we should think
about it and have an actual plan.  I think you will agree that it's
significantly more nuanced than the client code we have so far, which more
or less wrote itself.

FWIW, in my own work on top of DocumentManagerClient I have just set myself
up with one DocumentManagerClient per thread and left it at that.  It won't
work for all applications, but it happens to work for mine at the moment.

-D

-----Original Message-----
From: Yonik Seeley [mailto:[hidden email]]
Sent: Friday, June 02, 2006 10:39 AM
To: [hidden email]
Subject: Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs
for add(), delete(), commit() and optimize().

Thanks for contributing this Darren!

Some thoughts about future directions:
How would someone go about maximizing throughput by using multiple
connections and multiple requests to a Solr server?

There are two cases from the client perspective:
  1) client has a single thread calling add()
  2) client has multiple threads generating documents and calling add()

Interface:
  1) We could have a pool of DocumentManagerClient objects, and create
a *new* interface that accepts updates and dispatches to an available
DocumentManagerClient.
  2) We could make DocumentManagerClient an interface, and have both a
single threaded version and a multi-threaded, multi-connection version
(which could simply use multiple single threaded versions).
  - if we opted for (2), then we should make the single-threaded
version MT safe so the client wouldn't have to know that one
implementation is and one isn't.  Wouldn't be too hard - might need to
get rid of getLastServerException().


Implementation:
  Java5 concurrency classes!

-Yonik

Reply | Threaded
Open this post in threaded view
|

RE: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

Chris Hostetter-3

: implementation super-simple, though arguably it would have been better to
: skip getLastServerException() and just throw an exception on the client side
: when a non-zero server side status came back.  That would improve thread
: safety.

yeah .. if i remember ight,you have a SOlrClientException right/ .. you
could also have a SolrServerException to distinguish the two cases.

: comes to dealing with errors coming back from the server.  In the standard
: async kind of model with one or more producers (those calling add())
: dropping work into a queue, and one or more consumers pulling work from the
: queue and actually contacting the server, there is little or nothing a
: consumer can do to alert the original producer of an add that a server-side
: error occurred.  Indeed the consumer is unlikely to be able to do much other
: than log the error, hope that is enough, and move on.  In many cases this is

1) given enough information about hte nature of hte error, the producer
could choose to reattempt the operation.

2) i've been told that this is the use case the "Future" interface in
Java1.5 attempts to solve, but i've never had a chance (or a serious need)
to play with it.

: FWIW, in my own work on top of DocumentManagerClient I have just set myself
: up with one DocumentManagerClient per thread and left it at that.  It won't
: work for all applications, but it happens to work for mine at the moment.

I think that may be a pretty legitimate approach to take provided your
goal is for DocumentManagerClient to be a "low level" API (whatever that
may mean) ... "high level" APIs can worry about consuming/queing actions
and managing pools of DocumentManagerClient objects. (or something like
that)

The one thing that you may want to incorporate at the low level is
persistent HTTP connections when clients ask DocumentManagerClient to add
multiple documents at the same time.




: -----Original Message-----
: From: Yonik Seeley [mailto:[hidden email]]
: Sent: Friday, June 02, 2006 10:39 AM
: To: [hidden email]
: Subject: Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs
: for add(), delete(), commit() and optimize().
:
: Thanks for contributing this Darren!
:
: Some thoughts about future directions:
: How would someone go about maximizing throughput by using multiple
: connections and multiple requests to a Solr server?
:
: There are two cases from the client perspective:
:   1) client has a single thread calling add()
:   2) client has multiple threads generating documents and calling add()
:
: Interface:
:   1) We could have a pool of DocumentManagerClient objects, and create
: a *new* interface that accepts updates and dispatches to an available
: DocumentManagerClient.
:   2) We could make DocumentManagerClient an interface, and have both a
: single threaded version and a multi-threaded, multi-connection version
: (which could simply use multiple single threaded versions).
:   - if we opted for (2), then we should make the single-threaded
: version MT safe so the client wouldn't have to know that one
: implementation is and one isn't.  Wouldn't be too hard - might need to
: get rid of getLastServerException().
:
:
: Implementation:
:   Java5 concurrency classes!
:
: -Yonik
:



-Hoss

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-20?page=all ]

Darren Erik Vengroff updated SOLR-20:
-------------------------------------

    Attachment: SolrServerException.java

New exception type for reporting server-side exceptions.

> A simple Java client with Java APIs for add(), delete(), commit() and optimize().
> ---------------------------------------------------------------------------------
>
>          Key: SOLR-20
>          URL: http://issues.apache.org/jira/browse/SOLR-20
>      Project: Solr
>         Type: New Feature

>   Components: update
>  Environment: all
>     Reporter: Darren Erik Vengroff
>     Priority: Minor
>  Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java, SolrServerException.java
>
> I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods.  I'm posting here for review and comments as suggested by Yonik.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

12345