solr newbie

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

solr newbie

Tim Archambault
Trying to run the test tutorial to index an xml file and keep getting an
error message: curl: command not found?

Any help is greatly appreciated.
Reply | Threaded
Open this post in threaded view
|

RE: solr newbie

Darren Vengroff-2
You can download curl from http://curl.haxx.se/ if you don't have it on your
machine.

-D

-----Original Message-----
From: Tim Archambault [mailto:[hidden email]]
Sent: Thursday, June 01, 2006 9:42 AM
To: [hidden email]
Subject: solr newbie

Trying to run the test tutorial to index an xml file and keep getting an
error message: curl: command not found?

Any help is greatly appreciated.

Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Yonik Seeley
In reply to this post by Tim Archambault
Hi Tim,

Curl is a little command-line networking tool.  The easiest way to get
it is cygwin if you are not on a UNIX system.

See the 'Requirements" section of the tutorial:
3. On Win32, cygwin, for shell support. (If you plan to use Subversion
on Win32, be sure to select the subversion package when you install,
in the "Devel" category.) This tutorial will assume that "sh" is in
your PATH, and that you have "curl" installed from the "Web" category.

-Yonik

On 6/1/06, Tim Archambault <[hidden email]> wrote:
> Trying to run the test tutorial to index an xml file and keep getting an
> error message: curl: command not found?
>
> Any help is greatly appreciated.
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Tim Archambault-2
Thanks Yonik. All looks good except for the statement: curl installed from
the "Web" category.

Don't understand what "web category" means. SH.

On 6/1/06, Yonik Seeley <[hidden email]> wrote:

>
> Hi Tim,
>
> Curl is a little command-line networking tool.  The easiest way to get
> it is cygwin if you are not on a UNIX system.
>
> See the 'Requirements" section of the tutorial:
> 3. On Win32, cygwin, for shell support. (If you plan to use Subversion
> on Win32, be sure to select the subversion package when you install,
> in the "Devel" category.) This tutorial will assume that "sh" is in
> your PATH, and that you have "curl" installed from the "Web" category.
>
> -Yonik
>
> On 6/1/06, Tim Archambault <[hidden email]> wrote:
> > Trying to run the test tutorial to index an xml file and keep getting an
> > error message: curl: command not found?
> >
> > Any help is greatly appreciated.
>
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Yonik Seeley
On 6/1/06, Tim Archambault <[hidden email]> wrote:
> Don't understand what "web category" means. SH.

The cygwin installer has different categories of packages... "base","devel",etc.
If you are looking for the curl package, it should be filed under
"web".  It's not installed by default, so you need to select it.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Tim Archambault-2
I'll need to install cygwin again I think. Thanks.


On 6/1/06, Yonik Seeley <[hidden email]> wrote:

>
> On 6/1/06, Tim Archambault <[hidden email]> wrote:
> > Don't understand what "web category" means. SH.
>
> The cygwin installer has different categories of packages...
> "base","devel",etc.
> If you are looking for the curl package, it should be filed under
> "web".  It's not installed by default, so you need to select it.
>
> -Yonik
>
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Yonik Seeley
On 6/1/06, Tim Archambault <[hidden email]> wrote:
> I'll need to install cygwin again I think. Thanks.

Don't uninstall cygwin... just re-run the cygwin setup.exe and it will
do incremental updates, installing packages that have changed, and
allowing you to select new packages to install.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Tim Archambault-2
I found the "web" options. Thank you very much. While that is installing
incrementally, two last questions.

Are there any example stylesheets to review to see how the data flows into
the layout?
How would one go about injecting database information into the indexs
without having to create XML files for each one?

Thanks again.

Tim

On 6/1/06, Yonik Seeley <[hidden email]> wrote:

>
> On 6/1/06, Tim Archambault <[hidden email]> wrote:
> > I'll need to install cygwin again I think. Thanks.
>
> Don't uninstall cygwin... just re-run the cygwin setup.exe and it will
> do incremental updates, installing packages that have changed, and
> allowing you to select new packages to install.
>
> -Yonik
>
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Yonik Seeley
On 6/1/06, Tim Archambault <[hidden email]> wrote:
> I found the "web" options. Thank you very much. While that is installing
> incrementally, two last questions.
>
> Are there any example stylesheets to review to see how the data flows into
> the layout?
> How would one go about injecting database information into the indexs
> without having to create XML files for each one?

It's most efficient to make a "builder" application that reads from
the database, constructs XML documents *in memory* and sends them to
the Solr server.  Multiple threads/connections open to the Solr server
will speed up indexing and hide any request-response latency of
individual adds.

We don't have it yet, but there really should be a simple Java client
library that creates the XML add commands and handles sending them to
the server.

Also on the "todo" list is indexing directly from a SQL database w/o
the user having to write any code except select statements.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Tim Archambault-2
Great thanks. I manage a newspaper website in Maine USA with about
400,000-500,000 documents/database records (if not more) and I am going to
try and create a solr search engine for the site. We'll see how it goes.
I've been using a "bastardized" lucene search for my site up to now, but
this looks much better.

On 6/1/06, Yonik Seeley <[hidden email]> wrote:

>
> On 6/1/06, Tim Archambault <[hidden email]> wrote:
> > I found the "web" options. Thank you very much. While that is installing
> > incrementally, two last questions.
> >
> > Are there any example stylesheets to review to see how the data flows
> into
> > the layout?
> > How would one go about injecting database information into the indexs
> > without having to create XML files for each one?
>
> It's most efficient to make a "builder" application that reads from
> the database, constructs XML documents *in memory* and sends them to
> the Solr server.  Multiple threads/connections open to the Solr server
> will speed up indexing and hide any request-response latency of
> individual adds.
>
> We don't have it yet, but there really should be a simple Java client
> library that creates the XML add commands and handles sending them to
> the server.
>
> Also on the "todo" list is indexing directly from a SQL database w/o
> the user having to write any code except select statements.
>
> -Yonik
>
Reply | Threaded
Open this post in threaded view
|

RE: solr newbie

Darren Vengroff-2
In reply to this post by Yonik Seeley
I wrote just such a client within the last 24h to support load-testing Solr
for my application.  The client stub is simple and independent of my
particular application, so it would be easy for me to contribute it if there
is interest.  It has methods to add() a document or collection of documents,
and commit() and optimize().

-D

-----Original Message-----
From: Yonik Seeley [mailto:[hidden email]]
Sent: Thursday, June 01, 2006 10:44 AM
To: [hidden email]
Subject: Re: solr newbie

We don't have it yet, but there really should be a simple Java client
library that creates the XML add commands and handles sending them to
the server.

Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Yonik Seeley
On 6/2/06, Darren Vengroff <[hidden email]> wrote:
> I wrote just such a client within the last 24h to support load-testing Solr
> for my application.  The client stub is simple and independent of my
> particular application, so it would be easy for me to contribute it if there
> is interest.  It has methods to add() a document or collection of documents,
> and commit() and optimize().

It would be great to see what you have!
If you would like to contribute it, or get feedback on the API, please
open a new JIRA bug (feature) and add the code there.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: solr newbie

Raghavendra Prabhu
In reply to this post by Darren Vengroff-2
I would definitely like to use the code to run the same set of tests with my
application

Regards,
Prabhu


On 6/2/06, Darren Vengroff <[hidden email]> wrote:

>
> I wrote just such a client within the last 24h to support load-testing
> Solr
> for my application.  The client stub is simple and independent of my
> particular application, so it would be easy for me to contribute it if
> there
> is interest.  It has methods to add() a document or collection of
> documents,
> and commit() and optimize().
>
> -D
>
> -----Original Message-----
> From: Yonik Seeley [mailto:[hidden email]]
> Sent: Thursday, June 01, 2006 10:44 AM
> To: [hidden email]
> Subject: Re: solr newbie
>
> We don't have it yet, but there really should be a simple Java client
> library that creates the XML add commands and handles sending them to
> the server.
>
>
Reply | Threaded
Open this post in threaded view
|

RE: solr newbie

Darren Vengroff-2
In reply to this post by Yonik Seeley
See http://issues.apache.org/jira/browse/SOLR-20.

-D

-----Original Message-----
From: Yonik Seeley [mailto:[hidden email]]
Sent: Thursday, June 01, 2006 10:15 PM
To: [hidden email]
Subject: Re: solr newbie

On 6/2/06, Darren Vengroff <[hidden email]> wrote:
> I wrote just such a client within the last 24h to support load-testing
Solr
> for my application.  The client stub is simple and independent of my
> particular application, so it would be easy for me to contribute it if
there
> is interest.  It has methods to add() a document or collection of
documents,
> and commit() and optimize().

It would be great to see what you have!
If you would like to contribute it, or get feedback on the API, please
open a new JIRA bug (feature) and add the code there.

-Yonik