Does Solr Have?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Does Solr Have?

Robert Young-5
Hi,

We're just about to start work on a project in Solr and there are a
couple of points which I haven't been able to find out from the wiki
which I'm interested in.

1. Is there a REST interface for getting index stats? I would
particularly like access to terms and their document frequencies,
prefereably filtered by a query.

2. Is it possible to use different synonym sets for different queries
OR is it possible to search across multiple indexes with a single
query?

3. Is it possible to change stopword and synonym sets at runtime?

I'm sure I'll have lots more questions as time goes by and, hopefully,
I'll be able to answer others' questions in the future.

Thanks
Rob
Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Erik Hatcher

On Oct 4, 2007, at 4:38 AM, Robert Young wrote:
> 1. Is there a REST interface for getting index stats? I would
> particularly like access to terms and their document frequencies,
> prefereably filtered by a query.

Yes, the Luke request handler provides deeper views into the index  
information:

   <http://wiki.apache.org/solr/LukeRequestHandler>

> 2. Is it possible to use different synonym sets for different queries

Not exactly, but it possible to have different synonym configurations  
for different field types configured in the schema.

> OR is it possible to search across multiple indexes with a single
> query?

Not with the current version of Solr, but there is a  
"federated" ("distributed" is a better term, I think) Solr patch in  
JIRA.

> 3. Is it possible to change stopword and synonym sets at runtime?

Only if the underlying text file is changed.

> I'm sure I'll have lots more questions as time goes by and, hopefully,
> I'll be able to answer others' questions in the future.

Welcome!

        Erik

Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Robert Young-5
Brilliant, thank you, that LukeRequestHandler looks very useful.

On 10/4/07, Erik Hatcher <[hidden email]> wrote:
> > 3. Is it possible to change stopword and synonym sets at runtime?
>
> Only if the underlying text file is changed.

Will Solr automatically reload the file if it changes or does it have
to be informed of the change? Is changing the underlying file while
Solr is running dangerous?

Cheers
Rob
Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Erik Hatcher

On Oct 4, 2007, at 6:10 AM, Robert Young wrote:

> Brilliant, thank you, that LukeRequestHandler looks very useful.
>
> On 10/4/07, Erik Hatcher <[hidden email]> wrote:
>>> 3. Is it possible to change stopword and synonym sets at runtime?
>>
>> Only if the underlying text file is changed.
>
> Will Solr automatically reload the file if it changes or does it have
> to be informed of the change?

I'll expose my confusion here and say that I don't know for sure, but  
I'm pretty sure that once it's been loaded it won't get reloaded  
without bouncing Solr altogether.  solr-dev's, please correct me with  
some code pointers if I'm wrong.  In my brief IntelliJ surfing this  
morning, it seems that once a SolrCore has been instantiated, that  
instance sticks around forever, and it holds the IndexSchema instance.

I'm guessing that the new multi-core stuff makes this a bit more  
dynamically controllable.

> Is changing the underlying file while
> Solr is running dangerous?

No - it only reads it once as far as I can tell.

Caveat to all technical details above: I'm still learning the ropes  
with the core of Solr so if I misspoke on any of this lifecycle  
stuff, let me know.  By trying to answer questions, though, I learn  
quicker :)

        Erik



Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Ryan McKinley
In reply to this post by Robert Young-5
Robert Young wrote:

> Hi,
>
> We're just about to start work on a project in Solr and there are a
> couple of points which I haven't been able to find out from the wiki
> which I'm interested in.
>
> 1. Is there a REST interface for getting index stats? I would
> particularly like access to terms and their document frequencies,
> prefereably filtered by a query.
>

have you checked the luke request handler?
http://wiki.apache.org/solr/LukeRequestHandler


> 2. Is it possible to use different synonym sets for different queries

the synonym's are linked to each field -- if differnt queires access
different fields, it will use differnt synonyms.

To automatically index things with different field types, check the
<copyField ...> stuff.

> OR is it possible to search across multiple indexes with a single
> query?
>

not currently.


> 3. Is it possible to change stopword and synonym sets at runtime?
>

By default no - but it is not hard to write a custom field type that
would.  (I have one that loads synonyms from a SQL table -- it can be
updated dynamically at runtime)


> I'm sure I'll have lots more questions as time goes by and, hopefully,
> I'll be able to answer others' questions in the future.
>

great!

ryan

Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Ryan McKinley
In reply to this post by Erik Hatcher
dooh, should check all my email first!

>>
>> Will Solr automatically reload the file if it changes or does it have
>> to be informed of the change?
>
> I'll expose my confusion here and say that I don't know for sure, but
> I'm pretty sure that once it's been loaded it won't get reloaded without
> bouncing Solr altogether.  

Correct.  The StopFilterFactory is initialized at startup, any changes
to the file won't take effect 'till solr restarts.

but you can write a custom FilterFactory based on StopFilterFactory that
lets you change it dynamically.  Most likely this would also require
writing a custom RequestHandler to manipulate it.

note - changing the stop words at runtime will only effect queries, the
index will keep whatever was there at index time.

ryan
Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Robert Young-5
Is there, or are there plans to start, a plugin and extension repository?

Cheers
Rob

On 10/4/07, Ryan McKinley <[hidden email]> wrote:

> dooh, should check all my email first!
>
> >>
> >> Will Solr automatically reload the file if it changes or does it have
> >> to be informed of the change?
> >
> > I'll expose my confusion here and say that I don't know for sure, but
> > I'm pretty sure that once it's been loaded it won't get reloaded without
> > bouncing Solr altogether.
>
> Correct.  The StopFilterFactory is initialized at startup, any changes
> to the file won't take effect 'till solr restarts.
>
> but you can write a custom FilterFactory based on StopFilterFactory that
> lets you change it dynamically.  Most likely this would also require
> writing a custom RequestHandler to manipulate it.
>
> note - changing the stop words at runtime will only effect queries, the
> index will keep whatever was there at index time.
>
> ryan
>
Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Matthew Runo
In reply to this post by Ryan McKinley
How does one set up the LukeRequestHandler? I didn't see a document  
in the wiki about how to add new handlers, and my install (a 1.1  
install upgraded to 1.2) does not have this handler available.

I'd like to see what we're talking about, it sounds very  
interesting.. but I can't find how to turn on this request handler.

+--------------------------------------------------------+
  | Matthew Runo
  | Zappos Development
  | [hidden email]
  | 702-943-7833
+--------------------------------------------------------+


On Oct 4, 2007, at 6:11 AM, Ryan McKinley wrote:

> Robert Young wrote:
>> Hi,
>> We're just about to start work on a project in Solr and there are a
>> couple of points which I haven't been able to find out from the wiki
>> which I'm interested in.
>> 1. Is there a REST interface for getting index stats? I would
>> particularly like access to terms and their document frequencies,
>> prefereably filtered by a query.
>
> have you checked the luke request handler?
> http://wiki.apache.org/solr/LukeRequestHandler
>
>
>> 2. Is it possible to use different synonym sets for different queries
>
> the synonym's are linked to each field -- if differnt queires  
> access different fields, it will use differnt synonyms.
>
> To automatically index things with different field types, check the  
> <copyField ...> stuff.
>
>> OR is it possible to search across multiple indexes with a single
>> query?
>
> not currently.
>
>
>> 3. Is it possible to change stopword and synonym sets at runtime?
>
> By default no - but it is not hard to write a custom field type  
> that would.  (I have one that loads synonyms from a SQL table -- it  
> can be updated dynamically at runtime)
>
>
>> I'm sure I'll have lots more questions as time goes by and,  
>> hopefully,
>> I'll be able to answer others' questions in the future.
>
> great!
>
> ryan
>

Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Ryan McKinley
add:
   <requestHandler name="/admin/luke"
class="org.apache.solr.handler.admin.LukeRequestHandler" />

to your solrconfig.xml

It is in the example solrconfig.xml that comes with 1.2


Matthew Runo wrote:

> How does one set up the LukeRequestHandler? I didn't see a document in
> the wiki about how to add new handlers, and my install (a 1.1 install
> upgraded to 1.2) does not have this handler available.
>
> I'd like to see what we're talking about, it sounds very interesting..
> but I can't find how to turn on this request handler.
>
> +--------------------------------------------------------+
>  | Matthew Runo
>  | Zappos Development
>  | [hidden email]
>  | 702-943-7833
> +--------------------------------------------------------+
>
>
> On Oct 4, 2007, at 6:11 AM, Ryan McKinley wrote:
>
>> Robert Young wrote:
>>> Hi,
>>> We're just about to start work on a project in Solr and there are a
>>> couple of points which I haven't been able to find out from the wiki
>>> which I'm interested in.
>>> 1. Is there a REST interface for getting index stats? I would
>>> particularly like access to terms and their document frequencies,
>>> prefereably filtered by a query.
>>
>> have you checked the luke request handler?
>> http://wiki.apache.org/solr/LukeRequestHandler
>>
>>
>>> 2. Is it possible to use different synonym sets for different queries
>>
>> the synonym's are linked to each field -- if differnt queires access
>> different fields, it will use differnt synonyms.
>>
>> To automatically index things with different field types, check the
>> <copyField ...> stuff.
>>
>>> OR is it possible to search across multiple indexes with a single
>>> query?
>>
>> not currently.
>>
>>
>>> 3. Is it possible to change stopword and synonym sets at runtime?
>>
>> By default no - but it is not hard to write a custom field type that
>> would.  (I have one that loads synonyms from a SQL table -- it can be
>> updated dynamically at runtime)
>>
>>
>>> I'm sure I'll have lots more questions as time goes by and, hopefully,
>>> I'll be able to answer others' questions in the future.
>>
>> great!
>>
>> ryan
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

Matthew Runo
Boo, thank you for the reply. That's what I get for customizing it  
and taking out all the other code I guess. Sorry about that.

+--------------------------------------------------------+
  | Matthew Runo
  | Zappos Development
  | [hidden email]
  | 702-943-7833
+--------------------------------------------------------+


On Oct 4, 2007, at 9:47 AM, Ryan McKinley wrote:

> add:
>   <requestHandler name="/admin/luke"  
> class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
> to your solrconfig.xml
>
> It is in the example solrconfig.xml that comes with 1.2
>
>
> Matthew Runo wrote:
>> How does one set up the LukeRequestHandler? I didn't see a  
>> document in the wiki about how to add new handlers, and my install  
>> (a 1.1 install upgraded to 1.2) does not have this handler available.
>> I'd like to see what we're talking about, it sounds very  
>> interesting.. but I can't find how to turn on this request handler.
>> +--------------------------------------------------------+
>>  | Matthew Runo
>>  | Zappos Development
>>  | [hidden email]
>>  | 702-943-7833
>> +--------------------------------------------------------+
>> On Oct 4, 2007, at 6:11 AM, Ryan McKinley wrote:
>>> Robert Young wrote:
>>>> Hi,
>>>> We're just about to start work on a project in Solr and there are a
>>>> couple of points which I haven't been able to find out from the  
>>>> wiki
>>>> which I'm interested in.
>>>> 1. Is there a REST interface for getting index stats? I would
>>>> particularly like access to terms and their document frequencies,
>>>> prefereably filtered by a query.
>>>
>>> have you checked the luke request handler?
>>> http://wiki.apache.org/solr/LukeRequestHandler
>>>
>>>
>>>> 2. Is it possible to use different synonym sets for different  
>>>> queries
>>>
>>> the synonym's are linked to each field -- if differnt queires  
>>> access different fields, it will use differnt synonyms.
>>>
>>> To automatically index things with different field types, check  
>>> the <copyField ...> stuff.
>>>
>>>> OR is it possible to search across multiple indexes with a single
>>>> query?
>>>
>>> not currently.
>>>
>>>
>>>> 3. Is it possible to change stopword and synonym sets at runtime?
>>>
>>> By default no - but it is not hard to write a custom field type  
>>> that would.  (I have one that loads synonyms from a SQL table --  
>>> it can be updated dynamically at runtime)
>>>
>>>
>>>> I'm sure I'll have lots more questions as time goes by and,  
>>>> hopefully,
>>>> I'll be able to answer others' questions in the future.
>>>
>>> great!
>>>
>>> ryan
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Does Solr Have?

hossman
In reply to this post by Robert Young-5

: Is there, or are there plans to start, a plugin and extension repository?

Strictly speaking, there is no reason why Solr Plugins would have to live
in an Apache repository.  if people write plugins that would be generally
usefull to several people, and they wish to contribute them to Apache,
they can be commited into the Solr repository -- but people could also
host/distribute a plugin on some other site (sourceforge, etc...) however
they see fit (using pretty much any license they see fit, i believe).  If
people start doing that, then we should definitely have anice wiki page
for people to list those things.

Note: there was another thread recently on solr-dev about adding a
"contrib" section to the solr repository and bundling them in individual
jars as an alternative to including them in the core solr.war...

http://www.nabble.com/Solr-contrib-tf4517606.html#a12886773



-Hoss