Solr Flare

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr Flare

Erik Hatcher
I'm finally kicking off that long dreamed of Ruby/Solr DSL, including  
an accompanying general purpose web interface to showcase the  
greatness of Solr's capabilities, including faceted browsing.  I've  
titled the project Flare, and have committed the bits of low-level  
code into Solr's repository.  The gory details are on the wiki, here:

        <http://wiki.apache.org/solr/Flare>

I'm dedicating the next couple of months of spare time to this in  
order to have it ready for a couple of big projects, one is to  
demonstrate a faceted browsing front-end on our libraries holdings  
(~3.7M records), and also to have a framework for even more easily  
demonstrating and teaching Lucene and Solr to a bunch of library geeks
[1].

For the Ruby-savvy of you out there, I'd love to have help in  
crafting a Ruby Solr DSL suitable for "gem install", and fleshing out  
a general purpose API and UI.  As for collaboration, it'll be via  
JIRA patches, wiki, and this e-mail list, and we'll go from there.  
Thanks in advance!

        Erik

[1] http://www.code4lib.org/node/139
Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Yonik Seeley-2
Wow, awesome way to kick of the new year!
You may end up converting me to Ruby while you're at it ;-)

-Yonik

On 1/2/07, Erik Hatcher <[hidden email]> wrote:

> I'm finally kicking off that long dreamed of Ruby/Solr DSL, including
> an accompanying general purpose web interface to showcase the
> greatness of Solr's capabilities, including faceted browsing.  I've
> titled the project Flare, and have committed the bits of low-level
> code into Solr's repository.  The gory details are on the wiki, here:
>
>         <http://wiki.apache.org/solr/Flare>
>
> I'm dedicating the next couple of months of spare time to this in
> order to have it ready for a couple of big projects, one is to
> demonstrate a faceted browsing front-end on our libraries holdings
> (~3.7M records), and also to have a framework for even more easily
> demonstrating and teaching Lucene and Solr to a bunch of library geeks
> [1].
>
> For the Ruby-savvy of you out there, I'd love to have help in
> crafting a Ruby Solr DSL suitable for "gem install", and fleshing out
> a general purpose API and UI.  As for collaboration, it'll be via
> JIRA patches, wiki, and this e-mail list, and we'll go from there.
> Thanks in advance!
>
>         Erik
>
> [1] http://www.code4lib.org/node/139
Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Erik Hatcher

On Jan 2, 2007, at 4:15 PM, Yonik Seeley wrote:
> Wow, awesome way to kick of the new year!

All part of my plan for world dom^H^H^H peace!

> You may end up converting me to Ruby while you're at it ;-)

Ruby is lots of fun... and with the goodness of Solr behind it, the  
UI experience will be truly amazing.  The success of Flare will  
change the face of a lot of library systems in the world, at the very  
least.

On a more pragmatic level, I'd love for you and others to review how  
I've put the ASL in some of the files, but not all.  I'm not sure if  
generated code (by the Rails application generator) needs to have the  
ASL attached to it, or how this sort of thing should be handled.

Thanks!
        Erik


>
> -Yonik
>
> On 1/2/07, Erik Hatcher <[hidden email]> wrote:
>> I'm finally kicking off that long dreamed of Ruby/Solr DSL, including
>> an accompanying general purpose web interface to showcase the
>> greatness of Solr's capabilities, including faceted browsing.  I've
>> titled the project Flare, and have committed the bits of low-level
>> code into Solr's repository.  The gory details are on the wiki, here:
>>
>>         <http://wiki.apache.org/solr/Flare>
>>
>> I'm dedicating the next couple of months of spare time to this in
>> order to have it ready for a couple of big projects, one is to
>> demonstrate a faceted browsing front-end on our libraries holdings
>> (~3.7M records), and also to have a framework for even more easily
>> demonstrating and teaching Lucene and Solr to a bunch of library  
>> geeks
>> [1].
>>
>> For the Ruby-savvy of you out there, I'd love to have help in
>> crafting a Ruby Solr DSL suitable for "gem install", and fleshing out
>> a general purpose API and UI.  As for collaboration, it'll be via
>> JIRA patches, wiki, and this e-mail list, and we'll go from there.
>> Thanks in advance!
>>
>>         Erik
>>
>> [1] http://www.code4lib.org/node/139

Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Zaheed Haque
Great! Wow!

What are the difference between flare and act_as_solr as of today?

I been playing around with ruby/rails for about a month and its fun!
my thoughts were (i.e. after my playing around period) to create a
frontend presentation framework and seems like you have already put
the project into motion with flare :-) and it will be nice to
collaborate and contribute.

As you have already set things into motion I wonder if you have
thought about how you would like to execute the presentation
framework.. i.e do you have any frontend  rails app that I can start
hacking away... Bottom line I am trying to figure out where and how
should I start... I have gone through the flare wiki but I am bit
confused cos script/server webrick to localhost:3000 gives me "link to
wiki" and app/controller/ doesn't have any code as of todays checkout
so I am not sure what I am missing from the wiki todo list... Maybe I
am completely lost :=)

Are you planning to use the Solr Jira for issues? or will flare have
its own jira?

Cheers
Zaheed

On 1/2/07, Erik Hatcher <[hidden email]> wrote:

>
> On Jan 2, 2007, at 4:15 PM, Yonik Seeley wrote:
> > Wow, awesome way to kick of the new year!
>
> All part of my plan for world dom^H^H^H peace!
>
> > You may end up converting me to Ruby while you're at it ;-)
>
> Ruby is lots of fun... and with the goodness of Solr behind it, the
> UI experience will be truly amazing.  The success of Flare will
> change the face of a lot of library systems in the world, at the very
> least.
>
> On a more pragmatic level, I'd love for you and others to review how
> I've put the ASL in some of the files, but not all.  I'm not sure if
> generated code (by the Rails application generator) needs to have the
> ASL attached to it, or how this sort of thing should be handled.
>
> Thanks!
>         Erik
>
>
> >
> > -Yonik
> >
> > On 1/2/07, Erik Hatcher <[hidden email]> wrote:
> >> I'm finally kicking off that long dreamed of Ruby/Solr DSL, including
> >> an accompanying general purpose web interface to showcase the
> >> greatness of Solr's capabilities, including faceted browsing.  I've
> >> titled the project Flare, and have committed the bits of low-level
> >> code into Solr's repository.  The gory details are on the wiki, here:
> >>
> >>         <http://wiki.apache.org/solr/Flare>
> >>
> >> I'm dedicating the next couple of months of spare time to this in
> >> order to have it ready for a couple of big projects, one is to
> >> demonstrate a faceted browsing front-end on our libraries holdings
> >> (~3.7M records), and also to have a framework for even more easily
> >> demonstrating and teaching Lucene and Solr to a bunch of library
> >> geeks
> >> [1].
> >>
> >> For the Ruby-savvy of you out there, I'd love to have help in
> >> crafting a Ruby Solr DSL suitable for "gem install", and fleshing out
> >> a general purpose API and UI.  As for collaboration, it'll be via
> >> JIRA patches, wiki, and this e-mail list, and we'll go from there.
> >> Thanks in advance!
> >>
> >>         Erik
> >>
> >> [1] http://www.code4lib.org/node/139
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Erik Hatcher

On Jan 2, 2007, at 5:23 PM, Zaheed Haque wrote:
> Great! Wow!

Well, as you've noted, it's still in its infancy.  Being impressed  
will come a few weeks later, at least :)

> What are the difference between flare and act_as_solr as of today?

As of today, tons... since acts_as_solr actually does something  
useful, whereas Flare is just some basic Solr communication wrapping,  
and syntactic sugar coating on making the requests.

In the future, perhaps acts_as_solr will be something Flare does as  
well, though mapping ActiveRecord objects to Solr is not currently  
something of strong interest to me personally.  With a slick enough  
Solr DSL, acts_as_solr could leverage it under the covers rather than  
doing the HTTP communication directly, or perhaps Flare could take  
over as the predominant way folks would wire Solr and ActiveRecord  
objects together?  I would at least like to see the lower-level Solr  
clients glue code to be maintained along with the Solr project, so  
things can be kept in sync and distributed together eventually even.

> I been playing around with ruby/rails for about a month and its fun!
> my thoughts were (i.e. after my playing around period) to create a
> frontend presentation framework and seems like you have already put
> the project into motion with flare :-) and it will be nice to
> collaborate and contribute.

Excellent!

> As you have already set things into motion I wonder if you have
> thought about how you would like to execute the presentation
> framework.. i.e do you have any frontend  rails app that I can start
> hacking away...

No, the Rails application isn't in place yet - that is going to  
evolve over the next few weeks.  Jump in, I need your help!


> Bottom line I am trying to figure out where and how
> should I start... I have gone through the flare wiki but I am bit
> confused cos script/server webrick to localhost:3000 gives me "link to
> wiki" and app/controller/ doesn't have any code as of todays checkout
> so I am not sure what I am missing from the wiki todo list... Maybe I
> am completely lost :=)

You're not lost at all... you've got the gist of it thus far.  My  
focus was to get some basic Solr API going, and build up the UI from  
there, in a strongly test-centric fashion.

> Are you planning to use the Solr Jira for issues? or will flare have
> its own jira?

We'll use the Solr JIRA.  Thanks for mentioning this - I just added a  
JIRA component called "clients - ruby -flare" under the Solr project,  
so we can file issues there.

        Erik


Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Erik Hatcher
In reply to this post by Erik Hatcher
Based on a chat with an early adopter of Flare, we decided to split  
out the Ruby-Solr DSL sooner rather than later.  I've pulled out the  
library code into a separate directory under client/ruby/solrb ("rb"  
being the standard Ruby file extension).  Info for this library is here:

        <http://wiki.apache.org/solr/solrb>

Flare will be the user interface that leverages solrb to communicate  
with Solr, though currently the Flare code is just an empty Rails  
generated application.  It'll start being useful very soon.  There  
are quite a few folks already dipping into this, so this all should  
evolve fairly quickly over the next few weeks.

        Erik



>> On 1/2/07, Erik Hatcher <[hidden email]> wrote:
>>> I'm finally kicking off that long dreamed of Ruby/Solr DSL,  
>>> including
>>> an accompanying general purpose web interface to showcase the
>>> greatness of Solr's capabilities, including faceted browsing.  I've
>>> titled the project Flare, and have committed the bits of low-level
>>> code into Solr's repository.  The gory details are on the wiki,  
>>> here:
>>>
>>>         <http://wiki.apache.org/solr/Flare>
>>>
>>> I'm dedicating the next couple of months of spare time to this in
>>> order to have it ready for a couple of big projects, one is to
>>> demonstrate a faceted browsing front-end on our libraries holdings
>>> (~3.7M records), and also to have a framework for even more easily
>>> demonstrating and teaching Lucene and Solr to a bunch of library  
>>> geeks
>>> [1].
>>>
>>> For the Ruby-savvy of you out there, I'd love to have help in
>>> crafting a Ruby Solr DSL suitable for "gem install", and fleshing  
>>> out
>>> a general purpose API and UI.  As for collaboration, it'll be via
>>> JIRA patches, wiki, and this e-mail list, and we'll go from there.
>>> Thanks in advance!
>>>
>>>         Erik
>>>
>>> [1] http://www.code4lib.org/node/139

Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Zaheed Haque
Erik:

Thank you for your answer. I think I will be focusing more on the
rails application part. Some random thoughts and questions for
brainstorming .. Some of this questions are really way down .. but I
am trying to understand your roadmap rather ..

1. Are you thinking of any user/password/my page/settings what not...
for folksonomy/tagging etc..? i.e. user management/login logout etc?
or you think the end provider should do this customization after
downloading the app

2. Will the rails app be language independent? i.e. using plugins like
globalize, localize etc?

3. Have you given any thought regarding Erubis an alternative to ERB
.. Its a matter of speed..

http://www.kuwata-lab.com/erubis/

4. Also are you thinking of any admin tool like (streamlined
http://streamlined.relevancellc.com/ ) for managing the UI
applications?

The above is actually somewhat true for any application on the web.. I
just want to know will flare address the above .. or are you thinking
that  the "End developer i.e. one who will download flare"  will do
the above.. Or I guess better question would be "Will flare be a
complete download and go app" or somewhat "app that needs modification
further" :=).. of course anyone can customize anything its open..

I appreciate your feedback... I  know I have more questions.. :=)  if
you prefer I can send you mail directly..

Regards
Zaheed.
Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Erik Hatcher
Zaheed,

All great questions.  My answers will be vague, since I'm still in  
the early stages of how this will work, but I'll provide what I can  
below....

On Jan 3, 2007, at 2:28 PM, Zaheed Haque wrote:
> 1. Are you thinking of any user/password/my page/settings what not...
> for folksonomy/tagging etc..? i.e. user management/login logout etc?
> or you think the end provider should do this customization after
> downloading the app

First the application will be purely read-only, as the search and  
facet views get refined.  I see the tagging/annotating feature added  
much later, after the read-only stuff is good enough.

But, definitely, for tagging there will need to be user management  
features, a relational database, and various customizations.  How we  
build this into Flare remains to be seen.  I view the Flare  
application as a starting point, with the core functionality made as  
Rails plugins that could be added to any Rails application.

> 2. Will the rails app be language independent? i.e. using plugins like
> globalize, localize etc?

That's certainly a laudable goal.  So yeah, i18n-savvy would make sense.

> 3. Have you given any thought regarding Erubis an alternative to ERB
> .. Its a matter of speed..
>
> http://www.kuwata-lab.com/erubis/

This would be a later optimization.  I've not had any problems with  
ERB itself, so pulling in Erubis doesn't seem an immediate need.  
Again, Flare the web application will be a good starting point for  
folks that don't want to build a front-end themselves, but putting  
the functionality into plugins would allow folks to plug in the  
features into an existing Rails application.

We also have to consider the Apache Software License compatibility  
with any other libraries we pull in, at least for it to be in our  
repository.

> 4. Also are you thinking of any admin tool like (streamlined
> http://streamlined.relevancellc.com/ ) for managing the UI
> applications?

Streamlined would make since for the administrative interface, I  
think.  I'm good pals with Stu and Justin, and will be seeing them  
later this month.  I'll discuss it with them then.

> The above is actually somewhat true for any application on the web.. I
> just want to know will flare address the above .. or are you thinking
> that  the "End developer i.e. one who will download flare"  will do
> the above.. Or I guess better question would be "Will flare be a
> complete download and go app" or somewhat "app that needs modification
> further" :=).. of course anyone can customize anything its open..

I think I covered this above.  We'll have to all see how it evolves.  
I don't have it all planned out in my head, and that is one reason I  
kick started this publicly to solicit the collective expertise of the  
Solr/Ruby community.

> I appreciate your feedback... I  know I have more questions.. :=)  if
> you prefer I can send you mail directly..

Let's keep it here for all to see and participate in.

Thanks for your interest and questions.  I look forward to us all  
making solrb/Flare be the stuff we can't live without!

        Erik

Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Mel Riffe
In reply to this post by Erik Hatcher
Hi Erik (and others),

I would like to help with this effort. But I want to make sure I understand the landscape (forgive my noobness):

Solr Flare is actual two pieces of software: 1) a Rails application [flare] that uses 2) a Ruby DSL [solrb] to communicate with a Solr Server.  Is this correct?

Plus, the current state/status of Solr Flare is that the UI is yet to be developed but the communication piece works.  Is this also correct?

What work is next?

Thanks,
Mel Riffe

Erik Hatcher wrote
Based on a chat with an early adopter of Flare, we decided to split  
out the Ruby-Solr DSL sooner rather than later.  I've pulled out the  
library code into a separate directory under client/ruby/solrb ("rb"  
being the standard Ruby file extension).  Info for this library is here:

        <http://wiki.apache.org/solr/solrb>

Flare will be the user interface that leverages solrb to communicate  
with Solr, though currently the Flare code is just an empty Rails  
generated application.  It'll start being useful very soon.  There  
are quite a few folks already dipping into this, so this all should  
evolve fairly quickly over the next few weeks.

        Erik

>> On 1/2/07, Erik Hatcher <erik@ehatchersolutions.com> wrote:
>>> I'm finally kicking off that long dreamed of Ruby/Solr DSL,  
>>> including
>>> an accompanying general purpose web interface to showcase the
>>> greatness of Solr's capabilities, including faceted browsing.  I've
>>> titled the project Flare, and have committed the bits of low-level
>>> code into Solr's repository.  The gory details are on the wiki,  
>>> here:
>>>
>>>         <http://wiki.apache.org/solr/Flare>
>>>
>>> I'm dedicating the next couple of months of spare time to this in
>>> order to have it ready for a couple of big projects, one is to
>>> demonstrate a faceted browsing front-end on our libraries holdings
>>> (~3.7M records), and also to have a framework for even more easily
>>> demonstrating and teaching Lucene and Solr to a bunch of library  
>>> geeks
>>> [1].
>>>
>>> For the Ruby-savvy of you out there, I'd love to have help in
>>> crafting a Ruby Solr DSL suitable for "gem install", and fleshing  
>>> out
>>> a general purpose API and UI.  As for collaboration, it'll be via
>>> JIRA patches, wiki, and this e-mail list, and we'll go from there.
>>> Thanks in advance!
>>>
>>>         Erik
>>>
>>> [1] http://www.code4lib.org/node/139
Reply | Threaded
Open this post in threaded view
|

Re: Solr Flare

Erik Hatcher
Mel,

Welcome!

On Jan 11, 2007, at 12:41 AM, Mel Riffe wrote:
> Solr Flare is actual two pieces of software: 1) a Rails application  
> [flare]
> that uses 2) a Ruby DSL [solrb] to communicate with a Solr Server.  
> Is this
> correct?

That is correct.  I'm envisioning Flare being more about Rails  
plugins than the Rails application itself, so others could easily  
incorporate Solr into their own Rails applications.  However, I do  
think a general purpose web application will be built up around these  
plugins that folks can simply point at Solr, configure a few things  
(but maybe not even that), and voila, an search interface is up and  
running.

> Plus, the current state/status of Solr Flare is that the UI is yet  
> to be
> developed but the communication piece works.  Is this also correct?

That is correct.  I'm focusing on getting the UI up and running in  
prototypical stages by the end of this weekend, so others can give it  
a whirl.  solrb, the communication stuff, is working as is, but needs  
work to handle the various query types.

> What work is next?

solrb's ToDo is here: http://wiki.apache.org/solr/solrb/ToDo
Flare's is here: http://wiki.apache.org/solr/Flare/ToDo

        Erik