SOC: Lulu, a Lua implementation of Lucene

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2
A proposal for a Lua entry for the "Google Summer of Code" '08:

lu·lu (lū'lū) n. Slang.

A remarkable person, object, or idea.

A very attractive or seductive looking woman.

A Lua implementation of Lucene.


Skimpy details bellow:

http://svr225.stepx.com:3388/lulu
http://lua-users.org/wiki/GoogleSummerOfCodeIdeas


Feel free to drop by the Lua mailing list if you are interested.

http://www.lua.org/lua-l.html


Cheers,

PA.

 
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Mathieu Lecarme
Petite Abeille a écrit :
> A proposal for a Lua entry for the "Google Summer of Code" '08:
> A Lua implementation of Lucene.
For me, Lua is just a glue between C coded object, a super config file.
Like used in lighttpd or WoW.
Lulu will work on top of Lucy?
Did I miss something?

M.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Grant Ingersoll-2

On Feb 29, 2008, at 5:39 AM, Mathieu Lecarme wrote:

> Petite Abeille a écrit :
>> A proposal for a Lua entry for the "Google Summer of Code" '08:
>> A Lua implementation of Lucene.
> For me, Lua is just a glue between C coded object, a super config  
> file. Like used in lighttpd or WoW.
> Lulu will work on top of Lucy?

That implies the Lucy actually is under development...  Perhaps they  
will take up work on Lucy...
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Mathieu Lecarme
Grant Ingersoll a écrit :

>
> On Feb 29, 2008, at 5:39 AM, Mathieu Lecarme wrote:
>
>> Petite Abeille a écrit :
>>> A proposal for a Lua entry for the "Google Summer of Code" '08:
>>> A Lua implementation of Lucene.
>> For me, Lua is just a glue between C coded object, a super config
>> file. Like used in lighttpd or WoW.
>> Lulu will work on top of Lucy?
>
> That implies the Lucy actually is under development...  Perhaps they
> will take up work on Lucy...
In other hands, a Lucy with C for persistance and parsing, and Lua for
filter and other fine configuration can be great.

M.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2
In reply to this post by Mathieu Lecarme

On Feb 29, 2008, at 11:39 AM, Mathieu Lecarme wrote:

> For me, Lua is just a glue between C coded object, a super config  
> file. Like used in lighttpd or WoW.

This is a rather very narrow view of what one could do with Lua.

For example, this wiki engine is written exclusively in Lua:

http://alt.textdrive.com/nanoki/
http://dev.alt.textdrive.com/browser/HTTP

>
> Lulu will work on top of Lucy?

No.

> Did I miss something?

Yes.

Lulu is a meant to be a pure Lua implementation of Lucene.

Cheers,

PA.




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2
In reply to this post by Grant Ingersoll-2

On Feb 29, 2008, at 1:09 PM, Grant Ingersoll wrote:

> That implies the Lucy actually is under development...  Perhaps they  
> will take up work on Lucy...

Lulu has nothing to do with Lucy... goes back to something at high  
school or something...


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2
In reply to this post by Mathieu Lecarme

On Feb 29, 2008, at 3:42 PM, Mathieu Lecarme wrote:

> In other hands, a Lucy with C for persistance and parsing, and Lua  
> for filter and other fine configuration can be great.

Who is that Lucy you keep talking about? :P

Cheers,

PA.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Simon Willnauer
goto www.google.com
type Lucy lucene

feel lucky

or go to http://lucene.apache.org/lucy/

simon



On Fri, Feb 29, 2008 at 8:24 PM, Petite Abeille <[hidden email]> wrote:

>
>  On Feb 29, 2008, at 3:42 PM, Mathieu Lecarme wrote:
>
>  > In other hands, a Lucy with C for persistance and parsing, and Lua
>  > for filter and other fine configuration can be great.
>
>  Who is that Lucy you keep talking about? :P
>
>  Cheers,
>
>  PA.
>
>
>
>
>  ---------------------------------------------------------------------
>  To unsubscribe, e-mail: [hidden email]
>  For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2

On Feb 29, 2008, at 8:37 PM, Simon Willnauer wrote:

> or go to http://lucene.apache.org/lucy/

Looks rather, hmmm, inactive:

http://svn.apache.org/viewvc/lucene/lucy/

Is there any working code anywhere?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2
In reply to this post by Mathieu Lecarme

On Feb 29, 2008, at 11:39 AM, Mathieu Lecarme wrote:

> For me, Lua is just a glue between C coded object, a super config  
> file. Like used in lighttpd or WoW.

Here is a an online demo of a wiki engine implemented purely in Lua:

http://svr225.stepx.com:3388/a
http://svr225.stepx.com:3388/nanoki


Cheers,

PA.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Marvin Humphrey
In reply to this post by Grant Ingersoll-2

On Feb 29, 2008, at 4:09 AM, Grant Ingersoll wrote:

> That implies the Lucy actually is under development...  Perhaps they  
> will take up work on Lucy...


:)

I haven't been feeding back my KinoSearch commits into the Lucy  
repository, true.  Not much has changed status-wise since this: <http://xrl.us/bgyu6 
 > (Link to mail-archives.apache.org).  I miss Dave :( but work  
continues apace.  I just figured I wouldn't bring anything up here  
until I had some experimental Java bindings for you folks to play  
with.  Some of the stuff I'm working on with KS isn't compatible with  
Lucene, and it doesn't make sense to impede progress by imposing that  
constraint.  Better to finish it up, present a polished version and a  
coherent explanation of the design, then have Mike McCandless riff off  
of it -- that worked pretty well for indexing speed improvements in  
Lucene 2.3.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Marvin Humphrey
In reply to this post by Petite Abeille-2-2

On Feb 29, 2008, at 11:22 AM, Petite Abeille wrote:

> Lulu is a meant to be a pure Lua implementation of Lucene.

How fast is Lua's method dispatch, compared to Java's?  That has a  
huge impact on performance, since *everything* is a method in Lucene  
-- down to writeByte().

There have been several attempts at pure dynamic language ports of  
Lucene, including Plucene (Perl), Lupy (Python), and the original pure-
Ruby implementation of Ferret.  All of those have proven unacceptably  
slow.  The ports which have achieved good performance, including  
CLucene (C++), Lucene.NET, the latter-day Ferret (C/Ruby), and  
KinoSearch (C/Perl) (and possibly others, I haven't reviewed them all)  
all get close to the metal.  None rely on hash-based method dispatch  
for inner-loop code, and all manipulate primitive types directly.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Petite Abeille-2-2
Hi Marvin,

On Mar 1, 2008, at 2:33 AM, Marvin Humphrey wrote:

> How fast is Lua's method dispatch, compared to Java's?

Fast enough.

http://luajit.org/luajit_performance.html

>  That has a huge impact on performance, since *everything* is a  
> method in Lucene -- down to writeByte().

The plan is not to mimic Lucene's Java implementation, but rather it's  
file format and query syntax:

http://lucene.apache.org/java/2_3_1/fileformats.html
http://lucene.apache.org/java/2_3_1/queryparsersyntax.html

> There have been several attempts at pure dynamic language ports of  
> Lucene, including Plucene (Perl), Lupy (Python), and the original  
> pure-Ruby implementation of Ferret.  All of those have proven  
> unacceptably slow.  The ports which have achieved good performance,  
> including CLucene (C++), Lucene.NET, the latter-day Ferret (C/Ruby),  
> and KinoSearch (C/Perl) (and possibly others, I haven't reviewed  
> them all) all get close to the metal.  None rely on hash-based  
> method dispatch for inner-loop code, and all manipulate primitive  
> types directly.

Thanks for the head up :)

Cheers,

PA.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Michael McCandless-2

I am looking forward to a pure Lua version of Lucene.  Lua looks like  
a neat and amazingly compact programming language.  I especially like  
Lua's "table" container type which is both a list and a hash/dictionary.

One issue to watch out for is LUCENE-510, which has been an issue  
with other Lucene implementations trying to match the Lucene index  
format exactly:

   http://issues.apache.org/jira/browse/LUCENE-510

I'm working on this one but not sure when exactly it will be fixed  
since it's a difficult tradeoff (loses some indexing performance)...

Mike

Petite Abeille wrote:

> Hi Marvin,
>
> On Mar 1, 2008, at 2:33 AM, Marvin Humphrey wrote:
>
>> How fast is Lua's method dispatch, compared to Java's?
>
> Fast enough.
>
> http://luajit.org/luajit_performance.html
>
>>  That has a huge impact on performance, since *everything* is a  
>> method in Lucene -- down to writeByte().
>
> The plan is not to mimic Lucene's Java implementation, but rather  
> it's file format and query syntax:
>
> http://lucene.apache.org/java/2_3_1/fileformats.html
> http://lucene.apache.org/java/2_3_1/queryparsersyntax.html
>
>> There have been several attempts at pure dynamic language ports of  
>> Lucene, including Plucene (Perl), Lupy (Python), and the original  
>> pure-Ruby implementation of Ferret.  All of those have proven  
>> unacceptably slow.  The ports which have achieved good  
>> performance, including CLucene (C++), Lucene.NET, the latter-day  
>> Ferret (C/Ruby), and KinoSearch (C/Perl) (and possibly others, I  
>> haven't reviewed them all) all get close to the metal.  None rely  
>> on hash-based method dispatch for inner-loop code, and all  
>> manipulate primitive types directly.
>
> Thanks for the head up :)
>
> Cheers,
>
> PA.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SOC: Lulu, a Lua implementation of Lucene

Michael McCandless-2
In reply to this post by Marvin Humphrey

Marvin Humphrey wrote:

> I haven't been feeding back my KinoSearch commits into the Lucy  
> repository, true.  Not much has changed status-wise since this:  
> <http://xrl.us/bgyu6> (Link to mail-archives.apache.org).  I miss  
> Dave :( but work continues apace.  I just figured I wouldn't bring  
> anything up here until I had some experimental Java bindings for  
> you folks to play with.  Some of the stuff I'm working on with KS  
> isn't compatible with Lucene, and it doesn't make sense to impede  
> progress by imposing that constraint.  Better to finish it up,  
> present a polished version and a coherent explanation of the  
> design, then have Mike McCandless riff off of it -- that worked  
> pretty well for indexing speed improvements in Lucene 2.3.

Looking forward to it ;)  Though it'd be nice to be more iterative  
here, if possible, rather than the "big bang" approach.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]