1.9 RC1

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

1.9 RC1

Doug Cutting
I'd like to push out a 1.9 release candidate in the next week or so.
Are there any patches folks are really hoping to sneak into 1.9?  If so,
now's the time.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Doug Cutting
Doug Cutting wrote:
> I'd like to push out a 1.9 release candidate in the next week or so. Are
> there any patches folks are really hoping to sneak into 1.9?  If so,
> now's the time.

This is a great time to improve the javadoc.  I see lots of blank boxes
which could use a bit of descriptive text, for example:

Most of the "Analysis" packages lack a package.html file, as does the
Spellchecker.

The new paramters classes lack class-level javadoc, namely:
   DateTools.Resolution
   Field.Index
   Field.Store
   Field.TermVector
   BooleanClause.Occur

Does anyone want to volunteer to fix these?

Doug



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Chris Hostetter-3

: This is a great time to improve the javadoc.  I see lots of blank boxes
: which could use a bit of descriptive text, for example:

That reminds me about a documentation/release issue that's been rolling
arround in the back of my mind that seems like it's only going to get
worse as future releases are made...

I think moving forward the query parser and fileformat docs should be
moved into docfile directories within the java source, so they are
reved/tagged with the individual releases.  That way when people have
questions about the file format of their index built with 1.9 they don't
have to worry about changes that the version on the website documents the
fileformat in 2.4 which is different.  Likwise for questions about what
syntax is valid in the version of QueryParser they are using.

The only problem is that since 1.4.3 and previous versions didn't include
these docs, the URL is the only resource they have. so i would suggest..

 1) Migrate the current versions of fileformats.html and
    queryparsersyntax.html into src/java/**/docfiles somwhere.
 2) Revert the version of fileformats.html and
    queryparsersyntax.html on the website to their state when 1.4.3 was
    released.
 3) Add a disclaimer at the top of those files indicating that they are
    for 1.4.3 and earlier, and are online for legacy purposes only.
    Developers using newer versions of lucene should consult the
    documentation included in the release they are using.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Chris Hostetter-3
In reply to this post by Doug Cutting

: I'd like to push out a 1.9 release candidate in the next week or so.

I'm not sure what the ASF/Lucene policy is on keeping Copyright/License
statements in source files up to date, but should they all be updated to
say "Copyright 2006 The Apache Software Foundation" prior to a 1.9
release?

I've seen a few that say 2005 and 2004.


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Doug Cutting
Chris Hostetter wrote:
> I'm not sure what the ASF/Lucene policy is on keeping Copyright/License
> statements in source files up to date, but should they all be updated to
> say "Copyright 2006 The Apache Software Foundation" prior to a 1.9
> release?

It shouldn't hurt!

This week is pretty booked for me, so, barring major objections, I will
make a 1.9 RC1 release next Monday, February 20th.  If there are no
problems discovered, I'll aim to make a 1.9 final release a week later,
around the 27th.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Erik Hatcher
I've been away from constant e-mail for several days (nice while it  
lasted, but rough to come back to!)...

I'm +1 for 1.9 RC1, just for the record.  As for the copyright years  
- those should reflect only the years those files were touched, at  
least that is how it is carefully done with Ant and now Tapestry.

        Erik


On Feb 14, 2006, at 8:16 PM, Doug Cutting wrote:

> Chris Hostetter wrote:
>> I'm not sure what the ASF/Lucene policy is on keeping Copyright/
>> License
>> statements in source files up to date, but should they all be  
>> updated to
>> say "Copyright 2006 The Apache Software Foundation" prior to a 1.9
>> release?
>
> It shouldn't hurt!
>
> This week is pretty booked for me, so, barring major objections, I  
> will make a 1.9 RC1 release next Monday, February 20th.  If there  
> are no problems discovered, I'll aim to make a 1.9 final release a  
> week later, around the 27th.
>
> Doug
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Maxim Patramanskij
In reply to this post by Doug Cutting
Doug,

what about including optimization of BuffereIndexOutput.writeBytes()
method:

[ http://issues.apache.org/jira/browse/LUCENE-435?page=all ]


made by Lukas Zapletal, into 1.9?

I'm wondering, because this can decrease index creation time, which I
discovered as critical when using Lucene together with JDBCDirectory.

Max


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

DM Smith
In reply to this post by Doug Cutting
Not to get to far ahead, but what is the schedule relation between 1.9
and 2.0?
What are the dependencies on releasing 2.0?

Doug Cutting wrote:
> I'd like to push out a 1.9 release candidate in the next week or so.
> Are there any patches folks are really hoping to sneak into 1.9?  If
> so, now's the time.
>
> Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Erik Hatcher
On Feb 15, 2006, at 9:11 AM, DM Smith wrote:
> Not to get to far ahead, but what is the schedule relation between  
> 1.9 and 2.0?
> What are the dependencies on releasing 2.0?

My understanding is that 2.0 will be 1.9 with all the deprecated API  
removed.  Maybe there are other features planned?

        Erik


>
> Doug Cutting wrote:
>> I'd like to push out a 1.9 release candidate in the next week or  
>> so. Are there any patches folks are really hoping to sneak into  
>> 1.9?  If so, now's the time.
>>
>> Doug
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

DM Smith
Erik Hatcher wrote:
> On Feb 15, 2006, at 9:11 AM, DM Smith wrote:
>> Not to get to far ahead, but what is the schedule relation between
>> 1.9 and 2.0?
>> What are the dependencies on releasing 2.0?
>
> My understanding is that 2.0 will be 1.9 with all the deprecated API
> removed.  Maybe there are other features planned?

Would that mean that 1.9 and 2.0 will be released at the same time?

>
>     Erik
>
>
>>
>> Doug Cutting wrote:
>>> I'd like to push out a 1.9 release candidate in the next week or so.
>>> Are there any patches folks are really hoping to sneak into 1.9?  If
>>> so, now's the time.
>>>
>>> Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Doug Cutting
DM Smith wrote:
> Would that mean that 1.9 and 2.0 will be released at the same time?

No.  2.0 will be released after 1.9.  The primary change will be that
all deprecated methods are removed, but there may be other changes, but
probably not many.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Jeff Breidenbach
In reply to this post by Doug Cutting
> This week is pretty booked for me, so, barring major objections, I will
> make a 1.9 RC1 release next Monday, February 20th.  If there are no
> problems discovered, I'll aim to make a 1.9 final release a week later,
> around the 27th.

Has anyone tested if 1.9 can build with a Free Software toolchain? (e.g.
kaffe)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Otis Gospodnetic-2
In reply to this post by Maxim Patramanskij
Maxim - vote for it.  Not guaranteed to get tihngs in, but votes helps us see what people need/want/like.

Otis

----- Original Message ----
From: Maxim Patramanskij <[hidden email]>
To: Doug Cutting <[hidden email]>
Sent: Wednesday, February 15, 2006 7:25:52 AM
Subject: Re: 1.9 RC1

Doug,

what about including optimization of BuffereIndexOutput.writeBytes()
method:

[ http://issues.apache.org/jira/browse/LUCENE-435?page=all ]


made by Lukas Zapletal, into 1.9?

I'm wondering, because this can decrease index creation time, which I
discovered as critical when using Lucene together with JDBCDirectory.

Max


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Dan Armbrust
I'd like to see this improvement request implemented - but I'm not sure
if 1.9 or 2.0 would be a better place to do it:

http://issues.apache.org/jira/browse/LUCENE-301

Short summary - The Constructor for IndexWriter currently will only
create an index in a folder if you set the boolean create flag to true.
  But then, if you want to append to that index, you have to set the
create flag to false (otherwise it overwrites)

In my use cases, I seldom want to overwrite an index - but I often
create new ones, and append to existing ones.  Forgetting to switch the
boolean flag between the initial create and the append causes data loss.

To me, a better, safer API would be to change the parameter named
"create" into "clear" - and then change the behavior so that is always
creates a new index at the specified location if one doesn't already exist.

If clear is false - it would append (same as current behavior) - and if
clear is true is would clear first, and then create a new index.  So
nobody using the API should break.

Dan

--
****************************
Daniel Armbrust
Biomedical Informatics
Mayo Clinic Rochester
daniel.armbrust(at)mayo.edu
http://informatics.mayo.edu/

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Nadav Har'El-2
Dan Armbrust <[hidden email]> wrote on 17/02/2006 08:50:53
PM:
>...
> Short summary - The Constructor for IndexWriter currently will only
> create an index in a folder if you set the boolean create flag to true.
>   But then, if you want to append to that index, you have to set the
> create flag to false (otherwise it overwrites)
>
> In my use cases, I seldom want to overwrite an index - but I often
> create new ones, and append to existing ones.  Forgetting to switch the
> boolean flag between the initial create and the append causes data loss.

Hi,

I agree: as a new user of Lucene, the first thing I wanted to do in my
program
was to open an existing index, or if one doesn't yet exist, create it. I
found
it very strange that this natural usage pattern wasn't naturally supported
by
Lucene. I ended up doing something complex like opening the index once with
create=false, and if that failed, try again with create=true, although this
had
additional problems like trying to recreate the index when we actually got
an
error which was not caused by a non-existant index.

So I'm not sure the solution is to change the semantics of the existing
constructor, but I think Lucene definitely need a new constructor or
convenience
function that will do "the right thing" for opening a potentially-existing
index.

--
Nadav Har'El.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Tatu Saloranta
--- Nadav Har'El <[hidden email]> wrote:

> Dan Armbrust <[hidden email]> wrote
> on 17/02/2006 08:50:53
> PM:
...
> So I'm not sure the solution is to change the
> semantics of the existing
> constructor, but I think Lucene definitely need a
> new constructor or
> convenience
> function that will do "the right thing" for opening
> a potentially-existing
> index.

Or maybe get away from the vanilla constructor
mindset, and add more explicitly named factory
methods? Perhaps something like
IndexWriter.openOrCreate(), create() and open() (first
similar to append methods in streams, second for
overwrite, third for opening only if one exists). And
internally constructor could take set of arguments().

I mean, this same thing is done for Field(), with
somewhat improved construction semantics (although
semantics of Field object are bit messy, bundling both
field definition and value).

-+ Tatu +-



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Doug Cutting
In reply to this post by Chris Hostetter-3
Chris Hostetter wrote:
> I think moving forward the query parser and fileformat docs should be
> moved into docfile directories within the java source, so they are
> reved/tagged with the individual releases.  That way when people have
> questions about the file format of their index built with 1.9 they don't
> have to worry about changes that the version on the website documents the
> fileformat in 2.4 which is different.  Likwise for questions about what
> syntax is valid in the version of QueryParser they are using.

I agree that it would be useful to have multiple versions of the
documentation on the website.  (Any volunteers to re-architect the website?)

Currently the website is only meant to describe the current release.
Note that the fileformat and query parser syntax documentation is
included with releases, so that one can always refer to the
documentation downloaded, rather than the website.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Doug Cutting
In reply to this post by Maxim Patramanskij
Maxim Patramanskij wrote:
> Doug,
>
> what about including optimization of BuffereIndexOutput.writeBytes()
> method:
>
> [ http://issues.apache.org/jira/browse/LUCENE-435?page=all ]
>
>
> made by Lukas Zapletal, into 1.9?

I just committed this to trunk.  If no issues arise with it there then
perhaps we can include it in 1.9 final.  It will certainly go into the
2.0 release, which will follow quite soon.

Thanks,

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Shay Banon-2
In reply to this post by Maxim Patramanskij
Hi,

        I have just updated to lucene 1.9, and hit a problem with the  
mentioned optimization. I have applied it to the my  
JdbcBufferedOutput (I only duplicate the code because the BUFFER_SIZE  
is final), and I hit a problem. In the following code fragment (the  
method is writeBytes):

...
     } else {
       // is data larger then buffer?
       if (length > BUFFER_SIZE) {
         // we flush the buffer
         if (bufferPosition > 0)
           flush();
         // and write data at once
         flushBuffer(b, length);
       } else {
...

the bufferStart is not incremented after the flushBuffer method is  
called. So if someone calls getFilePointer just afterwards, it will  
give the wrong result (hit it with the compound format). A simple fix  
would be to add bufferStart += length; just after flushBuffer.

By the way, is there a chance that BufferedIndexInput and  
BufferedIndexOutput will have the buffer size as a member variable,  
with a setter (or at least protected visibility), so I won't have to  
duplicate the buffer support in Jdbc?

Shay

On 15 Feb 2006, at 12:25, Maxim Patramanskij wrote:

> Doug,
>
> what about including optimization of BuffereIndexOutput.writeBytes()
> method:
>
> [ http://issues.apache.org/jira/browse/LUCENE-435?page=all ]
>
>
> made by Lukas Zapletal, into 1.9?
>
> I'm wondering, because this can decrease index creation time, which I
> discovered as critical when using Lucene together with JDBCDirectory.
>
> Max
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: 1.9 RC1

Doug Cutting
Shay Banon wrote:

> ...
>     } else {
>       // is data larger then buffer?
>       if (length > BUFFER_SIZE) {
>         // we flush the buffer
>         if (bufferPosition > 0)
>           flush();
>         // and write data at once
>         flushBuffer(b, length);
>       } else {
> ...
>
> the bufferStart is not incremented after the flushBuffer method is  
> called. So if someone calls getFilePointer just afterwards, it will  
> give the wrong result (hit it with the compound format). A simple fix  
> would be to add bufferStart += length; just after flushBuffer.

Can you please file a bug for this and attach a bug to it with a unit
test that illustrates the problem?  This looks like something that could
warrant a 1.9.1 release, so we must proceed carefully.

Thanks,

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12