Lucene/Solr 7.5

classic Classic list List threaded Threaded
43 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Lucene/Solr 7.5

jim ferenczi
Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Alan Woodward-3
+1

I’d like to get Matches added to intervals, if anybody feels like reviewing https://issues.apache.org/jira/browse/LUCENE-8422 before the branch is cut.

On 3 Sep 2018, at 09:42, jim ferenczi <[hidden email]> wrote:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Adrien Grand
+1 too

Alan, I just left comments on LUCENE-8422.

Le lun. 3 sept. 2018 à 10:50, Alan Woodward <[hidden email]> a écrit :
+1

I’d like to get Matches added to intervals, if anybody feels like reviewing https://issues.apache.org/jira/browse/LUCENE-8422 before the branch is cut.


On 3 Sep 2018, at 09:42, jim ferenczi <[hidden email]> wrote:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Erick Erickson
I'm working on "SOLR-12727: Upgrade ZooKeeper dependency to 3.4.13...
will make the ZK client re-resolve the server hostnames when a
connection fails. This will fix issues where a failed ZK container is
replaced with a new one that has a different IP address and DNS gets
updated with the new address."

I've already done the upgrade locally, but got distracted when one of
the test failures turned out to be related to async logging
(SOLR-12728)...

I guess my question is whether to try to get this in 7.5 in the next
day or two or wait until 7.6. We've lived with this issue since the
first release of SolrCloud, so it's not pressing....

WDYT?
On Mon, Sep 3, 2018 at 5:53 AM Adrien Grand <[hidden email]> wrote:

>
> +1 too
>
> Alan, I just left comments on LUCENE-8422.
>
> Le lun. 3 sept. 2018 à 10:50, Alan Woodward <[hidden email]> a écrit :
>>
>> +1
>>
>> I’d like to get Matches added to intervals, if anybody feels like reviewing https://issues.apache.org/jira/browse/LUCENE-8422 before the branch is cut.
>>
>>
>> On 3 Sep 2018, at 09:42, jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>>
>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>> https://issues.apache.org/jira/browse/LUCENE-8466
>>
>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>>
>> Cheers,
>> Jim
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Varun Thacker-4
In reply to this post by jim ferenczi
+1

Erick - ZK 3.4.13 looks like a BugFix release and was released on July 15th. So based on those facts +1 to upgrade if you plan on tackling this.


On Mon, Sep 3, 2018 at 2:43 AM jim ferenczi <[hidden email]> wrote:
Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Jan Høydahl / Cominvent
In reply to this post by jim ferenczi
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
Sure Jan, this is a nice cleanup, +1 to backport in 7x.

Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
Thanks all,
since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.

Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
Sure Jan, this is a nice cleanup, +1 to backport in 7x.

Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Joel Bernstein
+1,

I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.




On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
Thanks all,
since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.

Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
Sure Jan, this is a nice cleanup, +1 to backport in 7x.

Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Cassandra Targett
I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed. 

Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.

Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.

I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?

On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
+1,

I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.




On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
Thanks all,
since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.

Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
Sure Jan, this is a nice cleanup, +1 to backport in 7x.

Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.

Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed. 

Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.

Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.

I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?

On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
+1,

I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.




On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
Thanks all,
since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.

Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
Sure Jan, this is a nice cleanup, +1 to backport in 7x.

Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Cassandra Targett
It's not so much the building of the RC as giving the content a detailed editorial review.

The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.

In terms of preparing the content, there are a number of things I do:

First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.

Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.

Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:

- sentences that don't end in periods
- inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.) 
- no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
- used sentence case for section titles instead of headline case
- used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
- misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
- config file names and parameter names/values not in monospace
- lists of parameters are not properly formatted (should not be in tables)

These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.

Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)

It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.

(as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)

On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.

Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed. 

Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.

Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.

I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?

On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
+1,

I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.




On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
Thanks all,
since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.

Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
Sure Jan, this is a nice cleanup, +1 to backport in 7x.

Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:

Hi all,

7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.

Cheers,
Jim

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Erick Erickson
Jim:

I know it's the 11th hour, but WDYT about cutting the branch next
Monday? We see a flurry of activity (announcing a release does
that....) and waiting to cut the branch might be easiest all 'round.

Up to you of course, I can backport the test fixes I'd like for
instance and I'd like to get the upgraded ZooKeeper in 7.5.

Erick
On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:

>
> It's not so much the building of the RC as giving the content a detailed editorial review.
>
> The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>
> In terms of preparing the content, there are a number of things I do:
>
> First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>
> Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>
> Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>
> - sentences that don't end in periods
> - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
> - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
> - used sentence case for section titles instead of headline case
> - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
> - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
> - config file names and parameter names/values not in monospace
> - lists of parameters are not properly formatted (should not be in tables)
>
> These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>
> Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>
> It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>
> (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>
> On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>>
>> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>>
>> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>>>
>>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>>>
>>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>>>
>>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>>>
>>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>>>
>>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>>>>
>>>> +1,
>>>>
>>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>>>>
>>>>
>>>> Joel Bernstein
>>>> http://joelsolr.blogspot.com/
>>>>
>>>>
>>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>>>>>
>>>>> Thanks all,
>>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>>>>>
>>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>>>>>>
>>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>>>>>>
>>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>>>>>>>
>>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>>>>>>>
>>>>>>> --
>>>>>>> Jan Høydahl, search solution architect
>>>>>>> Cominvent AS - www.cominvent.com
>>>>>>>
>>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>>>>>>>
>>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Jim
>>>>>>>
>>>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Noble Paul നോബിള്‍  नोब्ळ्
I need to fix https://issues.apache.org/jira/browse/SOLR-12117 for 7.5
On Thu, Sep 6, 2018 at 1:27 AM Erick Erickson <[hidden email]> wrote:

>
> Jim:
>
> I know it's the 11th hour, but WDYT about cutting the branch next
> Monday? We see a flurry of activity (announcing a release does
> that....) and waiting to cut the branch might be easiest all 'round.
>
> Up to you of course, I can backport the test fixes I'd like for
> instance and I'd like to get the upgraded ZooKeeper in 7.5.
>
> Erick
> On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
> >
> > It's not so much the building of the RC as giving the content a detailed editorial review.
> >
> > The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
> >
> > In terms of preparing the content, there are a number of things I do:
> >
> > First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
> >
> > Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
> >
> > Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
> >
> > - sentences that don't end in periods
> > - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
> > - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
> > - used sentence case for section titles instead of headline case
> > - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
> > - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
> > - config file names and parameter names/values not in monospace
> > - lists of parameters are not properly formatted (should not be in tables)
> >
> > These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
> >
> > Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
> >
> > It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
> >
> > (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
> >
> > On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
> >>
> >> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
> >> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
> >>
> >> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
> >>>
> >>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
> >>>
> >>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
> >>>
> >>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
> >>>
> >>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
> >>>
> >>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
> >>>>
> >>>> +1,
> >>>>
> >>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
> >>>>
> >>>>
> >>>> Joel Bernstein
> >>>> http://joelsolr.blogspot.com/
> >>>>
> >>>>
> >>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
> >>>>>
> >>>>> Thanks all,
> >>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
> >>>>>
> >>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
> >>>>>>
> >>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
> >>>>>>
> >>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
> >>>>>>>
> >>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
> >>>>>>>
> >>>>>>> --
> >>>>>>> Jan Høydahl, search solution architect
> >>>>>>> Cominvent AS - www.cominvent.com
> >>>>>>>
> >>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
> >>>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
> >>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
> >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
> >>>>>>>
> >>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Jim
> >>>>>>>
> >>>>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


--
-----------------------------------------------------
Noble Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
In reply to this post by Erick Erickson
Sure it can wait a few days. Let's cut the branch next Monday and we can sync with Cassandra to create the first RC when the ref guide is ready. 

Le mer. 5 sept. 2018 à 17:27, Erick Erickson <[hidden email]> a écrit :
Jim:

I know it's the 11th hour, but WDYT about cutting the branch next
Monday? We see a flurry of activity (announcing a release does
that....) and waiting to cut the branch might be easiest all 'round.

Up to you of course, I can backport the test fixes I'd like for
instance and I'd like to get the upgraded ZooKeeper in 7.5.

Erick
On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
>
> It's not so much the building of the RC as giving the content a detailed editorial review.
>
> The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>
> In terms of preparing the content, there are a number of things I do:
>
> First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>
> Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>
> Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>
> - sentences that don't end in periods
> - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
> - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
> - used sentence case for section titles instead of headline case
> - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
> - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
> - config file names and parameter names/values not in monospace
> - lists of parameters are not properly formatted (should not be in tables)
>
> These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>
> Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>
> It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>
> (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>
> On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>>
>> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>>
>> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>>>
>>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>>>
>>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>>>
>>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>>>
>>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>>>
>>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>>>>
>>>> +1,
>>>>
>>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>>>>
>>>>
>>>> Joel Bernstein
>>>> http://joelsolr.blogspot.com/
>>>>
>>>>
>>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>>>>>
>>>>> Thanks all,
>>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>>>>>
>>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>>>>>>
>>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>>>>>>
>>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>>>>>>>
>>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>>>>>>>
>>>>>>> --
>>>>>>> Jan Høydahl, search solution architect
>>>>>>> Cominvent AS - www.cominvent.com
>>>>>>>
>>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>>>>>>>
>>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Jim
>>>>>>>
>>>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Erick Erickson
Great, thanks!
On Wed, Sep 5, 2018 at 8:44 AM jim ferenczi <[hidden email]> wrote:

>
> Sure it can wait a few days. Let's cut the branch next Monday and we can sync with Cassandra to create the first RC when the ref guide is ready.
>
> Le mer. 5 sept. 2018 à 17:27, Erick Erickson <[hidden email]> a écrit :
>>
>> Jim:
>>
>> I know it's the 11th hour, but WDYT about cutting the branch next
>> Monday? We see a flurry of activity (announcing a release does
>> that....) and waiting to cut the branch might be easiest all 'round.
>>
>> Up to you of course, I can backport the test fixes I'd like for
>> instance and I'd like to get the upgraded ZooKeeper in 7.5.
>>
>> Erick
>> On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
>> >
>> > It's not so much the building of the RC as giving the content a detailed editorial review.
>> >
>> > The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>> >
>> > In terms of preparing the content, there are a number of things I do:
>> >
>> > First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>> >
>> > Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>> >
>> > Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>> >
>> > - sentences that don't end in periods
>> > - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
>> > - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
>> > - used sentence case for section titles instead of headline case
>> > - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
>> > - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
>> > - config file names and parameter names/values not in monospace
>> > - lists of parameters are not properly formatted (should not be in tables)
>> >
>> > These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>> >
>> > Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>> >
>> > It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>> >
>> > (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>> >
>> > On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>> >>
>> >> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> >> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>> >>
>> >> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>> >>>
>> >>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>> >>>
>> >>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>> >>>
>> >>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>> >>>
>> >>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>> >>>
>> >>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>> >>>>
>> >>>> +1,
>> >>>>
>> >>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>> >>>>
>> >>>>
>> >>>> Joel Bernstein
>> >>>> http://joelsolr.blogspot.com/
>> >>>>
>> >>>>
>> >>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> Thanks all,
>> >>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>> >>>>>
>> >>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>> >>>>>>
>> >>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>> >>>>>>>
>> >>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Jan Høydahl, search solution architect
>> >>>>>>> Cominvent AS - www.cominvent.com
>> >>>>>>>
>> >>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>> >>>>>>>
>> >>>>>>> Hi all,
>> >>>>>>>
>> >>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>> >>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>> >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>> >>>>>>>
>> >>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> Jim
>> >>>>>>>
>> >>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
Hi,

Feature freeze for 7.5 has started, I just created a branch_7_5.:

* No new features may be committed to the branch.
* Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all patches you want to commit to Jira first to give others the chance to review and possibly vote against the patch. Keep in mind that it is our main intention to keep the branch as stable as possible.
* All patches that are intended for the branch should first be committed to the unstable branch, merged into the stable branch, and then into the current release branch.
* Normal unstable and stable branch development may continue as usual. However, if you plan to commit a big change to the unstable branch while the branch feature freeze is in effect, think twice: can't the addition wait a couple more days? Merges of bug fixes into the branch may become more difficult.
* Only Jira issues with Fix version "7.5" and priority "Blocker" will delay a release candidate build.

I'll create the first RC later this week depending on the status of the Solr ref guide. Cassandra, can you update the status when you think that the ref guide is ready (no rush just a reminder that we need to sync during this release ;) ) ?

Cheers,
Jim

Le mer. 5 sept. 2018 à 17:57, Erick Erickson <[hidden email]> a écrit :
Great, thanks!
On Wed, Sep 5, 2018 at 8:44 AM jim ferenczi <[hidden email]> wrote:
>
> Sure it can wait a few days. Let's cut the branch next Monday and we can sync with Cassandra to create the first RC when the ref guide is ready.
>
> Le mer. 5 sept. 2018 à 17:27, Erick Erickson <[hidden email]> a écrit :
>>
>> Jim:
>>
>> I know it's the 11th hour, but WDYT about cutting the branch next
>> Monday? We see a flurry of activity (announcing a release does
>> that....) and waiting to cut the branch might be easiest all 'round.
>>
>> Up to you of course, I can backport the test fixes I'd like for
>> instance and I'd like to get the upgraded ZooKeeper in 7.5.
>>
>> Erick
>> On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
>> >
>> > It's not so much the building of the RC as giving the content a detailed editorial review.
>> >
>> > The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>> >
>> > In terms of preparing the content, there are a number of things I do:
>> >
>> > First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>> >
>> > Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>> >
>> > Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>> >
>> > - sentences that don't end in periods
>> > - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
>> > - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
>> > - used sentence case for section titles instead of headline case
>> > - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
>> > - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
>> > - config file names and parameter names/values not in monospace
>> > - lists of parameters are not properly formatted (should not be in tables)
>> >
>> > These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>> >
>> > Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>> >
>> > It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>> >
>> > (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>> >
>> > On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>> >>
>> >> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> >> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>> >>
>> >> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>> >>>
>> >>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>> >>>
>> >>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>> >>>
>> >>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>> >>>
>> >>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>> >>>
>> >>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>> >>>>
>> >>>> +1,
>> >>>>
>> >>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>> >>>>
>> >>>>
>> >>>> Joel Bernstein
>> >>>> http://joelsolr.blogspot.com/
>> >>>>
>> >>>>
>> >>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> Thanks all,
>> >>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>> >>>>>
>> >>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>> >>>>>>
>> >>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>> >>>>>>>
>> >>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Jan Høydahl, search solution architect
>> >>>>>>> Cominvent AS - www.cominvent.com
>> >>>>>>>
>> >>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>> >>>>>>>
>> >>>>>>> Hi all,
>> >>>>>>>
>> >>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>> >>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>> >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>> >>>>>>>
>> >>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> Jim
>> >>>>>>>
>> >>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
I just fixed the invalid version (7.5.1) that I added in master and 7x. The next version on these branches should be 7.6.0, sorry for the noise.

Le lun. 10 sept. 2018 à 09:26, jim ferenczi <[hidden email]> a écrit :
Hi,

Feature freeze for 7.5 has started, I just created a branch_7_5.:

* No new features may be committed to the branch.
* Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all patches you want to commit to Jira first to give others the chance to review and possibly vote against the patch. Keep in mind that it is our main intention to keep the branch as stable as possible.
* All patches that are intended for the branch should first be committed to the unstable branch, merged into the stable branch, and then into the current release branch.
* Normal unstable and stable branch development may continue as usual. However, if you plan to commit a big change to the unstable branch while the branch feature freeze is in effect, think twice: can't the addition wait a couple more days? Merges of bug fixes into the branch may become more difficult.
* Only Jira issues with Fix version "7.5" and priority "Blocker" will delay a release candidate build.

I'll create the first RC later this week depending on the status of the Solr ref guide. Cassandra, can you update the status when you think that the ref guide is ready (no rush just a reminder that we need to sync during this release ;) ) ?

Cheers,
Jim

Le mer. 5 sept. 2018 à 17:57, Erick Erickson <[hidden email]> a écrit :
Great, thanks!
On Wed, Sep 5, 2018 at 8:44 AM jim ferenczi <[hidden email]> wrote:
>
> Sure it can wait a few days. Let's cut the branch next Monday and we can sync with Cassandra to create the first RC when the ref guide is ready.
>
> Le mer. 5 sept. 2018 à 17:27, Erick Erickson <[hidden email]> a écrit :
>>
>> Jim:
>>
>> I know it's the 11th hour, but WDYT about cutting the branch next
>> Monday? We see a flurry of activity (announcing a release does
>> that....) and waiting to cut the branch might be easiest all 'round.
>>
>> Up to you of course, I can backport the test fixes I'd like for
>> instance and I'd like to get the upgraded ZooKeeper in 7.5.
>>
>> Erick
>> On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
>> >
>> > It's not so much the building of the RC as giving the content a detailed editorial review.
>> >
>> > The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>> >
>> > In terms of preparing the content, there are a number of things I do:
>> >
>> > First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>> >
>> > Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>> >
>> > Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>> >
>> > - sentences that don't end in periods
>> > - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
>> > - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
>> > - used sentence case for section titles instead of headline case
>> > - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
>> > - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
>> > - config file names and parameter names/values not in monospace
>> > - lists of parameters are not properly formatted (should not be in tables)
>> >
>> > These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>> >
>> > Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>> >
>> > It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>> >
>> > (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>> >
>> > On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>> >>
>> >> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> >> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>> >>
>> >> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>> >>>
>> >>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>> >>>
>> >>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>> >>>
>> >>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>> >>>
>> >>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>> >>>
>> >>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>> >>>>
>> >>>> +1,
>> >>>>
>> >>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>> >>>>
>> >>>>
>> >>>> Joel Bernstein
>> >>>> http://joelsolr.blogspot.com/
>> >>>>
>> >>>>
>> >>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> Thanks all,
>> >>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>> >>>>>
>> >>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>> >>>>>>
>> >>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>> >>>>>>>
>> >>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Jan Høydahl, search solution architect
>> >>>>>>> Cominvent AS - www.cominvent.com
>> >>>>>>>
>> >>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>> >>>>>>>
>> >>>>>>> Hi all,
>> >>>>>>>
>> >>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>> >>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>> >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>> >>>>>>>
>> >>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> Jim
>> >>>>>>>
>> >>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

Cassandra Targett
Sorry, Jim, I should have replied yesterday about the state of things with the Ref Guide - it's close. I'm doing the last bit of big review I need to do and am nearly done with that, then I have a couple more small things done (including SOLR-12763 which I just created since I forgot to do it earlier). My goal is to be done by the end of my day today so you could do the RC tomorrow, but who knows what the day will bring work-wise, so I'll send another mail at the end of the day my time to let you know for sure.

On Mon, Sep 10, 2018 at 9:07 AM jim ferenczi <[hidden email]> wrote:
I just fixed the invalid version (7.5.1) that I added in master and 7x. The next version on these branches should be 7.6.0, sorry for the noise.

Le lun. 10 sept. 2018 à 09:26, jim ferenczi <[hidden email]> a écrit :
Hi,

Feature freeze for 7.5 has started, I just created a branch_7_5.:

* No new features may be committed to the branch.
* Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all patches you want to commit to Jira first to give others the chance to review and possibly vote against the patch. Keep in mind that it is our main intention to keep the branch as stable as possible.
* All patches that are intended for the branch should first be committed to the unstable branch, merged into the stable branch, and then into the current release branch.
* Normal unstable and stable branch development may continue as usual. However, if you plan to commit a big change to the unstable branch while the branch feature freeze is in effect, think twice: can't the addition wait a couple more days? Merges of bug fixes into the branch may become more difficult.
* Only Jira issues with Fix version "7.5" and priority "Blocker" will delay a release candidate build.

I'll create the first RC later this week depending on the status of the Solr ref guide. Cassandra, can you update the status when you think that the ref guide is ready (no rush just a reminder that we need to sync during this release ;) ) ?

Cheers,
Jim

Le mer. 5 sept. 2018 à 17:57, Erick Erickson <[hidden email]> a écrit :
Great, thanks!
On Wed, Sep 5, 2018 at 8:44 AM jim ferenczi <[hidden email]> wrote:
>
> Sure it can wait a few days. Let's cut the branch next Monday and we can sync with Cassandra to create the first RC when the ref guide is ready.
>
> Le mer. 5 sept. 2018 à 17:27, Erick Erickson <[hidden email]> a écrit :
>>
>> Jim:
>>
>> I know it's the 11th hour, but WDYT about cutting the branch next
>> Monday? We see a flurry of activity (announcing a release does
>> that....) and waiting to cut the branch might be easiest all 'round.
>>
>> Up to you of course, I can backport the test fixes I'd like for
>> instance and I'd like to get the upgraded ZooKeeper in 7.5.
>>
>> Erick
>> On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
>> >
>> > It's not so much the building of the RC as giving the content a detailed editorial review.
>> >
>> > The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>> >
>> > In terms of preparing the content, there are a number of things I do:
>> >
>> > First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>> >
>> > Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>> >
>> > Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>> >
>> > - sentences that don't end in periods
>> > - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
>> > - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
>> > - used sentence case for section titles instead of headline case
>> > - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
>> > - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
>> > - config file names and parameter names/values not in monospace
>> > - lists of parameters are not properly formatted (should not be in tables)
>> >
>> > These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>> >
>> > Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>> >
>> > It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>> >
>> > (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>> >
>> > On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>> >>
>> >> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> >> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>> >>
>> >> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>> >>>
>> >>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>> >>>
>> >>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>> >>>
>> >>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>> >>>
>> >>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>> >>>
>> >>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>> >>>>
>> >>>> +1,
>> >>>>
>> >>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>> >>>>
>> >>>>
>> >>>> Joel Bernstein
>> >>>> http://joelsolr.blogspot.com/
>> >>>>
>> >>>>
>> >>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> Thanks all,
>> >>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>> >>>>>
>> >>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>> >>>>>>
>> >>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>> >>>>>>>
>> >>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Jan Høydahl, search solution architect
>> >>>>>>> Cominvent AS - www.cominvent.com
>> >>>>>>>
>> >>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>> >>>>>>>
>> >>>>>>> Hi all,
>> >>>>>>>
>> >>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>> >>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>> >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>> >>>>>>>
>> >>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> Jim
>> >>>>>>>
>> >>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 7.5

jim ferenczi
No worries at all Cassandra. What do you think of building the first RC on Friday and start the vote on Monday next week ? This will leave some
room to finish the missing bits. 
Could someone help to setup the Jenkins releases build ? It seems that I cannot create jobs with my account.

Le mar. 11 sept. 2018 à 14:08, Cassandra Targett <[hidden email]> a écrit :
Sorry, Jim, I should have replied yesterday about the state of things with the Ref Guide - it's close. I'm doing the last bit of big review I need to do and am nearly done with that, then I have a couple more small things done (including SOLR-12763 which I just created since I forgot to do it earlier). My goal is to be done by the end of my day today so you could do the RC tomorrow, but who knows what the day will bring work-wise, so I'll send another mail at the end of the day my time to let you know for sure.

On Mon, Sep 10, 2018 at 9:07 AM jim ferenczi <[hidden email]> wrote:
I just fixed the invalid version (7.5.1) that I added in master and 7x. The next version on these branches should be 7.6.0, sorry for the noise.

Le lun. 10 sept. 2018 à 09:26, jim ferenczi <[hidden email]> a écrit :
Hi,

Feature freeze for 7.5 has started, I just created a branch_7_5.:

* No new features may be committed to the branch.
* Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all patches you want to commit to Jira first to give others the chance to review and possibly vote against the patch. Keep in mind that it is our main intention to keep the branch as stable as possible.
* All patches that are intended for the branch should first be committed to the unstable branch, merged into the stable branch, and then into the current release branch.
* Normal unstable and stable branch development may continue as usual. However, if you plan to commit a big change to the unstable branch while the branch feature freeze is in effect, think twice: can't the addition wait a couple more days? Merges of bug fixes into the branch may become more difficult.
* Only Jira issues with Fix version "7.5" and priority "Blocker" will delay a release candidate build.

I'll create the first RC later this week depending on the status of the Solr ref guide. Cassandra, can you update the status when you think that the ref guide is ready (no rush just a reminder that we need to sync during this release ;) ) ?

Cheers,
Jim

Le mer. 5 sept. 2018 à 17:57, Erick Erickson <[hidden email]> a écrit :
Great, thanks!
On Wed, Sep 5, 2018 at 8:44 AM jim ferenczi <[hidden email]> wrote:
>
> Sure it can wait a few days. Let's cut the branch next Monday and we can sync with Cassandra to create the first RC when the ref guide is ready.
>
> Le mer. 5 sept. 2018 à 17:27, Erick Erickson <[hidden email]> a écrit :
>>
>> Jim:
>>
>> I know it's the 11th hour, but WDYT about cutting the branch next
>> Monday? We see a flurry of activity (announcing a release does
>> that....) and waiting to cut the branch might be easiest all 'round.
>>
>> Up to you of course, I can backport the test fixes I'd like for
>> instance and I'd like to get the upgraded ZooKeeper in 7.5.
>>
>> Erick
>> On Tue, Sep 4, 2018 at 1:04 PM Cassandra Targett <[hidden email]> wrote:
>> >
>> > It's not so much the building of the RC as giving the content a detailed editorial review.
>> >
>> > The build/release process itself is well-documented and published with every Ref Guide: https://lucene.apache.org/solr/guide/how-to-contribute.html#building-publishing-the-guide. It was designed from the artifact process, so it's nearly identical as a process. It's really barely a burden.
>> >
>> > In terms of preparing the content, there are a number of things I do:
>> >
>> > First, I try to ensure that every issue in CHANGES.txt that should be documented has been documented. That involves an intensive review of CHANGES.txt and a comparison with commits to find what might be missing, then chasing people down to see if they intend to make changes or not. Assuming the person responds, then it's waiting for them to get their stuff done. This is usually about 2-3 days of effort, before the waiting around for answers and/or commits.
>> >
>> > Then I review every commit and read it for clarity and correct English usage. Does it fit where someone put it? Does it explain what the author is hoping it explains? Also, many of our authors are not native English writers, and deserve the assistance of an editor to help put their work in the best possible light. In some cases, I feel I should extensively edit the contribution, which occasionally involves also immersing myself into the change itself. This is another 2-4 days of effort.
>> >
>> > Then there's this list of problems people commit all the time, many of which I can often resolve reasonably quickly with find/replace:
>> >
>> > - sentences that don't end in periods
>> > - inconsistency with instances of "i.e.," and "e.g.," (not "i.e.", "ie:", "IE", etc.)
>> > - no spaces between words and punctuation (commas, colons, periods), such as "here is :" or "word , word"
>> > - used sentence case for section titles instead of headline case
>> > - used abbreviations instead of the correct word ("ZK" instead of "ZooKeeper" being the biggest one here, but also "params" instead of "parameters" is quite common)
>> > - misspellings like "Zookeeper" instead of "ZooKeeper, or "solr" instead of "Solr"
>> > - config file names and parameter names/values not in monospace
>> > - lists of parameters are not properly formatted (should not be in tables)
>> >
>> > These are all to make the Ref Guide as consistent, cohesive, and easy to read as possible. It may be written by 30 people but it shouldn't read like it is.
>> >
>> > Should I do all this while the commits are coming through? Sure, but the reality is I can't. If we want to release the moment someone proposes a release, then most of my find/replace list above needs to go into precommit so these problems don't make it into the Guide to begin with. (Which might be onerous since we'd all get stalled waiting for someone to fix a typo...but really, precommit is meant in part to find your typos so why should this be different?)
>> >
>> > It would always still need editorial review, however, and that's not something we'll ever be able to fully automate. I'm more than happy to have a little help there, but assume since people aren't doing it today they don't have time, don't feel they have the skills, or don't want to bother. Or maybe I just kill myself for a level of quality no one else cares about...not sure I can stop doing it though if I'm the RM.
>> >
>> > (as a side note on that though, if we do merge the releases someday, then whoever RMs is going to have to wait for these editorial processes to be completed or the vote may fail because the Ref Guide reads like crap.)
>> >
>> > On Tue, Sep 4, 2018 at 11:33 AM jim ferenczi <[hidden email]> wrote:
>> >>
>> >> Thanks for explaining the situation Cassandra. I was planning to build the first RC beginning of next week to give people a week to discover blockers. I can certainly slow down things but I don't think that the timing
>> >> differs from other releases. I am not aware of the operations that are required for the Ref guide release process but what do you think of sharing the tasks with the RM ? We could even merge the two releases and make the RM responsible of both if the process is documented.  I'd be happy to experiment this for the 7.5 release if you want.
>> >>
>> >> Le mar. 4 sept. 2018 à 17:55, Cassandra Targett <[hidden email]> a écrit :
>> >>>
>> >>> I'm not objecting per se, but I feel like we used to propose a version and then give people a week before the branch was cut. Maybe that was just RM choice? From a personal perspective, I much prefer that model because the Ref Guide requires A LOT of my attention and my work there kicks into high gear as soon as a release is proposed.
>> >>>
>> >>> Even though the artifact and Ref Guide release processes are separate today, we want them to be a single process, so I need to act as though your timeframe for the RC is the deadline for Ref Guide edits to do an RC of the Ref Guide at the same time. That means I'm on your timetable, no matter what else I may have promised to my bosses and colleagues. It's stressful already to try to get it all done - I usually don't finish everything I want to do - and adding the burden of having to backport everything to 2 branches instead of 1 just makes it tedious as well.
>> >>>
>> >>> Also, yesterday was a major holiday in the US, and as of this moment it's not even noon on the East Coast, so there's a percentage of the community who may not even have seen your proposal yet.
>> >>>
>> >>> I greatly appreciate that you've volunteered to do the release and are energized to get it rolling, but is there a reason an RC has to be done by the beginning of next week?
>> >>>
>> >>> On Tue, Sep 4, 2018 at 10:36 AM Joel Bernstein <[hidden email]> wrote:
>> >>>>
>> >>>> +1,
>> >>>>
>> >>>> I'll likely be adding some Solr RefGuide changes later in the week to the 7.5 branch but I'll make sure they don't effect the build.
>> >>>>
>> >>>>
>> >>>> Joel Bernstein
>> >>>> http://joelsolr.blogspot.com/
>> >>>>
>> >>>>
>> >>>> On Tue, Sep 4, 2018 at 10:52 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> Thanks all,
>> >>>>> since there are no objections I am planning to cut the branch for 7.5 tomorrow. I'll build the first RC early next week so there will be some room to merge important bug fixes later this week. All blockers except SOLR-12727 seem to be merged/resolved, I'll watch the remaining solr issue for updates.
>> >>>>>
>> >>>>> Le mar. 4 sept. 2018 à 10:21, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> Sure Jan, this is a nice cleanup, +1 to backport in 7x.
>> >>>>>>
>> >>>>>> Le mar. 4 sept. 2018 à 10:16, Jan Høydahl <[hidden email]> a écrit :
>> >>>>>>>
>> >>>>>>> Jim, we have some release process improvements in LUCENE-5143. Basically, we'll only have one KEYS file instead of three plus those in the versioned folders that we have today. And the release py script will start checking that the RM's key is present in the KEYS file. Would you be ok with that being committed and you being the first RM to use it for 7.5.0?
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Jan Høydahl, search solution architect
>> >>>>>>> Cominvent AS - www.cominvent.com
>> >>>>>>>
>> >>>>>>> 3. sep. 2018 kl. 10:42 skrev jim ferenczi <[hidden email]>:
>> >>>>>>>
>> >>>>>>> Hi all,
>> >>>>>>>
>> >>>>>>> 7.4 has been released two months ago on June 29th and we have new features, enhancements and fixes that are not released yet so I'd like to start working on releasing Lucene/Solr 7.5.0.
>> >>>>>>> There's also a bad bug with index sorting that deletes the wrong documents when delete by query is used:
>> >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8466
>> >>>>>>>
>> >>>>>>> I can create the 7.5 branch later this week and build the first RC early next week if that works for everyone. Please let me know if there are bug fixes that needs to be fixed in 7.5 and might not be ready by then.
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> Jim
>> >>>>>>>
>> >>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

123