Lucene/Solr 8.0

classic Classic list List threaded Threaded
105 messages Options
123456
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

caomanhdat
Thanks Cassandra and Jim,

I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week. 

On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
> But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.

Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.

It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.

Cassandra

On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Nicholas Knize
If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?

- Nick


On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
Thanks Cassandra and Jim,

I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week. 

On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
> But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.

Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.

It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.

Cassandra

On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--

Nicholas Knize, Ph.D., GISP
Geospatial Software Guy  |  Elasticsearch
Apache Lucene Committer
[hidden email]  

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

david.w.smiley@gmail.com
+1 to a 7.6 —lots of stuff in there
On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?

- Nick


On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
Thanks Cassandra and Jim,

I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week. 

On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
> But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.

Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.

It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.

Cassandra

On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--

Nicholas Knize, Ph.D., GISP
Geospatial Software Guy  |  Elasticsearch
Apache Lucene Committer
[hidden email]  

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
+1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?

Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
+1 to a 7.6 —lots of stuff in there
On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?

- Nick


On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
Thanks Cassandra and Jim,

I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week. 

On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
> But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.

Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.

It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.

Cassandra

On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--

Nicholas Knize, Ph.D., GISP
Geospatial Software Guy  |  Elasticsearch
Apache Lucene Committer
[hidden email]  

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Erick Erickson
+1, this gives us all a chance to prioritize getting the blockers out
of the way in a careful manner.
On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:

>
> +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>
> Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>>
>> +1 to a 7.6 —lots of stuff in there
>> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>>>
>>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>>>
>>> - Nick
>>>
>>>
>>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>
>>>> Thanks Cassandra and Jim,
>>>>
>>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>>>>
>>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>>>>>
>>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>>>>>
>>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>>>>>
>>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>>>>>>
>>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>>>>>>
>>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>>>>>>
>>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>>>>>>
>>>>>> Cassandra
>>>>>>
>>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>>>>>>>
>>>>>>> Ok thanks for answering.
>>>>>>>
>>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>>>>>>>
>>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>>>>>>>
>>>>>>>
>>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>>>>>>>>
>>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>>>>>>>>
>>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>>>>>>>>
>>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>>>>>>>>
>>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>>>>>>>>
>>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>>>>>>>>
>>>>>>>> Cassandra
>>>>>>>>
>>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> I find 9 open blockers for 8.0:
>>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>>>>>>>>>
>>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi,
>>>>>>>>> >
>>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>>>>>>>>> >
>>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>>>>>>>>> >
>>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>>>>>>>>> >
>>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>>>>>>>>> >
>>>>>>>>> > ~ David
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Hi,
>>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>
>>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>>>>>>> >>>
>>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>>
>>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>>>>>> >>>>
>>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>>>>> >>>>>> I think there's a blocker JIRA.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> Shows 6 blockers
>>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Hi Jim,
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Thanks!
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Hi all,
>>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Cheers,
>>>>>>>>> >>>>>> >> Jim
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>
>>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>>>>> >>>>>> >>> ~ David
>>>>>>>>> >>>>>> >>>
>>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>
>>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>>>>> >>>>>> >>>>
>>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>>>>> >>>>>> >>>>> > 2018?
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>>>>> >>>>>> >>>>> >>> too invasive.
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>>>>> >>>>>> >>>>> >>> > already
>>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>>>>> >>>>>> >>>>> >>> > when
>>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>>>>> >>>>>> >>>>> >>> > only in
>>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>>>>> >>>>>> >>>>> >>> > which
>>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>>>>> >>>>>> >>>>> >>> > the
>>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>>>>> >>>>>> >>>>> >>> > API
>>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>>>>> >>>>>> >>>>> >>> > as we
>>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>>>>> >>>>>> >>>>> >>> > be
>>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>>>>> >>>>>> >>>>> >>> > worth
>>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>>>>> >>>>>> >>>>> >>> > to
>>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > Adrien
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>> --
>>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>>
>>>>>>>>> > --
>>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>>
>>> --
>>>
>>> Nicholas Knize, Ph.D., GISP
>>> Geospatial Software Guy  |  Elasticsearch
>>> Apache Lucene Committer
>>> [hidden email]
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Cassandra Targett
I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.

On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
+1, this gives us all a chance to prioritize getting the blockers out
of the way in a careful manner.
On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>
> +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>
> Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>>
>> +1 to a 7.6 —lots of stuff in there
>> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>>>
>>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>>>
>>> - Nick
>>>
>>>
>>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>
>>>> Thanks Cassandra and Jim,
>>>>
>>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>>>>
>>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>>>>>
>>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>>>>>
>>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>>>>>
>>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>>>>>>
>>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>>>>>>
>>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>>>>>>
>>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>>>>>>
>>>>>> Cassandra
>>>>>>
>>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>>>>>>>
>>>>>>> Ok thanks for answering.
>>>>>>>
>>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>>>>>>>
>>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>>>>>>>
>>>>>>>
>>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>>>>>>>>
>>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>>>>>>>>
>>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>>>>>>>>
>>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>>>>>>>>
>>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>>>>>>>>
>>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>>>>>>>>
>>>>>>>> Cassandra
>>>>>>>>
>>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> I find 9 open blockers for 8.0:
>>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>>>>>>>>>
>>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi,
>>>>>>>>> >
>>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>>>>>>>>> >
>>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>>>>>>>>> >
>>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>>>>>>>>> >
>>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>>>>>>>>> >
>>>>>>>>> > ~ David
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Hi,
>>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>
>>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>>>>>>> >>>
>>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>>
>>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>>>>>> >>>>
>>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>>>>> >>>>>> I think there's a blocker JIRA.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> Shows 6 blockers
>>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Hi Jim,
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Thanks!
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Hi all,
>>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Cheers,
>>>>>>>>> >>>>>> >> Jim
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>
>>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>>>>> >>>>>> >>> ~ David
>>>>>>>>> >>>>>> >>>
>>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>
>>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>>>>> >>>>>> >>>>
>>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>>>>> >>>>>> >>>>> > 2018?
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>>>>> >>>>>> >>>>> >>> too invasive.
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>>>>> >>>>>> >>>>> >>> > already
>>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>>>>> >>>>>> >>>>> >>> > when
>>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>>>>> >>>>>> >>>>> >>> > only in
>>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>>>>> >>>>>> >>>>> >>> > which
>>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>>>>> >>>>>> >>>>> >>> > the
>>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>>>>> >>>>>> >>>>> >>> > API
>>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>>>>> >>>>>> >>>>> >>> > as we
>>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>>>>> >>>>>> >>>>> >>> > be
>>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>>>>> >>>>>> >>>>> >>> > worth
>>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>>>>> >>>>>> >>>>> >>> > to
>>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > Adrien
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>> --
>>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>>
>>>>>>>>> > --
>>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>>
>>> --
>>>
>>> Nicholas Knize, Ph.D., GISP
>>> Geospatial Software Guy  |  Elasticsearch
>>> Apache Lucene Committer
>>> [hidden email]
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Alan Woodward-3
Hi all,

Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.

On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:

I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.

On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
+1, this gives us all a chance to prioritize getting the blockers out
of the way in a careful manner.
On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>
> +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>
> Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>>
>> +1 to a 7.6 —lots of stuff in there
>> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>>>
>>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>>>
>>> - Nick
>>>
>>>
>>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>
>>>> Thanks Cassandra and Jim,
>>>>
>>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>>>>
>>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>>>>>
>>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>>>>>
>>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>>>>>
>>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>>>>>>
>>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>>>>>>
>>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>>>>>>
>>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>>>>>>
>>>>>> Cassandra
>>>>>>
>>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>>>>>>>
>>>>>>> Ok thanks for answering.
>>>>>>>
>>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>>>>>>>
>>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>>>>>>>
>>>>>>>
>>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>>>>>>>>
>>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>>>>>>>>
>>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>>>>>>>>
>>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>>>>>>>>
>>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>>>>>>>>
>>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>>>>>>>>
>>>>>>>> Cassandra
>>>>>>>>
>>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> I find 9 open blockers for 8.0:
>>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>>>>>>>>>
>>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi,
>>>>>>>>> >
>>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>>>>>>>>> >
>>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>>>>>>>>> >
>>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>>>>>>>>> >
>>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>>>>>>>>> >
>>>>>>>>> > ~ David
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Hi,
>>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>
>>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>>>>>>> >>>
>>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>>
>>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>>>>>> >>>>
>>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>>>>> >>>>>> I think there's a blocker JIRA.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> Shows 6 blockers
>>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Hi Jim,
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Thanks!
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Hi all,
>>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Cheers,
>>>>>>>>> >>>>>> >> Jim
>>>>>>>>> >>>>>> >>
>>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>
>>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>>>>> >>>>>> >>> ~ David
>>>>>>>>> >>>>>> >>>
>>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>
>>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>>>>> >>>>>> >>>>
>>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>>>>> >>>>>> >>>>> > 2018?
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>>>>>>>>> >>>>>> >>>>> >>
>>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>>>>> >>>>>> >>>>> >>> too invasive.
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>>>>> >>>>>> >>>>> >>> > already
>>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>>>>> >>>>>> >>>>> >>> > when
>>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>>>>> >>>>>> >>>>> >>> > only in
>>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>>>>> >>>>>> >>>>> >>> > which
>>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>>>>> >>>>>> >>>>> >>> > the
>>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>>>>> >>>>>> >>>>> >>> > API
>>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>>>>> >>>>>> >>>>> >>> > as we
>>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>>>>> >>>>>> >>>>> >>> > be
>>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>>>>> >>>>>> >>>>> >>> > worth
>>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>>>>> >>>>>> >>>>> >>> > to
>>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>>>>>>>>> >>>>>> >>>>> >>> >
>>>>>>>>> >>>>>> >>>>> >>> > Adrien
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> >>>
>>>>>>>>> >>>>>> >>>>> >
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>> >>>>>
>>>>>>>>> >>>>>> >>> --
>>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> ---------------------------------------------------------------------
>>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>> >>>>>>
>>>>>>>>> > --
>>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>>
>>> --
>>>
>>> Nicholas Knize, Ph.D., GISP
>>> Geospatial Software Guy  |  Elasticsearch
>>> Apache Lucene Committer
>>> [hidden email]
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand
+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:

>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

david.w.smiley@gmail.com
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:

Thru the end of the month, I intend to work on those issues not yet assigned. 

On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:
+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:
>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
S G
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

S G
It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
Any idea when Solr 8 would be out ?

Thx
SG

On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:

Thru the end of the month, I intend to work on those issues not yet assigned. 

On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:
+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:
>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

david.w.smiley@gmail.com
January.

On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:
It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
Any idea when Solr 8 would be out ?

Thx
SG

On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:

Thru the end of the month, I intend to work on those issues not yet assigned. 

On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:
+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:
>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
--
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Alan Woodward-3
OK, Christmas caught up with me a bit… I’ve just created a branch for 8x from master, and am in the process of updating the master branch to version 9.  New commits that should be included in the 8.0 release should also be back-ported to branch_8x from master.

This is not intended as a feature freeze, as I know there are still some things being worked on for 8.0; however, it should let us clean up master by removing as much deprecated code as possible, and give us an idea of any replacement work that needs to be done.

On 19 Dec 2018, at 15:13, David Smiley <[hidden email]> wrote:

January.

On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:
It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
Any idea when Solr 8 would be out ?

Thx
SG

On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:

Thru the end of the month, I intend to work on those issues not yet assigned. 

On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:
+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:
>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
--
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Chris Hostetter-3
: OK, Christmas caught up with me a bit… I’ve just created a branch for 8x
: from master, and am in the process of updating the master branch to
: version 9.  New commits that should be included in the 8.0 release
: should also be back-ported to branch_8x from master.

It looks like someone renamed the "master (8.0)" version of SOLR & LUCENE
in Jira to "master (9.0)" but IIUC that's definitely *NOT* correct ...
because it means all the stuff that's been committed to origin/master over
the past X months won't be listed as "fixed in '8.0'" when people look
at jira in the future.

I'm pretty sure "master (8.0)" should have been renamed "8.0" and a completely
new version (with a new internal ID in jira) should have been added for
"master (9.0)"

        Right?

(In the meantime, it seems folks have already added new "8.0"
versions for SOLR/LUCENE to Jira, which have a handful of issues mapped to
them, that will need cleaned up)



: > >> >>>>>>
: > >> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>
: > >> >>>>>>> Ok thanks for answering.
: > >> >>>>>>>
: > >> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
: > >> >>>>>>>
: > >> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
: > >> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
: > >> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
: > >> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
: > >> >>>>>>>
: > >> >>>>>>>
: > >> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>
: > >> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
: > >> >>>>>>>>
: > >> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
: > >> >>>>>>>>
: > >> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
: > >> >>>>>>>>
: > >> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
: > >> >>>>>>>>
: > >> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
: > >> >>>>>>>>
: > >> >>>>>>>> Cassandra
: > >> >>>>>>>>
: > >> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>>
: > >> >>>>>>>>> I find 9 open blockers for 8.0:
: > >> >>>>>>>>>
: > >> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN <https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN>
: > >> >>>>>>>>>
: > >> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
: > >> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
: > >> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > Hi,
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 <https://issues.apache.org/jira/browse/LUCENE-7875> RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > ~ David
: > >> >>>>>>>>> >
: > >> >>>>>>>>> >
: > >> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >>
: > >> >>>>>>>>> >> Hi,
: > >> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
: > >> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20 <https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20>
: > >> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
: > >> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
: > >> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
: > >> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
: > >> >>>>>>>>> >>
: > >> >>>>>>>>> >>
: > >> >>>>>>>>> >>
: > >> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>
: > >> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 <https://issues.apache.org/jira/browse/SOLR-12639> the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
: > >> >>>>>>>>> >>>
: > >> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>
: > >> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20 <https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20>
: > >> >>>>>>>>> >>>>
: > >> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>>
: > >> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
: > >> >>>>>>>>> >>>>>
: > >> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>>>
: > >> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
: > >> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
: > >> >>>>>>>>> >>>>>>
: > >> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
: > >> >>>>>>>>> >>>>>>
: > >> >>>>>>>>> >>>>>> Shows 6 blockers
: > >> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >>>>>> >
: > >> >>>>>>>>> >>>>>> > Hi Jim,
: > >> >>>>>>>>> >>>>>> >
: > >> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
: > >> >>>>>>>>> >>>>>> >
: > >> >>>>>>>>> >>>>>> > Thanks!
: > >> >>>>>>>>> >>>>>> >
: > >> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >>>>>> >>
: > >> >>>>>>>>> >>>>>> >> Hi all,
: > >> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
: > >> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
: > >> >>>>>>>>> >>>>>> >>
: > >> >>>>>>>>> >>>>>> >> Cheers,
: > >> >>>>>>>>> >>>>>> >> Jim
: > >> >>>>>>>>> >>>>>> >>
: > >> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>>> >>>
: > >> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
: > >> >>>>>>>>> >>>>>> >>> ~ David
: > >> >>>>>>>>> >>>>>> >>>
: > >> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >>>>>> >>>>
: > >> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
: > >> >>>>>>>>> >>>>>> >>>>
: > >> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>>> >>>>>
: > >> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
: > >> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
: > >> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
: > >> >>>>>>>>> >>>>>> >>>>>
: > >> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
: > >> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
: > >> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060 <https://issues.apache.org/jira/browse/LUCENE-8060>). Any
: > >> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
: > >> >>>>>>>>> >>>>>> >>>>> > 2018?
: > >> >>>>>>>>> >>>>>> >>>>> >
: > >> >>>>>>>>> >>>>>> >>>>> >
: > >> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>>> >>>>> >>
: > >> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
: > >> >>>>>>>>> >>>>>> >>>>> >>
: > >> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
: > >> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204 <https://issues.apache.org/jira/browse/LUCENE-8204>)
: > >> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
: > >> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197 <https://issues.apache.org/jira/browse/LUCENE-8197>) in an optional
: > >> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
: > >> >>>>>>>>> >>>>>> >>>>> >>
: > >> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email] <mailto:[hidden email]>> a écrit :
: > >> >>>>>>>>> >>>>>> >>>>> >>>
: > >> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
: > >> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
: > >> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
: > >> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
: > >> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
: > >> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
: > >> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
: > >> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
: > >> >>>>>>>>> >>>>>> >>>>> >>>
: > >> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
: > >> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
: > >> >>>>>>>>> >>>>>> >>>>> >>> > already
: > >> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
: > >> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
: > >> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
: > >> >>>>>>>>> >>>>>> >>>>> >>> > when
: > >> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116 <https://issues.apache.org/jira/browse/LUCENE-8116>
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020 <https://issues.apache.org/jira/browse/LUCENE-8020>
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007 <https://issues.apache.org/jira/browse/LUCENE-8007>
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198 <https://issues.apache.org/jira/browse/LUCENE-4198>
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135 <https://issues.apache.org/jira/browse/LUCENE-8135>
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
: > >> >>>>>>>>> >>>>>> >>>>> >>> > only in
: > >> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031 <https://issues.apache.org/jira/browse/LUCENE-8031>
: > >> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134 <https://issues.apache.org/jira/browse/LUCENE-8134>
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
: > >> >>>>>>>>> >>>>>> >>>>> >>> > which
: > >> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
: > >> >>>>>>>>> >>>>>> >>>>> >>> > the
: > >> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
: > >> >>>>>>>>> >>>>>> >>>>> >>> > API
: > >> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
: > >> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
: > >> >>>>>>>>> >>>>>> >>>>> >>> > as we
: > >> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
: > >> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
: > >> >>>>>>>>> >>>>>> >>>>> >>> > be
: > >> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
: > >> >>>>>>>>> >>>>>> >>>>> >>> > worth
: > >> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
: > >> >>>>>>>>> >>>>>> >>>>> >>> > to
: > >> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
: > >> >>>>>>>>> >>>>>> >>>>> >>> >
: > >> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
: > >> >>>>>>>>> >>>>>> >>>>> >>>
: > >> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
: > >> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> >>>>>> >>>>> >>>
: > >> >>>>>>>>> >>>>>> >>>>> >
: > >> >>>>>>>>> >>>>>> >>>>>
: > >> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
: > >> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> >>>>>> >>>>>
: > >> >>>>>>>>> >>>>>> >>> --
: > >> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
: > >> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
: > >> >>>>>>>>> >>>>>>
: > >> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
: > >> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> >>>>>>
: > >> >>>>>>>>> > --
: > >> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
: > >> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
: > >> >>>>>>>>>
: > >> >>>>>>>>> ---------------------------------------------------------------------
: > >> >>>>>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
: > >> >>>>>>>>>
: > >> >>> --
: > >> >>>
: > >> >>> Nicholas Knize, Ph.D., GISP
: > >> >>> Geospatial Software Guy  |  Elasticsearch
: > >> >>> Apache Lucene Committer
: > >> >>> [hidden email] <mailto:[hidden email]>
: > >> >>
: > >> >> --
: > >> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
: > >> >> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
: > >>
: > >> ---------------------------------------------------------------------
: > >> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
: > >> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
: > >>
: > >
: >
: >
: > --
: > Adrien
: >
: > ---------------------------------------------------------------------
: > To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
: > For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
: >
: > --
: > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
: > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>--
: > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
: > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
:

-Hoss
http://www.lucidworks.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Lucene/Solr 8.0

Uwe Schindler
In reply to this post by Alan Woodward-3

Hi,

 

I will start and add the branch_8x jobs to Jenkins once I have some time later today.

 

The question: How to proceed with branch_7x? Should we stop using it and release 7.6.x only (so we would use branch_7_6 only for bugfixes), or are we planning to one more Lucene/Solr 7.7? In the latter case I would keep the jenkins jobs enabled for a while.

 

Uwe

 

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de

eMail: [hidden email]

 

From: Alan Woodward <[hidden email]>
Sent: Monday, January 7, 2019 11:30 AM
To: [hidden email]
Subject: Re: Lucene/Solr 8.0

 

OK, Christmas caught up with me a bit… I’ve just created a branch for 8x from master, and am in the process of updating the master branch to version 9.  New commits that should be included in the 8.0 release should also be back-ported to branch_8x from master.

 

This is not intended as a feature freeze, as I know there are still some things being worked on for 8.0; however, it should let us clean up master by removing as much deprecated code as possible, and give us an idea of any replacement work that needs to be done.



On 19 Dec 2018, at 15:13, David Smiley <[hidden email]> wrote:

 

January.

 

On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:

It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.

Any idea when Solr 8 would be out ?

 

Thx

SG

 

On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:

I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 

   click here:

 

Thru the end of the month, I intend to work on those issues not yet assigned. 

 

On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:

+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:


>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--

Lucene/Solr Search Committer (PMC), Developer, Author, Speaker

--

Lucene/Solr Search Committer (PMC), Developer, Author, Speaker

 

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Alan Woodward-3
In reply to this post by Chris Hostetter-3
> It looks like someone renamed the "master (8.0)" version of SOLR & LUCENE
> in Jira to "master (9.0)" but IIUC that's definitely *NOT* correct ...
> because it means all the stuff that's been committed to origin/master over
> the past X months won't be listed as "fixed in '8.0'" when people look
> at jira in the future.

That would be me… I’ll clean it up, thanks for pointing it out Hoss.

> On 7 Jan 2019, at 23:45, Chris Hostetter <[hidden email]> wrote:
>
> : OK, Christmas caught up with me a bit… I’ve just created a branch for 8x
> : from master, and am in the process of updating the master branch to
> : version 9.  New commits that should be included in the 8.0 release
> : should also be back-ported to branch_8x from master.
>
> It looks like someone renamed the "master (8.0)" version of SOLR & LUCENE
> in Jira to "master (9.0)" but IIUC that's definitely *NOT* correct ...
> because it means all the stuff that's been committed to origin/master over
> the past X months won't be listed as "fixed in '8.0'" when people look
> at jira in the future.
>
> I'm pretty sure "master (8.0)" should have been renamed "8.0" and a completely
> new version (with a new internal ID in jira) should have been added for
> "master (9.0)"
>
> Right?
>
> (In the meantime, it seems folks have already added new "8.0"
> versions for SOLR/LUCENE to Jira, which have a handful of issues mapped to
> them, that will need cleaned up)
>
>
>
> : > >> >>>>>>
> : > >> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>
> : > >> >>>>>>> Ok thanks for answering.
> : > >> >>>>>>>
> : > >> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
> : > >> >>>>>>>
> : > >> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
> : > >> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
> : > >> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
> : > >> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
> : > >> >>>>>>>
> : > >> >>>>>>>
> : > >> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>
> : > >> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
> : > >> >>>>>>>>
> : > >> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
> : > >> >>>>>>>>
> : > >> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
> : > >> >>>>>>>>
> : > >> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
> : > >> >>>>>>>>
> : > >> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
> : > >> >>>>>>>>
> : > >> >>>>>>>> Cassandra
> : > >> >>>>>>>>
> : > >> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>>
> : > >> >>>>>>>>> I find 9 open blockers for 8.0:
> : > >> >>>>>>>>>
> : > >> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN <https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN>
> : > >> >>>>>>>>>
> : > >> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
> : > >> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
> : > >> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > Hi,
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 <https://issues.apache.org/jira/browse/LUCENE-7875> RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > ~ David
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> >
> : > >> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >>
> : > >> >>>>>>>>> >> Hi,
> : > >> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
> : > >> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20 <https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20>
> : > >> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
> : > >> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
> : > >> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
> : > >> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
> : > >> >>>>>>>>> >>
> : > >> >>>>>>>>> >>
> : > >> >>>>>>>>> >>
> : > >> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>
> : > >> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 <https://issues.apache.org/jira/browse/SOLR-12639> the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
> : > >> >>>>>>>>> >>>
> : > >> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>
> : > >> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20 <https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20>
> : > >> >>>>>>>>> >>>>
> : > >> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>>
> : > >> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
> : > >> >>>>>>>>> >>>>>
> : > >> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>>>
> : > >> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
> : > >> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
> : > >> >>>>>>>>> >>>>>>
> : > >> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
> : > >> >>>>>>>>> >>>>>>
> : > >> >>>>>>>>> >>>>>> Shows 6 blockers
> : > >> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >>>>>> >
> : > >> >>>>>>>>> >>>>>> > Hi Jim,
> : > >> >>>>>>>>> >>>>>> >
> : > >> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
> : > >> >>>>>>>>> >>>>>> >
> : > >> >>>>>>>>> >>>>>> > Thanks!
> : > >> >>>>>>>>> >>>>>> >
> : > >> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >>>>>> >>
> : > >> >>>>>>>>> >>>>>> >> Hi all,
> : > >> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
> : > >> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
> : > >> >>>>>>>>> >>>>>> >>
> : > >> >>>>>>>>> >>>>>> >> Cheers,
> : > >> >>>>>>>>> >>>>>> >> Jim
> : > >> >>>>>>>>> >>>>>> >>
> : > >> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>>> >>>
> : > >> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
> : > >> >>>>>>>>> >>>>>> >>> ~ David
> : > >> >>>>>>>>> >>>>>> >>>
> : > >> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >>>>>> >>>>
> : > >> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
> : > >> >>>>>>>>> >>>>>> >>>>
> : > >> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>>> >>>>>
> : > >> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
> : > >> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
> : > >> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
> : > >> >>>>>>>>> >>>>>> >>>>>
> : > >> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
> : > >> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
> : > >> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060 <https://issues.apache.org/jira/browse/LUCENE-8060>). Any
> : > >> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
> : > >> >>>>>>>>> >>>>>> >>>>> > 2018?
> : > >> >>>>>>>>> >>>>>> >>>>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >
> : > >> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>>> >>>>> >>
> : > >> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
> : > >> >>>>>>>>> >>>>>> >>>>> >>
> : > >> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
> : > >> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204 <https://issues.apache.org/jira/browse/LUCENE-8204>)
> : > >> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
> : > >> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197 <https://issues.apache.org/jira/browse/LUCENE-8197>) in an optional
> : > >> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
> : > >> >>>>>>>>> >>>>>> >>>>> >>
> : > >> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email] <mailto:[hidden email]>> a écrit :
> : > >> >>>>>>>>> >>>>>> >>>>> >>>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
> : > >> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
> : > >> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
> : > >> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
> : > >> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
> : > >> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
> : > >> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
> : > >> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
> : > >> >>>>>>>>> >>>>>> >>>>> >>>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > already
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > when
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116 <https://issues.apache.org/jira/browse/LUCENE-8116>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020 <https://issues.apache.org/jira/browse/LUCENE-8020>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007 <https://issues.apache.org/jira/browse/LUCENE-8007>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198 <https://issues.apache.org/jira/browse/LUCENE-4198>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135 <https://issues.apache.org/jira/browse/LUCENE-8135>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > only in
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031 <https://issues.apache.org/jira/browse/LUCENE-8031>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134 <https://issues.apache.org/jira/browse/LUCENE-8134>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > which
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > the
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > API
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > as we
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > be
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > worth
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > to
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
> : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> : > >> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
> : > >> >>>>>>>>> >>>>>> >>>>> >>>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
> : > >> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> >>>>>> >>>>> >>>
> : > >> >>>>>>>>> >>>>>> >>>>> >
> : > >> >>>>>>>>> >>>>>> >>>>>
> : > >> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
> : > >> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> >>>>>> >>>>>
> : > >> >>>>>>>>> >>>>>> >>> --
> : > >> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> : > >> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> : > >> >>>>>>>>> >>>>>>
> : > >> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
> : > >> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> >>>>>>
> : > >> >>>>>>>>> > --
> : > >> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> : > >> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> : > >> >>>>>>>>>
> : > >> >>>>>>>>> ---------------------------------------------------------------------
> : > >> >>>>>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> >>>>>>>>>
> : > >> >>> --
> : > >> >>>
> : > >> >>> Nicholas Knize, Ph.D., GISP
> : > >> >>> Geospatial Software Guy  |  Elasticsearch
> : > >> >>> Apache Lucene Committer
> : > >> >>> [hidden email] <mailto:[hidden email]>
> : > >> >>
> : > >> >> --
> : > >> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> : > >> >> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> : > >>
> : > >> ---------------------------------------------------------------------
> : > >> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> : > >> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> : > >>
> : > >
> : >
> : >
> : > --
> : > Adrien
> : >
> : > ---------------------------------------------------------------------
> : > To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> : > For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> : >
> : > --
> : > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
> : > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>--
> : > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
> : > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Dawid Weiss-2
Thanks for doing this Alan. I'll handle RAMDirectory* removals from
the new master (LUCENE-8474)

D.

On Tue, Jan 8, 2019 at 10:11 AM Alan Woodward <[hidden email]> wrote:

>
> > It looks like someone renamed the "master (8.0)" version of SOLR & LUCENE
> > in Jira to "master (9.0)" but IIUC that's definitely *NOT* correct ...
> > because it means all the stuff that's been committed to origin/master over
> > the past X months won't be listed as "fixed in '8.0'" when people look
> > at jira in the future.
>
> That would be me… I’ll clean it up, thanks for pointing it out Hoss.
>
> > On 7 Jan 2019, at 23:45, Chris Hostetter <[hidden email]> wrote:
> >
> > : OK, Christmas caught up with me a bit… I’ve just created a branch for 8x
> > : from master, and am in the process of updating the master branch to
> > : version 9.  New commits that should be included in the 8.0 release
> > : should also be back-ported to branch_8x from master.
> >
> > It looks like someone renamed the "master (8.0)" version of SOLR & LUCENE
> > in Jira to "master (9.0)" but IIUC that's definitely *NOT* correct ...
> > because it means all the stuff that's been committed to origin/master over
> > the past X months won't be listed as "fixed in '8.0'" when people look
> > at jira in the future.
> >
> > I'm pretty sure "master (8.0)" should have been renamed "8.0" and a completely
> > new version (with a new internal ID in jira) should have been added for
> > "master (9.0)"
> >
> >       Right?
> >
> > (In the meantime, it seems folks have already added new "8.0"
> > versions for SOLR/LUCENE to Jira, which have a handful of issues mapped to
> > them, that will need cleaned up)
> >
> >
> >
> > : > >> >>>>>>
> > : > >> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>
> > : > >> >>>>>>> Ok thanks for answering.
> > : > >> >>>>>>>
> > : > >> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
> > : > >> >>>>>>>
> > : > >> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
> > : > >> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
> > : > >> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
> > : > >> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
> > : > >> >>>>>>>
> > : > >> >>>>>>>
> > : > >> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> Cassandra
> > : > >> >>>>>>>>
> > : > >> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>>
> > : > >> >>>>>>>>> I find 9 open blockers for 8.0:
> > : > >> >>>>>>>>>
> > : > >> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN <https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN>
> > : > >> >>>>>>>>>
> > : > >> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
> > : > >> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
> > : > >> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > Hi,
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 <https://issues.apache.org/jira/browse/LUCENE-7875> RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > ~ David
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> >
> > : > >> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >>
> > : > >> >>>>>>>>> >> Hi,
> > : > >> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
> > : > >> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20 <https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20>
> > : > >> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
> > : > >> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
> > : > >> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
> > : > >> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
> > : > >> >>>>>>>>> >>
> > : > >> >>>>>>>>> >>
> > : > >> >>>>>>>>> >>
> > : > >> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>
> > : > >> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 <https://issues.apache.org/jira/browse/SOLR-12639> the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
> > : > >> >>>>>>>>> >>>
> > : > >> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>
> > : > >> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20 <https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20>
> > : > >> >>>>>>>>> >>>>
> > : > >> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>>
> > : > >> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
> > : > >> >>>>>>>>> >>>>>
> > : > >> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>>>
> > : > >> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
> > : > >> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
> > : > >> >>>>>>>>> >>>>>>
> > : > >> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
> > : > >> >>>>>>>>> >>>>>>
> > : > >> >>>>>>>>> >>>>>> Shows 6 blockers
> > : > >> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >>>>>> >
> > : > >> >>>>>>>>> >>>>>> > Hi Jim,
> > : > >> >>>>>>>>> >>>>>> >
> > : > >> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
> > : > >> >>>>>>>>> >>>>>> >
> > : > >> >>>>>>>>> >>>>>> > Thanks!
> > : > >> >>>>>>>>> >>>>>> >
> > : > >> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >>>>>> >>
> > : > >> >>>>>>>>> >>>>>> >> Hi all,
> > : > >> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
> > : > >> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
> > : > >> >>>>>>>>> >>>>>> >>
> > : > >> >>>>>>>>> >>>>>> >> Cheers,
> > : > >> >>>>>>>>> >>>>>> >> Jim
> > : > >> >>>>>>>>> >>>>>> >>
> > : > >> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>>> >>>
> > : > >> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
> > : > >> >>>>>>>>> >>>>>> >>> ~ David
> > : > >> >>>>>>>>> >>>>>> >>>
> > : > >> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >>>>>> >>>>
> > : > >> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
> > : > >> >>>>>>>>> >>>>>> >>>>
> > : > >> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>>> >>>>>
> > : > >> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
> > : > >> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
> > : > >> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
> > : > >> >>>>>>>>> >>>>>> >>>>>
> > : > >> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
> > : > >> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
> > : > >> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060 <https://issues.apache.org/jira/browse/LUCENE-8060>). Any
> > : > >> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
> > : > >> >>>>>>>>> >>>>>> >>>>> > 2018?
> > : > >> >>>>>>>>> >>>>>> >>>>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>>> >>>>> >>
> > : > >> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
> > : > >> >>>>>>>>> >>>>>> >>>>> >>
> > : > >> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
> > : > >> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204 <https://issues.apache.org/jira/browse/LUCENE-8204>)
> > : > >> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
> > : > >> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197 <https://issues.apache.org/jira/browse/LUCENE-8197>) in an optional
> > : > >> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
> > : > >> >>>>>>>>> >>>>>> >>>>> >>
> > : > >> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email] <mailto:[hidden email]>> a écrit :
> > : > >> >>>>>>>>> >>>>>> >>>>> >>>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
> > : > >> >>>>>>>>> >>>>>> >>>>> >>>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email] <mailto:[hidden email]>> wrote:
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > already
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > when
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116 <https://issues.apache.org/jira/browse/LUCENE-8116>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020 <https://issues.apache.org/jira/browse/LUCENE-8020>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007 <https://issues.apache.org/jira/browse/LUCENE-8007>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198 <https://issues.apache.org/jira/browse/LUCENE-4198>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135 <https://issues.apache.org/jira/browse/LUCENE-8135>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > only in
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031 <https://issues.apache.org/jira/browse/LUCENE-8031>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134 <https://issues.apache.org/jira/browse/LUCENE-8134>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > which
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > the
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > API
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > as we
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > be
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > worth
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > to
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> >
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
> > : > >> >>>>>>>>> >>>>>> >>>>> >>>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> >>>>>> >>>>> >>>
> > : > >> >>>>>>>>> >>>>>> >>>>> >
> > : > >> >>>>>>>>> >>>>>> >>>>>
> > : > >> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
> > : > >> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> >>>>>> >>>>>
> > : > >> >>>>>>>>> >>>>>> >>> --
> > : > >> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> > : > >> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> > : > >> >>>>>>>>> >>>>>>
> > : > >> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
> > : > >> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> >>>>>>
> > : > >> >>>>>>>>> > --
> > : > >> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> > : > >> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> > : > >> >>>>>>>>>
> > : > >> >>>>>>>>> ---------------------------------------------------------------------
> > : > >> >>>>>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> >>>>>>>>>
> > : > >> >>> --
> > : > >> >>>
> > : > >> >>> Nicholas Knize, Ph.D., GISP
> > : > >> >>> Geospatial Software Guy  |  Elasticsearch
> > : > >> >>> Apache Lucene Committer
> > : > >> >>> [hidden email] <mailto:[hidden email]>
> > : > >> >>
> > : > >> >> --
> > : > >> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> > : > >> >> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> > : > >>
> > : > >> ---------------------------------------------------------------------
> > : > >> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> > : > >>
> > : > >
> > : >
> > : >
> > : > --
> > : > Adrien
> > : >
> > : > ---------------------------------------------------------------------
> > : > To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
> > : > For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
> > : >
> > : > --
> > : > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
> > : > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>--
> > : > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
> > : > LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
> > :
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Alan Woodward-3
In reply to this post by Uwe Schindler
I think the current plan is to do a 7.7 release at the same time as 8.0, to handle any last-minute deprecations etc.  So let’s keep those jobs enabled for now.

On 8 Jan 2019, at 09:10, Uwe Schindler <[hidden email]> wrote:

Hi,
 
I will start and add the branch_8x jobs to Jenkins once I have some time later today.
 
The question: How to proceed with branch_7x? Should we stop using it and release 7.6.x only (so we would use branch_7_6 only for bugfixes), or are we planning to one more Lucene/Solr 7.7? In the latter case I would keep the jenkins jobs enabled for a while.
 
Uwe
 
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
 
From: Alan Woodward <[hidden email]> 
Sent: Monday, January 7, 2019 11:30 AM
To: [hidden email]
Subject: Re: Lucene/Solr 8.0
 
OK, Christmas caught up with me a bit… I’ve just created a branch for 8x from master, and am in the process of updating the master branch to version 9.  New commits that should be included in the 8.0 release should also be back-ported to branch_8x from master.
 
This is not intended as a feature freeze, as I know there are still some things being worked on for 8.0; however, it should let us clean up master by removing as much deprecated code as possible, and give us an idea of any replacement work that needs to be done.


On 19 Dec 2018, at 15:13, David Smiley <[hidden email]> wrote:
 
January.
 
On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:
It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
Any idea when Solr 8 would be out ?
 
Thx
SG
 
On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:
 
Thru the end of the month, I intend to work on those issues not yet assigned. 
 
On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:

+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:


>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


-- 
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

-- 
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
-- 
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Alan Woodward-3
I’ve started to work through the various deprecations on the new master branch.  There are a lot of them, and I’m going to need some assistance for several of them, as it’s not entirely clear what to do.

I’ll open two overarching issues in JIRA, one for lucene and one for Solr, with lists of the deprecations that need to be removed in each one.  I’ll create a shared branch on gitbox to work against, and push the changes I’ve already done there.  We can then create individual JIRA issues for any changes that are more involved than just deleting code.

All assistance gratefully received, particularly for the Solr deprecations where there’s a lot of code I’m unfamiliar with.

On 8 Jan 2019, at 09:21, Alan Woodward <[hidden email]> wrote:

I think the current plan is to do a 7.7 release at the same time as 8.0, to handle any last-minute deprecations etc.  So let’s keep those jobs enabled for now.

On 8 Jan 2019, at 09:10, Uwe Schindler <[hidden email]> wrote:

Hi,
 
I will start and add the branch_8x jobs to Jenkins once I have some time later today.
 
The question: How to proceed with branch_7x? Should we stop using it and release 7.6.x only (so we would use branch_7_6 only for bugfixes), or are we planning to one more Lucene/Solr 7.7? In the latter case I would keep the jenkins jobs enabled for a while.
 
Uwe
 
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
 
From: Alan Woodward <[hidden email]> 
Sent: Monday, January 7, 2019 11:30 AM
To: [hidden email]
Subject: Re: Lucene/Solr 8.0
 
OK, Christmas caught up with me a bit… I’ve just created a branch for 8x from master, and am in the process of updating the master branch to version 9.  New commits that should be included in the 8.0 release should also be back-ported to branch_8x from master.
 
This is not intended as a feature freeze, as I know there are still some things being worked on for 8.0; however, it should let us clean up master by removing as much deprecated code as possible, and give us an idea of any replacement work that needs to be done.


On 19 Dec 2018, at 15:13, David Smiley <[hidden email]> wrote:
 
January.
 
On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:
It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
Any idea when Solr 8 would be out ?
 
Thx
SG
 
On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:
 
Thru the end of the month, I intend to work on those issues not yet assigned. 
 
On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:

+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:


>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


-- 
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

-- 
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
-- 
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker


Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
Hi,
As we agreed some time ago I'd like to start on releasing 8.0. The branch is already created so we can start the process anytime now. Unless there are objections I'd like to start the feature freeze next week in order to build the first candidate the week after. 
We'll also need a 7.7 release but I think we can handle both with Alan so the question now is whether we are ok to start the release process or if there are any blockers left ;).


Le mar. 15 janv. 2019 à 11:35, Alan Woodward <[hidden email]> a écrit :
I’ve started to work through the various deprecations on the new master branch.  There are a lot of them, and I’m going to need some assistance for several of them, as it’s not entirely clear what to do.

I’ll open two overarching issues in JIRA, one for lucene and one for Solr, with lists of the deprecations that need to be removed in each one.  I’ll create a shared branch on gitbox to work against, and push the changes I’ve already done there.  We can then create individual JIRA issues for any changes that are more involved than just deleting code.

All assistance gratefully received, particularly for the Solr deprecations where there’s a lot of code I’m unfamiliar with.

On 8 Jan 2019, at 09:21, Alan Woodward <[hidden email]> wrote:

I think the current plan is to do a 7.7 release at the same time as 8.0, to handle any last-minute deprecations etc.  So let’s keep those jobs enabled for now.

On 8 Jan 2019, at 09:10, Uwe Schindler <[hidden email]> wrote:

Hi,
 
I will start and add the branch_8x jobs to Jenkins once I have some time later today.
 
The question: How to proceed with branch_7x? Should we stop using it and release 7.6.x only (so we would use branch_7_6 only for bugfixes), or are we planning to one more Lucene/Solr 7.7? In the latter case I would keep the jenkins jobs enabled for a while.
 
Uwe
 
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
 
From: Alan Woodward <[hidden email]> 
Sent: Monday, January 7, 2019 11:30 AM
To: [hidden email]
Subject: Re: Lucene/Solr 8.0
 
OK, Christmas caught up with me a bit… I’ve just created a branch for 8x from master, and am in the process of updating the master branch to version 9.  New commits that should be included in the 8.0 release should also be back-ported to branch_8x from master.
 
This is not intended as a feature freeze, as I know there are still some things being worked on for 8.0; however, it should let us clean up master by removing as much deprecated code as possible, and give us an idea of any replacement work that needs to be done.


On 19 Dec 2018, at 15:13, David Smiley <[hidden email]> wrote:
 
January.
 
On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:
It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
Any idea when Solr 8 would be out ?
 
Thx
SG
 
On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)" 
   click here:
 
Thru the end of the month, I intend to work on those issues not yet assigned. 
 
On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:

+1

On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:


>
> Hi all,
>
> Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>
> On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>
> I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>
> On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>>
>> +1, this gives us all a chance to prioritize getting the blockers out
>> of the way in a careful manner.
>> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >
>> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >
>> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >>
>> >> +1 to a 7.6 —lots of stuff in there
>> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >>>
>> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >>>
>> >>> - Nick
>> >>>
>> >>>
>> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>
>> >>>> Thanks Cassandra and Jim,
>> >>>>
>> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >>>>
>> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>
>> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>
>> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >>>>>
>> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>
>> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >>>>>>
>> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >>>>>>
>> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >>>>>>
>> >>>>>> Cassandra
>> >>>>>>
>> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>
>> >>>>>>> Ok thanks for answering.
>> >>>>>>>
>> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>
>> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >>>>>>>>
>> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >>>>>>>>
>> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >>>>>>>>
>> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >>>>>>>>
>> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >>>>>>>>
>> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >>>>>>>>
>> >>>>>>>> Cassandra
>> >>>>>>>>
>> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I find 9 open blockers for 8.0:
>> >>>>>>>>>
>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
>> >>>>>>>>>
>> >>>>>>>>> As David mentioned, many of the SOlr committers are at Activate, which
>> >>>>>>>>> ends Thursday so feedback (and work) may be a bit delayed.
>> >>>>>>>>> On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > Hi,
>> >>>>>>>>> >
>> >>>>>>>>> > Thanks for volunteering to do the 8.0 release Jim!
>> >>>>>>>>> >
>> >>>>>>>>> > Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>> >>>>>>>>> >
>> >>>>>>>>> > On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>> >>>>>>>>> >
>> >>>>>>>>> > I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>> >>>>>>>>> >
>> >>>>>>>>> > ~ David
>> >>>>>>>>> >
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Hi,
>> >>>>>>>>> >> We still have two blockers for the Lucene 8 release:
>> >>>>>>>>> >> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> >>>>>>>>> >> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> >>>>>>>>> >> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> >>>>>>>>> >> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> >>>>>>>>> >>>>
>> >>>>>>>>> >>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>> >>>>>>>>> >>>>>
>> >>>>>>>>> >>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> There's also the issue of what to do as far as removing Trie* support.
>> >>>>>>>>> >>>>>> I think there's a blocker JIRA.
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> Shows 6 blockers
>> >>>>>>>>> >>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Hi Jim,
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > Thanks!
>> >>>>>>>>> >>>>>> >
>> >>>>>>>>> >>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Hi all,
>> >>>>>>>>> >>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> >>>>>>>>> >>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Cheers,
>> >>>>>>>>> >>>>>> >> Jim
>> >>>>>>>>> >>>>>> >>
>> >>>>>>>>> >>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>> >>>>>>>>> >>>>>> >>> ~ David
>> >>>>>>>>> >>>>>> >>>
>> >>>>>>>>> >>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>> >>>>>>>>> >>>>>> >>>>
>> >>>>>>>>> >>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>> >>>>>>>>> >>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>> >>>>>>>>> >>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>> >>>>>>>>> >>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>> >>>>>>>>> >>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>> >>>>>>>>> >>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>> >>>>>>>>> >>>>>> >>>>> > 2018?
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Hi Robert,
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>> >>>>>>>>> >>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> >>>>>>>>> >>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>> >>>>>>>>> >>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> >>>>>>>>> >>>>>> >>>>> >> clause are also fast.
>> >>>>>>>>> >>>>>> >>>>> >>
>> >>>>>>>>> >>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>> >>>>>>>>> >>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>> >>>>>>>>> >>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>> >>>>>>>>> >>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>> >>>>>>>>> >>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>> >>>>>>>>> >>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>> >>>>>>>>> >>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>> >>>>>>>>> >>>>>> >>>>> >>> too invasive.
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>> >>>>>>>>> >>>>>> >>>>> >>> > Hi all,
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>> >>>>>>>>> >>>>>> >>>>> >>> > already
>> >>>>>>>>> >>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>> >>>>>>>>> >>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>> >>>>>>>>> >>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>> >>>>>>>>> >>>>>> >>>>> >>> > when
>> >>>>>>>>> >>>>>> >>>>> >>> > total hit counts are not requested.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>> >>>>>>>>> >>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>> >>>>>>>>> >>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>> >>>>>>>>> >>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>> >>>>>>>>> >>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>> >>>>>>>>> >>>>>> >>>>> >>> > only in
>> >>>>>>>>> >>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>> >>>>>>>>> >>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>> >>>>>>>>> >>>>>> >>>>> >>> > which
>> >>>>>>>>> >>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>> >>>>>>>>> >>>>>> >>>>> >>> > the
>> >>>>>>>>> >>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>> >>>>>>>>> >>>>>> >>>>> >>> > API
>> >>>>>>>>> >>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>> >>>>>>>>> >>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>> >>>>>>>>> >>>>>> >>>>> >>> > as we
>> >>>>>>>>> >>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>> >>>>>>>>> >>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>> >>>>>>>>> >>>>>> >>>>> >>> > be
>> >>>>>>>>> >>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>> >>>>>>>>> >>>>>> >>>>> >>> > worth
>> >>>>>>>>> >>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>> >>>>>>>>> >>>>>> >>>>> >>> > to
>> >>>>>>>>> >>>>>> >>>>> >>> > get in for 8.0?
>> >>>>>>>>> >>>>>> >>>>> >>> >
>> >>>>>>>>> >>>>>> >>>>> >>> > Adrien
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> >>>
>> >>>>>>>>> >>>>>> >>>>> >
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> >>>>>
>> >>>>>>>>> >>>>>> >>> --
>> >>>>>>>>> >>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> >>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> >>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> >>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> >>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>> >>>>>>
>> >>>>>>>>> > --
>> >>>>>>>>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >>>>>>>>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>> >>>>>>>>>
>> >>>>>>>>> ---------------------------------------------------------------------
>> >>>>>>>>> To unsubscribe, e-mail: [hidden email]
>> >>>>>>>>> For additional commands, e-mail: [hidden email]
>> >>>>>>>>>
>> >>> --
>> >>>
>> >>> Nicholas Knize, Ph.D., GISP
>> >>> Geospatial Software Guy  |  Elasticsearch
>> >>> Apache Lucene Committer
>> >>> [hidden email]
>> >>
>> >> --
>> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>


-- 
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

-- 
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker
-- 
Lucene/Solr Search Committer (PMC), Developer, Author, Speaker


Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand
+1 to release 7.7 and 8.0 in a row starting on the week of February 4th.

On Wed, Jan 23, 2019 at 4:23 PM jim ferenczi <[hidden email]> wrote:

>
> Hi,
> As we agreed some time ago I'd like to start on releasing 8.0. The branch is already created so we can start the process anytime now. Unless there are objections I'd like to start the feature freeze next week in order to build the first candidate the week after.
> We'll also need a 7.7 release but I think we can handle both with Alan so the question now is whether we are ok to start the release process or if there are any blockers left ;).
>
>
> Le mar. 15 janv. 2019 à 11:35, Alan Woodward <[hidden email]> a écrit :
>>
>> I’ve started to work through the various deprecations on the new master branch.  There are a lot of them, and I’m going to need some assistance for several of them, as it’s not entirely clear what to do.
>>
>> I’ll open two overarching issues in JIRA, one for lucene and one for Solr, with lists of the deprecations that need to be removed in each one.  I’ll create a shared branch on gitbox to work against, and push the changes I’ve already done there.  We can then create individual JIRA issues for any changes that are more involved than just deleting code.
>>
>> All assistance gratefully received, particularly for the Solr deprecations where there’s a lot of code I’m unfamiliar with.
>>
>> On 8 Jan 2019, at 09:21, Alan Woodward <[hidden email]> wrote:
>>
>> I think the current plan is to do a 7.7 release at the same time as 8.0, to handle any last-minute deprecations etc.  So let’s keep those jobs enabled for now.
>>
>> On 8 Jan 2019, at 09:10, Uwe Schindler <[hidden email]> wrote:
>>
>> Hi,
>>
>> I will start and add the branch_8x jobs to Jenkins once I have some time later today.
>>
>> The question: How to proceed with branch_7x? Should we stop using it and release 7.6.x only (so we would use branch_7_6 only for bugfixes), or are we planning to one more Lucene/Solr 7.7? In the latter case I would keep the jenkins jobs enabled for a while.
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> http://www.thetaphi.de
>> eMail: [hidden email]
>>
>> From: Alan Woodward <[hidden email]>
>> Sent: Monday, January 7, 2019 11:30 AM
>> To: [hidden email]
>> Subject: Re: Lucene/Solr 8.0
>>
>> OK, Christmas caught up with me a bit… I’ve just created a branch for 8x from master, and am in the process of updating the master branch to version 9.  New commits that should be included in the 8.0 release should also be back-ported to branch_8x from master.
>>
>> This is not intended as a feature freeze, as I know there are still some things being worked on for 8.0; however, it should let us clean up master by removing as much deprecated code as possible, and give us an idea of any replacement work that needs to be done.
>>
>>
>> On 19 Dec 2018, at 15:13, David Smiley <[hidden email]> wrote:
>>
>> January.
>>
>> On Wed, Dec 19, 2018 at 2:04 AM S G <[hidden email]> wrote:
>>
>> It would be nice to see Solr 8 in January soon as there is an enhancement on nested-documents we are waiting to get our hands on.
>> Any idea when Solr 8 would be out ?
>>
>> Thx
>> SG
>>
>> On Mon, Dec 17, 2018 at 1:34 PM David Smiley <[hidden email]> wrote:
>>
>> I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND priority = Blocker and status = open and fixVersion = "master (8.0)"
>>    click here:
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(SOLR%2C%20LUCENE)%20AND%20priority%20%3D%20Blocker%20and%20status%20%3D%20open%20and%20fixVersion%20%3D%20%22master%20(8.0)%22%20
>>
>> Thru the end of the month, I intend to work on those issues not yet assigned.
>>
>> On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand <[hidden email]> wrote:
>>
>> +1
>>
>> On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward <[hidden email]> wrote:
>> >
>> > Hi all,
>> >
>> > Now that 7.6 is out of the door (thanks Nick!) we should think about cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the branch this week - say Wednesday?  Then we should have some time to clean up the master branch and uncover anything that still needs to be done on 8.0 before we start the release process next year.
>> >
>> > On 22 Oct 2018, at 18:12, Cassandra Targett <[hidden email]> wrote:
>> >
>> > I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>> >
>> > On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson <[hidden email]> wrote:
>> >>
>> >> +1, this gives us all a chance to prioritize getting the blockers out
>> >> of the way in a careful manner.
>> >> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi <[hidden email]> wrote:
>> >> >
>> >> > +1 too. With this new perspective we could create the branch just after the 7.6 release and target the 8.0 release for January 2019 which gives almost 3 month to finish the blockers ?
>> >> >
>> >> > Le jeu. 18 oct. 2018 à 23:56, David Smiley <[hidden email]> a écrit :
>> >> >>
>> >> >> +1 to a 7.6 —lots of stuff in there
>> >> >> On Thu, Oct 18, 2018 at 4:47 PM Nicholas Knize <[hidden email]> wrote:
>> >> >>>
>> >> >>> If we're planning to postpone cutting an 8.0 branch until a few weeks from now then I'd like to propose (and volunteer to RM) a 7.6 release targeted for late November or early December (following the typical 2 month release pattern). It feels like this might give a little breathing room for finishing up 8.0 blockers? And looking at the change log there appear to be a healthy list of features, bug fixes, and improvements to both Solr and Lucene that warrant a 7.6 release? Personally I wouldn't mind releasing the LatLonShape encoding changes in LUCENE-8521 and selective indexing work done in LUCENE-8496. Any objections or thoughts?
>> >> >>>
>> >> >>> - Nick
>> >> >>>
>> >> >>>
>> >> >>> On Thu, Oct 18, 2018 at 5:32 AM Đạt Cao Mạnh <[hidden email]> wrote:
>> >> >>>>
>> >> >>>> Thanks Cassandra and Jim,
>> >> >>>>
>> >> >>>> I created a blocker issue for Solr 8.0 SOLR-12883, currently in jira/http2 branch there are a draft-unmature implementation of SPNEGO authentication which enough to makes the test pass, this implementation will be removed when SOLR-12883 gets resolved . Therefore I don't see any problem on merging jira/http2 to master branch in the next week.
>> >> >>>>
>> >> >>>> On Thu, Oct 18, 2018 at 2:33 AM jim ferenczi <[hidden email]> wrote:
>> >> >>>>>
>> >> >>>>> > But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >> >>>>>
>> >> >>>>> Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.
>> >> >>>>>
>> >> >>>>> Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
>> >> >>>>>>
>> >> >>>>>> OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.
>> >> >>>>>>
>> >> >>>>>> It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.
>> >> >>>>>>
>> >> >>>>>> If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.
>> >> >>>>>>
>> >> >>>>>> Cassandra
>> >> >>>>>>
>> >> >>>>>> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
>> >> >>>>>>>
>> >> >>>>>>> Ok thanks for answering.
>> >> >>>>>>>
>> >> >>>>>>> > - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >> >>>>>>>
>> >> >>>>>>> We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
>> >> >>>>>>> HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
>> >> >>>>>>> we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
>> >> >>>>>>> Next week was just a proposal, what I meant was soon because we target a release in a few months.
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
>> >> >>>>>>>>
>> >> >>>>>>>> IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.
>> >> >>>>>>>>
>> >> >>>>>>>> Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit.
>> >> >>>>>>>>
>> >> >>>>>>>> He can hopefully reply with more details on his status and what else needs to be done.
>> >> >>>>>>>>
>> >> >>>>>>>> Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.
>> >> >>>>>>>>
>> >> >>>>>>>> Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.
>> >> >>>>>>>>
>> >> >>>>>>>> Cassandra
>> >> >>>>>>>>
>> >> >>>>>>>> On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
>> >> >>>>>>>>>
>> >> >>>>>