Lucene/Solr 8.0

classic Classic list List threaded Threaded
105 messages Options
1234 ... 6
Reply | Threaded
Open this post in threaded view
|

Lucene/Solr 8.0

Adrien Grand
Hi all,

I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8 already has some good changes around scoring, notably cleanups to similarities[1][2][3], indexing of impacts[4], and an implementation of Block-Max WAND[5] which, once combined, allow to run queries faster when total hit counts are not requested.

[1] https://issues.apache.org/jira/browse/LUCENE-8116
[2] https://issues.apache.org/jira/browse/LUCENE-8020
[3] https://issues.apache.org/jira/browse/LUCENE-8007
[4] https://issues.apache.org/jira/browse/LUCENE-4198
[5] https://issues.apache.org/jira/browse/LUCENE-8135

In terms of bug fixes, there is also a bad relevancy bug[6] which is only in 8.0 because it required a breaking change[7] to be implemented.

[6] https://issues.apache.org/jira/browse/LUCENE-8031
[7] https://issues.apache.org/jira/browse/LUCENE-8134

As usual, doing a new major release will also help age out old codecs, which in-turn make maintenance easier: 8.0 will no longer need to care about the fact that some codecs were initially implemented with a random-access API for doc values, that pre-7.0 indices encoded norms differently, or that pre-6.2 indices could not record an index sort.

I also expect that we will come up with ideas of things to do for 8.0 as we feel that the next major is getting closer. In terms of planning, I was thinking that we could target something like october 2018, which would be 12-13 months after 7.0 and 3-4 months from now.

From a Solr perspective, the main change I'm aware of that would be worth releasing a new major is the Star Burst effort. Is it something we want to get in for 8.0?

Adrien
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Robert Muir
How can the end user actually use the biggest new feature: impacts and
BMW? As far as I can tell, the issue to actually implement the
necessary API changes (IndexSearcher/TopDocs/etc) is still open and
unresolved, although there are some interesting ideas on it. This
seems like a really big missing piece, without a proper API, the stuff
is not really usable. I also can't imagine a situation where the API
could be introduced in a followup minor release because it would be
too invasive.

On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:

> Hi all,
>
> I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8 already
> has some good changes around scoring, notably cleanups to
> similarities[1][2][3], indexing of impacts[4], and an implementation of
> Block-Max WAND[5] which, once combined, allow to run queries faster when
> total hit counts are not requested.
>
> [1] https://issues.apache.org/jira/browse/LUCENE-8116
> [2] https://issues.apache.org/jira/browse/LUCENE-8020
> [3] https://issues.apache.org/jira/browse/LUCENE-8007
> [4] https://issues.apache.org/jira/browse/LUCENE-4198
> [5] https://issues.apache.org/jira/browse/LUCENE-8135
>
> In terms of bug fixes, there is also a bad relevancy bug[6] which is only in
> 8.0 because it required a breaking change[7] to be implemented.
>
> [6] https://issues.apache.org/jira/browse/LUCENE-8031
> [7] https://issues.apache.org/jira/browse/LUCENE-8134
>
> As usual, doing a new major release will also help age out old codecs, which
> in-turn make maintenance easier: 8.0 will no longer need to care about the
> fact that some codecs were initially implemented with a random-access API
> for doc values, that pre-7.0 indices encoded norms differently, or that
> pre-6.2 indices could not record an index sort.
>
> I also expect that we will come up with ideas of things to do for 8.0 as we
> feel that the next major is getting closer. In terms of planning, I was
> thinking that we could target something like october 2018, which would be
> 12-13 months after 7.0 and 3-4 months from now.
>
> From a Solr perspective, the main change I'm aware of that would be worth
> releasing a new major is the Star Burst effort. Is it something we want to
> get in for 8.0?
>
> Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand
Hi Robert,

I agree we need to make it more usable before 8.0. I would also like to improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204) to leverage impacts so that queries that incorporate queries on feature fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional clause are also fast.

Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
How can the end user actually use the biggest new feature: impacts and
BMW? As far as I can tell, the issue to actually implement the
necessary API changes (IndexSearcher/TopDocs/etc) is still open and
unresolved, although there are some interesting ideas on it. This
seems like a really big missing piece, without a proper API, the stuff
is not really usable. I also can't imagine a situation where the API
could be introduced in a followup minor release because it would be
too invasive.

On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
> Hi all,
>
> I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8 already
> has some good changes around scoring, notably cleanups to
> similarities[1][2][3], indexing of impacts[4], and an implementation of
> Block-Max WAND[5] which, once combined, allow to run queries faster when
> total hit counts are not requested.
>
> [1] https://issues.apache.org/jira/browse/LUCENE-8116
> [2] https://issues.apache.org/jira/browse/LUCENE-8020
> [3] https://issues.apache.org/jira/browse/LUCENE-8007
> [4] https://issues.apache.org/jira/browse/LUCENE-4198
> [5] https://issues.apache.org/jira/browse/LUCENE-8135
>
> In terms of bug fixes, there is also a bad relevancy bug[6] which is only in
> 8.0 because it required a breaking change[7] to be implemented.
>
> [6] https://issues.apache.org/jira/browse/LUCENE-8031
> [7] https://issues.apache.org/jira/browse/LUCENE-8134
>
> As usual, doing a new major release will also help age out old codecs, which
> in-turn make maintenance easier: 8.0 will no longer need to care about the
> fact that some codecs were initially implemented with a random-access API
> for doc values, that pre-7.0 indices encoded norms differently, or that
> pre-6.2 indices could not record an index sort.
>
> I also expect that we will come up with ideas of things to do for 8.0 as we
> feel that the next major is getting closer. In terms of planning, I was
> thinking that we could target something like october 2018, which would be
> 12-13 months after 7.0 and 3-4 months from now.
>
> From a Solr perspective, the main change I'm aware of that would be worth
> releasing a new major is the Star Burst effort. Is it something we want to
> get in for 8.0?
>
> Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand
I'd like to revive this thread now that these new optimizations for collection of top docs are more usable and enabled by default in IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any feedback about starting to work towards releasing 8.0 and targeting October 2018?

Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
Hi Robert,

I agree we need to make it more usable before 8.0. I would also like to improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204) to leverage impacts so that queries that incorporate queries on feature fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional clause are also fast.

Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
How can the end user actually use the biggest new feature: impacts and
BMW? As far as I can tell, the issue to actually implement the
necessary API changes (IndexSearcher/TopDocs/etc) is still open and
unresolved, although there are some interesting ideas on it. This
seems like a really big missing piece, without a proper API, the stuff
is not really usable. I also can't imagine a situation where the API
could be introduced in a followup minor release because it would be
too invasive.

On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
> Hi all,
>
> I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8 already
> has some good changes around scoring, notably cleanups to
> similarities[1][2][3], indexing of impacts[4], and an implementation of
> Block-Max WAND[5] which, once combined, allow to run queries faster when
> total hit counts are not requested.
>
> [1] https://issues.apache.org/jira/browse/LUCENE-8116
> [2] https://issues.apache.org/jira/browse/LUCENE-8020
> [3] https://issues.apache.org/jira/browse/LUCENE-8007
> [4] https://issues.apache.org/jira/browse/LUCENE-4198
> [5] https://issues.apache.org/jira/browse/LUCENE-8135
>
> In terms of bug fixes, there is also a bad relevancy bug[6] which is only in
> 8.0 because it required a breaking change[7] to be implemented.
>
> [6] https://issues.apache.org/jira/browse/LUCENE-8031
> [7] https://issues.apache.org/jira/browse/LUCENE-8134
>
> As usual, doing a new major release will also help age out old codecs, which
> in-turn make maintenance easier: 8.0 will no longer need to care about the
> fact that some codecs were initially implemented with a random-access API
> for doc values, that pre-7.0 indices encoded norms differently, or that
> pre-6.2 indices could not record an index sort.
>
> I also expect that we will come up with ideas of things to do for 8.0 as we
> feel that the next major is getting closer. In terms of planning, I was
> thinking that we could target something like october 2018, which would be
> 12-13 months after 7.0 and 3-4 months from now.
>
> From a Solr perspective, the main change I'm aware of that would be worth
> releasing a new major is the Star Burst effort. Is it something we want to
> get in for 8.0?
>
> Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Robert Muir
My only other suggestion is we may want to get Nick's shape stuff into
the sandbox module at least for 8.0 so that it can be tested out. I
think it looks like that wouldn't delay any October target though?

On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:

> I'd like to revive this thread now that these new optimizations for
> collection of top docs are more usable and enabled by default in
> IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
> feedback about starting to work towards releasing 8.0 and targeting October
> 2018?
>
>
> Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>
>> Hi Robert,
>>
>> I agree we need to make it more usable before 8.0. I would also like to
>> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> to leverage impacts so that queries that incorporate queries on feature
>> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> clause are also fast.
>>
>> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>
>>> How can the end user actually use the biggest new feature: impacts and
>>> BMW? As far as I can tell, the issue to actually implement the
>>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>> unresolved, although there are some interesting ideas on it. This
>>> seems like a really big missing piece, without a proper API, the stuff
>>> is not really usable. I also can't imagine a situation where the API
>>> could be introduced in a followup minor release because it would be
>>> too invasive.
>>>
>>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>> > Hi all,
>>> >
>>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>> > already
>>> > has some good changes around scoring, notably cleanups to
>>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>> > when
>>> > total hit counts are not requested.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>> >
>>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>> > only in
>>> > 8.0 because it required a breaking change[7] to be implemented.
>>> >
>>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>> >
>>> > As usual, doing a new major release will also help age out old codecs,
>>> > which
>>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>> > the
>>> > fact that some codecs were initially implemented with a random-access
>>> > API
>>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>> > pre-6.2 indices could not record an index sort.
>>> >
>>> > I also expect that we will come up with ideas of things to do for 8.0
>>> > as we
>>> > feel that the next major is getting closer. In terms of planning, I was
>>> > thinking that we could target something like october 2018, which would
>>> > be
>>> > 12-13 months after 7.0 and 3-4 months from now.
>>> >
>>> > From a Solr perspective, the main change I'm aware of that would be
>>> > worth
>>> > releasing a new major is the Star Burst effort. Is it something we want
>>> > to
>>> > get in for 8.0?
>>> >
>>> > Adrien
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand
I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.

Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
My only other suggestion is we may want to get Nick's shape stuff into
the sandbox module at least for 8.0 so that it can be tested out. I
think it looks like that wouldn't delay any October target though?

On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
> I'd like to revive this thread now that these new optimizations for
> collection of top docs are more usable and enabled by default in
> IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
> feedback about starting to work towards releasing 8.0 and targeting October
> 2018?
>
>
> Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>
>> Hi Robert,
>>
>> I agree we need to make it more usable before 8.0. I would also like to
>> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> to leverage impacts so that queries that incorporate queries on feature
>> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> clause are also fast.
>>
>> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>
>>> How can the end user actually use the biggest new feature: impacts and
>>> BMW? As far as I can tell, the issue to actually implement the
>>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>> unresolved, although there are some interesting ideas on it. This
>>> seems like a really big missing piece, without a proper API, the stuff
>>> is not really usable. I also can't imagine a situation where the API
>>> could be introduced in a followup minor release because it would be
>>> too invasive.
>>>
>>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>> > Hi all,
>>> >
>>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>> > already
>>> > has some good changes around scoring, notably cleanups to
>>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>> > when
>>> > total hit counts are not requested.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>> >
>>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>> > only in
>>> > 8.0 because it required a breaking change[7] to be implemented.
>>> >
>>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>> >
>>> > As usual, doing a new major release will also help age out old codecs,
>>> > which
>>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>> > the
>>> > fact that some codecs were initially implemented with a random-access
>>> > API
>>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>> > pre-6.2 indices could not record an index sort.
>>> >
>>> > I also expect that we will come up with ideas of things to do for 8.0
>>> > as we
>>> > feel that the next major is getting closer. In terms of planning, I was
>>> > thinking that we could target something like october 2018, which would
>>> > be
>>> > 12-13 months after 7.0 and 3-4 months from now.
>>> >
>>> > From a Solr perspective, the main change I'm aware of that would be
>>> > worth
>>> > releasing a new major is the Star Burst effort. Is it something we want
>>> > to
>>> > get in for 8.0?
>>> >
>>> > Adrien
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

david.w.smiley@gmail.com
Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
~ David

On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.

Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
My only other suggestion is we may want to get Nick's shape stuff into
the sandbox module at least for 8.0 so that it can be tested out. I
think it looks like that wouldn't delay any October target though?

On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
> I'd like to revive this thread now that these new optimizations for
> collection of top docs are more usable and enabled by default in
> IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
> feedback about starting to work towards releasing 8.0 and targeting October
> 2018?
>
>
> Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>
>> Hi Robert,
>>
>> I agree we need to make it more usable before 8.0. I would also like to
>> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> to leverage impacts so that queries that incorporate queries on feature
>> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> clause are also fast.
>>
>> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>
>>> How can the end user actually use the biggest new feature: impacts and
>>> BMW? As far as I can tell, the issue to actually implement the
>>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>> unresolved, although there are some interesting ideas on it. This
>>> seems like a really big missing piece, without a proper API, the stuff
>>> is not really usable. I also can't imagine a situation where the API
>>> could be introduced in a followup minor release because it would be
>>> too invasive.
>>>
>>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>> > Hi all,
>>> >
>>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>> > already
>>> > has some good changes around scoring, notably cleanups to
>>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>> > when
>>> > total hit counts are not requested.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>> >
>>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>> > only in
>>> > 8.0 because it required a breaking change[7] to be implemented.
>>> >
>>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>> >
>>> > As usual, doing a new major release will also help age out old codecs,
>>> > which
>>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>> > the
>>> > fact that some codecs were initially implemented with a random-access
>>> > API
>>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>> > pre-6.2 indices could not record an index sort.
>>> >
>>> > I also expect that we will come up with ideas of things to do for 8.0
>>> > as we
>>> > feel that the next major is getting closer. In terms of planning, I was
>>> > thinking that we could target something like october 2018, which would
>>> > be
>>> > 12-13 months after 7.0 and 3-4 months from now.
>>> >
>>> > From a Solr perspective, the main change I'm aware of that would be
>>> > worth
>>> > releasing a new major is the Star Burst effort. Is it something we want
>>> > to
>>> > get in for 8.0?
>>> >
>>> > Adrien
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
Hi all,
I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved. 
From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?

Cheers,
Jim

Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
~ David

On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.

Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
My only other suggestion is we may want to get Nick's shape stuff into
the sandbox module at least for 8.0 so that it can be tested out. I
think it looks like that wouldn't delay any October target though?

On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
> I'd like to revive this thread now that these new optimizations for
> collection of top docs are more usable and enabled by default in
> IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
> feedback about starting to work towards releasing 8.0 and targeting October
> 2018?
>
>
> Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>
>> Hi Robert,
>>
>> I agree we need to make it more usable before 8.0. I would also like to
>> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> to leverage impacts so that queries that incorporate queries on feature
>> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> clause are also fast.
>>
>> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>
>>> How can the end user actually use the biggest new feature: impacts and
>>> BMW? As far as I can tell, the issue to actually implement the
>>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>> unresolved, although there are some interesting ideas on it. This
>>> seems like a really big missing piece, without a proper API, the stuff
>>> is not really usable. I also can't imagine a situation where the API
>>> could be introduced in a followup minor release because it would be
>>> too invasive.
>>>
>>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>> > Hi all,
>>> >
>>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>> > already
>>> > has some good changes around scoring, notably cleanups to
>>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>> > when
>>> > total hit counts are not requested.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>> >
>>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>> > only in
>>> > 8.0 because it required a breaking change[7] to be implemented.
>>> >
>>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>> >
>>> > As usual, doing a new major release will also help age out old codecs,
>>> > which
>>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>> > the
>>> > fact that some codecs were initially implemented with a random-access
>>> > API
>>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>> > pre-6.2 indices could not record an index sort.
>>> >
>>> > I also expect that we will come up with ideas of things to do for 8.0
>>> > as we
>>> > feel that the next major is getting closer. In terms of planning, I was
>>> > thinking that we could target something like october 2018, which would
>>> > be
>>> > 12-13 months after 7.0 and 3-4 months from now.
>>> >
>>> > From a Solr perspective, the main change I'm aware of that would be
>>> > worth
>>> > releasing a new major is the Star Burst effort. Is it something we want
>>> > to
>>> > get in for 8.0?
>>> >
>>> > Adrien
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

caomanhdat
Hi Jim,

I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.

Thanks!

On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
Hi all,
I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved. 
From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?

Cheers,
Jim

Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
~ David

On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.

Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
My only other suggestion is we may want to get Nick's shape stuff into
the sandbox module at least for 8.0 so that it can be tested out. I
think it looks like that wouldn't delay any October target though?

On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
> I'd like to revive this thread now that these new optimizations for
> collection of top docs are more usable and enabled by default in
> IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
> feedback about starting to work towards releasing 8.0 and targeting October
> 2018?
>
>
> Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>
>> Hi Robert,
>>
>> I agree we need to make it more usable before 8.0. I would also like to
>> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>> to leverage impacts so that queries that incorporate queries on feature
>> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>> clause are also fast.
>>
>> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>
>>> How can the end user actually use the biggest new feature: impacts and
>>> BMW? As far as I can tell, the issue to actually implement the
>>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>> unresolved, although there are some interesting ideas on it. This
>>> seems like a really big missing piece, without a proper API, the stuff
>>> is not really usable. I also can't imagine a situation where the API
>>> could be introduced in a followup minor release because it would be
>>> too invasive.
>>>
>>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>> > Hi all,
>>> >
>>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>> > already
>>> > has some good changes around scoring, notably cleanups to
>>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>> > when
>>> > total hit counts are not requested.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>> >
>>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>> > only in
>>> > 8.0 because it required a breaking change[7] to be implemented.
>>> >
>>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>> >
>>> > As usual, doing a new major release will also help age out old codecs,
>>> > which
>>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>> > the
>>> > fact that some codecs were initially implemented with a random-access
>>> > API
>>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>> > pre-6.2 indices could not record an index sort.
>>> >
>>> > I also expect that we will come up with ideas of things to do for 8.0
>>> > as we
>>> > feel that the next major is getting closer. In terms of planning, I was
>>> > thinking that we could target something like october 2018, which would
>>> > be
>>> > 12-13 months after 7.0 and 3-4 months from now.
>>> >
>>> > From a Solr perspective, the main change I'm aware of that would be
>>> > worth
>>> > releasing a new major is the Star Burst effort. Is it something we want
>>> > to
>>> > get in for 8.0?
>>> >
>>> > Adrien
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Erick Erickson
There's also the issue of what to do as far as removing Trie* support.
I think there's a blocker JIRA.

project = SOLR AND priority = Blocker AND resolution = Unresolved

Shows 6 blockers
On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:

>
> Hi Jim,
>
> I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>
> Thanks!
>
> On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>
>> Cheers,
>> Jim
>>
>> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>
>>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>> ~ David
>>>
>>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>
>>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>
>>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>
>>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>> think it looks like that wouldn't delay any October target though?
>>>>>
>>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>> > I'd like to revive this thread now that these new optimizations for
>>>>> > collection of top docs are more usable and enabled by default in
>>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>> > 2018?
>>>>> >
>>>>> >
>>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>> >>
>>>>> >> Hi Robert,
>>>>> >>
>>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>> >> clause are also fast.
>>>>> >>
>>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>> >>>
>>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>> >>> could be introduced in a followup minor release because it would be
>>>>> >>> too invasive.
>>>>> >>>
>>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>> >>> > Hi all,
>>>>> >>> >
>>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>> >>> > already
>>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>> >>> > when
>>>>> >>> > total hit counts are not requested.
>>>>> >>> >
>>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>> >>> >
>>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>> >>> > only in
>>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>> >>> >
>>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>> >>> >
>>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>> >>> > which
>>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>> >>> > the
>>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>> >>> > API
>>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>> >>> >
>>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>> >>> > as we
>>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>> >>> > thinking that we could target something like october 2018, which would
>>>>> >>> > be
>>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>> >>> >
>>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>> >>> > worth
>>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>> >>> > to
>>>>> >>> > get in for 8.0?
>>>>> >>> >
>>>>> >>> > Adrien
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>> >>> For additional commands, e-mail: [hidden email]
>>>>> >>>
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?

Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
There's also the issue of what to do as far as removing Trie* support.
I think there's a blocker JIRA.

project = SOLR AND priority = Blocker AND resolution = Unresolved

Shows 6 blockers
On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>
> Hi Jim,
>
> I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>
> Thanks!
>
> On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>
>> Cheers,
>> Jim
>>
>> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>
>>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>> ~ David
>>>
>>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>
>>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>
>>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>
>>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>> think it looks like that wouldn't delay any October target though?
>>>>>
>>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>> > I'd like to revive this thread now that these new optimizations for
>>>>> > collection of top docs are more usable and enabled by default in
>>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>> > 2018?
>>>>> >
>>>>> >
>>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>> >>
>>>>> >> Hi Robert,
>>>>> >>
>>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>> >> clause are also fast.
>>>>> >>
>>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>> >>>
>>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>> >>> could be introduced in a followup minor release because it would be
>>>>> >>> too invasive.
>>>>> >>>
>>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>> >>> > Hi all,
>>>>> >>> >
>>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>> >>> > already
>>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>> >>> > when
>>>>> >>> > total hit counts are not requested.
>>>>> >>> >
>>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>> >>> >
>>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>> >>> > only in
>>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>> >>> >
>>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>> >>> >
>>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>> >>> > which
>>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>> >>> > the
>>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>> >>> > API
>>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>> >>> >
>>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>> >>> > as we
>>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>> >>> > thinking that we could target something like october 2018, which would
>>>>> >>> > be
>>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>> >>> >
>>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>> >>> > worth
>>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>> >>> > to
>>>>> >>> > get in for 8.0?
>>>>> >>> >
>>>>> >>> > Adrien
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>> >>> For additional commands, e-mail: [hidden email]
>>>>> >>>
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand

Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?

Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
There's also the issue of what to do as far as removing Trie* support.
I think there's a blocker JIRA.

project = SOLR AND priority = Blocker AND resolution = Unresolved

Shows 6 blockers
On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>
> Hi Jim,
>
> I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>
> Thanks!
>
> On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>
>> Cheers,
>> Jim
>>
>> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>
>>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>> ~ David
>>>
>>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>
>>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>
>>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>
>>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>> think it looks like that wouldn't delay any October target though?
>>>>>
>>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>> > I'd like to revive this thread now that these new optimizations for
>>>>> > collection of top docs are more usable and enabled by default in
>>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>> > 2018?
>>>>> >
>>>>> >
>>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>> >>
>>>>> >> Hi Robert,
>>>>> >>
>>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>> >> clause are also fast.
>>>>> >>
>>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>> >>>
>>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>> >>> could be introduced in a followup minor release because it would be
>>>>> >>> too invasive.
>>>>> >>>
>>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>> >>> > Hi all,
>>>>> >>> >
>>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>> >>> > already
>>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>> >>> > when
>>>>> >>> > total hit counts are not requested.
>>>>> >>> >
>>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>> >>> >
>>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>> >>> > only in
>>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>> >>> >
>>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>> >>> >
>>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>> >>> > which
>>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>> >>> > the
>>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>> >>> > API
>>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>> >>> >
>>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>> >>> > as we
>>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>> >>> > thinking that we could target something like october 2018, which would
>>>>> >>> > be
>>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>> >>> >
>>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>> >>> > worth
>>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>> >>> > to
>>>>> >>> > get in for 8.0?
>>>>> >>> >
>>>>> >>> > Adrien
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>> >>> For additional commands, e-mail: [hidden email]
>>>>> >>>
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Adrien Grand
Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?

Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :

Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?

Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
There's also the issue of what to do as far as removing Trie* support.
I think there's a blocker JIRA.

project = SOLR AND priority = Blocker AND resolution = Unresolved

Shows 6 blockers
On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>
> Hi Jim,
>
> I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>
> Thanks!
>
> On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>
>> Cheers,
>> Jim
>>
>> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>
>>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>> ~ David
>>>
>>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>
>>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>
>>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>
>>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>> think it looks like that wouldn't delay any October target though?
>>>>>
>>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>> > I'd like to revive this thread now that these new optimizations for
>>>>> > collection of top docs are more usable and enabled by default in
>>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>> > 2018?
>>>>> >
>>>>> >
>>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>> >>
>>>>> >> Hi Robert,
>>>>> >>
>>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>> >> clause are also fast.
>>>>> >>
>>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>> >>>
>>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>> >>> could be introduced in a followup minor release because it would be
>>>>> >>> too invasive.
>>>>> >>>
>>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>> >>> > Hi all,
>>>>> >>> >
>>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>> >>> > already
>>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>> >>> > when
>>>>> >>> > total hit counts are not requested.
>>>>> >>> >
>>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>> >>> >
>>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>> >>> > only in
>>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>> >>> >
>>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>> >>> >
>>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>> >>> > which
>>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>> >>> > the
>>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>> >>> > API
>>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>> >>> >
>>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>> >>> > as we
>>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>> >>> > thinking that we could target something like october 2018, which would
>>>>> >>> > be
>>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>> >>> >
>>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>> >>> > worth
>>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>> >>> > to
>>>>> >>> > get in for 8.0?
>>>>> >>> >
>>>>> >>> > Adrien
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>> >>> For additional commands, e-mail: [hidden email]
>>>>> >>>
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
Hi,
We still have two blockers for the Lucene 8 release:
We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side. 
Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
we can discuss the best date for the release when all blockers are resolved. What do you think ?



Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?

Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :

Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?

Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
There's also the issue of what to do as far as removing Trie* support.
I think there's a blocker JIRA.

project = SOLR AND priority = Blocker AND resolution = Unresolved

Shows 6 blockers
On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>
> Hi Jim,
>
> I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>
> Thanks!
>
> On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>
>> Cheers,
>> Jim
>>
>> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>
>>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>> ~ David
>>>
>>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>
>>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>
>>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>
>>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>> think it looks like that wouldn't delay any October target though?
>>>>>
>>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>> > I'd like to revive this thread now that these new optimizations for
>>>>> > collection of top docs are more usable and enabled by default in
>>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>> > 2018?
>>>>> >
>>>>> >
>>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>> >>
>>>>> >> Hi Robert,
>>>>> >>
>>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>> >> clause are also fast.
>>>>> >>
>>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>> >>>
>>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>> >>> could be introduced in a followup minor release because it would be
>>>>> >>> too invasive.
>>>>> >>>
>>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>> >>> > Hi all,
>>>>> >>> >
>>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>> >>> > already
>>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>> >>> > when
>>>>> >>> > total hit counts are not requested.
>>>>> >>> >
>>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>> >>> >
>>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>> >>> > only in
>>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>> >>> >
>>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>> >>> >
>>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>> >>> > which
>>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>> >>> > the
>>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>> >>> > API
>>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>> >>> >
>>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>> >>> > as we
>>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>> >>> > thinking that we could target something like october 2018, which would
>>>>> >>> > be
>>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>> >>> >
>>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>> >>> > worth
>>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>> >>> > to
>>>>> >>> > get in for 8.0?
>>>>> >>> >
>>>>> >>> > Adrien
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>> >>> For additional commands, e-mail: [hidden email]
>>>>> >>>
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

david.w.smiley@gmail.com
Hi,

Thanks for volunteering to do the 8.0 release Jim!

Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.

On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.

I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.

~ David


On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
Hi,
We still have two blockers for the Lucene 8 release:
We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side. 
Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
we can discuss the best date for the release when all blockers are resolved. What do you think ?



Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?

Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :

Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?

Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
There's also the issue of what to do as far as removing Trie* support.
I think there's a blocker JIRA.

project = SOLR AND priority = Blocker AND resolution = Unresolved

Shows 6 blockers
On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>
> Hi Jim,
>
> I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>
> Thanks!
>
> On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>
>> Hi all,
>> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>
>> Cheers,
>> Jim
>>
>> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>
>>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>> ~ David
>>>
>>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>
>>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>
>>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>
>>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>> think it looks like that wouldn't delay any October target though?
>>>>>
>>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>> > I'd like to revive this thread now that these new optimizations for
>>>>> > collection of top docs are more usable and enabled by default in
>>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>> > 2018?
>>>>> >
>>>>> >
>>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>> >>
>>>>> >> Hi Robert,
>>>>> >>
>>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>> >> clause are also fast.
>>>>> >>
>>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>> >>>
>>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>> >>> could be introduced in a followup minor release because it would be
>>>>> >>> too invasive.
>>>>> >>>
>>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>> >>> > Hi all,
>>>>> >>> >
>>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>> >>> > already
>>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>> >>> > when
>>>>> >>> > total hit counts are not requested.
>>>>> >>> >
>>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>> >>> >
>>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>> >>> > only in
>>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>> >>> >
>>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>> >>> >
>>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>> >>> > which
>>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>> >>> > the
>>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>> >>> > API
>>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>> >>> >
>>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>> >>> > as we
>>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>> >>> > thinking that we could target something like october 2018, which would
>>>>> >>> > be
>>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>> >>> >
>>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>> >>> > worth
>>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>> >>> > to
>>>>> >>> > get in for 8.0?
>>>>> >>> >
>>>>> >>> > Adrien
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>> >>> For additional commands, e-mail: [hidden email]
>>>>> >>>
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Erick Erickson
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:

>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Cassandra Targett
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

Cassandra Targett
OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.

It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.

Cassandra

On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene/Solr 8.0

jim ferenczi
> But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

Yes that's my reasoning. This issue is a blocker so we won't release without it but we can work on the branch in the meantime and let other people work on new features that are not targeted to 8.

Le mer. 17 oct. 2018 à 20:51, Cassandra Targett <[hidden email]> a écrit :
OK - I was making an assumption that the timeline for the first 8.0 RC would be ASAP after the branch is created.

It's a common perception that making a branch freezes adding new features to the release, perhaps in an unofficial way (more of a courtesy rather than a rule). But if you're working with a different assumption - that just the existence of the branch does not stop Dat from still merging his work and the work being included in 8.0 - then I agree, waiting for him to merge doesn't need to stop the creation of the branch.

If, however, once the branch is there people object to Dat merging his work because it's "too late", then the branch shouldn't be created yet because we want to really try to clear that blocker for 8.0.

Cassandra

On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi <[hidden email]> wrote:
Ok thanks for answering. 

> - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

We can wait a few more weeks to create the branch but I don't think that one action (creating the branch) prevents the other (the work Dat is doing).
HTTP/2 is one of the blocker for the release but it can be done in master and backported to the appropriate branch as any other feature ? We just need an issue with the blocker label to ensure that
we don't miss it ;). Creating the branch early would also help in case you don't want to release all the work at once in 8.0.0.
Next week was just a proposal, what I meant was soon because we target a release in a few months.


Le mer. 17 oct. 2018 à 17:52, Cassandra Targett <[hidden email]> a écrit :
IMO next week is a bit too soon for the branch - I think Solr needs a couple more weeks since the work Dat is doing isn't quite done yet.

Solr needs the HTTP/2 work Dat has been doing, and he told me yesterday he feels it is nearly ready to be merged into master. However, it does require a new release of Jetty to Solr is able to retain Kerberos authentication support (Dat has been working with that team to help test the changes Jetty needs to support Kerberos with HTTP/2). They should get that release out soon, but we are dependent on them a little bit. 

He can hopefully reply with more details on his status and what else needs to be done.

Once Dat merges his work, IMO we should leave it in master for a little bit. While he has been beasting and testing with Jenkins as he goes along, I think it would be good to have all the regular master builds work on it for a little bit also.

Of the other blockers, the only other large-ish one is to fully remove Trie* fields, which some of us also discussed yesterday and it seemed we concluded that Solr isn't really ready to do that. The performance issues with single value lookups are a major obstacle. It would be nice if someone with a bit more experience with that could comment in the issue (SOLR-12632) and/or unmark it as a blocker.

Cassandra

On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson <[hidden email]> wrote:
I find 9 open blockers for 8.0:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN

As David mentioned, many of the SOlr committers are at Activate, which
ends Thursday so feedback (and work) may be a bit delayed.
On Wed, Oct 17, 2018 at 8:11 AM David Smiley <[hidden email]> wrote:
>
> Hi,
>
> Thanks for volunteering to do the 8.0 release Jim!
>
> Many of us are at the Activate Conference in Montreal.  We had a committers meeting where we discussed some of the blockers.  I think only a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On the Solr nested docs front, I articulated one and we mostly came to a decision on how to do it.  It's not "hard" just a matter of how to hook in some functionality so that it's user-friendly.  I'll file an issue for this.  Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  I'll file that issue and look at another issue or two that ought to be blockers.  Nothing is "hard" or tons of work that is in my sphere of work.
>
> On the Lucene side, I will commit https://issues.apache.org/jira/browse/LUCENE-7875 RE MultiFields either late tonight or tomorrow when I have time.  It's ready to be committed; just sitting there.  It's a minor thing but important to make this change now before 8.0.
>
> I personally plan to spend more time on the upcoming weeks on a few of these 8.0 things.
>
> ~ David
>
>
> On Wed, Oct 17, 2018 at 4:21 AM jim ferenczi <[hidden email]> wrote:
>>
>> Hi,
>> We still have two blockers for the Lucene 8 release:
>> https://issues.apache.org/jira/browse/LUCENE-7075?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>> We're planning to work on these issues in the coming days, are there any other blockers (not in the list) on Solr side.
>> Now that Lucene 7.5 is released I'd like to create a Lucene 8 branch soon (next week for instance ? ). There are some work to do to make sure that all tests pass, add the new version...
>> I can take care of it if there are no objections. Creating the branch in advance would help to stabilize this version (people can continue to work on new features that are not targeted for 8.0) and
>> we can discuss the best date for the release when all blockers are resolved. What do you think ?
>>
>>
>>
>> Le mar. 18 sept. 2018 à 11:32, Adrien Grand <[hidden email]> a écrit :
>>>
>>> Đạt, is https://issues.apache.org/jira/browse/SOLR-12639 the right issue for HTTP/2 support? Should we make it a blocker for 8.0?
>>>
>>> Le lun. 3 sept. 2018 à 23:37, Adrien Grand <[hidden email]> a écrit :
>>>>
>>>> For the record here is the JIRA query for blockers that Erick referred to: https://issues.apache.org/jira/browse/SOLR-12720?jql=(project%3D%22Lucene%20-%20Core%22%20%20OR%20project%3DSOLR)%20AND%20priority%3DBlocker%20and%20resolution%20%3D%20Unresolved%20
>>>>
>>>> Le lun. 3 sept. 2018 à 10:36, jim ferenczi <[hidden email]> a écrit :
>>>>>
>>>>> Ok thanks Đạt and Erick. I'll follow the blockers on Jira.  Đạt do you have an issue opened for the HTTP/2 support ?
>>>>>
>>>>> Le ven. 31 août 2018 à 16:40, Erick Erickson <[hidden email]> a écrit :
>>>>>>
>>>>>> There's also the issue of what to do as far as removing Trie* support.
>>>>>> I think there's a blocker JIRA.
>>>>>>
>>>>>> project = SOLR AND priority = Blocker AND resolution = Unresolved
>>>>>>
>>>>>> Shows 6 blockers
>>>>>> On Fri, Aug 31, 2018 at 4:12 AM Đạt Cao Mạnh <[hidden email]> wrote:
>>>>>> >
>>>>>> > Hi Jim,
>>>>>> >
>>>>>> > I really want to introduce the support of HTTP/2 into Solr 8.0 (currently cooked in jira/http2 branch). The changes of that branch are less than Star Burst effort and closer to be merged into master branch.
>>>>>> >
>>>>>> > Thanks!
>>>>>> >
>>>>>> > On Fri, Aug 31, 2018 at 3:55 PM jim ferenczi <[hidden email]> wrote:
>>>>>> >>
>>>>>> >> Hi all,
>>>>>> >> I'd like to get some feedback regarding the upcoming Lucene/Solr 8 release. There are still some cleanups and docs to add on the Lucene side but it seems that all blockers are resolved.
>>>>>> >> From a Solr perspective are there any important changes that need to be done or are we still good with the October target for the release ? Adrien mentioned the Star Burst effort some time ago, is it something that is planned for 8 ?
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Jim
>>>>>> >>
>>>>>> >> Le mer. 1 août 2018 à 19:02, David Smiley <[hidden email]> a écrit :
>>>>>> >>>
>>>>>> >>> Yes, that new BKD/Points based code is definitely something we want in 8 or 7.5 -- it's a big deal.  I think it would also be awesome if we had highlighter that could use the Weight.matches() API -- again for either 7.5 or 8.  I'm working on this on the UnifiedHighlighter front and Alan from other aspects.
>>>>>> >>> ~ David
>>>>>> >>>
>>>>>> >>> On Wed, Aug 1, 2018 at 12:51 PM Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>
>>>>>> >>>> I was hoping that we would release some bits of this new support for geo shapes in 7.5 already. We are already very close to being able to index points, lines and polygons and query for intersection with an envelope. It would be nice to add support for other relations (eg. disjoint) and queries (eg. polygon) but the current work looks already useful to me.
>>>>>> >>>>
>>>>>> >>>> Le mer. 1 août 2018 à 17:00, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>>
>>>>>> >>>>> My only other suggestion is we may want to get Nick's shape stuff into
>>>>>> >>>>> the sandbox module at least for 8.0 so that it can be tested out. I
>>>>>> >>>>> think it looks like that wouldn't delay any October target though?
>>>>>> >>>>>
>>>>>> >>>>> On Wed, Aug 1, 2018 at 9:51 AM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> > I'd like to revive this thread now that these new optimizations for
>>>>>> >>>>> > collection of top docs are more usable and enabled by default in
>>>>>> >>>>> > IndexSearcher (https://issues.apache.org/jira/browse/LUCENE-8060). Any
>>>>>> >>>>> > feedback about starting to work towards releasing 8.0 and targeting October
>>>>>> >>>>> > 2018?
>>>>>> >>>>> >
>>>>>> >>>>> >
>>>>>> >>>>> > Le jeu. 21 juin 2018 à 09:31, Adrien Grand <[hidden email]> a écrit :
>>>>>> >>>>> >>
>>>>>> >>>>> >> Hi Robert,
>>>>>> >>>>> >>
>>>>>> >>>>> >> I agree we need to make it more usable before 8.0. I would also like to
>>>>>> >>>>> >> improve ReqOptSumScorer (https://issues.apache.org/jira/browse/LUCENE-8204)
>>>>>> >>>>> >> to leverage impacts so that queries that incorporate queries on feature
>>>>>> >>>>> >> fields (https://issues.apache.org/jira/browse/LUCENE-8197) in an optional
>>>>>> >>>>> >> clause are also fast.
>>>>>> >>>>> >>
>>>>>> >>>>> >> Le jeu. 21 juin 2018 à 03:06, Robert Muir <[hidden email]> a écrit :
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> How can the end user actually use the biggest new feature: impacts and
>>>>>> >>>>> >>> BMW? As far as I can tell, the issue to actually implement the
>>>>>> >>>>> >>> necessary API changes (IndexSearcher/TopDocs/etc) is still open and
>>>>>> >>>>> >>> unresolved, although there are some interesting ideas on it. This
>>>>>> >>>>> >>> seems like a really big missing piece, without a proper API, the stuff
>>>>>> >>>>> >>> is not really usable. I also can't imagine a situation where the API
>>>>>> >>>>> >>> could be introduced in a followup minor release because it would be
>>>>>> >>>>> >>> too invasive.
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> On Mon, Jun 18, 2018 at 1:19 PM, Adrien Grand <[hidden email]> wrote:
>>>>>> >>>>> >>> > Hi all,
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I would like to start discussing releasing Lucene/Solr 8.0. Lucene 8
>>>>>> >>>>> >>> > already
>>>>>> >>>>> >>> > has some good changes around scoring, notably cleanups to
>>>>>> >>>>> >>> > similarities[1][2][3], indexing of impacts[4], and an implementation of
>>>>>> >>>>> >>> > Block-Max WAND[5] which, once combined, allow to run queries faster
>>>>>> >>>>> >>> > when
>>>>>> >>>>> >>> > total hit counts are not requested.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [1] https://issues.apache.org/jira/browse/LUCENE-8116
>>>>>> >>>>> >>> > [2] https://issues.apache.org/jira/browse/LUCENE-8020
>>>>>> >>>>> >>> > [3] https://issues.apache.org/jira/browse/LUCENE-8007
>>>>>> >>>>> >>> > [4] https://issues.apache.org/jira/browse/LUCENE-4198
>>>>>> >>>>> >>> > [5] https://issues.apache.org/jira/browse/LUCENE-8135
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > In terms of bug fixes, there is also a bad relevancy bug[6] which is
>>>>>> >>>>> >>> > only in
>>>>>> >>>>> >>> > 8.0 because it required a breaking change[7] to be implemented.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > [6] https://issues.apache.org/jira/browse/LUCENE-8031
>>>>>> >>>>> >>> > [7] https://issues.apache.org/jira/browse/LUCENE-8134
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > As usual, doing a new major release will also help age out old codecs,
>>>>>> >>>>> >>> > which
>>>>>> >>>>> >>> > in-turn make maintenance easier: 8.0 will no longer need to care about
>>>>>> >>>>> >>> > the
>>>>>> >>>>> >>> > fact that some codecs were initially implemented with a random-access
>>>>>> >>>>> >>> > API
>>>>>> >>>>> >>> > for doc values, that pre-7.0 indices encoded norms differently, or that
>>>>>> >>>>> >>> > pre-6.2 indices could not record an index sort.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > I also expect that we will come up with ideas of things to do for 8.0
>>>>>> >>>>> >>> > as we
>>>>>> >>>>> >>> > feel that the next major is getting closer. In terms of planning, I was
>>>>>> >>>>> >>> > thinking that we could target something like october 2018, which would
>>>>>> >>>>> >>> > be
>>>>>> >>>>> >>> > 12-13 months after 7.0 and 3-4 months from now.
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > From a Solr perspective, the main change I'm aware of that would be
>>>>>> >>>>> >>> > worth
>>>>>> >>>>> >>> > releasing a new major is the Star Burst effort. Is it something we want
>>>>>> >>>>> >>> > to
>>>>>> >>>>> >>> > get in for 8.0?
>>>>>> >>>>> >>> >
>>>>>> >>>>> >>> > Adrien
>>>>>> >>>>> >>>
>>>>>> >>>>> >>> ---------------------------------------------------------------------
>>>>>> >>>>> >>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> >>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>> >>>
>>>>>> >>>>> >
>>>>>> >>>>>
>>>>>> >>>>> ---------------------------------------------------------------------
>>>>>> >>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> >>>>> For additional commands, e-mail: [hidden email]
>>>>>> >>>>>
>>>>>> >>> --
>>>>>> >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>>>>> >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

1234 ... 6