Poll: Merge jira/http2 to master branch

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Poll: Merge jira/http2 to master branch

caomanhdat
Hi guys,

The changes in jira/http2 branch is very close to be commit now, but the current state of tests make me quite nervous about merging. I have a local Jenkins job for testing jira/http2 and the result is quite bad so far. I'm not sure failures caused by jira/http2 changes or the master branch. I wish I have more time on hardening and debugging the failures in jira/http2 branch. But the recent changes in my job do not allow me to spend as much time as I want. 

Therefore I'm thinking about merging jira/http2 to the master branch now and with the support of the community we can make tests more reliable. (not just me watching and working on it).
I know that Mark Miller is working on hardness tests, do you want me to wait for your work to be committed first?

Thanks!

--
Best regards,
Cao Mạnh Đạt
D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

Simon Willnauer-4
I don't know what the state of the tests is for solr. I'm also not on top of your work so please correct me if I am wrong. You are saying you won't have enough time for the feature in the near future but you want to merge it still to master so others can chime in? I don't think this is an approach that works. I think if others need to chime in it should be done on the branch? My definition for being ready to be merged to master is that the code and tests are stable. If there are minor tasks to be done then I guess I am fine with that. To me this sounds like it will put a ton of work on somebody else plate? I mean mark is spending a lot of time on the tests and this would make his life even harder?

I don't think we should do this. 

simon

On Mon, Nov 26, 2018 at 12:08 PM Đạt Cao Mạnh <[hidden email]> wrote:
Hi guys,

The changes in jira/http2 branch is very close to be commit now, but the current state of tests make me quite nervous about merging. I have a local Jenkins job for testing jira/http2 and the result is quite bad so far. I'm not sure failures caused by jira/http2 changes or the master branch. I wish I have more time on hardening and debugging the failures in jira/http2 branch. But the recent changes in my job do not allow me to spend as much time as I want. 

Therefore I'm thinking about merging jira/http2 to the master branch now and with the support of the community we can make tests more reliable. (not just me watching and working on it).
I know that Mark Miller is working on hardness tests, do you want me to wait for your work to be committed first?

Thanks!

--
Best regards,
Cao Mạnh Đạt
D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler

I agree with Simon,

 

I’d prefer to just add jenkins jobs for the “http2” branch and not yet merge it. That’s possible to setup in short time. If the number of failures don’t grow beyond the current state, we can merge. If it get’s worse on same hardware, then we should not merge yet.

 

The problem with Solr tests is that the reliability differs from hardware to hardware (because of some timing/multithreading issues). So to make a comparison, you should compare with and without patch on same hardware.

 

I can setup a jenkins job on ASF jenkins and Policeman jenkins (copy of master job with different branch) and let it run. I can configure the dev mailing list or Cao Mạnh Đạt’s mail address there. What do you prefer?

 

Uwe

 

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de

eMail: [hidden email]

 

From: Simon Willnauer <[hidden email]>
Sent: Monday, November 26, 2018 3:03 PM
To: Lucene/Solr dev <[hidden email]>
Subject: Re: Poll: Merge jira/http2 to master branch

 

I don't know what the state of the tests is for solr. I'm also not on top of your work so please correct me if I am wrong. You are saying you won't have enough time for the feature in the near future but you want to merge it still to master so others can chime in? I don't think this is an approach that works. I think if others need to chime in it should be done on the branch? My definition for being ready to be merged to master is that the code and tests are stable. If there are minor tasks to be done then I guess I am fine with that. To me this sounds like it will put a ton of work on somebody else plate? I mean mark is spending a lot of time on the tests and this would make his life even harder?

 

I don't think we should do this. 

 

simon

 

On Mon, Nov 26, 2018 at 12:08 PM Đạt Cao Mạnh <[hidden email]> wrote:

Hi guys,

 

The changes in jira/http2 branch is very close to be commit now, but the current state of tests make me quite nervous about merging. I have a local Jenkins job for testing jira/http2 and the result is quite bad so far. I'm not sure failures caused by jira/http2 changes or the master branch. I wish I have more time on hardening and debugging the failures in jira/http2 branch. But the recent changes in my job do not allow me to spend as much time as I want. 

 

Therefore I'm thinking about merging jira/http2 to the master branch now and with the support of the community we can make tests more reliable. (not just me watching and working on it).

I know that Mark Miller is working on hardness tests, do you want me to wait for your work to be committed first?

 

Thanks!

 

--

Best regards,

Cao Mạnh Đạt

D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

caomanhdat
Hi Uwe, 

That would be nice if we have a public Jenkins report that everyone can look. Because of this change gonna be a blocker for the release of 8.0, so it is worth to notice to all people about any failures of the branch.

Thanks!


On Mon, Nov 26, 2018 at 2:30 PM Uwe Schindler <[hidden email]> wrote:

I agree with Simon,

 

I’d prefer to just add jenkins jobs for the “http2” branch and not yet merge it. That’s possible to setup in short time. If the number of failures don’t grow beyond the current state, we can merge. If it get’s worse on same hardware, then we should not merge yet.

 

The problem with Solr tests is that the reliability differs from hardware to hardware (because of some timing/multithreading issues). So to make a comparison, you should compare with and without patch on same hardware.

 

I can setup a jenkins job on ASF jenkins and Policeman jenkins (copy of master job with different branch) and let it run. I can configure the dev mailing list or Cao Mạnh Đạt’s mail address there. What do you prefer?

 

Uwe

 

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de

eMail: [hidden email]

 

From: Simon Willnauer <[hidden email]>
Sent: Monday, November 26, 2018 3:03 PM
To: Lucene/Solr dev <[hidden email]>
Subject: Re: Poll: Merge jira/http2 to master branch

 

I don't know what the state of the tests is for solr. I'm also not on top of your work so please correct me if I am wrong. You are saying you won't have enough time for the feature in the near future but you want to merge it still to master so others can chime in? I don't think this is an approach that works. I think if others need to chime in it should be done on the branch? My definition for being ready to be merged to master is that the code and tests are stable. If there are minor tasks to be done then I guess I am fine with that. To me this sounds like it will put a ton of work on somebody else plate? I mean mark is spending a lot of time on the tests and this would make his life even harder?

 

I don't think we should do this. 

 

simon

 

On Mon, Nov 26, 2018 at 12:08 PM Đạt Cao Mạnh <[hidden email]> wrote:

Hi guys,

 

The changes in jira/http2 branch is very close to be commit now, but the current state of tests make me quite nervous about merging. I have a local Jenkins job for testing jira/http2 and the result is quite bad so far. I'm not sure failures caused by jira/http2 changes or the master branch. I wish I have more time on hardening and debugging the failures in jira/http2 branch. But the recent changes in my job do not allow me to spend as much time as I want. 

 

Therefore I'm thinking about merging jira/http2 to the master branch now and with the support of the community we can make tests more reliable. (not just me watching and working on it).

I know that Mark Miller is working on hardness tests, do you want me to wait for your work to be committed first?

 

Thanks!

 

--

Best regards,

Cao Mạnh Đạt

D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]



--
Best regards,
Cao Mạnh Đạt
D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Chris Hostetter-3
In reply to this post by Uwe Schindler

: I’d prefer to just add jenkins jobs for the “http2” branch and not yet

Uwe: note that in particular it would be really helpful to have a
jira/http2 jenkins job setup on your policeman's jenkins box, where java11
and java12 are randomized, since you seem to hit the java>9 SSL related
bugs the most, and AFAICT those problems are fixed on the jira/http2
branch -- see comments in SOLR-12990 (and related SOLR-12988)


-Hoss
http://www.lucidworks.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
Hi,

The job would use the usual randomization anyways, so what's your special request? So we should see an improvement asap.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Chris Hostetter <[hidden email]>
> Sent: Monday, November 26, 2018 8:04 PM
> To: Lucene Dev <[hidden email]>
> Cc: Uwe Schindler <[hidden email]>
> Subject: RE: Poll: Merge jira/http2 to master branch
>
>
> : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
>
> Uwe: note that in particular it would be really helpful to have a
> jira/http2 jenkins job setup on your policeman's jenkins box, where java11
> and java12 are randomized, since you seem to hit the java>9 SSL related
> bugs the most, and AFAICT those problems are fixed on the jira/http2
> branch -- see comments in SOLR-12990 (and related SOLR-12988)
>
>
> -Hoss
> http://www.lucidworks.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Chris Hostetter-3

: The job would use the usual randomization anyways, so what's your
: special request? So we should see an improvement asap.

No special request beyond asking you to setup a job on your personal
jenkins server -- just pointing out that i tried asking you for this via
jira ping 2 weeks ago :)

And yes: if my experimentation is correct, we should see a much lower rate
of failures from your box when testing Dat's http2 branch with java>9 vs
what we see w/master & 7x

: > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
: >
: > Uwe: note that in particular it would be really helpful to have a
: > jira/http2 jenkins job setup on your policeman's jenkins box, where java11
: > and java12 are randomized, since you seem to hit the java>9 SSL related
: > bugs the most, and AFAICT those problems are fixed on the jira/http2
: > branch -- see comments in SOLR-12990 (and related SOLR-12988)


-Hoss
http://www.lucidworks.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
Ah sorry, I did not see the ping. Where did you try to contact me?

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Chris Hostetter <[hidden email]>
> Sent: Monday, November 26, 2018 10:59 PM
> To: [hidden email]
> Subject: RE: Poll: Merge jira/http2 to master branch
>
>
> : The job would use the usual randomization anyways, so what's your
> : special request? So we should see an improvement asap.
>
> No special request beyond asking you to setup a job on your personal
> jenkins server -- just pointing out that i tried asking you for this via
> jira ping 2 weeks ago :)
>
> And yes: if my experimentation is correct, we should see a much lower rate
> of failures from your box when testing Dat's http2 branch with java>9 vs
> what we see w/master & 7x
>
> : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> : >
> : > Uwe: note that in particular it would be really helpful to have a
> : > jira/http2 jenkins job setup on your policeman's jenkins box, where java11
> : > and java12 are randomized, since you seem to hit the java>9 SSL related
> : > bugs the most, and AFAICT those problems are fixed on the jira/http2
> : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
>
>
> -Hoss
> http://www.lucidworks.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
OK,

I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris). On ASF Jenkins I created the standard "tests" job for now, others can be added later. They are called "http2" at the place of the version (instead of "7.x", "7.6", "master").

Let's see how it behaves,
Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Uwe Schindler <[hidden email]>
> Sent: Tuesday, November 27, 2018 12:22 AM
> To: [hidden email]
> Subject: RE: Poll: Merge jira/http2 to master branch
>
> Ah sorry, I did not see the ping. Where did you try to contact me?
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Chris Hostetter <[hidden email]>
> > Sent: Monday, November 26, 2018 10:59 PM
> > To: [hidden email]
> > Subject: RE: Poll: Merge jira/http2 to master branch
> >
> >
> > : The job would use the usual randomization anyways, so what's your
> > : special request? So we should see an improvement asap.
> >
> > No special request beyond asking you to setup a job on your personal
> > jenkins server -- just pointing out that i tried asking you for this via
> > jira ping 2 weeks ago :)
> >
> > And yes: if my experimentation is correct, we should see a much lower rate
> > of failures from your box when testing Dat's http2 branch with java>9 vs
> > what we see w/master & 7x
> >
> > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> > : >
> > : > Uwe: note that in particular it would be really helpful to have a
> > : > jira/http2 jenkins job setup on your policeman's jenkins box, where
> java11
> > : > and java12 are randomized, since you seem to hit the java>9 SSL related
> > : > bugs the most, and AFAICT those problems are fixed on the jira/http2
> > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> >
> >
> > -Hoss
> > http://www.lucidworks.com/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Chris Hostetter-3
In reply to this post by Uwe Schindler

: Ah sorry, I did not see the ping. Where did you try to contact me?

: > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)

https://issues.apache.org/jira/browse/SOLR-12990?focusedCommentId=16688433&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16688433


-Hoss
http://www.lucidworks.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
In reply to this post by Uwe Schindler
It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests fail. Reason is a mising JAR library: conscrypt.jar (which seems to be used by Jetty to support some HTTP/2 requires stuff not included in the JDK).

We should at least disable HTTP/2 in Java 8 or add this library (seems to contain native code): https://github.com/google/conscrypt#uber-jar

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Uwe Schindler <[hidden email]>
> Sent: Tuesday, November 27, 2018 12:38 AM
> To: [hidden email]
> Subject: RE: Poll: Merge jira/http2 to master branch
>
> OK,
>
> I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris). On
> ASF Jenkins I created the standard "tests" job for now, others can be added
> later. They are called "http2" at the place of the version (instead of "7.x",
> "7.6", "master").
>
> Let's see how it behaves,
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Uwe Schindler <[hidden email]>
> > Sent: Tuesday, November 27, 2018 12:22 AM
> > To: [hidden email]
> > Subject: RE: Poll: Merge jira/http2 to master branch
> >
> > Ah sorry, I did not see the ping. Where did you try to contact me?
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Chris Hostetter <[hidden email]>
> > > Sent: Monday, November 26, 2018 10:59 PM
> > > To: [hidden email]
> > > Subject: RE: Poll: Merge jira/http2 to master branch
> > >
> > >
> > > : The job would use the usual randomization anyways, so what's your
> > > : special request? So we should see an improvement asap.
> > >
> > > No special request beyond asking you to setup a job on your personal
> > > jenkins server -- just pointing out that i tried asking you for this via
> > > jira ping 2 weeks ago :)
> > >
> > > And yes: if my experimentation is correct, we should see a much lower
> rate
> > > of failures from your box when testing Dat's http2 branch with java>9 vs
> > > what we see w/master & 7x
> > >
> > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> > > : >
> > > : > Uwe: note that in particular it would be really helpful to have a
> > > : > jira/http2 jenkins job setup on your policeman's jenkins box, where
> > java11
> > > : > and java12 are randomized, since you seem to hit the java>9 SSL
> related
> > > : > bugs the most, and AFAICT those problems are fixed on the jira/http2
> > > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> > >
> > >
> > > -Hoss
> > > http://www.lucidworks.com/
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
Ah I figured out, there is an issue open already:
https://github.com/google/conscrypt/issues/560

Seems to be closed, so we have to wait for a new release, right?

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Uwe Schindler <[hidden email]>
> Sent: Tuesday, November 27, 2018 8:48 AM
> To: [hidden email]
> Subject: RE: Poll: Merge jira/http2 to master branch
>
> It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests fail.
> Reason is a mising JAR library: conscrypt.jar (which seems to be used by Jetty
> to support some HTTP/2 requires stuff not included in the JDK).
>
> We should at least disable HTTP/2 in Java 8 or add this library (seems to
> contain native code): https://github.com/google/conscrypt#uber-jar
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Uwe Schindler <[hidden email]>
> > Sent: Tuesday, November 27, 2018 12:38 AM
> > To: [hidden email]
> > Subject: RE: Poll: Merge jira/http2 to master branch
> >
> > OK,
> >
> > I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris). On
> > ASF Jenkins I created the standard "tests" job for now, others can be added
> > later. They are called "http2" at the place of the version (instead of "7.x",
> > "7.6", "master").
> >
> > Let's see how it behaves,
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Uwe Schindler <[hidden email]>
> > > Sent: Tuesday, November 27, 2018 12:22 AM
> > > To: [hidden email]
> > > Subject: RE: Poll: Merge jira/http2 to master branch
> > >
> > > Ah sorry, I did not see the ping. Where did you try to contact me?
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > http://www.thetaphi.de
> > > eMail: [hidden email]
> > >
> > > > -----Original Message-----
> > > > From: Chris Hostetter <[hidden email]>
> > > > Sent: Monday, November 26, 2018 10:59 PM
> > > > To: [hidden email]
> > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > >
> > > >
> > > > : The job would use the usual randomization anyways, so what's your
> > > > : special request? So we should see an improvement asap.
> > > >
> > > > No special request beyond asking you to setup a job on your personal
> > > > jenkins server -- just pointing out that i tried asking you for this via
> > > > jira ping 2 weeks ago :)
> > > >
> > > > And yes: if my experimentation is correct, we should see a much lower
> > rate
> > > > of failures from your box when testing Dat's http2 branch with java>9 vs
> > > > what we see w/master & 7x
> > > >
> > > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> > > > : >
> > > > : > Uwe: note that in particular it would be really helpful to have a
> > > > : > jira/http2 jenkins job setup on your policeman's jenkins box, where
> > > java11
> > > > : > and java12 are randomized, since you seem to hit the java>9 SSL
> > related
> > > > : > bugs the most, and AFAICT those problems are fixed on the
> jira/http2
> > > > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> > > >
> > > >
> > > > -Hoss
> > > > http://www.lucidworks.com/
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

Simon Willnauer-4
folks,

I just want to ask if this is an issue or not but this branch is
associated with a blocker issue for 8.0. If it doesn't have an owner
or is actively worked on do we still consider it a blocker? I mean
don't get me wrong but if this is something that we don't dedicate
time and committer to I think we should re-think the decision if this
is a blocker or not?

simon

On Tue, Nov 27, 2018 at 9:03 AM Uwe Schindler <[hidden email]> wrote:

>
> Ah I figured out, there is an issue open already:
> https://github.com/google/conscrypt/issues/560
>
> Seems to be closed, so we have to wait for a new release, right?
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Uwe Schindler <[hidden email]>
> > Sent: Tuesday, November 27, 2018 8:48 AM
> > To: [hidden email]
> > Subject: RE: Poll: Merge jira/http2 to master branch
> >
> > It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests fail.
> > Reason is a mising JAR library: conscrypt.jar (which seems to be used by Jetty
> > to support some HTTP/2 requires stuff not included in the JDK).
> >
> > We should at least disable HTTP/2 in Java 8 or add this library (seems to
> > contain native code): https://github.com/google/conscrypt#uber-jar
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Uwe Schindler <[hidden email]>
> > > Sent: Tuesday, November 27, 2018 12:38 AM
> > > To: [hidden email]
> > > Subject: RE: Poll: Merge jira/http2 to master branch
> > >
> > > OK,
> > >
> > > I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris). On
> > > ASF Jenkins I created the standard "tests" job for now, others can be added
> > > later. They are called "http2" at the place of the version (instead of "7.x",
> > > "7.6", "master").
> > >
> > > Let's see how it behaves,
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > http://www.thetaphi.de
> > > eMail: [hidden email]
> > >
> > > > -----Original Message-----
> > > > From: Uwe Schindler <[hidden email]>
> > > > Sent: Tuesday, November 27, 2018 12:22 AM
> > > > To: [hidden email]
> > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > >
> > > > Ah sorry, I did not see the ping. Where did you try to contact me?
> > > >
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: [hidden email]
> > > >
> > > > > -----Original Message-----
> > > > > From: Chris Hostetter <[hidden email]>
> > > > > Sent: Monday, November 26, 2018 10:59 PM
> > > > > To: [hidden email]
> > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > >
> > > > >
> > > > > : The job would use the usual randomization anyways, so what's your
> > > > > : special request? So we should see an improvement asap.
> > > > >
> > > > > No special request beyond asking you to setup a job on your personal
> > > > > jenkins server -- just pointing out that i tried asking you for this via
> > > > > jira ping 2 weeks ago :)
> > > > >
> > > > > And yes: if my experimentation is correct, we should see a much lower
> > > rate
> > > > > of failures from your box when testing Dat's http2 branch with java>9 vs
> > > > > what we see w/master & 7x
> > > > >
> > > > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> > > > > : >
> > > > > : > Uwe: note that in particular it would be really helpful to have a
> > > > > : > jira/http2 jenkins job setup on your policeman's jenkins box, where
> > > > java11
> > > > > : > and java12 are randomized, since you seem to hit the java>9 SSL
> > > related
> > > > > : > bugs the most, and AFAICT those problems are fixed on the
> > jira/http2
> > > > > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> > > > >
> > > > >
> > > > > -Hoss
> > > > > http://www.lucidworks.com/
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

caomanhdat
Hi Uwe,

It seems that is conflict between settings default security provider of conscrypt and httpclient. I will try to resolve that problem today.

Thanks!

On Tue, Nov 27, 2018 at 9:11 AM Simon Willnauer <[hidden email]> wrote:
folks,

I just want to ask if this is an issue or not but this branch is
associated with a blocker issue for 8.0. If it doesn't have an owner
or is actively worked on do we still consider it a blocker? I mean
don't get me wrong but if this is something that we don't dedicate
time and committer to I think we should re-think the decision if this
is a blocker or not?

simon

On Tue, Nov 27, 2018 at 9:03 AM Uwe Schindler <[hidden email]> wrote:
>
> Ah I figured out, there is an issue open already:
> https://github.com/google/conscrypt/issues/560
>
> Seems to be closed, so we have to wait for a new release, right?
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Uwe Schindler <[hidden email]>
> > Sent: Tuesday, November 27, 2018 8:48 AM
> > To: [hidden email]
> > Subject: RE: Poll: Merge jira/http2 to master branch
> >
> > It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests fail.
> > Reason is a mising JAR library: conscrypt.jar (which seems to be used by Jetty
> > to support some HTTP/2 requires stuff not included in the JDK).
> >
> > We should at least disable HTTP/2 in Java 8 or add this library (seems to
> > contain native code): https://github.com/google/conscrypt#uber-jar
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Uwe Schindler <[hidden email]>
> > > Sent: Tuesday, November 27, 2018 12:38 AM
> > > To: [hidden email]
> > > Subject: RE: Poll: Merge jira/http2 to master branch
> > >
> > > OK,
> > >
> > > I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris). On
> > > ASF Jenkins I created the standard "tests" job for now, others can be added
> > > later. They are called "http2" at the place of the version (instead of "7.x",
> > > "7.6", "master").
> > >
> > > Let's see how it behaves,
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > http://www.thetaphi.de
> > > eMail: [hidden email]
> > >
> > > > -----Original Message-----
> > > > From: Uwe Schindler <[hidden email]>
> > > > Sent: Tuesday, November 27, 2018 12:22 AM
> > > > To: [hidden email]
> > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > >
> > > > Ah sorry, I did not see the ping. Where did you try to contact me?
> > > >
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: [hidden email]
> > > >
> > > > > -----Original Message-----
> > > > > From: Chris Hostetter <[hidden email]>
> > > > > Sent: Monday, November 26, 2018 10:59 PM
> > > > > To: [hidden email]
> > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > >
> > > > >
> > > > > : The job would use the usual randomization anyways, so what's your
> > > > > : special request? So we should see an improvement asap.
> > > > >
> > > > > No special request beyond asking you to setup a job on your personal
> > > > > jenkins server -- just pointing out that i tried asking you for this via
> > > > > jira ping 2 weeks ago :)
> > > > >
> > > > > And yes: if my experimentation is correct, we should see a much lower
> > > rate
> > > > > of failures from your box when testing Dat's http2 branch with java>9 vs
> > > > > what we see w/master & 7x
> > > > >
> > > > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> > > > > : >
> > > > > : > Uwe: note that in particular it would be really helpful to have a
> > > > > : > jira/http2 jenkins job setup on your policeman's jenkins box, where
> > > > java11
> > > > > : > and java12 are randomized, since you seem to hit the java>9 SSL
> > > related
> > > > > : > bugs the most, and AFAICT those problems are fixed on the
> > jira/http2
> > > > > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> > > > >
> > > > >
> > > > > -Hoss
> > > > > http://www.lucidworks.com/
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Best regards,
Cao Mạnh Đạt
D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

Malcolm Upayavira Holmes
In reply to this post by Uwe Schindler
Uwe - there is already a release newer than the commit

On Tue, 27 Nov 2018, at 8:03 AM, Uwe Schindler wrote:

> Ah I figured out, there is an issue open already:
> https://github.com/google/conscrypt/issues/560
>
> Seems to be closed, so we have to wait for a new release, right?
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Uwe Schindler <[hidden email]>
> > Sent: Tuesday, November 27, 2018 8:48 AM
> > To: [hidden email]
> > Subject: RE: Poll: Merge jira/http2 to master branch
> >
> > It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests fail.
> > Reason is a mising JAR library: conscrypt.jar (which seems to be used by Jetty
> > to support some HTTP/2 requires stuff not included in the JDK).
> >
> > We should at least disable HTTP/2 in Java 8 or add this library (seems to
> > contain native code): https://github.com/google/conscrypt#uber-jar
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Uwe Schindler <[hidden email]>
> > > Sent: Tuesday, November 27, 2018 12:38 AM
> > > To: [hidden email]
> > > Subject: RE: Poll: Merge jira/http2 to master branch
> > >
> > > OK,
> > >
> > > I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris). On
> > > ASF Jenkins I created the standard "tests" job for now, others can be added
> > > later. They are called "http2" at the place of the version (instead of "7.x",
> > > "7.6", "master").
> > >
> > > Let's see how it behaves,
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > http://www.thetaphi.de
> > > eMail: [hidden email]
> > >
> > > > -----Original Message-----
> > > > From: Uwe Schindler <[hidden email]>
> > > > Sent: Tuesday, November 27, 2018 12:22 AM
> > > > To: [hidden email]
> > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > >
> > > > Ah sorry, I did not see the ping. Where did you try to contact me?
> > > >
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: [hidden email]
> > > >
> > > > > -----Original Message-----
> > > > > From: Chris Hostetter <[hidden email]>
> > > > > Sent: Monday, November 26, 2018 10:59 PM
> > > > > To: [hidden email]
> > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > >
> > > > >
> > > > > : The job would use the usual randomization anyways, so what's your
> > > > > : special request? So we should see an improvement asap.
> > > > >
> > > > > No special request beyond asking you to setup a job on your personal
> > > > > jenkins server -- just pointing out that i tried asking you for this via
> > > > > jira ping 2 weeks ago :)
> > > > >
> > > > > And yes: if my experimentation is correct, we should see a much lower
> > > rate
> > > > > of failures from your box when testing Dat's http2 branch with java>9 vs
> > > > > what we see w/master & 7x
> > > > >
> > > > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not yet
> > > > > : >
> > > > > : > Uwe: note that in particular it would be really helpful to have a
> > > > > : > jira/http2 jenkins job setup on your policeman's jenkins box, where
> > > > java11
> > > > > : > and java12 are randomized, since you seem to hit the java>9 SSL
> > > related
> > > > > : > bugs the most, and AFAICT those problems are fixed on the
> > jira/http2
> > > > > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> > > > >
> > > > >
> > > > > -Hoss
> > > > > http://www.lucidworks.com/
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
Thanks Upayavira,

I leave that open to Dat to update it. BTW, there are still checksum files missing for many jars.

I'd like to bring one thing into attention: This library conscrypt is ASF2-licensed, but the JAR files contain binary versions of an OpenSSL fork named BoringSSL. I'd recommend to check the licensing, because OpenSSL licenses are a bit strange (some BSD fork, also ASF2, but some code is still outdated - I am not sure what the fork actually uses). I have the feeling we should include ASF legal department.

Nevertheless, I am not 100% sure if conscrypt should really be inclued by default in SolrJ. As SolrJ is a client library for Solr and can be used by external projects, there is the problem of incompatibilities with the native C code included. E.g., if one uses SolrJ with IBM J9 or other Java versions different from openjdk. So I'd be careful. My question is: Do we really need that library. To me it looks like it improves speed only, or is there something that requires its use?

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Malcolm Upayavira Holmes <[hidden email]>
> Sent: Tuesday, November 27, 2018 11:17 AM
> To: [hidden email]
> Subject: Re: Poll: Merge jira/http2 to master branch
>
> Uwe - there is already a release newer than the commit
>
> On Tue, 27 Nov 2018, at 8:03 AM, Uwe Schindler wrote:
> > Ah I figured out, there is an issue open already:
> > https://github.com/google/conscrypt/issues/560
> >
> > Seems to be closed, so we have to wait for a new release, right?
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Uwe Schindler <[hidden email]>
> > > Sent: Tuesday, November 27, 2018 8:48 AM
> > > To: [hidden email]
> > > Subject: RE: Poll: Merge jira/http2 to master branch
> > >
> > > It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests
> fail.
> > > Reason is a mising JAR library: conscrypt.jar (which seems to be used by
> Jetty
> > > to support some HTTP/2 requires stuff not included in the JDK).
> > >
> > > We should at least disable HTTP/2 in Java 8 or add this library (seems to
> > > contain native code): https://github.com/google/conscrypt#uber-jar
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > http://www.thetaphi.de
> > > eMail: [hidden email]
> > >
> > > > -----Original Message-----
> > > > From: Uwe Schindler <[hidden email]>
> > > > Sent: Tuesday, November 27, 2018 12:38 AM
> > > > To: [hidden email]
> > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > >
> > > > OK,
> > > >
> > > > I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos, Solaris).
> On
> > > > ASF Jenkins I created the standard "tests" job for now, others can be
> added
> > > > later. They are called "http2" at the place of the version (instead of
> "7.x",
> > > > "7.6", "master").
> > > >
> > > > Let's see how it behaves,
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: [hidden email]
> > > >
> > > > > -----Original Message-----
> > > > > From: Uwe Schindler <[hidden email]>
> > > > > Sent: Tuesday, November 27, 2018 12:22 AM
> > > > > To: [hidden email]
> > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > >
> > > > > Ah sorry, I did not see the ping. Where did you try to contact me?
> > > > >
> > > > > Uwe
> > > > >
> > > > > -----
> > > > > Uwe Schindler
> > > > > Achterdiek 19, D-28357 Bremen
> > > > > http://www.thetaphi.de
> > > > > eMail: [hidden email]
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Chris Hostetter <[hidden email]>
> > > > > > Sent: Monday, November 26, 2018 10:59 PM
> > > > > > To: [hidden email]
> > > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > > >
> > > > > >
> > > > > > : The job would use the usual randomization anyways, so what's
> your
> > > > > > : special request? So we should see an improvement asap.
> > > > > >
> > > > > > No special request beyond asking you to setup a job on your
> personal
> > > > > > jenkins server -- just pointing out that i tried asking you for this via
> > > > > > jira ping 2 weeks ago :)
> > > > > >
> > > > > > And yes: if my experimentation is correct, we should see a much
> lower
> > > > rate
> > > > > > of failures from your box when testing Dat's http2 branch with
> java>9 vs
> > > > > > what we see w/master & 7x
> > > > > >
> > > > > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and not
> yet
> > > > > > : >
> > > > > > : > Uwe: note that in particular it would be really helpful to have a
> > > > > > : > jira/http2 jenkins job setup on your policeman's jenkins box,
> where
> > > > > java11
> > > > > > : > and java12 are randomized, since you seem to hit the java>9 SSL
> > > > related
> > > > > > : > bugs the most, and AFAICT those problems are fixed on the
> > > jira/http2
> > > > > > : > branch -- see comments in SOLR-12990 (and related SOLR-12988)
> > > > > >
> > > > > >
> > > > > > -Hoss
> > > > > > http://www.lucidworks.com/
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: [hidden email]
> > > > > For additional commands, e-mail: [hidden email]
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler
Hi,

Because if you install Jetty, to renable conscrypt, you have to start a special command and accept the non-eclipse license:
https://www.eclipse.org/jetty/documentation/9.4.x/jetty-ssl-distribution.html#jetty-conscrypt-distribution

Conscrypt SSL Configuration
Enabling Conscrypt SSL is just as easy as default SSL - enable both the conscrypt and ssl modules:

$ cd ${JETTY_HOME}
$ java -jar ../start.jar --add-to-start=ssl,conscrypt

ALERT: There are enabled module(s) with licenses.
The following 1 module(s):
 + contains software not provided by the Eclipse Foundation!
 + contains software not covered by the Eclipse Public License!
 + has not been audited for compliance with its license

 Module: conscrypt
  + Conscrypt is distributed under the Apache Licence 2.0
  + https://github.com/google/conscrypt/blob/master/LICENSE

Proceed (y/N)? y
INFO  : server          transitively enabled, ini template available with --add-to-start=server
INFO  : conscrypt       initialized in ${jetty.base}/start.d/conscrypt.ini
INFO  : ssl             initialized in ${jetty.base}/start.d/ssl.ini
MKDIR : ${jetty.base}/lib/conscrypt
DOWNLD: https://repo1.maven.org/maven2/org/conscrypt/conscrypt-openjdk-uber/1.0.0.RC11/conscrypt-openjdk-uber-1.0.0.RC11.jar to ${jetty.base}/lib/conscrypt/conscrypt-uber-1.0.0.RC11.jar
MKDIR : ${jetty.base}/etc
COPY  : ${jetty.home}/modules/conscrypt/conscrypt.xml to ${jetty.base}/etc/conscrypt.xml
COPY  : ${jetty.home}/modules/ssl/keystore to ${jetty.base}/etc/keystore
INFO  : Base directory was modified
No additional Conscrypt configuration is needed. SSL-specific parameters, like keyStorePath and keyStorePassword can still configured as in the example above, making use of the ${JETTY_BASE}/start.d/ssl.ini file.

Maybe we should do the same for solr: If you want full performance you may install conscrypt in your Solr working dir or the SolrJ client, but otherwise it should use non-native code shipped with JDK only. Maybe we should disable HTTP2 for Java 8 by default and only enable it if user installs conscrypt.

I have the feeling, we have to include ASF Legal department!
Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Uwe Schindler <[hidden email]>
> Sent: Tuesday, November 27, 2018 11:32 AM
> To: [hidden email]
> Subject: RE: Poll: Merge jira/http2 to master branch
>
> Thanks Upayavira,
>
> I leave that open to Dat to update it. BTW, there are still checksum files
> missing for many jars.
>
> I'd like to bring one thing into attention: This library conscrypt is ASF2-
> licensed, but the JAR files contain binary versions of an OpenSSL fork named
> BoringSSL. I'd recommend to check the licensing, because OpenSSL licenses
> are a bit strange (some BSD fork, also ASF2, but some code is still outdated - I
> am not sure what the fork actually uses). I have the feeling we should include
> ASF legal department.
>
> Nevertheless, I am not 100% sure if conscrypt should really be inclued by
> default in SolrJ. As SolrJ is a client library for Solr and can be used by external
> projects, there is the problem of incompatibilities with the native C code
> included. E.g., if one uses SolrJ with IBM J9 or other Java versions different
> from openjdk. So I'd be careful. My question is: Do we really need that
> library. To me it looks like it improves speed only, or is there something that
> requires its use?
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Malcolm Upayavira Holmes <[hidden email]>
> > Sent: Tuesday, November 27, 2018 11:17 AM
> > To: [hidden email]
> > Subject: Re: Poll: Merge jira/http2 to master branch
> >
> > Uwe - there is already a release newer than the commit
> >
> > On Tue, 27 Nov 2018, at 8:03 AM, Uwe Schindler wrote:
> > > Ah I figured out, there is an issue open already:
> > > https://github.com/google/conscrypt/issues/560
> > >
> > > Seems to be closed, so we have to wait for a new release, right?
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > http://www.thetaphi.de
> > > eMail: [hidden email]
> > >
> > > > -----Original Message-----
> > > > From: Uwe Schindler <[hidden email]>
> > > > Sent: Tuesday, November 27, 2018 8:48 AM
> > > > To: [hidden email]
> > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > >
> > > > It seems to work with Java 9/10/11, but with Java 8 almost all Solr tests
> > fail.
> > > > Reason is a mising JAR library: conscrypt.jar (which seems to be used by
> > Jetty
> > > > to support some HTTP/2 requires stuff not included in the JDK).
> > > >
> > > > We should at least disable HTTP/2 in Java 8 or add this library (seems to
> > > > contain native code): https://github.com/google/conscrypt#uber-jar
> > > >
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: [hidden email]
> > > >
> > > > > -----Original Message-----
> > > > > From: Uwe Schindler <[hidden email]>
> > > > > Sent: Tuesday, November 27, 2018 12:38 AM
> > > > > To: [hidden email]
> > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > >
> > > > > OK,
> > > > >
> > > > > I created 4 Jobs on Policeman Jenkins (Linux, Windows, macos,
> Solaris).
> > On
> > > > > ASF Jenkins I created the standard "tests" job for now, others can be
> > added
> > > > > later. They are called "http2" at the place of the version (instead of
> > "7.x",
> > > > > "7.6", "master").
> > > > >
> > > > > Let's see how it behaves,
> > > > > Uwe
> > > > >
> > > > > -----
> > > > > Uwe Schindler
> > > > > Achterdiek 19, D-28357 Bremen
> > > > > http://www.thetaphi.de
> > > > > eMail: [hidden email]
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Uwe Schindler <[hidden email]>
> > > > > > Sent: Tuesday, November 27, 2018 12:22 AM
> > > > > > To: [hidden email]
> > > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > > >
> > > > > > Ah sorry, I did not see the ping. Where did you try to contact me?
> > > > > >
> > > > > > Uwe
> > > > > >
> > > > > > -----
> > > > > > Uwe Schindler
> > > > > > Achterdiek 19, D-28357 Bremen
> > > > > > http://www.thetaphi.de
> > > > > > eMail: [hidden email]
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Chris Hostetter <[hidden email]>
> > > > > > > Sent: Monday, November 26, 2018 10:59 PM
> > > > > > > To: [hidden email]
> > > > > > > Subject: RE: Poll: Merge jira/http2 to master branch
> > > > > > >
> > > > > > >
> > > > > > > : The job would use the usual randomization anyways, so what's
> > your
> > > > > > > : special request? So we should see an improvement asap.
> > > > > > >
> > > > > > > No special request beyond asking you to setup a job on your
> > personal
> > > > > > > jenkins server -- just pointing out that i tried asking you for this via
> > > > > > > jira ping 2 weeks ago :)
> > > > > > >
> > > > > > > And yes: if my experimentation is correct, we should see a much
> > lower
> > > > > rate
> > > > > > > of failures from your box when testing Dat's http2 branch with
> > java>9 vs
> > > > > > > what we see w/master & 7x
> > > > > > >
> > > > > > > : > : I’d prefer to just add jenkins jobs for the “http2” branch and
> not
> > yet
> > > > > > > : >
> > > > > > > : > Uwe: note that in particular it would be really helpful to have a
> > > > > > > : > jira/http2 jenkins job setup on your policeman's jenkins box,
> > where
> > > > > > java11
> > > > > > > : > and java12 are randomized, since you seem to hit the java>9
> SSL
> > > > > related
> > > > > > > : > bugs the most, and AFAICT those problems are fixed on the
> > > > jira/http2
> > > > > > > : > branch -- see comments in SOLR-12990 (and related SOLR-
> 12988)
> > > > > > >
> > > > > > >
> > > > > > > -Hoss
> > > > > > > http://www.lucidworks.com/
> > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: [hidden email]
> > > > > > For additional commands, e-mail: [hidden email]
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: [hidden email]
> > > > > For additional commands, e-mail: [hidden email]
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

Shawn Heisey-2
In reply to this post by Uwe Schindler
On 11/27/2018 3:32 AM, Uwe Schindler wrote:
> I'd like to bring one thing into attention: This library conscrypt is ASF2-licensed, but the JAR files contain binary versions of an OpenSSL fork named BoringSSL. I'd recommend to check the licensing, because OpenSSL licenses are a bit strange (some BSD fork, also ASF2, but some code is still outdated - I am not sure what the fork actually uses). I have the feeling we should include ASF legal department.
>
> Nevertheless, I am not 100% sure if conscrypt should really be inclued by default in SolrJ. As SolrJ is a client library for Solr and can be used by external projects, there is the problem of incompatibilities with the native C code included. E.g., if one uses SolrJ with IBM J9 or other Java versions different from openjdk. So I'd be careful. My question is: Do we really need that library. To me it looks like it improves speed only, or is there something that requires its use?

As you might know ... full and proper http/2 support with Java 8
requires the conscrypt library.  With Java 9 or higher, the
functionality is provided natively by Java.  If I remember right, what
conscrypt provides that Java 8 lacks is ALPN.

If there are license issues with conscrypt, perhaps it might make sense
to require a newer major version of Java instead ... but I think that
upgrading the project's Java requirement to 9, 10, or 11 probably
requires a wider discussion.  Lucene probably has no burning need for it
right now.  I have no idea whether there are language features in Java
9+ that we as a group are wanting to use.  Another point for
consideration is that the effective EOL for Java 8 is fast approaching,
and EOL dates have already come and gone for 9 and 10.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Poll: Merge jira/http2 to master branch

caomanhdat
Hi,

I will try to summary all the options here
1. No support for HTTP/2 + SSL at all, if users want to run SSL they must stick with HTTP/1.1
2. Set Java requirement to 9 and use ALPN implementation of JDK 9
3. If users want to use HTTP/2, we will notice about license problem then download and use Conscrypt library. Of course that we still test the Conscrypt implementation in tests.


On Wed, Nov 28, 2018 at 1:06 AM Shawn Heisey <[hidden email]> wrote:
On 11/27/2018 3:32 AM, Uwe Schindler wrote:
> I'd like to bring one thing into attention: This library conscrypt is ASF2-licensed, but the JAR files contain binary versions of an OpenSSL fork named BoringSSL. I'd recommend to check the licensing, because OpenSSL licenses are a bit strange (some BSD fork, also ASF2, but some code is still outdated - I am not sure what the fork actually uses). I have the feeling we should include ASF legal department.
>
> Nevertheless, I am not 100% sure if conscrypt should really be inclued by default in SolrJ. As SolrJ is a client library for Solr and can be used by external projects, there is the problem of incompatibilities with the native C code included. E.g., if one uses SolrJ with IBM J9 or other Java versions different from openjdk. So I'd be careful. My question is: Do we really need that library. To me it looks like it improves speed only, or is there something that requires its use?

As you might know ... full and proper http/2 support with Java 8
requires the conscrypt library.  With Java 9 or higher, the
functionality is provided natively by Java.  If I remember right, what
conscrypt provides that Java 8 lacks is ALPN.

If there are license issues with conscrypt, perhaps it might make sense
to require a newer major version of Java instead ... but I think that
upgrading the project's Java requirement to 9, 10, or 11 probably
requires a wider discussion.  Lucene probably has no burning need for it
right now.  I have no idea whether there are language features in Java
9+ that we as a group are wanting to use.  Another point for
consideration is that the effective EOL for Java 8 is fast approaching,
and EOL dates have already come and gone for 9 and 10.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Best regards,
Cao Mạnh Đạt
D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Poll: Merge jira/http2 to master branch

Uwe Schindler

Hi Dat, hi Shawn,

 

Thanks for the confirmation! This is what I had in mind (missing ALPN support).

 

IMHO, we should not yet switch away from Java 8 as minimum requirement. With multi-release JARs we are at the moment perfectly able to handle users with newer Java versions, but we should still support Java 8 (as its still widely deployed), although EOL is close. As Redhat, AdoptOpenJDK, and several Linux distributions still provide support for Java 8 at least for 2 or 3 more years, I think it’s to early to switch. With recent changes, the support by Oracle is no longer the way to go, as there are many alternatives.

 

My proposal would be to wait until master is branched to “branch_8x” (stays on Java 8 + MR-JAR support), then change “master” to have Java 11 as minimum requirement.

 

About HTTP2: I have no problem with bundling conscrypt (is it needed for both Jetty Server and Jetty Client)?, but disable it by default and only register it in the Java TLS SPI, if Java 8 is used. On Java 9 and later use the shipped HTTP2 support. My proposal would be to do it with a Multirelease JAR (this is currently enabled in Lucene already).

 

If there are license problems, we can do the following: Disable HTTP2 on Java 8 completely but provide documentation in the Solr Ref Guide how to enable it (something like: download conscrypt-uber.jar from maven and save in solr/lib directory). We can enable it automatically, if class.forName() does not throw CNFE). Maybe add a sysprop, but that’s not needed. I think enabling conscrypt is easy with reflection only, or do we need to compile hard against it. From what I figured out, it’s only used at a few places.

 

SolrJ should by default ship without conscyrpt (make it “optional” maven dependency). If it’s in classpath, enable it.

 

Uwe

 

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de

eMail: [hidden email]

 

From: Đạt Cao Mạnh <[hidden email]>
Sent: Wednesday, November 28, 2018 11:01 AM
To: Solr/Lucene Dev <[hidden email]>
Subject: Re: Poll: Merge jira/http2 to master branch

 

Hi,

 

I will try to summary all the options here

1. No support for HTTP/2 + SSL at all, if users want to run SSL they must stick with HTTP/1.1

2. Set Java requirement to 9 and use ALPN implementation of JDK 9

3. If users want to use HTTP/2, we will notice about license problem then download and use Conscrypt library. Of course that we still test the Conscrypt implementation in tests.

 

 

On Wed, Nov 28, 2018 at 1:06 AM Shawn Heisey <[hidden email]> wrote:

On 11/27/2018 3:32 AM, Uwe Schindler wrote:
> I'd like to bring one thing into attention: This library conscrypt is ASF2-licensed, but the JAR files contain binary versions of an OpenSSL fork named BoringSSL. I'd recommend to check the licensing, because OpenSSL licenses are a bit strange (some BSD fork, also ASF2, but some code is still outdated - I am not sure what the fork actually uses). I have the feeling we should include ASF legal department.
>
> Nevertheless, I am not 100% sure if conscrypt should really be inclued by default in SolrJ. As SolrJ is a client library for Solr and can be used by external projects, there is the problem of incompatibilities with the native C code included. E.g., if one uses SolrJ with IBM J9 or other Java versions different from openjdk. So I'd be careful. My question is: Do we really need that library. To me it looks like it improves speed only, or is there something that requires its use?

As you might know ... full and proper http/2 support with Java 8
requires the conscrypt library.  With Java 9 or higher, the
functionality is provided natively by Java.  If I remember right, what
conscrypt provides that Java 8 lacks is ALPN.

If there are license issues with conscrypt, perhaps it might make sense
to require a newer major version of Java instead ... but I think that
upgrading the project's Java requirement to 9, 10, or 11 probably
requires a wider discussion.  Lucene probably has no burning need for it
right now.  I have no idea whether there are language features in Java
9+ that we as a group are wanting to use.  Another point for
consideration is that the effective EOL for Java 8 is fast approaching,
and EOL dates have already come and gone for 9 and 10.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


 

--

Best regards,

Cao Mạnh Đạt

D.O.B : 31-07-1991
Cell: (+84) 946.328.329
E-mail: [hidden email]

12