Solr jetty.port vs hostPort and host vs jetty.host

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr jetty.port vs hostPort and host vs jetty.host

david.w.smiley@gmail.com
While using MiniSolrCloudCluster on another project with a solr.xml that looks like the standard solr.xml, I ran into some problems.

The default solr.xml has a solrCloud element with these elements:
<str name="host">${host:}</str>
<int name="hostPort">${jetty.port:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
And MiniSolrCloudCluster.DEFAULT_CLOUD_SOLR_XML, a string literal, has the equivalent of this:
<str name="host">127.0.0.1</str>
<int name="hostPort">${hostPort:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
Notice the system property for the <hostPort> element is either "jetty.port" or "hostPort".  I submit that the system properties used here ought to be consistent.  So for example do away with jetty.port in lieu of hostPort given what appears more natural and consistent here.  This "hostPort" System property is used in a bunch of places in tests.  I've also seen a jetty.host lingering somewhere in some jetty xml files.

Also notice that the default solr.xml "host" has an empty string default, and AFAICT this is not equivalent to 127.0.0.1.

Any thoughts on this from folks in terms of understanding if/why we can't do this, or why this inconsistency is this way today?
Two relevant issues:

~ David Smiley
Apache Lucene/Solr Search Developer
Reply | Threaded
Open this post in threaded view
|

Re: Solr jetty.port vs hostPort and host vs jetty.host

Houston Putman
Thanks for pointing this out David.

I was confused about the difference between the two when I tried to make solr work with kubernetes networking. From my understanding now, "jetty.port" is the actual port that the jetty solr server listens on. "hostPort" is the port that the solr node advertises itself as, since the sole nodeName is just the host and port. This allowed me to run solr actually listening on port 8983, but advertising itself as listening on port 80. Then kubernetes services could map port 80 to port 8983.

I'm not sure if this is the official distinction between "hostPort" and "jetty.port", but it's worked for us. I would really like to keep this functionality available (I was very surprised to see it was possible when I first found it), because using solr within kubernetes will be harder without it.

- Houston

On Tue, Oct 8, 2019 at 1:44 PM David Smiley <[hidden email]> wrote:
While using MiniSolrCloudCluster on another project with a solr.xml that looks like the standard solr.xml, I ran into some problems.

The default solr.xml has a solrCloud element with these elements:
<str name="host">${host:}</str>
<int name="hostPort">${jetty.port:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
And MiniSolrCloudCluster.DEFAULT_CLOUD_SOLR_XML, a string literal, has the equivalent of this:
<str name="host">127.0.0.1</str>
<int name="hostPort">${hostPort:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
Notice the system property for the <hostPort> element is either "jetty.port" or "hostPort".  I submit that the system properties used here ought to be consistent.  So for example do away with jetty.port in lieu of hostPort given what appears more natural and consistent here.  This "hostPort" System property is used in a bunch of places in tests.  I've also seen a jetty.host lingering somewhere in some jetty xml files.

Also notice that the default solr.xml "host" has an empty string default, and AFAICT this is not equivalent to 127.0.0.1.

Any thoughts on this from folks in terms of understanding if/why we can't do this, or why this inconsistency is this way today?
Two relevant issues:

~ David Smiley
Apache Lucene/Solr Search Developer
Reply | Threaded
Open this post in threaded view
|

Re: Solr jetty.port vs hostPort and host vs jetty.host

Mark Miller-3
Yup, it's two different things, one tells jetty what port to start on and the other advertises a port in ZK.

I was pro renaming jetty.port to solr.port back in the day I believe, but I think jetty.port had some history or something.

Given we try to keep Jetty an impl detail best we can (not entirely possible), I'm not a big fan of leakage like this. I'd say fix it for 9. I'm guessing others would say it's not worth the disruption, but really we could allow jetty.port for a while as well if we had to.

- Mark

On Mon, Oct 14, 2019 at 10:56 AM Houston Putman <[hidden email]> wrote:
Thanks for pointing this out David.

I was confused about the difference between the two when I tried to make solr work with kubernetes networking. From my understanding now, "jetty.port" is the actual port that the jetty solr server listens on. "hostPort" is the port that the solr node advertises itself as, since the sole nodeName is just the host and port. This allowed me to run solr actually listening on port 8983, but advertising itself as listening on port 80. Then kubernetes services could map port 80 to port 8983.

I'm not sure if this is the official distinction between "hostPort" and "jetty.port", but it's worked for us. I would really like to keep this functionality available (I was very surprised to see it was possible when I first found it), because using solr within kubernetes will be harder without it.

- Houston

On Tue, Oct 8, 2019 at 1:44 PM David Smiley <[hidden email]> wrote:
While using MiniSolrCloudCluster on another project with a solr.xml that looks like the standard solr.xml, I ran into some problems.

The default solr.xml has a solrCloud element with these elements:
<str name="host">${host:}</str>
<int name="hostPort">${jetty.port:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
And MiniSolrCloudCluster.DEFAULT_CLOUD_SOLR_XML, a string literal, has the equivalent of this:
<str name="host">127.0.0.1</str>
<int name="hostPort">${hostPort:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
Notice the system property for the <hostPort> element is either "jetty.port" or "hostPort".  I submit that the system properties used here ought to be consistent.  So for example do away with jetty.port in lieu of hostPort given what appears more natural and consistent here.  This "hostPort" System property is used in a bunch of places in tests.  I've also seen a jetty.host lingering somewhere in some jetty xml files.

Also notice that the default solr.xml "host" has an empty string default, and AFAICT this is not equivalent to 127.0.0.1.

Any thoughts on this from folks in terms of understanding if/why we can't do this, or why this inconsistency is this way today?
Two relevant issues:

~ David Smiley
Apache Lucene/Solr Search Developer


--
Reply | Threaded
Open this post in threaded view
|

Re: Solr jetty.port vs hostPort and host vs jetty.host

david.w.smiley@gmail.com
Okay I think I'm appreciating this better now.  Thanks Houston & Mark.  So the <solrCloud> section is how Solr advertises itself... and it's default settings are inherited from Jetty, at least in the case of "jetty.port".  The distinction is definitely something we want to retain for containerized scenarios and perhaps others.   format-of-solr-xml.adoc could make this point a bit more explicit.  Maybe if MiniSolrCloudCluster or whatever other test component involved set jetty.port AND hostPort, then I wouldn't have tripped over this.  The host tripped me up too.

~ David Smiley
Apache Lucene/Solr Search Developer


On Wed, Oct 23, 2019 at 9:31 PM Mark Miller <[hidden email]> wrote:
Yup, it's two different things, one tells jetty what port to start on and the other advertises a port in ZK.

I was pro renaming jetty.port to solr.port back in the day I believe, but I think jetty.port had some history or something.

Given we try to keep Jetty an impl detail best we can (not entirely possible), I'm not a big fan of leakage like this. I'd say fix it for 9. I'm guessing others would say it's not worth the disruption, but really we could allow jetty.port for a while as well if we had to.

- Mark

On Mon, Oct 14, 2019 at 10:56 AM Houston Putman <[hidden email]> wrote:
Thanks for pointing this out David.

I was confused about the difference between the two when I tried to make solr work with kubernetes networking. From my understanding now, "jetty.port" is the actual port that the jetty solr server listens on. "hostPort" is the port that the solr node advertises itself as, since the sole nodeName is just the host and port. This allowed me to run solr actually listening on port 8983, but advertising itself as listening on port 80. Then kubernetes services could map port 80 to port 8983.

I'm not sure if this is the official distinction between "hostPort" and "jetty.port", but it's worked for us. I would really like to keep this functionality available (I was very surprised to see it was possible when I first found it), because using solr within kubernetes will be harder without it.

- Houston

On Tue, Oct 8, 2019 at 1:44 PM David Smiley <[hidden email]> wrote:
While using MiniSolrCloudCluster on another project with a solr.xml that looks like the standard solr.xml, I ran into some problems.

The default solr.xml has a solrCloud element with these elements:
<str name="host">${host:}</str>
<int name="hostPort">${jetty.port:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
And MiniSolrCloudCluster.DEFAULT_CLOUD_SOLR_XML, a string literal, has the equivalent of this:
<str name="host">127.0.0.1</str>
<int name="hostPort">${hostPort:8983}</int>
<str name="hostContext">${hostContext:solr}</str>
Notice the system property for the <hostPort> element is either "jetty.port" or "hostPort".  I submit that the system properties used here ought to be consistent.  So for example do away with jetty.port in lieu of hostPort given what appears more natural and consistent here.  This "hostPort" System property is used in a bunch of places in tests.  I've also seen a jetty.host lingering somewhere in some jetty xml files.

Also notice that the default solr.xml "host" has an empty string default, and AFAICT this is not equivalent to 127.0.0.1.

Any thoughts on this from folks in terms of understanding if/why we can't do this, or why this inconsistency is this way today?
Two relevant issues:

~ David Smiley
Apache Lucene/Solr Search Developer


--