HDInsight with Solr 4.9.0 Create Collection

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

HDInsight with Solr 4.9.0 Create Collection

Abhi Basu
Folks:

I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no Solrctl
commands installed. So, I am doing the following to create a collection.

I have my collection schema in a location:



/home/sshuser/abhi/ems-collection/conf



Using this command to create a collection:



http://headnode1:8983/solr/admin/cores?action=CREATE&name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
<http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.cloudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
/



Get an error like:

Error CREATEing SolrCore 'ems-collection': Unable to create core:
ems-collection Caused by: Could not find configName for collection
ems-collection found:[collection1, hditestconfig]



I guess i need to register my config name with Zk. How do I register the
collection schema with Zookeeper?


Is there way to bypass the registration with zk and build the collection
directly from my schema files at that folder location, like I was able to
do in Solr 4.10 in CDH 5.14:

solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
ems-collection /home/sshuser/abhi/ems-collection/

solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
ems-collection -s 3 -r 2


Your help is appreciated.



Thanks,

Abhi



--
Abhi Basu
Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Erick Erickson
First I have no idea what HDInsight is about, and CDH is a Cloudera
product so anything related to that specifically would best be asked
on the Cloudera lists. Solrctl is also a Cloudera-provided script
IIUC.

ButI can say that the command you showed is...odd. If you're saying
_you_ entered it then its about _core_ creation, not collection
creation and you should be using the Collections API, not the Core
Admin API.

Best,
Erick

On Thu, Mar 8, 2018 at 12:26 PM, Abhi Basu <[hidden email]> wrote:

> Folks:
>
> I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no Solrctl
> commands installed. So, I am doing the following to create a collection.
>
> I have my collection schema in a location:
>
>
>
> /home/sshuser/abhi/ems-collection/conf
>
>
>
> Using this command to create a collection:
>
>
>
> http://headnode1:8983/solr/admin/cores?action=CREATE&name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
> <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.cloudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
> /
>
>
>
> Get an error like:
>
> Error CREATEing SolrCore 'ems-collection': Unable to create core:
> ems-collection Caused by: Could not find configName for collection
> ems-collection found:[collection1, hditestconfig]
>
>
>
> I guess i need to register my config name with Zk. How do I register the
> collection schema with Zookeeper?
>
>
> Is there way to bypass the registration with zk and build the collection
> directly from my schema files at that folder location, like I was able to
> do in Solr 4.10 in CDH 5.14:
>
> solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
> ems-collection /home/sshuser/abhi/ems-collection/
>
> solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
> ems-collection -s 3 -r 2
>
>
> Your help is appreciated.
>
>
>
> Thanks,
>
> Abhi
>
>
>
> --
> Abhi Basu
Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Shawn Heisey-2
In reply to this post by Abhi Basu
On 3/8/2018 1:26 PM, Abhi Basu wrote:
> I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no Solrctl
> commands installed. So, I am doing the following to create a collection.

This 'solrctl' command is NOT part of Solr.  Google tells me it's part
of software from Cloudera.

You need to talk to Cloudera for support on that software.

> I have my collection schema in a location:
>
> /home/sshuser/abhi/ems-collection/conf
>
> Using this command to create a collection:
>
> http://headnode1:8983/solr/admin/cores?action=CREATE&name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
> <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.cloudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
> /

You're using the term "collection".  And later you mention ZooKeeper. So
you're almost certainly running in SolrCloud mode.  If your Solr is
running in SolrCloud mode, do not try to use the CoreAdmin API
(/solr/admin/cores).  Use the Collections API instead.  But before that,
you need to get the configuration into ZooKeeper.  For standard Solr
without Cloudera's tools, you would typically use the "zkcli" script
(either zkcli.sh or zkcli.bat).  See page 376 of the reference guide for
that specific version of Solr for help with the "upconfig" command for
that script:

http://archive.apache.org/dist/lucene/solr/ref-guide/apache-solr-ref-guide-4.9.pdf

> I guess i need to register my config name with Zk. How do I register the
> collection schema with Zookeeper?
>
> Is there way to bypass the registration with zk and build the collection
> directly from my schema files at that folder location, like I was able to
> do in Solr 4.10 in CDH 5.14:
>
> solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
> ems-collection /home/sshuser/abhi/ems-collection/
>
> solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
> ems-collection -s 3 -r 2

The solrctl command is not something we can help you with on this
mailing list.  Cloudera customizes Solr to the point where only they are
able to really provide support for their version.  Your best bet will be
to talk to Cloudera.

When Solr is running with ZooKeeper, it's in SolrCloud mode.  In
SolrCloud mode, you cannot create cores in the same way that you can in
standalone mode -- you MUST create collections, and all configuration
will be in zookeeper, not on the disk.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Abhi Basu
Thanks for the reply, this really helped me.

For Solr 4.9, what is the actual zkcli command to upload config?

java -classpath example/solr-webapp/WEB-INF/lib/*
 org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983
 -confdir example/solr/collection1/conf -confname conf1 -solrhome
example/solr

OR

./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd
upconfig -confname my_new_config -confdir
server/solr/configsets/basic_configs/conf

I dont know why HDP/HDInsight does not provide something like solrctl
commands to make life easier for all!




On Thu, Mar 8, 2018 at 5:43 PM, Shawn Heisey <[hidden email]> wrote:

> On 3/8/2018 1:26 PM, Abhi Basu wrote:
> > I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no Solrctl
> > commands installed. So, I am doing the following to create a collection.
>
> This 'solrctl' command is NOT part of Solr.  Google tells me it's part
> of software from Cloudera.
>
> You need to talk to Cloudera for support on that software.
>
> > I have my collection schema in a location:
> >
> > /home/sshuser/abhi/ems-collection/conf
> >
> > Using this command to create a collection:
> >
> > http://headnode1:8983/solr/admin/cores?action=CREATE&
> name=ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
> > <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.
> internal.cloudapp.net:8983/solr/admin/cores?action=
> CREATE&name=ems-collection&instanceDir=/home/sshuser/
> abhi/ems-collection/conf/>
> > /
>
> You're using the term "collection".  And later you mention ZooKeeper. So
> you're almost certainly running in SolrCloud mode.  If your Solr is
> running in SolrCloud mode, do not try to use the CoreAdmin API
> (/solr/admin/cores).  Use the Collections API instead.  But before that,
> you need to get the configuration into ZooKeeper.  For standard Solr
> without Cloudera's tools, you would typically use the "zkcli" script
> (either zkcli.sh or zkcli.bat).  See page 376 of the reference guide for
> that specific version of Solr for help with the "upconfig" command for
> that script:
>
> http://archive.apache.org/dist/lucene/solr/ref-guide/
> apache-solr-ref-guide-4.9.pdf
>
> > I guess i need to register my config name with Zk. How do I register the
> > collection schema with Zookeeper?
> >
> > Is there way to bypass the registration with zk and build the collection
> > directly from my schema files at that folder location, like I was able to
> > do in Solr 4.10 in CDH 5.14:
> >
> > solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
> > ems-collection /home/sshuser/abhi/ems-collection/
> >
> > solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
> > ems-collection -s 3 -r 2
>
> The solrctl command is not something we can help you with on this
> mailing list.  Cloudera customizes Solr to the point where only they are
> able to really provide support for their version.  Your best bet will be
> to talk to Cloudera.
>
> When Solr is running with ZooKeeper, it's in SolrCloud mode.  In
> SolrCloud mode, you cannot create cores in the same way that you can in
> standalone mode -- you MUST create collections, and all configuration
> will be in zookeeper, not on the disk.
>
> Thanks,
> Shawn
>
>


--
Abhi Basu
Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Abhi Basu
Ok, so I tried the following:

/usr/hdp/current/solr/example/scripts/cloud-scripts/zkcli.sh -cmd upconfig
-zkhost zk0-esohad.mzwz3dh4pb1evcdwc1lcsddrbe.jx.internal.cloudapp.net:2181
-confdir /home/sshuser/abhi/ems-collection/conf -confname ems-collection

And got this exception:
java.lang.IllegalArgumentException: Illegal directory:
/home/sshuser/abhi/ems-collection/conf


On Fri, Mar 9, 2018 at 10:43 AM, Abhi Basu <[hidden email]> wrote:

> Thanks for the reply, this really helped me.
>
> For Solr 4.9, what is the actual zkcli command to upload config?
>
> java -classpath example/solr-webapp/WEB-INF/lib/*
>  org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983
>  -confdir example/solr/collection1/conf -confname conf1 -solrhome
> example/solr
>
> OR
>
> ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd
> upconfig -confname my_new_config -confdir server/solr/configsets/basic_
> configs/conf
>
> I dont know why HDP/HDInsight does not provide something like solrctl
> commands to make life easier for all!
>
>
>
>
> On Thu, Mar 8, 2018 at 5:43 PM, Shawn Heisey <[hidden email]> wrote:
>
>> On 3/8/2018 1:26 PM, Abhi Basu wrote:
>> > I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no Solrctl
>> > commands installed. So, I am doing the following to create a collection.
>>
>> This 'solrctl' command is NOT part of Solr.  Google tells me it's part
>> of software from Cloudera.
>>
>> You need to talk to Cloudera for support on that software.
>>
>> > I have my collection schema in a location:
>> >
>> > /home/sshuser/abhi/ems-collection/conf
>> >
>> > Using this command to create a collection:
>> >
>> > http://headnode1:8983/solr/admin/cores?action=CREATE&name=
>> ems-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
>> > <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.
>> cloudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-
>> collection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
>> > /
>>
>> You're using the term "collection".  And later you mention ZooKeeper. So
>> you're almost certainly running in SolrCloud mode.  If your Solr is
>> running in SolrCloud mode, do not try to use the CoreAdmin API
>> (/solr/admin/cores).  Use the Collections API instead.  But before that,
>> you need to get the configuration into ZooKeeper.  For standard Solr
>> without Cloudera's tools, you would typically use the "zkcli" script
>> (either zkcli.sh or zkcli.bat).  See page 376 of the reference guide for
>> that specific version of Solr for help with the "upconfig" command for
>> that script:
>>
>> http://archive.apache.org/dist/lucene/solr/ref-guide/apache-
>> solr-ref-guide-4.9.pdf
>>
>> > I guess i need to register my config name with Zk. How do I register the
>> > collection schema with Zookeeper?
>> >
>> > Is there way to bypass the registration with zk and build the collection
>> > directly from my schema files at that folder location, like I was able
>> to
>> > do in Solr 4.10 in CDH 5.14:
>> >
>> > solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
>> > ems-collection /home/sshuser/abhi/ems-collection/
>> >
>> > solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
>> > ems-collection -s 3 -r 2
>>
>> The solrctl command is not something we can help you with on this
>> mailing list.  Cloudera customizes Solr to the point where only they are
>> able to really provide support for their version.  Your best bet will be
>> to talk to Cloudera.
>>
>> When Solr is running with ZooKeeper, it's in SolrCloud mode.  In
>> SolrCloud mode, you cannot create cores in the same way that you can in
>> standalone mode -- you MUST create collections, and all configuration
>> will be in zookeeper, not on the disk.
>>
>> Thanks,
>> Shawn
>>
>>
>
>
> --
> Abhi Basu
>



--
Abhi Basu
Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Abhi Basu
That was due to a folder not being present. Is this something to do with
version?

http://hn0-esohad.mzwz3dh4pb1evcdwc1lcsddrbe.jx.internal.cloudapp.net:8983/solr/admin/collections?action=CREATE&name=ems-collection2&numShards=2&replicationFactor=2&maxShardsPerNode=1


org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
CREATEing SolrCore 'ems-collection2_shard2_replica2': Unable to create
core: ems-collection2_shard2_replica2 Caused by: No enum constant
org.apache.lucene.util.Version.4.10.3

On Fri, Mar 9, 2018 at 11:11 AM, Abhi Basu <[hidden email]> wrote:

> Ok, so I tried the following:
>
> /usr/hdp/current/solr/example/scripts/cloud-scripts/zkcli.sh -cmd
> upconfig -zkhost zk0-esohad.mzwz3dh4pb1evcdwc1lcsddrbe.jx.
> internal.cloudapp.net:2181 -confdir /home/sshuser/abhi/ems-collection/conf
> -confname ems-collection
>
> And got this exception:
> java.lang.IllegalArgumentException: Illegal directory:
> /home/sshuser/abhi/ems-collection/conf
>
>
> On Fri, Mar 9, 2018 at 10:43 AM, Abhi Basu <[hidden email]> wrote:
>
>> Thanks for the reply, this really helped me.
>>
>> For Solr 4.9, what is the actual zkcli command to upload config?
>>
>> java -classpath example/solr-webapp/WEB-INF/lib/*
>>  org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983
>>  -confdir example/solr/collection1/conf -confname conf1 -solrhome
>> example/solr
>>
>> OR
>>
>> ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd
>> upconfig -confname my_new_config -confdir server/solr/configsets/basic_c
>> onfigs/conf
>>
>> I dont know why HDP/HDInsight does not provide something like solrctl
>> commands to make life easier for all!
>>
>>
>>
>>
>> On Thu, Mar 8, 2018 at 5:43 PM, Shawn Heisey <[hidden email]> wrote:
>>
>>> On 3/8/2018 1:26 PM, Abhi Basu wrote:
>>> > I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no
>>> Solrctl
>>> > commands installed. So, I am doing the following to create a
>>> collection.
>>>
>>> This 'solrctl' command is NOT part of Solr.  Google tells me it's part
>>> of software from Cloudera.
>>>
>>> You need to talk to Cloudera for support on that software.
>>>
>>> > I have my collection schema in a location:
>>> >
>>> > /home/sshuser/abhi/ems-collection/conf
>>> >
>>> > Using this command to create a collection:
>>> >
>>> > http://headnode1:8983/solr/admin/cores?action=CREATE&name=em
>>> s-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
>>> > <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.cl
>>> oudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-coll
>>> ection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
>>> > /
>>>
>>> You're using the term "collection".  And later you mention ZooKeeper. So
>>> you're almost certainly running in SolrCloud mode.  If your Solr is
>>> running in SolrCloud mode, do not try to use the CoreAdmin API
>>> (/solr/admin/cores).  Use the Collections API instead.  But before that,
>>> you need to get the configuration into ZooKeeper.  For standard Solr
>>> without Cloudera's tools, you would typically use the "zkcli" script
>>> (either zkcli.sh or zkcli.bat).  See page 376 of the reference guide for
>>> that specific version of Solr for help with the "upconfig" command for
>>> that script:
>>>
>>> http://archive.apache.org/dist/lucene/solr/ref-guide/apache-
>>> solr-ref-guide-4.9.pdf
>>>
>>> > I guess i need to register my config name with Zk. How do I register
>>> the
>>> > collection schema with Zookeeper?
>>> >
>>> > Is there way to bypass the registration with zk and build the
>>> collection
>>> > directly from my schema files at that folder location, like I was able
>>> to
>>> > do in Solr 4.10 in CDH 5.14:
>>> >
>>> > solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
>>> > ems-collection /home/sshuser/abhi/ems-collection/
>>> >
>>> > solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
>>> > ems-collection -s 3 -r 2
>>>
>>> The solrctl command is not something we can help you with on this
>>> mailing list.  Cloudera customizes Solr to the point where only they are
>>> able to really provide support for their version.  Your best bet will be
>>> to talk to Cloudera.
>>>
>>> When Solr is running with ZooKeeper, it's in SolrCloud mode.  In
>>> SolrCloud mode, you cannot create cores in the same way that you can in
>>> standalone mode -- you MUST create collections, and all configuration
>>> will be in zookeeper, not on the disk.
>>>
>>> Thanks,
>>> Shawn
>>>
>>>
>>
>>
>> --
>> Abhi Basu
>>
>
>
>
> --
> Abhi Basu
>



--
Abhi Basu
Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Abhi Basu
This has been resolved!

Turned out to be schema and config file version diff between 4.10 and 4.9.

Thanks,

Abhi

On Fri, Mar 9, 2018 at 11:41 AM, Abhi Basu <[hidden email]> wrote:

> That was due to a folder not being present. Is this something to do with
> version?
>
> http://hn0-esohad.mzwz3dh4pb1evcdwc1lcsddrbe.jx.
> internal.cloudapp.net:8983/solr/admin/collections?action=
> CREATE&name=ems-collection2&numShards=2&replicationFactor=
> 2&maxShardsPerNode=1
>
>
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'ems-collection2_shard2_replica2': Unable to create
> core: ems-collection2_shard2_replica2 Caused by: No enum constant
> org.apache.lucene.util.Version.4.10.3
>
> On Fri, Mar 9, 2018 at 11:11 AM, Abhi Basu <[hidden email]> wrote:
>
>> Ok, so I tried the following:
>>
>> /usr/hdp/current/solr/example/scripts/cloud-scripts/zkcli.sh -cmd
>> upconfig -zkhost zk0-esohad.mzwz3dh4pb1evcdwc1l
>> csddrbe.jx.internal.cloudapp.net:2181 -confdir
>> /home/sshuser/abhi/ems-collection/conf -confname ems-collection
>>
>> And got this exception:
>> java.lang.IllegalArgumentException: Illegal directory:
>> /home/sshuser/abhi/ems-collection/conf
>>
>>
>> On Fri, Mar 9, 2018 at 10:43 AM, Abhi Basu <[hidden email]> wrote:
>>
>>> Thanks for the reply, this really helped me.
>>>
>>> For Solr 4.9, what is the actual zkcli command to upload config?
>>>
>>> java -classpath example/solr-webapp/WEB-INF/lib/*
>>>  org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983
>>>  -confdir example/solr/collection1/conf -confname conf1 -solrhome
>>> example/solr
>>>
>>> OR
>>>
>>> ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd
>>> upconfig -confname my_new_config -confdir server/solr/configsets/basic_c
>>> onfigs/conf
>>>
>>> I dont know why HDP/HDInsight does not provide something like solrctl
>>> commands to make life easier for all!
>>>
>>>
>>>
>>>
>>> On Thu, Mar 8, 2018 at 5:43 PM, Shawn Heisey <[hidden email]>
>>> wrote:
>>>
>>>> On 3/8/2018 1:26 PM, Abhi Basu wrote:
>>>> > I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no
>>>> Solrctl
>>>> > commands installed. So, I am doing the following to create a
>>>> collection.
>>>>
>>>> This 'solrctl' command is NOT part of Solr.  Google tells me it's part
>>>> of software from Cloudera.
>>>>
>>>> You need to talk to Cloudera for support on that software.
>>>>
>>>> > I have my collection schema in a location:
>>>> >
>>>> > /home/sshuser/abhi/ems-collection/conf
>>>> >
>>>> > Using this command to create a collection:
>>>> >
>>>> > http://headnode1:8983/solr/admin/cores?action=CREATE&name=em
>>>> s-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
>>>> > <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.cl
>>>> oudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-coll
>>>> ection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
>>>> > /
>>>>
>>>> You're using the term "collection".  And later you mention ZooKeeper. So
>>>> you're almost certainly running in SolrCloud mode.  If your Solr is
>>>> running in SolrCloud mode, do not try to use the CoreAdmin API
>>>> (/solr/admin/cores).  Use the Collections API instead.  But before that,
>>>> you need to get the configuration into ZooKeeper.  For standard Solr
>>>> without Cloudera's tools, you would typically use the "zkcli" script
>>>> (either zkcli.sh or zkcli.bat).  See page 376 of the reference guide for
>>>> that specific version of Solr for help with the "upconfig" command for
>>>> that script:
>>>>
>>>> http://archive.apache.org/dist/lucene/solr/ref-guide/apache-
>>>> solr-ref-guide-4.9.pdf
>>>>
>>>> > I guess i need to register my config name with Zk. How do I register
>>>> the
>>>> > collection schema with Zookeeper?
>>>> >
>>>> > Is there way to bypass the registration with zk and build the
>>>> collection
>>>> > directly from my schema files at that folder location, like I was
>>>> able to
>>>> > do in Solr 4.10 in CDH 5.14:
>>>> >
>>>> > solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
>>>> > ems-collection /home/sshuser/abhi/ems-collection/
>>>> >
>>>> > solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
>>>> > ems-collection -s 3 -r 2
>>>>
>>>> The solrctl command is not something we can help you with on this
>>>> mailing list.  Cloudera customizes Solr to the point where only they are
>>>> able to really provide support for their version.  Your best bet will be
>>>> to talk to Cloudera.
>>>>
>>>> When Solr is running with ZooKeeper, it's in SolrCloud mode.  In
>>>> SolrCloud mode, you cannot create cores in the same way that you can in
>>>> standalone mode -- you MUST create collections, and all configuration
>>>> will be in zookeeper, not on the disk.
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>>>
>>>
>>>
>>> --
>>> Abhi Basu
>>>
>>
>>
>>
>> --
>> Abhi Basu
>>
>
>
>
> --
> Abhi Basu
>



--
Abhi Basu
Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

spoonerk
would be nice to not get this email.


On 3/9/2018 1:23 PM, Abhi Basu wrote:

> This has been resolved!
>
> Turned out to be schema and config file version diff between 4.10 and 4.9.
>
> Thanks,
>
> Abhi
>
> On Fri, Mar 9, 2018 at 11:41 AM, Abhi Basu <[hidden email]> wrote:
>
>> That was due to a folder not being present. Is this something to do with
>> version?
>>
>> http://hn0-esohad.mzwz3dh4pb1evcdwc1lcsddrbe.jx.
>> internal.cloudapp.net:8983/solr/admin/collections?action=
>> CREATE&name=ems-collection2&numShards=2&replicationFactor=
>> 2&maxShardsPerNode=1
>>
>>
>> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>> CREATEing SolrCore 'ems-collection2_shard2_replica2': Unable to create
>> core: ems-collection2_shard2_replica2 Caused by: No enum constant
>> org.apache.lucene.util.Version.4.10.3
>>
>> On Fri, Mar 9, 2018 at 11:11 AM, Abhi Basu <[hidden email]> wrote:
>>
>>> Ok, so I tried the following:
>>>
>>> /usr/hdp/current/solr/example/scripts/cloud-scripts/zkcli.sh -cmd
>>> upconfig -zkhost zk0-esohad.mzwz3dh4pb1evcdwc1l
>>> csddrbe.jx.internal.cloudapp.net:2181 -confdir
>>> /home/sshuser/abhi/ems-collection/conf -confname ems-collection
>>>
>>> And got this exception:
>>> java.lang.IllegalArgumentException: Illegal directory:
>>> /home/sshuser/abhi/ems-collection/conf
>>>
>>>
>>> On Fri, Mar 9, 2018 at 10:43 AM, Abhi Basu <[hidden email]> wrote:
>>>
>>>> Thanks for the reply, this really helped me.
>>>>
>>>> For Solr 4.9, what is the actual zkcli command to upload config?
>>>>
>>>> java -classpath example/solr-webapp/WEB-INF/lib/*
>>>>   org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983
>>>>   -confdir example/solr/collection1/conf -confname conf1 -solrhome
>>>> example/solr
>>>>
>>>> OR
>>>>
>>>> ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 -cmd
>>>> upconfig -confname my_new_config -confdir server/solr/configsets/basic_c
>>>> onfigs/conf
>>>>
>>>> I dont know why HDP/HDInsight does not provide something like solrctl
>>>> commands to make life easier for all!
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Mar 8, 2018 at 5:43 PM, Shawn Heisey <[hidden email]>
>>>> wrote:
>>>>
>>>>> On 3/8/2018 1:26 PM, Abhi Basu wrote:
>>>>>> I'm in a bind. Added Solr 4.9.0 to HDInsight cluster and find no
>>>>> Solrctl
>>>>>> commands installed. So, I am doing the following to create a
>>>>> collection.
>>>>>
>>>>> This 'solrctl' command is NOT part of Solr.  Google tells me it's part
>>>>> of software from Cloudera.
>>>>>
>>>>> You need to talk to Cloudera for support on that software.
>>>>>
>>>>>> I have my collection schema in a location:
>>>>>>
>>>>>> /home/sshuser/abhi/ems-collection/conf
>>>>>>
>>>>>> Using this command to create a collection:
>>>>>>
>>>>>> http://headnode1:8983/solr/admin/cores?action=CREATE&name=em
>>>>> s-collection&instanceDir=/home/sshuser/abhi/ems-collection/conf
>>>>>> <http://hn0-esohad.iqz04pwsg24ulbodxuo51nheig.jx.internal.cl
>>>>> oudapp.net:8983/solr/admin/cores?action=CREATE&name=ems-coll
>>>>> ection&instanceDir=/home/sshuser/abhi/ems-collection/conf/>
>>>>>> /
>>>>> You're using the term "collection".  And later you mention ZooKeeper. So
>>>>> you're almost certainly running in SolrCloud mode.  If your Solr is
>>>>> running in SolrCloud mode, do not try to use the CoreAdmin API
>>>>> (/solr/admin/cores).  Use the Collections API instead.  But before that,
>>>>> you need to get the configuration into ZooKeeper.  For standard Solr
>>>>> without Cloudera's tools, you would typically use the "zkcli" script
>>>>> (either zkcli.sh or zkcli.bat).  See page 376 of the reference guide for
>>>>> that specific version of Solr for help with the "upconfig" command for
>>>>> that script:
>>>>>
>>>>> http://archive.apache.org/dist/lucene/solr/ref-guide/apache-
>>>>> solr-ref-guide-4.9.pdf
>>>>>
>>>>>> I guess i need to register my config name with Zk. How do I register
>>>>> the
>>>>>> collection schema with Zookeeper?
>>>>>>
>>>>>> Is there way to bypass the registration with zk and build the
>>>>> collection
>>>>>> directly from my schema files at that folder location, like I was
>>>>> able to
>>>>>> do in Solr 4.10 in CDH 5.14:
>>>>>>
>>>>>> solrctl --zk hadoop-dn6.eso.local:2181/solr instancedir --create
>>>>>> ems-collection /home/sshuser/abhi/ems-collection/
>>>>>>
>>>>>> solrctl --zk hadoop-dn6.eso.local:2181/solr collection --create
>>>>>> ems-collection -s 3 -r 2
>>>>> The solrctl command is not something we can help you with on this
>>>>> mailing list.  Cloudera customizes Solr to the point where only they are
>>>>> able to really provide support for their version.  Your best bet will be
>>>>> to talk to Cloudera.
>>>>>
>>>>> When Solr is running with ZooKeeper, it's in SolrCloud mode.  In
>>>>> SolrCloud mode, you cannot create cores in the same way that you can in
>>>>> standalone mode -- you MUST create collections, and all configuration
>>>>> will be in zookeeper, not on the disk.
>>>>>
>>>>> Thanks,
>>>>> Shawn
>>>>>
>>>>>
>>>>
>>>> --
>>>> Abhi Basu
>>>>
>>>
>>>
>>> --
>>> Abhi Basu
>>>
>>
>>
>> --
>> Abhi Basu
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: HDInsight with Solr 4.9.0 Create Collection

Shawn Heisey-2
In reply to this post by Abhi Basu
On 3/9/2018 10:41 AM, Abhi Basu wrote:
> That was due to a folder not being present. Is this something to do with
> version?
>
> http://hn0-esohad.mzwz3dh4pb1evcdwc1lcsddrbe.jx.internal.cloudapp.net:8983/solr/admin/collections?action=CREATE&name=ems-collection2&numShards=2&replicationFactor=2&maxShardsPerNode=1
>
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'ems-collection2_shard2_replica2': Unable to create
> core: ems-collection2_shard2_replica2 Caused by: No enum constant
> org.apache.lucene.util.Version.4.10.3

I see that you've resolved both problems.

If you have additional issues in the future with errors you don't
understand:  We need far more detail from your log.  It is very likely
that the full error message is several lines long, possibly even dozens
of lines long, and may include *multiple* java stacktraces.  Try looking
at solr.log instead of the admin UI.  For 4.x versions, we usually have
no way of knowing where the logfile ends up, because there are MANY
different ways 4.x versions can be installed and started.

Thanks,
Shawn