Solr Zookeeper error while connecting from Spark context

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Solr Zookeeper error while connecting from Spark context

Manjunath N S (mans3)

Hello,

I have been trying to read hive tables by invoking HiveContext within SparkContext.

The data is being read from hive tables,However when I try to connect it to Solr external zookeeper server from Hadoop edge node,I am facing an issue.

 

The same code works when I try to push sample json from SparkContext to Solr using same zookeeper server.I am using lucidworks spark-solr:2.1.0 package to do this. End goal is to push parquet files to Solr directly.

 

Please find the error below.

 

17/02/22 20:14:17 ERROR ZooKeeperSaslClient: SASL authentication failed using login context 'Client'.

17/02/22 20:14:17 WARN ConnectionManager: zkClient received AuthFailed

17/02/22 20:14:17 WARN SolrQuerySupport: Can't get uniqueKey for testspark due to: com.google.common.util.concurrent.UncheckedExecutionException: org.apache.solr.common.cloud.ZooKeeperException:

17/02/22 20:14:17 ERROR ZooKeeperSaslClient: SASL authentication failed using login context 'Client'.

17/02/22 20:14:17 WARN ConnectionManager: zkClient received AuthFailed

Exception in thread "main" com.google.common.util.concurrent.UncheckedExecutionException: org.apache.solr.common.cloud.ZooKeeperException:

        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2263)

        at com.google.common.cache.LocalCache.get(LocalCache.java:4000)

        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004)

        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)

        at com.lucidworks.spark.util.SolrSupport$.getCachedCloudClient(SolrSupport.scala:93)

        at com.lucidworks.spark.util.SolrSupport$.getSolrBaseUrl(SolrSupport.scala:97)

        at com.lucidworks.spark.util.SolrRelationUtil$.getBaseSchema(SolrRelationUtil.scala:34)

        at com.lucidworks.spark.SolrRelation.<init>(SolrRelation.scala:83)

        at solr.DefaultSource.createRelation(DefaultSource.scala:26)

        at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:222)

        at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:148)

        at com.cisco.SparkSolr.main(SparkSolr.java:58)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:742)

        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)

        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Caused by: org.apache.solr.common.cloud.ZooKeeperException:

        at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:475)

        at com.lucidworks.spark.util.SolrSupport$.getSolrCloudClient(SolrSupport.scala:83)

        at com.lucidworks.spark.util.SolrSupport$.getNewSolrCloudClient(SolrSupport.scala:89)

        at com.lucidworks.spark.util.CacheSolrClient$$anon$1.load(SolrSupport.scala:38)

        at com.lucidworks.spark.util.CacheSolrClient$$anon$1.load(SolrSupport.scala:36)

        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)

        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)

        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)

       at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)

        ... 20 more

Caused by: org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /clusterstate.json

        at org.apache.zookeeper.KeeperException.create(KeeperException.java:123)

        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)

        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)

        at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:311)

        at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:308)

        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)

        at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)

        at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:289)

        at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:467)

        ... 28 more

hdsvmg@hddev-c01-edge-01:/users/hdsvmg>