Facing Gora exception in Nutch 2.4

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Facing Gora exception in Nutch 2.4

Gajalakshmi G
Hi all,

I am using Nutch 2.4 with Hadoop 3.1.1 and hbase 2.0.2 along with Gora 0.9 version.

I am getting below error  while trying to run the code:

InjectorJob: org.apache.gora.util.GoraException: java.lang.RuntimeException: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:162)
        at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:130)
        at org.apache.nutch.storage.StorageUtils.createWebStore(StorageUtils.java:78)
        at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:267)
        at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:301)
        at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:324)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:334)
Caused by: java.lang.RuntimeException: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:138)
        at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:97)
        at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:156)
        ... 7 more
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
        at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
        at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
        at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:131)
        ... 9 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
        ... 12 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/io/LimitInputStream
        at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
        at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:905)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:648)
        ... 17 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.io.LimitInputStream
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 22 more



Please guide me on resolving this error.



Thanks & Regards,

Gajalakshmi.G

Assistant Consultant

Tata Consultancy Services
Mailto: [hidden email]<https://mail.tcs.com/owa/redir.aspx?C=15cf4bf65eff4bdab465e0a2dd682f11&URL=mailto%3agajalakshmi.g%40tcs.com>
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you


Reply | Threaded
Open this post in threaded view
|

Re: Facing Gora exception in Nutch 2.4

lewis john mcgibbney-2
Hi Gajalakshmi.G,
Firstly, it's important for me to state that Nutch 2.X is deprecated. No
more development ius happening on the 2.X branch.
That being said, please see my comments inline below

On Thu, Sep 17, 2020 at 7:45 AM <[hidden email]> wrote:

>
> I am using Nutch 2.4 with Hadoop 3.1.1


To the best of my knowledge, Nutch 2.4 was never tested against Hadoop 3.x
https://github.com/apache/nutch/blob/release-2.4/ivy/ivy.xml#L49-L61


> and hbase 2.0.2 along with Gora 0.9 version.
>

It was also not tested against Gora 0.9
https://github.com/apache/nutch/blob/release-2.4/ivy/ivy.xml#L107
However Gora 0.9 WAS tested against HBase 2.1.1
https://github.com/apache/gora/blob/apache-gora-0.9/pom.xml#L787
HOWEVER Gora 0.9 WAS NOT tested against Hadoop 3.X


>
> I am getting below error  while trying to run the code:
>

... <snip>


> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.io
> .LimitInputStream
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         ... 22 more
>

The ClassNotFoundException is most likely caused by an incompatible
dependency trail as I describe above. If you really wanted to use Nutch 2.X
(which I would advise ag=ainst at this stage) then you would need to update
the dependency chain.

Sorry I can't be of any further help.
Lewis