General question for warning message.

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

General question for warning message.

Erik Test
Hello,

I'm getting the following messages when I try to run a job I've developed.

 hadoop jar distanceCalc.jar DistanceCalc distanceCalculations
distanceCalculations/output9

10/07/28 09:25:37 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.

Exception in thread "main" java.lang.RuntimeException:
java.lang.NoSuchMethodException:
DistanceCalc$DistanceValuesInputFormat.<init>()
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:400)
        at
org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:810)
        at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:781)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1249)
        at DistanceCalc.main(DistanceCalc.java:228)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.NoSuchMethodException:
DistanceCalc$DistanceValuesInputFormat.<init>()
        at java.lang.Class.getConstructor0(Class.java:2706)
        at java.lang.Class.getDeclaredConstructor(Class.java:1985)
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
        ... 11 more

I'm obviously doing something wrong in the code but my question for this
thread is the warning message I'm receiving.  I'm not quite sure what it
means and what I've seen on google so far is the warning has something to do
with Pig but I'm not using Pig. Can someone tell me what the warning means
please?

Thanks for your time

Erik
Reply | Threaded
Open this post in threaded view
|

Re: General question for warning message.

Himanshu Vashishtha
Hello,
The exception is not related to the warning message.

It is a Reflection error. You have defined a custom Input format. Provide a
no arg constructor and then test.

On Wed, Jul 28, 2010 at 9:42 AM, Erik Test <[hidden email]> wrote:

> Hello,
>
> I'm getting the following messages when I try to run a job I've developed.
>
>  hadoop jar distanceCalc.jar DistanceCalc distanceCalculations
> distanceCalculations/output9
>
> 10/07/28 09:25:37 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
>
> Exception in thread "main" java.lang.RuntimeException:
> java.lang.NoSuchMethodException:
> DistanceCalc$DistanceValuesInputFormat.<init>()
>        at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
>        at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:400)
>        at
> org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:810)
>        at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:781)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1249)
>        at DistanceCalc.main(DistanceCalc.java:228)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NoSuchMethodException:
> DistanceCalc$DistanceValuesInputFormat.<init>()
>        at java.lang.Class.getConstructor0(Class.java:2706)
>        at java.lang.Class.getDeclaredConstructor(Class.java:1985)
>        at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
>        ... 11 more
>
> I'm obviously doing something wrong in the code but my question for this
> thread is the warning message I'm receiving.  I'm not quite sure what it
> means and what I've seen on google so far is the warning has something to
> do
> with Pig but I'm not using Pig. Can someone tell me what the warning means
> please?
>
> Thanks for your time
>
> Erik
>
Reply | Threaded
Open this post in threaded view
|

Re: General question for warning message.

Erik Test
Thank you for the answer. I am aware of this and have been looking how to
solve this issue in other places because I'm already providing a no arg
constructor.

However, I'm still curious what this warning message means. Can someone tell
me what this warning means?

Erik


On 28 July 2010 11:57, Himanshu Vashishtha <[hidden email]> wrote:

> Hello,
> The exception is not related to the warning message.
>
> It is a Reflection error. You have defined a custom Input format. Provide a
> no arg constructor and then test.
>
> On Wed, Jul 28, 2010 at 9:42 AM, Erik Test <[hidden email]> wrote:
>
> > Hello,
> >
> > I'm getting the following messages when I try to run a job I've
> developed.
> >
> >  hadoop jar distanceCalc.jar DistanceCalc distanceCalculations
> > distanceCalculations/output9
> >
> > 10/07/28 09:25:37 WARN mapred.JobClient: Use GenericOptionsParser for
> > parsing the arguments. Applications should implement Tool for the same.
> >
> > Exception in thread "main" java.lang.RuntimeException:
> > java.lang.NoSuchMethodException:
> > DistanceCalc$DistanceValuesInputFormat.<init>()
> >        at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> >        at
> org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:400)
> >        at
> > org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:810)
> >        at
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:781)
> >        at
> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
> >        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1249)
> >        at DistanceCalc.main(DistanceCalc.java:228)
> >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >        at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >        at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >        at java.lang.reflect.Method.invoke(Method.java:597)
> >        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Caused by: java.lang.NoSuchMethodException:
> > DistanceCalc$DistanceValuesInputFormat.<init>()
> >        at java.lang.Class.getConstructor0(Class.java:2706)
> >        at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> >        at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> >        ... 11 more
> >
> > I'm obviously doing something wrong in the code but my question for this
> > thread is the warning message I'm receiving.  I'm not quite sure what it
> > means and what I've seen on google so far is the warning has something to
> > do
> > with Pig but I'm not using Pig. Can someone tell me what the warning
> means
> > please?
> >
> > Thanks for your time
> >
> > Erik
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: General question for warning message.

Himanshu Vashishtha
And your custom input shd also be a static class.

The warning suggests you to implement the Tool interface. It is another way
to start your MR job.
The syntax is:
public class <YourClass> extends Configured implements Tool {

///    Define mapper/reducer classes....

and override the run(String s[]) method (in this method, you configure your
job object and then invoke the runJob method of JobClient class.

// in the main method, invoke the run method.

HTH,
Himanshu

On Wed, Jul 28, 2010 at 10:02 AM, Erik Test <[hidden email]> wrote:

> Thank you for the answer. I am aware of this and have been looking how to
> solve this issue in other places because I'm already providing a no arg
> constructor.
>
> However, I'm still curious what this warning message means. Can someone
> tell
> me what this warning means?
>
> Erik
>
>
> On 28 July 2010 11:57, Himanshu Vashishtha <[hidden email]> wrote:
>
> > Hello,
> > The exception is not related to the warning message.
> >
> > It is a Reflection error. You have defined a custom Input format. Provide
> a
> > no arg constructor and then test.
> >
> > On Wed, Jul 28, 2010 at 9:42 AM, Erik Test <[hidden email]>
> wrote:
> >
> > > Hello,
> > >
> > > I'm getting the following messages when I try to run a job I've
> > developed.
> > >
> > >  hadoop jar distanceCalc.jar DistanceCalc distanceCalculations
> > > distanceCalculations/output9
> > >
> > > 10/07/28 09:25:37 WARN mapred.JobClient: Use GenericOptionsParser for
> > > parsing the arguments. Applications should implement Tool for the same.
> > >
> > > Exception in thread "main" java.lang.RuntimeException:
> > > java.lang.NoSuchMethodException:
> > > DistanceCalc$DistanceValuesInputFormat.<init>()
> > >        at
> > >
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> > >        at
> > org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:400)
> > >        at
> > > org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:810)
> > >        at
> > >
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:781)
> > >        at
> > org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
> > >        at
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1249)
> > >        at DistanceCalc.main(DistanceCalc.java:228)
> > >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >        at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >        at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > >        at java.lang.reflect.Method.invoke(Method.java:597)
> > >        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > > Caused by: java.lang.NoSuchMethodException:
> > > DistanceCalc$DistanceValuesInputFormat.<init>()
> > >        at java.lang.Class.getConstructor0(Class.java:2706)
> > >        at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> > >        ... 11 more
> > >
> > > I'm obviously doing something wrong in the code but my question for
> this
> > > thread is the warning message I'm receiving.  I'm not quite sure what
> it
> > > means and what I've seen on google so far is the warning has something
> to
> > > do
> > > with Pig but I'm not using Pig. Can someone tell me what the warning
> > means
> > > please?
> > >
> > > Thanks for your time
> > >
> > > Erik
> > >
> >
>