[jira] Created: (MAHOUT-450) RowSimilarityJob error

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
RowSimilarityJob  error
-----------------------

                 Key: MAHOUT-450
                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
             Project: Mahout
          Issue Type: Test
          Components: Collaborative Filtering
    Affects Versions: 0.4
            Reporter: Hui Wen Han
             Fix For: 0.4


when ran RowSimilarityJob job as following:
 hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100

got following error:

Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

It seemed that here error:

public static class SimilarityReducer
      extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {

    private DistributedVectorSimilarity similarity;
    private int numberOfColumns;

    @Override
    protected void setup(Context ctx) throws IOException, InterruptedException {
      super.setup(ctx);
      similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
      numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
      if (numberOfColumns < 1) {
        throw new IllegalStateException("Number of columns was not correctly set!");
      }
    }

I guess that maybe be wrong here:
in SimilarityType:

SIMILARITY_COOCCURRENCE(DistributedCooccurrenceVectorSimilarity.class), //just the class name is needed,has no the ".class"
  SIMILARITY_EUCLIDEAN_DISTANCE(DistributedEuclideanDistanceVectorSimilarity.class),
  SIMILARITY_LOGLIKELIHOOD(DistributedLoglikelihoodVectorSimilarity.class),
  SIMILARITY_PEARSON_CORRELATION(DistributedPearsonCorrelationVectorSimilarity.class),
  SIMILARITY_TANIMOTO_COEFFICIENT(DistributedTanimotoCoefficientVectorSimilarity.class),
  SIMILARITY_UNCENTERED_COSINE(DistributedUncenteredCosineVectorSimilarity.class),
  SIMILARITY_UNCENTERED_ZERO_ASSUMING_COSINE(DistributedUncenteredZeroAssumingCosineVectorSimilarity.class);

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hui Wen Han updated MAHOUT-450:
-------------------------------

    Description:
when ran RowSimilarityJob job as following:
 hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100

got following error:

Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

It seemed that here error:

public static class SimilarityReducer
      extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {

    private DistributedVectorSimilarity similarity;
    private int numberOfColumns;

    @Override
    protected void setup(Context ctx) throws IOException, InterruptedException {
      super.setup(ctx);
      similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
      numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
      if (numberOfColumns < 1) {
        throw new IllegalStateException("Number of columns was not correctly set!");
      }
    }



  was:
when ran RowSimilarityJob job as following:
 hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100

got following error:

Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

It seemed that here error:

public static class SimilarityReducer
      extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {

    private DistributedVectorSimilarity similarity;
    private int numberOfColumns;

    @Override
    protected void setup(Context ctx) throws IOException, InterruptedException {
      super.setup(ctx);
      similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
      numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
      if (numberOfColumns < 1) {
        throw new IllegalStateException("Number of columns was not correctly set!");
      }
    }

I guess that maybe be wrong here:
in SimilarityType:

SIMILARITY_COOCCURRENCE(DistributedCooccurrenceVectorSimilarity.class), //just the class name is needed,has no the ".class"
  SIMILARITY_EUCLIDEAN_DISTANCE(DistributedEuclideanDistanceVectorSimilarity.class),
  SIMILARITY_LOGLIKELIHOOD(DistributedLoglikelihoodVectorSimilarity.class),
  SIMILARITY_PEARSON_CORRELATION(DistributedPearsonCorrelationVectorSimilarity.class),
  SIMILARITY_TANIMOTO_COEFFICIENT(DistributedTanimotoCoefficientVectorSimilarity.class),
  SIMILARITY_UNCENTERED_COSINE(DistributedUncenteredCosineVectorSimilarity.class),
  SIMILARITY_UNCENTERED_ZERO_ASSUMING_COSINE(DistributedUncenteredZeroAssumingCosineVectorSimilarity.class);


> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>             Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892855#action_12892855 ]

Sebastian Schelter commented on MAHOUT-450:
-------------------------------------------

from a first look I'd say that org.apache.commons.cli2.OptionException from commons cli cannot be loaded, are you sure that your jar-file is correctly built?

> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>             Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893116#action_12893116 ]

Hui Wen Han commented on MAHOUT-450:
------------------------------------

  public static class SimilarityReducer
      extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {

    private DistributedVectorSimilarity similarity;
    private int numberOfColumns;

    @Override
    protected void setup(Context ctx) throws IOException, InterruptedException {
      super.setup(ctx);
      String className=ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME); //error here ,but we cah get DISTRIBUTED_SIMILARITY_CLASSNAME in RowWeightMapper
      similarity = instantiateSimilarity(className);    // I separate this line to two lines
      numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
      if (numberOfColumns < 1) {
        throw new IllegalStateException("Number of columns was not correctly set!");
      }
    }

> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>             Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893120#action_12893120 ]

Sebastian Schelter commented on MAHOUT-450:
-------------------------------------------

I see that you use a customly built jar file, can you check whether it contains the class that cannot be loaded (org.apache.commons.cli2.OptionException)?

You can unzip jars as just like standard zip files. If you do this with core/target/mahout-core-0.4-SNAPSHOT.job you will see that the class we're talking about is contained in lib/commons-cli-2.0-mahout.jar under the path org/apache/commons/cli2/OptionException.class. Please check if you can find it in your jar too.

You could also try to run the RowSimilarityJob with the .job file and see if that works.


> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>             Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Issue Comment Edited: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893120#action_12893120 ]

Sebastian Schelter edited comment on MAHOUT-450 at 7/28/10 6:22 AM:
--------------------------------------------------------------------

I see that you use a customly built jar file, can you check whether it contains the class that cannot be loaded (org.apache.commons.cli2.OptionException)?

You can unzip jars just like standard zip files. If you do this with core/target/mahout-core-0.4-SNAPSHOT.job you will see that the class we're talking about is contained in lib/commons-cli-2.0-mahout.jar under the path org/apache/commons/cli2/OptionException.class. Please check if you can find it in your jar too.

You could also try to run the RowSimilarityJob with the .job file and see if that works.


      was (Author: ssc):
    I see that you use a customly built jar file, can you check whether it contains the class that cannot be loaded (org.apache.commons.cli2.OptionException)?

You can unzip jars as just like standard zip files. If you do this with core/target/mahout-core-0.4-SNAPSHOT.job you will see that the class we're talking about is contained in lib/commons-cli-2.0-mahout.jar under the path org/apache/commons/cli2/OptionException.class. Please check if you can find it in your jar too.

You could also try to run the RowSimilarityJob with the .job file and see if that works.

 

> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>             Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893197#action_12893197 ]

Hui Wen Han commented on MAHOUT-450:
------------------------------------

I ran using mahout-core-0.4-SNAPSHOT.job,it worked correctly.


> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>             Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (MAHOUT-450) RowSimilarityJob error

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved MAHOUT-450.
------------------------------

    Fix Version/s:     (was: 0.4)
       Resolution: Not A Problem

Yes, you were missing dependencies, which are in the .job file. THis is what must be used.

> RowSimilarityJob  error
> -----------------------
>
>                 Key: MAHOUT-450
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-450
>             Project: Mahout
>          Issue Type: Test
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>
> when ran RowSimilarityJob job as following:
>  hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
>       extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
>     private DistributedVectorSimilarity similarity;
>     private int numberOfColumns;
>     @Override
>     protected void setup(Context ctx) throws IOException, InterruptedException {
>       super.setup(ctx);
>       similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));  //this line,seems can not load the class.
>       numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
>       if (numberOfColumns < 1) {
>         throw new IllegalStateException("Number of columns was not correctly set!");
>       }
>     }

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.