Cmd line for running plugins

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Cmd line for running plugins

Andrzej Białecki-2
Hi,

I just found out that it's not possible to invoke main() methods of
plugins through the bin/nutch script. Sometimes it's useful for testing
and debugging - I can do it from within Eclipse, because I have all
plugins on the classpath, but from the command-line it's not possible -
in the code they are accessed through PluginRepository. So I added this:

    public static void main(String[] args) throws Exception {
      NutchConf conf = new NutchConf();
      PluginRepository repo = new PluginRepository(conf);
      // args[0] - plugin ID
      PluginDescriptor d = repo.getPluginDescriptor(args[0]);
      if (d == null) {
        System.err.println("Plugin '" + args[0] + "' not present or
inactive.");
        return;
      }
      ClassLoader cl = d.getClassLoader();
      // args[1] - class name
      Class clazz = Class.forName(args[1], true, cl);
      Method m = clazz.getMethod("main", new Class[]{args.getClass()});
      String[] subargs = new String[args.length - 2];
      System.arraycopy(args, 2, subargs, 0, subargs.length);
      m.invoke(null, new Object[]{subargs});
    }

It works rather nicely. If other people find it useful, I can add this
to PluginRepository.

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Reply | Threaded
Open this post in threaded view
|

Re: Cmd line for running plugins

kangas
+1

That would help NUTCH-87 (whitelist urlfilter). The code is all in  
one plugin, but there is a utility class that needs to be called from  
the command line.

On Feb 1, 2006, at 4:35 PM, Andrzej Bialecki wrote:

> Hi,
>
> I just found out that it's not possible to invoke main() methods of  
> plugins through the bin/nutch script. Sometimes it's useful for  
> testing and debugging - I can do it from within Eclipse, because I  
> have all plugins on the classpath, but from the command-line it's  
> not possible - in the code they are accessed through  
> PluginRepository. So I added this:
>
>    public static void main(String[] args) throws Exception {
>      NutchConf conf = new NutchConf();
>      PluginRepository repo = new PluginRepository(conf);
>      // args[0] - plugin ID
>      PluginDescriptor d = repo.getPluginDescriptor(args[0]);
>      if (d == null) {
>        System.err.println("Plugin '" + args[0] + "' not present or  
> inactive.");
>        return;
>      }
>      ClassLoader cl = d.getClassLoader();
>      // args[1] - class name
>      Class clazz = Class.forName(args[1], true, cl);
>      Method m = clazz.getMethod("main", new Class[]{args.getClass()});
>      String[] subargs = new String[args.length - 2];
>      System.arraycopy(args, 2, subargs, 0, subargs.length);
>      m.invoke(null, new Object[]{subargs});
>    }
>
> It works rather nicely. If other people find it useful, I can add  
> this to PluginRepository.
>
> --
> Best regards,
> Andrzej Bialecki     <><
> ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>

--
Matt Kangas / [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Cmd line for running plugins

Andrzej Białecki-2
Matt Kangas wrote:
> +1
>
> That would help NUTCH-87 (whitelist urlfilter). The code is all in one
> plugin, but there is a utility class that needs to be called from the
> command line.

Yes, the code that I posted allows you to call any class loaded by the
plugin descriptor's ClassLoader, not just the ones declared as
extensions in plugin.xml.

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Reply | Threaded
Open this post in threaded view
|

Re: Cmd line for running plugins

Stefan Groschupf-2
In reply to this post by Andrzej Białecki-2
+1

Am 01.02.2006 um 22:35 schrieb Andrzej Bialecki:

> Hi,
>
> I just found out that it's not possible to invoke main() methods of  
> plugins through the bin/nutch script. Sometimes it's useful for  
> testing and debugging - I can do it from within Eclipse, because I  
> have all plugins on the classpath, but from the command-line it's  
> not possible - in the code they are accessed through  
> PluginRepository. So I added this:
>
>    public static void main(String[] args) throws Exception {
>      NutchConf conf = new NutchConf();
>      PluginRepository repo = new PluginRepository(conf);
>      // args[0] - plugin ID
>      PluginDescriptor d = repo.getPluginDescriptor(args[0]);
>      if (d == null) {
>        System.err.println("Plugin '" + args[0] + "' not present or  
> inactive.");
>        return;
>      }
>      ClassLoader cl = d.getClassLoader();
>      // args[1] - class name
>      Class clazz = Class.forName(args[1], true, cl);
>      Method m = clazz.getMethod("main", new Class[]{args.getClass()});
>      String[] subargs = new String[args.length - 2];
>      System.arraycopy(args, 2, subargs, 0, subargs.length);
>      m.invoke(null, new Object[]{subargs});
>    }
>
> It works rather nicely. If other people find it useful, I can add  
> this to PluginRepository.
>
> --
> Best regards,
> Andrzej Bialecki     <><
> ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>
>

---------------------------------------------------------------
company:        http://www.media-style.com
forum:        http://www.text-mining.org
blog:            http://www.find23.net


Reply | Threaded
Open this post in threaded view
|

Re: Cmd line for running plugins

Jérôme Charron
+1

On 2/1/06, Stefan Groschupf <[hidden email]> wrote:

>
> +1
>
> Am 01.02.2006 um 22:35 schrieb Andrzej Bialecki:
>
> > Hi,
> >
> > I just found out that it's not possible to invoke main() methods of
> > plugins through the bin/nutch script. Sometimes it's useful for
> > testing and debugging - I can do it from within Eclipse, because I
> > have all plugins on the classpath, but from the command-line it's
> > not possible - in the code they are accessed through
> > PluginRepository. So I added this:
> >
> >    public static void main(String[] args) throws Exception {
> >      NutchConf conf = new NutchConf();
> >      PluginRepository repo = new PluginRepository(conf);
> >      // args[0] - plugin ID
> >      PluginDescriptor d = repo.getPluginDescriptor(args[0]);
> >      if (d == null) {
> >        System.err.println("Plugin '" + args[0] + "' not present or
> > inactive.");
> >        return;
> >      }
> >      ClassLoader cl = d.getClassLoader();
> >      // args[1] - class name
> >      Class clazz = Class.forName(args[1], true, cl);
> >      Method m = clazz.getMethod("main", new Class[]{args.getClass()});
> >      String[] subargs = new String[args.length - 2];
> >      System.arraycopy(args, 2, subargs, 0, subargs.length);
> >      m.invoke(null, new Object[]{subargs});
> >    }
> >
> > It works rather nicely. If other people find it useful, I can add
> > this to PluginRepository.
> >
> > --
> > Best regards,
> > Andrzej Bialecki     <><
> > ___. ___ ___ ___ _ _   __________________________________
> > [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> > ___|||__||  \|  ||  |  Embedded Unix, System Integration
> > http://www.sigram.com  Contact: info at sigram dot com
> >
> >
> >
>
> ---------------------------------------------------------------
> company:        http://www.media-style.com
> forum:        http://www.text-mining.org
> blog:            http://www.find23.net
>
>
>
>


--
http://motrech.free.fr/
http://www.frutch.org/
Reply | Threaded
Open this post in threaded view
|

Re: Cmd line for running plugins

Andrzej Białecki-2
In reply to this post by Andrzej Białecki-2
Andrzej Bialecki wrote:
> It works rather nicely. If other people find it useful, I can add this
> to PluginRepository.
>

Committed.

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com