Ant, Javadoc, and JUnit

[Chris Hostetter-3]

Okay, so i really thought I understood the *right* way to use JUnit with
Ant back when I added JUnit tests to the Solr code base -- all of the
information i found online suggested that the correct way to do things was
to *not* include junit.jar in your project's lib, instead encourage
developers to put it in their ANT_HOME/lib directory.

(if i recall correclty, putting it in the project's lib directory doesn't
allow <junit> tasks from working at all, because Ant's classloader doesn't
have it when it tries to run the <junit> task)

This worked fine at first, but now that I'm trying to cleanup javadocs,
we've got the anoying error included below.

As far as i can tell, the problem is that unlike the <junit> task, that
<javadoc> task is acctually executing that javadoc command on the shell --
so it's not running in the same JVM as ant, and doesn't have access to the
junit.jar from ant's classloader.

I suspect the "right" way to fix this, is to figure out some way to pass a
refrence to the classes already loaded by ant's classloader to javadocs
classpath option.

Does anyone have any ideas on how to make that work, or better suggestions
for dealing with the problem?  (perhaps someone who has written a book on
using ant?  *cough* erik *cough*)




laptop:~/lucene/solr/trunk> ant javadoc
Buildfile: build.xml
        ...
  [javadoc]
/home/hossman/lucene/solr/trunk/src/java/org/apache/solr/util/AbstractSolrTestCase.java:8:
package junit.framework does not exist
  [javadoc] import junit.framework.TestCase;
  [javadoc]                        ^
  [javadoc]
/home/hossman/lucene/solr/trunk/src/java/org/apache/solr/util/AbstractSolrTestCase.java:27:
cannot find symbol
  [javadoc] symbol: class TestCase
  [javadoc] public abstract class AbstractSolrTestCase extends TestCase {
  [javadoc]                                                    ^




-Hoss

[Paul Elschot]

On Saturday 27 May 2006 23:30, Chris Hostetter wrote:

>
>
>
> laptop:~/lucene/solr/trunk> ant javadoc
> Buildfile: build.xml
> ...
>   [javadoc]
> /home/hossman/lucene/solr/trunk/src/java/org/apache/solr/util/AbstractSolrTestCase.java:8:
> package junit.framework does not exist
>   [javadoc] import junit.framework.TestCase;
>   [javadoc]                        ^
>   [javadoc]
> /home/hossman/lucene/solr/trunk/src/java/org/apache/solr/util/AbstractSolrTestCase.java:27:
> cannot find symbol
>   [javadoc] symbol: class TestCase
>   [javadoc] public abstract class AbstractSolrTestCase extends TestCase {
>   [javadoc]                                                    ^

<repeat times="2" ><cough volume="medium"></cough></repeat>

How about adding junit.jar to the javadoc classpath?

http://ant.apache.org/manual/CoreTasks/javadoc.html

Regards,
Paul Elschot

[Chris Hostetter-3]

: How about adding junit.jar to the javadoc classpath?
:
: http://ant.apache.org/manual/CoreTasks/javadoc.html

okay ... you mean using either the "classpath" or "classpathref" of the
<javadoc> task right?

But how do we (in a way that will work for all users) know where the
junit.jar is to add it to the classpath?

The expectation is that developers have ant installed somewhere, and
wherever they have it installed they have the junit.jar in it's lib
directory -- how do we modify the build.xml, so that it knows what that
location is so it can specify it when execing the <javadoc> target?
(regardless of how people may have ant/junit installed on their machines)

        ?


-Hoss

[Yonik Seeley]

Is there a way to just exclude JUnit classes from javadoc?
It's not like we need the base classes documented, right?

-Yonik

On 5/28/06, Chris Hostetter <[hidden email]> wrote:

>
> : How about adding junit.jar to the javadoc classpath?
> :
> : http://ant.apache.org/manual/CoreTasks/javadoc.html
>
> okay ... you mean using either the "classpath" or "classpathref" of the
> <javadoc> task right?
>
> But how do we (in a way that will work for all users) know where the
> junit.jar is to add it to the classpath?
>
> The expectation is that developers have ant installed somewhere, and
> wherever they have it installed they have the junit.jar in it's lib
> directory -- how do we modify the build.xml, so that it knows what that
> location is so it can specify it when execing the <javadoc> target?
> (regardless of how people may have ant/junit installed on their machines)
>
>         ?
>
>
> -Hoss

[Chris Hostetter-3]

: Is there a way to just exclude JUnit classes from javadoc?
: It's not like we need the base classes documented, right?

Well, i supose we could ... but the whole reason I put it in src/java
instead of src/test was to make it easy for users to build JUnit tests for
their custom plugins -- having javadocs increases the visibility of the
class.



-Hoss

[Yonik Seeley]

On 5/28/06, Chris Hostetter <[hidden email]> wrote:
>
> : Is there a way to just exclude JUnit classes from javadoc?
> : It's not like we need the base classes documented, right?
>
> Well, i supose we could ... but the whole reason I put it in src/java
> instead of src/test was to make it easy for users to build JUnit tests for
> their custom plugins -- having javadocs increases the visibility of the
> class.

I meant keep the javadoc for your base classes derived from TestCase,
but skip it for TestCase itself.


-Yonik

[Chris Hostetter-3]

: I meant keep the javadoc for your base classes derived from TestCase,
: but skip it for TestCase itself.

OH, i'm not trying to generate javadocs for TestCase -- the error is
because when javadoc is processing AbstractSolrTestCase it doesn't know
how to resolve the inheritence from TestCase so it doesn't know what
methods/variables to inherit and display documentation for.

the basic class documentation works fine ... anybody familiar with
writing JUNit tests is already going to know what methods are available
to them in a subclass of TestCase -- i'm mainly just anoyed by the warning
message.

-Hoss

[Paul Elschot]

On Sunday 28 May 2006 20:11, Chris Hostetter wrote:

>
> : How about adding junit.jar to the javadoc classpath?
> :
> : http://ant.apache.org/manual/CoreTasks/javadoc.html
>
> okay ... you mean using either the "classpath" or "classpathref" of the
> <javadoc> task right?
>
> But how do we (in a way that will work for all users) know where the
> junit.jar is to add it to the classpath?
>
> The expectation is that developers have ant installed somewhere, and
> wherever they have it installed they have the junit.jar in it's lib
> directory -- how do we modify the build.xml, so that it knows what that
> location is so it can specify it when execing the <javadoc> target?
> (regardless of how people may have ant/junit installed on their machines)
>
> ?

Adding this path element might work:
${ant.library.dir}/junit.jar

(I could not make that work here. For some reason on my machine
${ant.library.dir} points to /usr/share/ant/lib even though the environment
variable ANT_HOME is set to ~/ant/apache-ant-1.6.5, and $ANT_HOME/bin
is on the shell PATH.)

Another way might be to set the build.sysclasspath property to "last", but
that has the disadvantage of adding more things to the classpath.

When ${ant.library.dir}/junit.jar does not work a specialist might
be needed...

Regards,
Paul Elschot

[Chris Hostetter-3]

: Adding this path element might work:
: ${ant.library.dir}/junit.jar

Ah HA! ... see, i didn't even know that property existed, it's not in the
list of built-in properties...
  http://ant.apache.org/manual/using.html#built-in-props
...probably becuase it's acctually just a system property set by the ant
wrapper script.

we wouldn't want to use the explicit path with "junit.jar" in it because
people might name the jar soemthing else (with a version number perhaps)
but we can certainly use that property in a fileset to get all the jars.

: (I could not make that work here. For some reason on my machine
: ${ant.library.dir} points to /usr/share/ant/lib even though the environment
: variable ANT_HOME is set to ~/ant/apache-ant-1.6.5, and $ANT_HOME/bin
: is on the shell PATH.)

that's strange ... now that i'm poking arround in the ant wrapper script,
i can't see any obvious way that ant.library.dir wouldn't point to
ANT_HOME/lib ... acctually, the property ${ant.home} gets set by the ant
wrapper script as well, try echoing it and ${ant.version} from a build.xml
file and make sure your using the instance of ant you think you are.

This is what i've currently got in my local solr build.xml, and it seems
to be working ok...

    <path id="javadoc.classpath">
       <path refid="compile.classpath"/>
       <!-- aparently ant.library.dir isn't allways set right? -->
       <fileset dir="${ant.home}/lib">
          <include name="**/*.jar"/>
       </fileset>
       <fileset dir="${ant.library.dir}">
          <include name="**/*.jar"/>
       </fileset>
    </path>

...anyone see any problems with that?

: Another way might be to set the build.sysclasspath property to "last", but
: that has the disadvantage of adding more things to the classpath.

Interesting, i've never seen that property used before...
   http://ant.apache.org/manual/sysclasspath.html

...if i'm understanding that correctly, using "last" adds everything to
the classpath that i added using ${ant.home}/lib and ${ant.library.dir},
but it would also add any other classes that might have been in the users
$ENV{CLASSPATH} when running ant correct?

Using ${ant.home}/lib and ${ant.library.dir} already feels a bit dirty to
me, adding build.sysclasspath=last seems like overkill.

-Hoss

[Darren Vengroff-2]

I hope it's not too off topic, but since you are hacking away at the build,
I'll ask.  Have you thought of moving from Ant to Maven2?  I'm new to solr,
so if it's been hashed over and rejected for valid reasons, excuse my
intrusion.

FWIW, I just put together a quick Maven2 pom.xml file for Solr and it seems
to work well.  It's probably not be practical for general use until Lucene
itself uses Maven2, but I'd be happy to contribute it if and when that
happens.

On the plus side, the file is compact and easy to understand. There is none
of the low-level config you are struggling with in ant.  The only remotely
custom things I added are overrides for the location of the source and test
directories and a pointer to the local repository where I put some of the
dependencies (Lucene and xpp).

I also got around the problem that started this thread by putting a
dependency on junit at "provided" scope instead of more typical "test"
scope.  It seems to have worked fine.
org.apache.solr.util.AbstractSolrTestCase ends up in the javadoc output as
desired.  Personally, I'd probably move it in with the test sources, but I
see your point in keeping it more visible.

-D

-----Original Message-----
From: Chris Hostetter [mailto:[hidden email]]
Sent: Sunday, May 28, 2006 1:19 PM
To: [hidden email]
Subject: Re: Ant, Javadoc, and JUnit


: Adding this path element might work:
: ${ant.library.dir}/junit.jar

Ah HA! ... see, i didn't even know that property existed, it's not in the
list of built-in properties...
  http://ant.apache.org/manual/using.html#built-in-props
...probably becuase it's acctually just a system property set by the ant
wrapper script.

we wouldn't want to use the explicit path with "junit.jar" in it because
people might name the jar soemthing else (with a version number perhaps)
but we can certainly use that property in a fileset to get all the jars.

: (I could not make that work here. For some reason on my machine
: ${ant.library.dir} points to /usr/share/ant/lib even though the
environment
: variable ANT_HOME is set to ~/ant/apache-ant-1.6.5, and $ANT_HOME/bin
: is on the shell PATH.)

that's strange ... now that i'm poking arround in the ant wrapper script,
i can't see any obvious way that ant.library.dir wouldn't point to
ANT_HOME/lib ... acctually, the property ${ant.home} gets set by the ant
wrapper script as well, try echoing it and ${ant.version} from a build.xml
file and make sure your using the instance of ant you think you are.

This is what i've currently got in my local solr build.xml, and it seems
to be working ok...

    <path id="javadoc.classpath">
       <path refid="compile.classpath"/>
       <!-- aparently ant.library.dir isn't allways set right? -->
       <fileset dir="${ant.home}/lib">
          <include name="**/*.jar"/>
       </fileset>
       <fileset dir="${ant.library.dir}">
          <include name="**/*.jar"/>
       </fileset>
    </path>

...anyone see any problems with that?

: Another way might be to set the build.sysclasspath property to "last", but
: that has the disadvantage of adding more things to the classpath.

Interesting, i've never seen that property used before...
   http://ant.apache.org/manual/sysclasspath.html

...if i'm understanding that correctly, using "last" adds everything to
the classpath that i added using ${ant.home}/lib and ${ant.library.dir},
but it would also add any other classes that might have been in the users
$ENV{CLASSPATH} when running ant correct?

Using ${ant.home}/lib and ${ant.library.dir} already feels a bit dirty to
me, adding build.sysclasspath=last seems like overkill.

-Hoss

[Chris Hostetter-3]

: I hope it's not too off topic, but since you are hacking away at the build,
: I'll ask.  Have you thought of moving from Ant to Maven2?  I'm new to solr,
: so if it's been hashed over and rejected for valid reasons, excuse my
: intrusion.

have i personally thought about it? no.
has anyone else thought about it? i don't know.
has it ever come up in discussion on the mailing lists? no.

I don't know about any one else, but I don't have any particular affinity
for ant other then: it's what i know, it works fairly well, and it's
fairly common so using it in the project doesn't create any immediate
hurdle for new developers wanting to build solr.

If maven(2) is better then ant in these respects, then i'd be happy to
switch, but it doesn't quite seem like there's any serious motivation to
do so -- particularly on that last point, maven isn't quite as common as
ant is it? (let alone maven2).  I get the "early adopter" chicken and egg
problem for tools, but at ths point Solr is still in the "trying to get
noticed" phase and supporting the least common denomiator is probably a
good idea.

: FWIW, I just put together a quick Maven2 pom.xml file for Solr and it seems
: to work well.  It's probably not be practical for general use until Lucene
: itself uses Maven2, but I'd be happy to contribute it if and when that
: happens.

Definitely! ... if you'd like to contribute it now, it could probably live
side by side with the build.xml so people comfortable with maven could use
it instead if if they prefer ... right? (or is my vast lack of
understanding about maven showing through here?)



-Hoss

[Darren Vengroff-2]

I think your side-by-side proposal is a good one.

Let me polish the pom a little and then I'll submit it.  This will happen
the first time I get a free moment in the next few days.

-D

-----Original Message-----
From: Chris Hostetter [mailto:[hidden email]]
Sent: Monday, May 29, 2006 1:35 PM
To: [hidden email]
Subject: RE: Ant, Javadoc, and JUnit


: I hope it's not too off topic, but since you are hacking away at the
build,
: I'll ask.  Have you thought of moving from Ant to Maven2?  I'm new to
solr,
: so if it's been hashed over and rejected for valid reasons, excuse my
: intrusion.

have i personally thought about it? no.
has anyone else thought about it? i don't know.
has it ever come up in discussion on the mailing lists? no.

I don't know about any one else, but I don't have any particular affinity
for ant other then: it's what i know, it works fairly well, and it's
fairly common so using it in the project doesn't create any immediate
hurdle for new developers wanting to build solr.

If maven(2) is better then ant in these respects, then i'd be happy to
switch, but it doesn't quite seem like there's any serious motivation to
do so -- particularly on that last point, maven isn't quite as common as
ant is it? (let alone maven2).  I get the "early adopter" chicken and egg
problem for tools, but at ths point Solr is still in the "trying to get
noticed" phase and supporting the least common denomiator is probably a
good idea.

: FWIW, I just put together a quick Maven2 pom.xml file for Solr and it
seems
: to work well.  It's probably not be practical for general use until Lucene
: itself uses Maven2, but I'd be happy to contribute it if and when that
: happens.

Definitely! ... if you'd like to contribute it now, it could probably live
side by side with the build.xml so people comfortable with maven could use
it instead if if they prefer ... right? (or is my vast lack of
understanding about maven showing through here?)



-Hoss

[Yoav Shapira-2]

Hola,
Like a couple of other people, I wasn't a big fan of Maven 1, so
always voted against it as the canonical build system, especially
since Ant is still much more widely supported.  However, I think Maven
2 brings great improvements, so I don't mind us at least trying it out
to see how it works.

FWIW, Maven 1 had a way (a plugin in maven terminology) to generate a
pure Ant build.xml file from a Maven POM.  I'm sure Maven 2 still has
this capability, which would obviously reduce our maintenance burden.

Yoav

On 5/30/06, Darren Vengroff <[hidden email]> wrote:

> I think your side-by-side proposal is a good one.
>
> Let me polish the pom a little and then I'll submit it.  This will happen
> the first time I get a free moment in the next few days.
>
> -D
>
> -----Original Message-----
> From: Chris Hostetter [mailto:[hidden email]]
> Sent: Monday, May 29, 2006 1:35 PM
> To: [hidden email]
> Subject: RE: Ant, Javadoc, and JUnit
>
>
> : I hope it's not too off topic, but since you are hacking away at the
> build,
> : I'll ask.  Have you thought of moving from Ant to Maven2?  I'm new to
> solr,
> : so if it's been hashed over and rejected for valid reasons, excuse my
> : intrusion.
>
> have i personally thought about it? no.
> has anyone else thought about it? i don't know.
> has it ever come up in discussion on the mailing lists? no.
>
> I don't know about any one else, but I don't have any particular affinity
> for ant other then: it's what i know, it works fairly well, and it's
> fairly common so using it in the project doesn't create any immediate
> hurdle for new developers wanting to build solr.
>
> If maven(2) is better then ant in these respects, then i'd be happy to
> switch, but it doesn't quite seem like there's any serious motivation to
> do so -- particularly on that last point, maven isn't quite as common as
> ant is it? (let alone maven2).  I get the "early adopter" chicken and egg
> problem for tools, but at ths point Solr is still in the "trying to get
> noticed" phase and supporting the least common denomiator is probably a
> good idea.
>
> : FWIW, I just put together a quick Maven2 pom.xml file for Solr and it
> seems
> : to work well.  It's probably not be practical for general use until Lucene
> : itself uses Maven2, but I'd be happy to contribute it if and when that
> : happens.
>
> Definitely! ... if you'd like to contribute it now, it could probably live
> side by side with the build.xml so people comfortable with maven could use
> it instead if if they prefer ... right? (or is my vast lack of
> understanding about maven showing through here?)
>
>
>
> -Hoss
>
>

--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
[hidden email] / www.yoavshapira.com