C/C++ interface for HDFS

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

C/C++ interface for HDFS

alakshman
Hi

I am assuming that if I need a C/C++ interface to HDFS I must build libhdfs.
This may be a problem very specific to my environment but would appreciate
if someone could tell me what is going on ? So in order to build libhdfs I
am running the command ant compile-libhdfs. But I get the following output :

compile-libhdfs:
     [exec] gcc -g -Wall -O2 -fPIC -m64 -m32
-I/home/alakshman/jdk1.6.0_01/include
-I/home/alakshman/jdk1.6.0_01/include/linux -c hdfs.c -o
/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o
     [exec] In file included from /usr/include/features.h:337,
     [exec]                  from /usr/include/sys/types.h:27,
     [exec]                  from hdfs.h:22,
     [exec]                  from hdfs.c:19:
     [exec] /usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No such
file or directory
     [exec] make: ***
[/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o]
Error 1
     [exec] Result: 2

BUILD SUCCESSFUL
Total time: 1 second

What is going on ? I have set my JAVA_HOME correctly and this is the version
of my Linux box  2.6.12-1.1398_FC4smp.

Thanks
A
Reply | Threaded
Open this post in threaded view
|

Re: C/C++ interface for HDFS

Neeraj Mahajan-3
If you look at the gcc options, then you can see that both 32-bit as well as
64-bit  environments  are specified. Probably only one of them should be
there. To fix this you might have to edit the Makefile in c++/src/libhdfs
and replace/remove the -m32 option.

~ Neeraj

On 6/11/07, Phantom <[hidden email]> wrote:

>
> Hi
>
> I am assuming that if I need a C/C++ interface to HDFS I must build
> libhdfs.
> This may be a problem very specific to my environment but would appreciate
> if someone could tell me what is going on ? So in order to build libhdfs I
> am running the command ant compile-libhdfs. But I get the following output
> :
>
> compile-libhdfs:
>      [exec] gcc -g -Wall -O2 -fPIC -m64 -m32
> -I/home/alakshman/jdk1.6.0_01/include
> -I/home/alakshman/jdk1.6.0_01/include/linux -c hdfs.c -o
> /home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o
>      [exec] In file included from /usr/include/features.h:337,
>      [exec]                  from /usr/include/sys/types.h:27,
>      [exec]                  from hdfs.h:22,
>      [exec]                  from hdfs.c:19:
>      [exec] /usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No such
> file or directory
>      [exec] make: ***
> [/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o]
> Error 1
>      [exec] Result: 2
>
> BUILD SUCCESSFUL
> Total time: 1 second
>
> What is going on ? I have set my JAVA_HOME correctly and this is the
> version
> of my Linux box  2.6.12-1.1398_FC4smp.
>
> Thanks
> A
>
Reply | Threaded
Open this post in threaded view
|

Re: C/C++ interface for HDFS

Arun C Murthy-2
On Mon, Jun 11, 2007 at 04:20:48PM -0700, Neeraj Mahajan wrote:

>If you look at the gcc options, then you can see that both 32-bit as well as
>64-bit  environments  are specified. Probably only one of them should be
>there. To fix this you might have to edit the Makefile in c++/src/libhdfs
>and replace/remove the -m32 option.
>
>~ Neeraj
>
>On 6/11/07, Phantom <[hidden email]> wrote:
>>
>>Hi
>>
>>I am assuming that if I need a C/C++ interface to HDFS I must build
>>libhdfs.
Yep.

>>This may be a problem very specific to my environment but would appreciate
>>if someone could tell me what is going on ? So in order to build libhdfs I
>>am running the command ant compile-libhdfs. But I get the following output
>>:
>>
>>compile-libhdfs:
>>     [exec] gcc -g -Wall -O2 -fPIC -m64 -m32

Neeraj is right, there is a bug in the Makefile of libhdfs i.e. doesn't handle 64-bit platforms correctly...

Could you try the attached patch and check if it works for you? Once you give the go-ahead I'll go ahead and get it committed...

thanks,
Arun

>>-I/home/alakshman/jdk1.6.0_01/include
>>-I/home/alakshman/jdk1.6.0_01/include/linux -c hdfs.c -o
>>/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o
>>     [exec] In file included from /usr/include/features.h:337,
>>     [exec]                  from /usr/include/sys/types.h:27,
>>     [exec]                  from hdfs.h:22,
>>     [exec]                  from hdfs.c:19:
>>     [exec] /usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No such
>>file or directory
>>     [exec] make: ***
>>[/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o]
>>Error 1
>>     [exec] Result: 2
>>
>>BUILD SUCCESSFUL
>>Total time: 1 second
>>
>>What is going on ? I have set my JAVA_HOME correctly and this is the
>>version
>>of my Linux box  2.6.12-1.1398_FC4smp.
>>
>>Thanks
>>A
>>

libhdfs.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: C/C++ interface for HDFS

Avinash Lakshman-2
The patch you sent me didn't seem to work complaining that some env
variable was empty. This is the Makefile I had used to fix my problem.

Thanks
Avinash


-----Original Message-----
From: Arun C Murthy [mailto:[hidden email]]
Sent: Monday, June 11, 2007 8:48 PM
To: [hidden email]
Subject: Re: C/C++ interface for HDFS

On Mon, Jun 11, 2007 at 04:20:48PM -0700, Neeraj Mahajan wrote:
>If you look at the gcc options, then you can see that both 32-bit as
well as
>64-bit  environments  are specified. Probably only one of them should
be
>there. To fix this you might have to edit the Makefile in
c++/src/libhdfs

>and replace/remove the -m32 option.
>
>~ Neeraj
>
>On 6/11/07, Phantom <[hidden email]> wrote:
>>
>>Hi
>>
>>I am assuming that if I need a C/C++ interface to HDFS I must build
>>libhdfs.
Yep.

>>This may be a problem very specific to my environment but would
appreciate
>>if someone could tell me what is going on ? So in order to build
libhdfs I
>>am running the command ant compile-libhdfs. But I get the following
output
>>:
>>
>>compile-libhdfs:
>>     [exec] gcc -g -Wall -O2 -fPIC -m64 -m32

Neeraj is right, there is a bug in the Makefile of libhdfs i.e. doesn't
handle 64-bit platforms correctly...

Could you try the attached patch and check if it works for you? Once you
give the go-ahead I'll go ahead and get it committed...

thanks,
Arun

>>-I/home/alakshman/jdk1.6.0_01/include
>>-I/home/alakshman/jdk1.6.0_01/include/linux -c hdfs.c -o
>>/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o
>>     [exec] In file included from /usr/include/features.h:337,
>>     [exec]                  from /usr/include/sys/types.h:27,
>>     [exec]                  from hdfs.h:22,
>>     [exec]                  from hdfs.c:19:
>>     [exec] /usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No
such

>>file or directory
>>     [exec] make: ***
>>[/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o]
>>Error 1
>>     [exec] Result: 2
>>
>>BUILD SUCCESSFUL
>>Total time: 1 second
>>
>>What is going on ? I have set my JAVA_HOME correctly and this is the
>>version
>>of my Linux box  2.6.12-1.1398_FC4smp.
>>
>>Thanks
>>A
>>
Reply | Threaded
Open this post in threaded view
|

Re: C/C++ interface for HDFS

Arun C Murthy-2
Avinash Lakshman wrote:
> The patch you sent me didn't seem to work complaining that some env
> variable was empty. This is the Makefile I had used to fix my problem.
>

It's useful if you can share some information about the actual issue on
hand... perhaps you forgot to attach the Makefile?

Arun

> Thanks
> Avinash
>
>
> -----Original Message-----
> From: Arun C Murthy [mailto:[hidden email]]
> Sent: Monday, June 11, 2007 8:48 PM
> To: [hidden email]
> Subject: Re: C/C++ interface for HDFS
>
> On Mon, Jun 11, 2007 at 04:20:48PM -0700, Neeraj Mahajan wrote:
>
>>If you look at the gcc options, then you can see that both 32-bit as
>
> well as
>
>>64-bit  environments  are specified. Probably only one of them should
>
> be
>
>>there. To fix this you might have to edit the Makefile in
>
> c++/src/libhdfs
>
>>and replace/remove the -m32 option.
>>
>>~ Neeraj
>>
>>On 6/11/07, Phantom <[hidden email]> wrote:
>>
>>>Hi
>>>
>>>I am assuming that if I need a C/C++ interface to HDFS I must build
>>>libhdfs.
>
>
> Yep.
>
>
>>>This may be a problem very specific to my environment but would
>
> appreciate
>
>>>if someone could tell me what is going on ? So in order to build
>
> libhdfs I
>
>>>am running the command ant compile-libhdfs. But I get the following
>
> output
>
>>>:
>>>
>>>compile-libhdfs:
>>>    [exec] gcc -g -Wall -O2 -fPIC -m64 -m32
>
>
> Neeraj is right, there is a bug in the Makefile of libhdfs i.e. doesn't
> handle 64-bit platforms correctly...
>
> Could you try the attached patch and check if it works for you? Once you
> give the go-ahead I'll go ahead and get it committed...
>
> thanks,
> Arun
>
>
>>>-I/home/alakshman/jdk1.6.0_01/include
>>>-I/home/alakshman/jdk1.6.0_01/include/linux -c hdfs.c -o
>>>/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o
>>>    [exec] In file included from /usr/include/features.h:337,
>>>    [exec]                  from /usr/include/sys/types.h:27,
>>>    [exec]                  from hdfs.h:22,
>>>    [exec]                  from hdfs.c:19:
>>>    [exec] /usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No
>
> such
>
>>>file or directory
>>>    [exec] make: ***
>>>[/home/alakshman/FB-Projects/hadoop-0.13.0/build/libhdfs/hdfs.o]
>>>Error 1
>>>    [exec] Result: 2
>>>
>>>BUILD SUCCESSFUL
>>>Total time: 1 second
>>>
>>>What is going on ? I have set my JAVA_HOME correctly and this is the
>>>version
>>>of my Linux box  2.6.12-1.1398_FC4smp.
>>>
>>>Thanks
>>>A
>
>>
>