Quantcast

Hbase with Hadoop

classic Classic list List threaded Threaded
36 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Hbase with Hadoop

Jignesh Patel
Can I integrate HBase 0.90.4 with hadoop 0.20.204.0 ?

-Jignesh
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Matt Foley-2
Hi Jignesh,
0.20.204.0 does not have hflush/sync support, but 0.20.205.0 does.
Without HDFS hsync, HBase will still work, but is subject to data loss if
the datanode is restarted.  In 205, this deficiency is fixed.

0.20.205.0-rc2 is up for vote in common-dev@.  Please try it out with HBase
:-)
We've been using it with HBase 0.90.3 and it works, with some config
adjustments.

--Matt, RM for 0.20.205.0


On Tue, Oct 11, 2011 at 2:11 PM, Jignesh Patel <[hidden email]> wrote:

> Can I integrate HBase 0.90.4 with hadoop 0.20.204.0 ?
>
> -Jignesh
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

jigneshmpatel
Matt,
Thanks a lot. Just wanted to have some more information. If hadoop 0.2.205.0 voted by the community members then will it become major release? And what if it is not approved by community members.

And as you said I do like to use 0.90.3 if it works. If it is ok, can you share the deails of those configuration changes?

-Jignesh
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Konstantin Boudnik-2
Matt,

I'd like to re-enforce the inquiry about posting (or blogging perhaps ;) details
about Hbase/0.20.205 coexistence. I am sure lotta people will benefit from
this.

Thanks in advance,
  Cos

On Tue, Oct 11, 2011 at 08:29PM, jigneshmpatel wrote:

> Matt,
> Thanks a lot. Just wanted to have some more information. If hadoop 0.2.205.0
> voted by the community members then will it become major release? And what
> if it is not approved by community members.
>
> And as you said I do like to use 0.90.3 if it works. If it is ok, can you
> share the deails of those configuration changes?
>
> -Jignesh
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

jigneshmpatel
This post has NOT been accepted by the mailing list yet.
Konstantin,
You sent this email to me  not to Matt.

-Jignesh

On Wed, Oct 12, 2011 at 12:55 AM, Konstantin Boudnik-2 [via Lucene] <[hidden email]> wrote:
Matt,

I'd like to re-enforce the inquiry about posting (or blogging perhaps ;) details
about Hbase/0.20.205 coexistence. I am sure lotta people will benefit from
this.

Thanks in advance,
  Cos

On Tue, Oct 11, 2011 at 08:29PM, jigneshmpatel wrote:

> Matt,
> Thanks a lot. Just wanted to have some more information. If hadoop 0.2.205.0
> voted by the community members then will it become major release? And what
> if it is not approved by community members.
>
> And as you said I do like to use 0.90.3 if it works. If it is ok, can you
> share the deails of those configuration changes?
>
> -Jignesh
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.



If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414823.html
To unsubscribe from Hbase with Hadoop, click here.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

T Vinod Gupta
In reply to this post by jigneshmpatel
for what its worth, i was in a similar situation/dilemma few days ago and
got frustrated figuring out what version combination of hadoop/hbase to use
and how to build hadoop manually to be compatible with hbase. the build
process didn't work for me either.
eventually, i ended up using cloudera distribution and i think it saved me a
lot of headache and time.

thanks

On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <[hidden email]>wrote:

> Matt,
> Thanks a lot. Just wanted to have some more information. If hadoop
> 0.2.205.0
> voted by the community members then will it become major release? And what
> if it is not approved by community members.
>
> And as you said I do like to use 0.90.3 if it works. If it is ok, can you
> share the deails of those configuration changes?
>
> -Jignesh
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Jignesh Patel
Sorry to here that.
Is CDH3 is a open source or a paid version?

-jignesh
On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:

> for what its worth, i was in a similar situation/dilemma few days ago and
> got frustrated figuring out what version combination of hadoop/hbase to use
> and how to build hadoop manually to be compatible with hbase. the build
> process didn't work for me either.
> eventually, i ended up using cloudera distribution and i think it saved me a
> lot of headache and time.
>
> thanks
>
> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <[hidden email]>wrote:
>
>> Matt,
>> Thanks a lot. Just wanted to have some more information. If hadoop
>> 0.2.205.0
>> voted by the community members then will it become major release? And what
>> if it is not approved by community members.
>>
>> And as you said I do like to use 0.90.3 if it works. If it is ok, can you
>> share the deails of those configuration changes?
>>
>> -Jignesh
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
>> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

T Vinod Gupta
its free and open source too.. basically, their releases are ahead of public
releases of hadoop/hbase - from what i understand, major bug fixes and
enhancements are checked in to their branch first and then eventually make
it to public release branches.

thanks

On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel <[hidden email]> wrote:

> Sorry to here that.
> Is CDH3 is a open source or a paid version?
>
> -jignesh
> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
>
> > for what its worth, i was in a similar situation/dilemma few days ago and
> > got frustrated figuring out what version combination of hadoop/hbase to
> use
> > and how to build hadoop manually to be compatible with hbase. the build
> > process didn't work for me either.
> > eventually, i ended up using cloudera distribution and i think it saved
> me a
> > lot of headache and time.
> >
> > thanks
> >
> > On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <[hidden email]
> >wrote:
> >
> >> Matt,
> >> Thanks a lot. Just wanted to have some more information. If hadoop
> >> 0.2.205.0
> >> voted by the community members then will it become major release? And
> what
> >> if it is not approved by community members.
> >>
> >> And as you said I do like to use 0.90.3 if it works. If it is ok, can
> you
> >> share the deails of those configuration changes?
> >>
> >> -Jignesh
> >>
> >> --
> >> View this message in context:
> >>
> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
> >> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
> >>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Jignesh Patel
I have installed Hadoop-0.20.205.0 but when I replace the hadoop 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing it.

-Jignesh
On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:

> its free and open source too.. basically, their releases are ahead of public
> releases of hadoop/hbase - from what i understand, major bug fixes and
> enhancements are checked in to their branch first and then eventually make
> it to public release branches.
>
> thanks
>
> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel <[hidden email]> wrote:
>
>> Sorry to here that.
>> Is CDH3 is a open source or a paid version?
>>
>> -jignesh
>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
>>
>>> for what its worth, i was in a similar situation/dilemma few days ago and
>>> got frustrated figuring out what version combination of hadoop/hbase to
>> use
>>> and how to build hadoop manually to be compatible with hbase. the build
>>> process didn't work for me either.
>>> eventually, i ended up using cloudera distribution and i think it saved
>> me a
>>> lot of headache and time.
>>>
>>> thanks
>>>
>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <[hidden email]
>>> wrote:
>>>
>>>> Matt,
>>>> Thanks a lot. Just wanted to have some more information. If hadoop
>>>> 0.2.205.0
>>>> voted by the community members then will it become major release? And
>> what
>>>> if it is not approved by community members.
>>>>
>>>> And as you said I do like to use 0.90.3 if it works. If it is ok, can
>> you
>>>> share the deails of those configuration changes?
>>>>
>>>> -Jignesh
>>>>
>>>> --
>>>> View this message in context:
>>>>
>> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
>>>> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Jignesh Patel
The new plugin works after deleting eclipse and reinstalling it.
On Oct 12, 2011, at 2:39 PM, Jignesh Patel wrote:

> I have installed Hadoop-0.20.205.0 but when I replace the hadoop 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing it.
>
> -Jignesh
> On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:
>
>> its free and open source too.. basically, their releases are ahead of public
>> releases of hadoop/hbase - from what i understand, major bug fixes and
>> enhancements are checked in to their branch first and then eventually make
>> it to public release branches.
>>
>> thanks
>>
>> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel <[hidden email]> wrote:
>>
>>> Sorry to here that.
>>> Is CDH3 is a open source or a paid version?
>>>
>>> -jignesh
>>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
>>>
>>>> for what its worth, i was in a similar situation/dilemma few days ago and
>>>> got frustrated figuring out what version combination of hadoop/hbase to
>>> use
>>>> and how to build hadoop manually to be compatible with hbase. the build
>>>> process didn't work for me either.
>>>> eventually, i ended up using cloudera distribution and i think it saved
>>> me a
>>>> lot of headache and time.
>>>>
>>>> thanks
>>>>
>>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <[hidden email]
>>>> wrote:
>>>>
>>>>> Matt,
>>>>> Thanks a lot. Just wanted to have some more information. If hadoop
>>>>> 0.2.205.0
>>>>> voted by the community members then will it become major release? And
>>> what
>>>>> if it is not approved by community members.
>>>>>
>>>>> And as you said I do like to use 0.90.3 if it works. If it is ok, can
>>> you
>>>>> share the deails of those configuration changes?
>>>>>
>>>>> -Jignesh
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>>
>>> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
>>>>> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>>>>>
>>>
>>>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Jignesh Patel
When I tried to run Hbase 0.90.4 with hadoop-.0.20.205.0 I got following error

Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011

hbase(main):001:0> status

ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately. This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable's javadoc for more information.


And when I tried to stop Hbase I continuously sees dot being printed and no sign of stopping it. Not sure why it just simply stop it.

stopping hbase...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................….


On Oct 12, 2011, at 3:19 PM, Jignesh Patel wrote:

> The new plugin works after deleting eclipse and reinstalling it.
> On Oct 12, 2011, at 2:39 PM, Jignesh Patel wrote:
>
>> I have installed Hadoop-0.20.205.0 but when I replace the hadoop 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing it.
>>
>> -Jignesh
>> On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:
>>
>>> its free and open source too.. basically, their releases are ahead of public
>>> releases of hadoop/hbase - from what i understand, major bug fixes and
>>> enhancements are checked in to their branch first and then eventually make
>>> it to public release branches.
>>>
>>> thanks
>>>
>>> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel <[hidden email]> wrote:
>>>
>>>> Sorry to here that.
>>>> Is CDH3 is a open source or a paid version?
>>>>
>>>> -jignesh
>>>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
>>>>
>>>>> for what its worth, i was in a similar situation/dilemma few days ago and
>>>>> got frustrated figuring out what version combination of hadoop/hbase to
>>>> use
>>>>> and how to build hadoop manually to be compatible with hbase. the build
>>>>> process didn't work for me either.
>>>>> eventually, i ended up using cloudera distribution and i think it saved
>>>> me a
>>>>> lot of headache and time.
>>>>>
>>>>> thanks
>>>>>
>>>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <[hidden email]
>>>>> wrote:
>>>>>
>>>>>> Matt,
>>>>>> Thanks a lot. Just wanted to have some more information. If hadoop
>>>>>> 0.2.205.0
>>>>>> voted by the community members then will it become major release? And
>>>> what
>>>>>> if it is not approved by community members.
>>>>>>
>>>>>> And as you said I do like to use 0.90.3 if it works. If it is ok, can
>>>> you
>>>>>> share the deails of those configuration changes?
>>>>>>
>>>>>> -Jignesh
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>>> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
>>>>>> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>>>>>>
>>>>
>>>>
>>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Matt Foley
Hi Jignesh,
Not clear what's going on with your ZK, but as a starting point, the
hsync/flush feature in 205 was implemented with an on-off switch.  Make sure
you've turned it on by setting  *dfs.support.append  *to true in the
hdfs-site.xml config file.

Also, are you installing Hadoop with security turned on or off?

I'll gather some other config info that should help.
--Matt


On Wed, Oct 12, 2011 at 1:47 PM, Jignesh Patel <[hidden email]> wrote:

> When I tried to run Hbase 0.90.4 with hadoop-.0.20.205.0 I got following
> error
>
> Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase shell
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011
>
> hbase(main):001:0> status
>
> ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able
> to connect to ZooKeeper but the connection closes immediately. This could be
> a sign that the server has too many connections (30 is the default).
> Consider inspecting your ZK server logs for that error and then make sure
> you are reusing HBaseConfiguration as often as you can. See HTable's javadoc
> for more information.
>
>
> And when I tried to stop Hbase I continuously sees dot being printed and no
> sign of stopping it. Not sure why it just simply stop it.
>
> stopping
> hbase...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................….
>
>
> On Oct 12, 2011, at 3:19 PM, Jignesh Patel wrote:
>
> > The new plugin works after deleting eclipse and reinstalling it.
> > On Oct 12, 2011, at 2:39 PM, Jignesh Patel wrote:
> >
> >> I have installed Hadoop-0.20.205.0 but when I replace the hadoop
> 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing
> it.
> >>
> >> -Jignesh
> >> On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:
> >>
> >>> its free and open source too.. basically, their releases are ahead of
> public
> >>> releases of hadoop/hbase - from what i understand, major bug fixes and
> >>> enhancements are checked in to their branch first and then eventually
> make
> >>> it to public release branches.
> >>>
> >>> thanks
> >>>
> >>> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel <[hidden email]>
> wrote:
> >>>
> >>>> Sorry to here that.
> >>>> Is CDH3 is a open source or a paid version?
> >>>>
> >>>> -jignesh
> >>>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
> >>>>
> >>>>> for what its worth, i was in a similar situation/dilemma few days ago
> and
> >>>>> got frustrated figuring out what version combination of hadoop/hbase
> to
> >>>> use
> >>>>> and how to build hadoop manually to be compatible with hbase. the
> build
> >>>>> process didn't work for me either.
> >>>>> eventually, i ended up using cloudera distribution and i think it
> saved
> >>>> me a
> >>>>> lot of headache and time.
> >>>>>
> >>>>> thanks
> >>>>>
> >>>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <
> [hidden email]
> >>>>> wrote:
> >>>>>
> >>>>>> Matt,
> >>>>>> Thanks a lot. Just wanted to have some more information. If hadoop
> >>>>>> 0.2.205.0
> >>>>>> voted by the community members then will it become major release?
> And
> >>>> what
> >>>>>> if it is not approved by community members.
> >>>>>>
> >>>>>> And as you said I do like to use 0.90.3 if it works. If it is ok,
> can
> >>>> you
> >>>>>> share the deails of those configuration changes?
> >>>>>>
> >>>>>> -Jignesh
> >>>>>>
> >>>>>> --
> >>>>>> View this message in context:
> >>>>>>
> >>>>
> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
> >>>>>> Sent from the Hadoop lucene-users mailing list archive at
> Nabble.com.
> >>>>>>
> >>>>
> >>>>
> >>
> >
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Ramya Sunil
Hi Jignesh,

I have been running quite a few hbase tests on Hadoop 0.20.205 without any
issues on both secure and non secure clusters.

I have seen the error you mentioned when one has not specified the hbase
config directory.

Can you please try "hbase --config <path to hbase config directory> shell"
and check if that solves the problem?

Thanks
Ramya


On Wed, Oct 12, 2011 at 4:50 PM, Matt Foley <[hidden email]> wrote:

> Hi Jignesh,
> Not clear what's going on with your ZK, but as a starting point, the
> hsync/flush feature in 205 was implemented with an on-off switch.  Make
> sure
> you've turned it on by setting  *dfs.support.append  *to true in the
> hdfs-site.xml config file.
>
> Also, are you installing Hadoop with security turned on or off?
>
> I'll gather some other config info that should help.
> --Matt
>
>
> On Wed, Oct 12, 2011 at 1:47 PM, Jignesh Patel <[hidden email]>
> wrote:
>
> > When I tried to run Hbase 0.90.4 with hadoop-.0.20.205.0 I got following
> > error
> >
> > Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase shell
> > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > Type "exit<RETURN>" to leave the HBase Shell
> > Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011
> >
> > hbase(main):001:0> status
> >
> > ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is
> able
> > to connect to ZooKeeper but the connection closes immediately. This could
> be
> > a sign that the server has too many connections (30 is the default).
> > Consider inspecting your ZK server logs for that error and then make sure
> > you are reusing HBaseConfiguration as often as you can. See HTable's
> javadoc
> > for more information.
> >
> >
> > And when I tried to stop Hbase I continuously sees dot being printed and
> no
> > sign of stopping it. Not sure why it just simply stop it.
> >
> > stopping
> >
> hbase...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................….
> >
> >
> > On Oct 12, 2011, at 3:19 PM, Jignesh Patel wrote:
> >
> > > The new plugin works after deleting eclipse and reinstalling it.
> > > On Oct 12, 2011, at 2:39 PM, Jignesh Patel wrote:
> > >
> > >> I have installed Hadoop-0.20.205.0 but when I replace the hadoop
> > 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing
> > it.
> > >>
> > >> -Jignesh
> > >> On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:
> > >>
> > >>> its free and open source too.. basically, their releases are ahead of
> > public
> > >>> releases of hadoop/hbase - from what i understand, major bug fixes
> and
> > >>> enhancements are checked in to their branch first and then eventually
> > make
> > >>> it to public release branches.
> > >>>
> > >>> thanks
> > >>>
> > >>> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel <[hidden email]>
> > wrote:
> > >>>
> > >>>> Sorry to here that.
> > >>>> Is CDH3 is a open source or a paid version?
> > >>>>
> > >>>> -jignesh
> > >>>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
> > >>>>
> > >>>>> for what its worth, i was in a similar situation/dilemma few days
> ago
> > and
> > >>>>> got frustrated figuring out what version combination of
> hadoop/hbase
> > to
> > >>>> use
> > >>>>> and how to build hadoop manually to be compatible with hbase. the
> > build
> > >>>>> process didn't work for me either.
> > >>>>> eventually, i ended up using cloudera distribution and i think it
> > saved
> > >>>> me a
> > >>>>> lot of headache and time.
> > >>>>>
> > >>>>> thanks
> > >>>>>
> > >>>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel <
> > [hidden email]
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Matt,
> > >>>>>> Thanks a lot. Just wanted to have some more information. If hadoop
> > >>>>>> 0.2.205.0
> > >>>>>> voted by the community members then will it become major release?
> > And
> > >>>> what
> > >>>>>> if it is not approved by community members.
> > >>>>>>
> > >>>>>> And as you said I do like to use 0.90.3 if it works. If it is ok,
> > can
> > >>>> you
> > >>>>>> share the deails of those configuration changes?
> > >>>>>>
> > >>>>>> -Jignesh
> > >>>>>>
> > >>>>>> --
> > >>>>>> View this message in context:
> > >>>>>>
> > >>>>
> >
> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
> > >>>>>> Sent from the Hadoop lucene-users mailing list archive at
> > Nabble.com.
> > >>>>>>
> > >>>>
> > >>>>
> > >>
> > >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Giridharan Kesavan
In reply to this post by Matt Foley
Jignesh,

passing --config <path_to_hbase_configs> would help.

Like:
bin/hbase --config <path_to_hbase_configs> shell

-Giri

On 10/12/11 4:50 PM, Matt Foley wrote:

> Hi Jignesh,
> Not clear what's going on with your ZK, but as a starting point, the
> hsync/flush feature in 205 was implemented with an on-off switch.  Make sure
> you've turned it on by setting  *dfs.support.append  *to true in the
> hdfs-site.xml config file.
>
> Also, are you installing Hadoop with security turned on or off?
>
> I'll gather some other config info that should help.
> --Matt
>
>
> On Wed, Oct 12, 2011 at 1:47 PM, Jignesh Patel<[hidden email]>  wrote:
>
>> When I tried to run Hbase 0.90.4 with hadoop-.0.20.205.0 I got following
>> error
>>
>> Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase shell
>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
>> Type "exit<RETURN>" to leave the HBase Shell
>> Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011
>>
>> hbase(main):001:0>  status
>>
>> ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able
>> to connect to ZooKeeper but the connection closes immediately. This could be
>> a sign that the server has too many connections (30 is the default).
>> Consider inspecting your ZK server logs for that error and then make sure
>> you are reusing HBaseConfiguration as often as you can. See HTable's javadoc
>> for more information.
>>
>>
>> And when I tried to stop Hbase I continuously sees dot being printed and no
>> sign of stopping it. Not sure why it just simply stop it.
>>
>> stopping
>> hbase...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................….
>>
>>
>> On Oct 12, 2011, at 3:19 PM, Jignesh Patel wrote:
>>
>>> The new plugin works after deleting eclipse and reinstalling it.
>>> On Oct 12, 2011, at 2:39 PM, Jignesh Patel wrote:
>>>
>>>> I have installed Hadoop-0.20.205.0 but when I replace the hadoop
>> 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing
>> it.
>>>> -Jignesh
>>>> On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:
>>>>
>>>>> its free and open source too.. basically, their releases are ahead of
>> public
>>>>> releases of hadoop/hbase - from what i understand, major bug fixes and
>>>>> enhancements are checked in to their branch first and then eventually
>> make
>>>>> it to public release branches.
>>>>>
>>>>> thanks
>>>>>
>>>>> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel<[hidden email]>
>> wrote:
>>>>>> Sorry to here that.
>>>>>> Is CDH3 is a open source or a paid version?
>>>>>>
>>>>>> -jignesh
>>>>>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
>>>>>>
>>>>>>> for what its worth, i was in a similar situation/dilemma few days ago
>> and
>>>>>>> got frustrated figuring out what version combination of hadoop/hbase
>> to
>>>>>> use
>>>>>>> and how to build hadoop manually to be compatible with hbase. the
>> build
>>>>>>> process didn't work for me either.
>>>>>>> eventually, i ended up using cloudera distribution and i think it
>> saved
>>>>>> me a
>>>>>>> lot of headache and time.
>>>>>>>
>>>>>>> thanks
>>>>>>>
>>>>>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel<
>> [hidden email]
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Matt,
>>>>>>>> Thanks a lot. Just wanted to have some more information. If hadoop
>>>>>>>> 0.2.205.0
>>>>>>>> voted by the community members then will it become major release?
>> And
>>>>>> what
>>>>>>>> if it is not approved by community members.
>>>>>>>>
>>>>>>>> And as you said I do like to use 0.90.3 if it works. If it is ok,
>> can
>>>>>> you
>>>>>>>> share the deails of those configuration changes?
>>>>>>>>
>>>>>>>> -Jignesh
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>>
>> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
>>>>>>>> Sent from the Hadoop lucene-users mailing list archive at
>> Nabble.com.
>>>>>>
>>


--
-Giri

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

jigneshmpatel
There is no command like -config see below

Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase -config ./config shell
Unrecognized option: -config
Could not create the Java virtual machine.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

jigneshmpatel
Another thing I am using hadoop in psuedo single node server. But even if I don't start Hbase I will have same error.

ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately. This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable's javadoc for more information.


 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

jigneshmpatel
This post has NOT been accepted by the mailing list yet.
My log shows following error

2011-10-13 12:17:13,118 WARN org.apache.zookeeper.ClientCnxn: Session 0x132fe0c15e90002 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2011-10-13 12:17:13,375 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/fe80:0:0:0:0:0:0:1%1:2181
2011-10-13 12:17:13,376 WARN org.apache.zookeeper.ClientCnxn: Session 0x132fe0c15e90001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

At this time I am not running hadoop.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

jigneshmpatel
This post has NOT been accepted by the mailing list yet.
Actually real problem is here.

2011-10-12 16:00:12,834 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
java.io.IOException: Call to localhost/127.0.0.1:9000 failed on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy6.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:113)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:215)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:177)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:364)
at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:81)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:346)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:282)
at org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.run(HMasterCommandLine.java:193)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
2011-10-12 16:00:12,836 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2011-10-12 16:00:12,836 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads
2011-10-12 16:00:12,836 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 54993
2011-10-12 16:00:12,836 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 54993
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 54993: exiting
2011-10-12 16:00:12,836 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 54993: exiting
2011-10-12 16:00:12,837 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 54993: exiting
2011-10-12 16:00:12,838 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2011-10-12 16:00:12,843 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x132f9b8865f0000
2011-10-12 16:00:12,843 INFO org.apache.zookeeper.ZooKeeper: Session: 0x132f9b8865f0000 closed
2011-10-12 16:00:12,843 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2011-10-12 16:00:12,843 INFO org.apache.hadoop.hbase.master.HMaster: HMaster main thread exiting
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:54997 which had sessionid 0x132f9b8865f0002
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x132f9b8865f0002, likely server has closed socket, closing socket connection and attempting reconnect
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /0:0:0:0:0:0:0:1%0:54995 which had sessionid 0x132f9b8865f0001
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x132f9b8865f0001, likely server has closed socket, closing socket connection and attempting reconnect
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /0:0:0:0:0:0:0:1%0:54994 which had sessionid 0x132f9b8865f0000
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.server.NIOServerCnxn: NIOServerCnxn factory exited run method
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.server.PrepRequestProcessor: PrepRequestProcessor exited loop!
2011-10-12 16:00:12,844 INFO org.apache.zookeeper.server.SyncRequestProcessor: SyncRequestProcessor exited!
2011-10-12 16:00:12,845 INFO org.apache.zookeeper.server.FinalRequestProcessor: shutdown of request processor complete
2011-10-12 16:00:13,786 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/fe80:0:0:0:0:0:0:1%1:2181
2011-10-12 16:00:13,786 WARN org.apache.zookeeper.ClientCnxn: Session 0x132f9b8865f0002 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2011-10-12 16:00:13,806 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/fe80:0:0:0:0:0:0:1%1:2181
2011-10-12 16:00:13,806 WARN org.apache.zookeeper.ClientCnxn: Session 0x132f9b8865f0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

On Thu, Oct 13, 2011 at 12:19 PM, jigneshmpatel [via Lucene] <[hidden email]> wrote:
My log shows following error

2011-10-13 12:17:13,118 WARN org.apache.zookeeper.ClientCnxn: Session 0x132fe0c15e90002 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2011-10-13 12:17:13,375 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/fe80:0:0:0:0:0:0:1%1:2181
2011-10-13 12:17:13,376 WARN org.apache.zookeeper.ClientCnxn: Session 0x132fe0c15e90001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

At this time I am not running hadoop.


If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3419080.html
To unsubscribe from Hbase with Hadoop, click here.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Jignesh Patel
In reply to this post by Giridharan Kesavan
Actually real problem is here

http://pastebin.com/jyvpivt6

Moreover I didn't find any command like -config.

-Jignesh

On Oct 13, 2011, at 2:02 AM, giridharan kesavan wrote:

> Jignesh,
>
> passing --config <path_to_hbase_configs> would help.
>
> Like:
> bin/hbase --config <path_to_hbase_configs> shell
>
> -Giri
>
> On 10/12/11 4:50 PM, Matt Foley wrote:
>> Hi Jignesh,
>> Not clear what's going on with your ZK, but as a starting point, the
>> hsync/flush feature in 205 was implemented with an on-off switch.  Make sure
>> you've turned it on by setting  *dfs.support.append  *to true in the
>> hdfs-site.xml config file.
>>
>> Also, are you installing Hadoop with security turned on or off?
>>
>> I'll gather some other config info that should help.
>> --Matt
>>
>>
>> On Wed, Oct 12, 2011 at 1:47 PM, Jignesh Patel<[hidden email]>  wrote:
>>
>>> When I tried to run Hbase 0.90.4 with hadoop-.0.20.205.0 I got following
>>> error
>>>
>>> Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase shell
>>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
>>> Type "exit<RETURN>" to leave the HBase Shell
>>> Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011
>>>
>>> hbase(main):001:0>  status
>>>
>>> ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able
>>> to connect to ZooKeeper but the connection closes immediately. This could be
>>> a sign that the server has too many connections (30 is the default).
>>> Consider inspecting your ZK server logs for that error and then make sure
>>> you are reusing HBaseConfiguration as often as you can. See HTable's javadoc
>>> for more information.
>>>
>>>
>>> And when I tried to stop Hbase I continuously sees dot being printed and no
>>> sign of stopping it. Not sure why it just simply stop it.
>>>
>>> stopping
>>> hbase...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................….
>>>
>>>
>>> On Oct 12, 2011, at 3:19 PM, Jignesh Patel wrote:
>>>
>>>> The new plugin works after deleting eclipse and reinstalling it.
>>>> On Oct 12, 2011, at 2:39 PM, Jignesh Patel wrote:
>>>>
>>>>> I have installed Hadoop-0.20.205.0 but when I replace the hadoop
>>> 0.20.204.0 eclipse plugin with the 0.20.205.0, eclipse is not recognizing
>>> it.
>>>>> -Jignesh
>>>>> On Oct 12, 2011, at 12:31 PM, Vinod Gupta Tankala wrote:
>>>>>
>>>>>> its free and open source too.. basically, their releases are ahead of
>>> public
>>>>>> releases of hadoop/hbase - from what i understand, major bug fixes and
>>>>>> enhancements are checked in to their branch first and then eventually
>>> make
>>>>>> it to public release branches.
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>> On Wed, Oct 12, 2011 at 9:26 AM, Jignesh Patel<[hidden email]>
>>> wrote:
>>>>>>> Sorry to here that.
>>>>>>> Is CDH3 is a open source or a paid version?
>>>>>>>
>>>>>>> -jignesh
>>>>>>> On Oct 12, 2011, at 11:58 AM, Vinod Gupta Tankala wrote:
>>>>>>>
>>>>>>>> for what its worth, i was in a similar situation/dilemma few days ago
>>> and
>>>>>>>> got frustrated figuring out what version combination of hadoop/hbase
>>> to
>>>>>>> use
>>>>>>>> and how to build hadoop manually to be compatible with hbase. the
>>> build
>>>>>>>> process didn't work for me either.
>>>>>>>> eventually, i ended up using cloudera distribution and i think it
>>> saved
>>>>>>> me a
>>>>>>>> lot of headache and time.
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> On Tue, Oct 11, 2011 at 8:29 PM, jigneshmpatel<
>>> [hidden email]
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Matt,
>>>>>>>>> Thanks a lot. Just wanted to have some more information. If hadoop
>>>>>>>>> 0.2.205.0
>>>>>>>>> voted by the community members then will it become major release?
>>> And
>>>>>>> what
>>>>>>>>> if it is not approved by community members.
>>>>>>>>>
>>>>>>>>> And as you said I do like to use 0.90.3 if it works. If it is ok,
>>> can
>>>>>>> you
>>>>>>>>> share the deails of those configuration changes?
>>>>>>>>>
>>>>>>>>> -Jignesh
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>>
>>> http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3414658.html
>>>>>>>>> Sent from the Hadoop lucene-users mailing list archive at
>>> Nabble.com.
>>>>>>>
>>>
>
>
> --
> -Giri
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hbase with Hadoop

Harsh J-2
In reply to this post by jigneshmpatel
You'll need two hyphens before 'config'.

On 13-Oct-2011, at 9:00 PM, jigneshmpatel wrote:

> There is no command like -config see below
>
> Jignesh-MacBookPro:hadoop-hbase hadoop-user$ bin/hbase -config ./config
> shell
> Unrecognized option: -config
> Could not create the Java virtual machine.
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Hbase-with-Hadoop-tp3413950p3418924.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

12
Loading...