Luke reading index in hdfs

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Luke reading index in hdfs

MilleBii
Guys is there a way you can get Luke to read the index from hdfs:// ???
Or you have to copy it out to the local filesystem?

--
-MilleBii-
Reply | Threaded
Open this post in threaded view
|

Re: Luke reading index in hdfs

Andrzej Białecki-2
On 2009-12-11 22:21, MilleBii wrote:
> Guys is there a way you can get Luke to read the index from hdfs:// ???
> Or you have to copy it out to the local filesystem?
>

Luke 0.9.9 can open indexes directly from HDFS hosted on Hadoop 0.19.x.
Luke 0.9.9.1 can do the same, but uses Hadoop 0.20.1.

Start Luke, dismiss the open dialog, and then go to Plugins / Hadoop,
and enter the full URL of the index directory (including the hdfs://
part). You can also open multiple parts of the index (e.g. if you follow
the Nutch naming convention, you can directly open the "indexes/"
directory that contains part-NNNNN partial indexes).


--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

Reply | Threaded
Open this post in threaded view
|

Re: Luke reading index in hdfs

MilleBii
Great thx I can open it will help,
 but I don't get the summary page to be populated is this normal ???

2009/12/11 Andrzej Bialecki <[hidden email]>

> On 2009-12-11 22:21, MilleBii wrote:
>
>> Guys is there a way you can get Luke to read the index from hdfs:// ???
>> Or you have to copy it out to the local filesystem?
>>
>>
> Luke 0.9.9 can open indexes directly from HDFS hosted on Hadoop 0.19.x.
> Luke 0.9.9.1 can do the same, but uses Hadoop 0.20.1.
>
> Start Luke, dismiss the open dialog, and then go to Plugins / Hadoop, and
> enter the full URL of the index directory (including the hdfs:// part). You
> can also open multiple parts of the index (e.g. if you follow the Nutch
> naming convention, you can directly open the "indexes/" directory that
> contains part-NNNNN partial indexes).
>
>
> --
> Best regards,
> Andrzej Bialecki     <><
>  ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>


--
-MilleBii-