[jira] Commented: (NUTCH-185) XMLParser is configurable xml parser plugin.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (NUTCH-185) XMLParser is configurable xml parser plugin.

JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/NUTCH-185?page=comments#action_12451452 ]
           
Rida Benjelloun commented on NUTCH-185:
---------------------------------------

Nutch doesn't support multifieds values, so I decided to merge the content in the same field. If you want to search the field you should index it as "Text" instead of "keyword".



> XMLParser is configurable xml parser plugin.
> --------------------------------------------
>
>                 Key: NUTCH-185
>                 URL: http://issues.apache.org/jira/browse/NUTCH-185
>             Project: Nutch
>          Issue Type: New Feature
>          Components: fetcher, indexer
>    Affects Versions: 0.7.2, 0.8.1, 0.8
>         Environment: OS Independent
>            Reporter: Rida Benjelloun
>         Attachments: parse-xml.patch, parse-xml.zip, parse-xml.zip
>
>
> Xml parser  is configurable plugin. It use XPath and namespaces to do the mapping between the XML elements and Lucene fields.
> Informations :
> 1- Copy "xmlparser-conf.xml" to the nutch/conf dir
> 2- To index your custom XML file, you have to modify the "xmlparser-conf.xml".
> This parser uses namespaces and XPATH to parse XML content
> The config file do the mapping between the XML noeds (using XPATH) and lucene field.
> Example : <field name="dctitle" xpath="//dc:title" type="Text" boost="1.4" />
> 3- The xmlIndexerProperties encapsulate a set of fields associated to a namespace.
> If the namespace is found in the xml document, the fields represented by the namespace will be indexed.
> Example :
> <xmlIndexerProperties type="filePerDocument" namespace=" http://purl.org/dc/elements/1.1/">
>   <field name="dctitle" xpath="//dc:title" type="Text" boost=" 1.4" />
>   <field name="dccreator" xpath="//dc:creator" type="keyword" boost=" 1.0" />
> </xmlIndexerProperties>
> 4- It is possible to define a default namespace that will be applied when the parser
> didn't find any namespace in the document or when the namespace found in the xml document doesn't match with the namespace defined in the xmlIndexerProperties.
> Example :
> <xmlIndexerProperties type="filePerDocument" namespace="default">
>   <field name="xmlcontent" xpath="//*" type="Unstored" boost="1.0" />
> </xmlIndexerProperties>

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

RE: [jira] Commented: (NUTCH-185) XMLParser is configurable xml parser plugin.

Armel T. Nene-2
Rida,

There is something I would like to clarify, when using a namespace and xpath
to store content in the index, can this be seen as multi-fields. For example
if we are storing customer name and customer address which are been declared
in a xml configuration file, is that multi-field. Please explain, sorry I am
quite new to the Nutch architecture.

Armel

-----Original Message-----
From: Rida Benjelloun (JIRA) [mailto:[hidden email]]
Sent: 20 November 2006 22:16
To: [hidden email]
Subject: [jira] Commented: (NUTCH-185) XMLParser is configurable xml parser
plugin.

    [
http://issues.apache.org/jira/browse/NUTCH-185?page=comments#action_12451452
]
           
Rida Benjelloun commented on NUTCH-185:
---------------------------------------

Nutch doesn't support multifieds values, so I decided to merge the content
in the same field. If you want to search the field you should index it as
"Text" instead of "keyword".



> XMLParser is configurable xml parser plugin.
> --------------------------------------------
>
>                 Key: NUTCH-185
>                 URL: http://issues.apache.org/jira/browse/NUTCH-185
>             Project: Nutch
>          Issue Type: New Feature
>          Components: fetcher, indexer
>    Affects Versions: 0.7.2, 0.8.1, 0.8
>         Environment: OS Independent
>            Reporter: Rida Benjelloun
>         Attachments: parse-xml.patch, parse-xml.zip, parse-xml.zip
>
>
> Xml parser  is configurable plugin. It use XPath and namespaces to do the
mapping between the XML elements and Lucene fields.
> Informations :
> 1- Copy "xmlparser-conf.xml" to the nutch/conf dir
> 2- To index your custom XML file, you have to modify the
"xmlparser-conf.xml".
> This parser uses namespaces and XPATH to parse XML content
> The config file do the mapping between the XML noeds (using XPATH) and
lucene field.
> Example : <field name="dctitle" xpath="//dc:title" type="Text" boost="1.4"
/>
> 3- The xmlIndexerProperties encapsulate a set of fields associated to a
namespace.
> If the namespace is found in the xml document, the fields represented by
the namespace will be indexed.
> Example :
> <xmlIndexerProperties type="filePerDocument" namespace="
http://purl.org/dc/elements/1.1/">
>   <field name="dctitle" xpath="//dc:title" type="Text" boost=" 1.4" />
>   <field name="dccreator" xpath="//dc:creator" type="keyword" boost=" 1.0"
/>
> </xmlIndexerProperties>
> 4- It is possible to define a default namespace that will be applied when
the parser
> didn't find any namespace in the document or when the namespace found in
the xml document doesn't match with the namespace defined in the
xmlIndexerProperties.
> Example :
> <xmlIndexerProperties type="filePerDocument" namespace="default">
>   <field name="xmlcontent" xpath="//*" type="Unstored" boost="1.0" />
> </xmlIndexerProperties>

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira