[jira] [Commented] (NUTCH-1403) Add default ScoringFilter for manipulating metadata

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (NUTCH-1403) Add default ScoringFilter for manipulating metadata

Steve Loughran (Jira)

    [ https://issues.apache.org/jira/browse/NUTCH-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262888#comment-17262888 ]

ASF GitHub Bot commented on NUTCH-1403:

lewismc commented on a change in pull request #458:
URL: https://github.com/apache/nutch/pull/458#discussion_r555283893

File path: src/plugin/scoring-metadata/src/test/org/apache/nutch/scoring/metadata/MetadataScoringFilterTest.java
@@ -0,0 +1,113 @@
+package org.apache.nutch.scoring.metadata;

Review comment:
       Excellent @aalbahem I'll begin reviewing. Thank you

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

> Add default ScoringFilter for manipulating metadata
> ----------------------------------------------------
>                 Key: NUTCH-1403
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1403
>             Project: Nutch
>          Issue Type: Improvement
>            Reporter: Julien Nioche
>            Assignee: Sebastian Nagel
>            Priority: Major
>             Fix For: 1.18
> This is currently done by the urlmeta plugin, which has too vague a name and a redundant indexing filter now that we have the index-metadata plugin. This scoring filter would help defining which metadata to pass from :
> - the crawl metadata to the content metadata
> - the content metadata to the parse metadata
> - the parse metadata to the crawldatum for the outlinks
> I'd make this scoring filter available by default i.e. not in a separate plugin as its functionalities are commonly used.  

This message was sent by Atlassian Jira