[jira] [Commented] (NUTCH-2495) Use -deleteGone instead of clean job in crawler script while indexing

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (NUTCH-2495) Use -deleteGone instead of clean job in crawler script while indexing

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/NUTCH-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324719#comment-16324719 ]

ASF GitHub Bot commented on NUTCH-2495:
---------------------------------------

mfeltscher opened a new pull request #275: NUTCH-2495: Use -deleteGone instead of clean job in crawler script while indexing
URL: https://github.com/apache/nutch/pull/275
 
 
   Instead of running `bin/nutch` clean after indexing the documents run `bin/nutch index` with the `-deleteGone` flag which instead of just deleting gone and duplicated documents also deletes redirects from the index.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


> Use -deleteGone instead of clean job in crawler script while indexing
> ---------------------------------------------------------------------
>
>                 Key: NUTCH-2495
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2495
>             Project: Nutch
>          Issue Type: Improvement
>            Reporter: Moreno Feltscher
>            Assignee: Moreno Feltscher
>
> Instead of running {{bin/nutch clean}} after indexing the documents run {{bin/nutch index}} with the {{-deleteGone}} flag which instead of just deleting gone and duplicated documents also deletes redirects from the index.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)