[jira] [Commented] (NUTCH-2759) bin/crawl: Rename option --num-slaves

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (NUTCH-2759) bin/crawl: Rename option --num-slaves

Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/NUTCH-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17011780#comment-17011780 ]

ASF GitHub Bot commented on NUTCH-2759:
---------------------------------------

sebastian-nagel commented on pull request #491: NUTCH-2759 bin/crawl: Rename option --num-slaves
URL: https://github.com/apache/nutch/pull/491
 
 
   Renamed option --num-slaves to --num-fetchers:
   ```
      --num-fetchers <num_fetchers>        Number of tasks used for fetching (fetcher map tasks) [default: 1]
                                           Note: This can only be set when running in distributed mode and
                                                 should correspond to the number of worker nodes in the cluster.
   ```
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


> bin/crawl: Rename option --num-slaves
> -------------------------------------
>
>                 Key: NUTCH-2759
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2759
>             Project: Nutch
>          Issue Type: Improvement
>          Components: bin
>    Affects Versions: 1.16
>            Reporter: Sebastian Nagel
>            Assignee: Sebastian Nagel
>            Priority: Minor
>             Fix For: 1.17
>
>
> The option {{--num-slaves}} of the script bin/crawl it isn't really precise: there is no need to set it to the actual number of nodes in the Hadoop cluster. In addition, Hadoop now uses the term "worker" , see HADOOP-13209.
> Renaming the option to {{\--num-fetcher-tasks}} or {{\--num-fetcher-workers}} would be more precise.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)