[jira] [Commented] (NUTCH-2746) Basic URL normalizer to normalize Unicode domain names

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (NUTCH-2746) Basic URL normalizer to normalize Unicode domain names

Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/NUTCH-2746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951950#comment-16951950 ]

ASF GitHub Bot commented on NUTCH-2746:
---------------------------------------

sebastian-nagel commented on pull request #480: NUTCH-2746 Basic URL normalizer to normalize Unicode domain names
URL: https://github.com/apache/nutch/pull/480
 
 
   - convert domain names IDN to ASCII or ASCII to IDN (configured by urlnormalizer.basic.host.idn)
   - strip trailing dot in host names (if urlnormalizer.basic.host.trim-trailing-dot is true)
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


> Basic URL normalizer to normalize Unicode domain names
> ------------------------------------------------------
>
>                 Key: NUTCH-2746
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2746
>             Project: Nutch
>          Issue Type: Improvement
>    Affects Versions: 1.16
>            Reporter: Sebastian Nagel
>            Priority: Major
>             Fix For: 1.17
>
>
> The BasicURLNormalizer (plugin urlnormalizer-basic) lacks the possibility to normalize IDNs (Unicode host/domain names).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)