[jira] Commented: (HADOOP-525) Need raw comparators for hadoop record types

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] Commented: (HADOOP-525) Need raw comparators for hadoop record types

JIRA jira@apache.org

    [ http://issues.apache.org/jira/browse/HADOOP-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12461831 ]

Milind Bhandarkar commented on HADOOP-525:

Yes, that would be a better test. I will file a separate bug for that, though.

> Need raw comparators for hadoop record types
> --------------------------------------------
>                 Key: HADOOP-525
>                 URL: http://issues.apache.org/jira/browse/HADOOP-525
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: record
>    Affects Versions: 0.6.0
>            Reporter: Sameer Paranjpye
>         Assigned To: Milind Bhandarkar
>             Fix For: 0.10.0
>         Attachments: raw-comparators.patch, TypeBuilder-support.tar, TypeBuilder.java, WordCountType.java
> Raw comparators are not generated for types that are generated with the Hadoop record framework. This could have a substantial performance impact when using hadoop record generated types in Map/Reduce. The record i/o framework should auto-generate raw comparators for types.
> Comparison for hadoop record i/o types is defined to be member wise comparison of objects. A possible implementation could only deserialize one member from each object at a time, compare them and either return or move on to the next member if the values are equal.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira