[jira] [Created] (HADOOP-16467) failed to adding erasure code policies with example template

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (HADOOP-16467) failed to adding erasure code policies with example template

Tim Allison (Jira)
Yuan Zhou created HADOOP-16467:
----------------------------------

             Summary: failed to adding erasure code policies with example template
                 Key: HADOOP-16467
                 URL: https://issues.apache.org/jira/browse/HADOOP-16467
             Project: Hadoop Common
          Issue Type: Bug
          Components: common
    Affects Versions: 3.1.2
            Reporter: Yuan Zhou
         Attachments: fix_adding_EC_policy_example.diff

Hi Hadoop developers,

 

Trying to do some quick tests with erasure coding feature and ran into a issue on adding policies. The example on adding erasure code policies with provided template failed:
{quote}./bin/hdfs ec -addPolicies -policyFile /tmp/user_ec_policies.xml.template
 2019-07-30 10:35:16,447 INFO util.ECPolicyLoader: Loading EC policy file /tmp/user_ec_policies.xml.template
 Add ErasureCodingPolicy XOR-2-1-128k succeed.
 Add ErasureCodingPolicy RS-LEGACY-12-4-256k failed and error message is Codec name RS-legacy is not supported
{quote}
The issue seems due to be the mismatching codec(upper case vs lower case). The codec is in upper case in the example template[1] while all available codecs are lower case[2]. A way to fix maybe just converting the codec to lower case when parsing the policy schema. Also attached a simple patch here. 

[1] [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template#L51]

[2][https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template#L51]

Thanks, -yuan



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]