Problems accessing the local file system on AWS MapReduce

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Problems accessing the local file system on AWS MapReduce

Steve Lewis-2
I am trying to use Distributes Cache on AWS Elastic Map educe

Using S3Fox I see I have two files in my bucket lordjoe

In My main I say
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
String[] otherArgs = new GenericOptionsParser(conf,
Job job = new Job(conf, "WholeFiles");
Path("s3n://lordjoe/Hello#Hello.x").toUri(), conf);
Path("s3n://lordjoe/FeeFie.txt#FeeFie.txt").toUri(), conf);
boolean ans = job.waitForCompletion(true);

In my reducer I say
protected void reduce(final KEYIN key, final Iterable<VALUEIN> values, final
Context context) throws IOException, InterruptedException {
Configuration configuration = context.getConfiguration();
Path[] files = DistributedCache.getLocalCacheFiles(configuration);
if (files != null) {
for (int i = 0; i < files.length; i++) {
Path file = files *" *
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA